CN105718484A - File writing method, file reading method, file deletion method, file query method and client - Google Patents
File writing method, file reading method, file deletion method, file query method and client Download PDFInfo
- Publication number
- CN105718484A CN105718484A CN201410733953.9A CN201410733953A CN105718484A CN 105718484 A CN105718484 A CN 105718484A CN 201410733953 A CN201410733953 A CN 201410733953A CN 105718484 A CN105718484 A CN 105718484A
- Authority
- CN
- China
- Prior art keywords
- metadata
- file
- request
- server
- local database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 96
- 238000012217 deletion Methods 0.000 title claims abstract description 39
- 230000037430 deletion Effects 0.000 title claims abstract description 39
- 239000012634 fragment Substances 0.000 claims abstract description 42
- 230000004044 response Effects 0.000 claims description 40
- 230000001360 synchronised effect Effects 0.000 claims description 14
- 238000013500 data storage Methods 0.000 claims description 7
- 230000008030 elimination Effects 0.000 claims description 4
- 238000003379 elimination reaction Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 10
- 230000003993 interaction Effects 0.000 description 9
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Embodiments of the invention disclose a file writing method which is applied to a file access client. The method comprises the following steps: receiving a write request sent by an application program, wherein the write request is used for requesting to write to-be-written data in a target file; obtaining metadata of the target file from a local database, wherein the metadata at least comprises fragment position information of the target file; and writing the to-be-written data into a corresponding data server according to the fragment position information. The embodiments of the invention furthermore provide a file reading method, a file deletion method, a file query method and corresponding equipment.
Description
Technical Field
The invention relates to the field of distributed file systems, in particular to a method and a client for writing files, reading files, deleting files and inquiring files.
Background
With the wide application of distributed file systems, the file level is also rapidly increased from the million level to the billion, billion or billion level. In the case of such a huge amount of files, especially in the case of an increasing number of small files, an application needs to write, read or modify its metadata with high frequency, and then the file access client frequently interacts with the metadata, so that the metadata server is overwhelmed.
However, there is no reasonable solution to relieve the stress on the metadata server in the prior art.
Disclosure of Invention
In view of this, embodiments of the present invention are expected to provide a method and a client for writing a file, reading a file, deleting a file, and querying a file, so as to reduce interaction overhead between a file access client and a metadata server and reduce pressure on the metadata server to the greatest extent.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a method for writing a file, which is applied to a file access client, and the method includes: receiving a write request sent by an application program, wherein the write request is used for requesting to write data to be written into a target file; obtaining metadata of the target file from a local database, wherein the metadata at least comprises fragment position information of the target file; and writing the data to be written into the corresponding data server according to the fragment position information.
Further, before the writing the data to be written into the corresponding data server according to the fragment location information, the method further includes: sending a write lock request to a remote metadata server, wherein the write lock request is used for requesting the remote metadata server to write lock the metadata; receiving a write lock response returned by the remote metadata server, wherein the write lock response carries a timestamp; and saving the time stamp to the local database.
Further, after the writing the data to be written into the corresponding data server according to the fragment location information, the method further includes: updating the metadata in the local database; the updated metadata is synchronized to the remote metadata server.
Further, the synchronizing the updated metadata to the remote metadata server includes: in the process of writing the data to be written, acquiring the updated metadata from the local database at intervals of preset duration, and synchronizing the metadata to the remote metadata server; or receiving a closing request from the application program, acquiring the updated metadata from the local database, and synchronizing to the remote metadata server.
Further, before the receiving a write request sent by an application program, the method further includes: receiving a file creation request sent by the application program, wherein the file creation request is used for requesting to create the target file for the application program; sending an index node creating request to a directory tree server, wherein the index node creating request is used for requesting the directory tree server to create an index node for the target file; and receiving and adding the index information of the index node and the metadata from the directory tree server in the local database to complete the creation of the target file.
In a second aspect, an embodiment of the present invention provides a method for reading a file, which is applied to a file access client, where the method includes: receiving a reading request sent by an application program, wherein the reading request is used for requesting to read data to be read in a target file; obtaining metadata of the target file from a local database, wherein the metadata at least comprises fragment position information of the target file; reading the data to be read from a corresponding data server according to the fragment position information; and returning the data to be read to the application program.
Further, before the receiving a read request sent by an application program, the method further includes: receiving a read lock request sent by the application program, wherein the read lock request is used for requesting to read lock the target file; when the metadata is stored in the local database, returning a lock reading success response to the application program; and when the metadata is not stored in the local database, the metadata is acquired from a remote metadata server, then is stored in the local database, and returns the response of successful lock reading to the application program.
Further, after the receiving the read lock request sent by the application program, the method further includes: sending a timestamp of the target file to the remote metadata server, wherein the timestamp is used by the remote metadata server to determine whether the metadata is stored in the local database.
Further, after the returning the data to be read to the application program, the method further comprises: and after receiving a closing request sent by the application program, informing a remote metadata server to release the read lock of the target file, wherein the closing request is used for requesting to close the target file.
In a third aspect, an embodiment of the present invention provides a method for deleting a file, where the method is applied to a file access client, and the method includes: receiving a file deletion request sent by the application program; and deleting the metadata of the target file in the local database.
Further, after the receiving a request for deleting a file sent by the application program, the method further includes: sending a first deletion request to a remote metadata server, wherein the first deletion request is used for instructing the remote metadata server to delete the metadata; and receiving a deletion response sent by the remote metadata server.
Further, before the sending the first deletion request to the remote metadata server, the method further comprises: and sending a second deletion request to a directory tree server, wherein the second deletion request is used for indicating the directory tree server to delete the index information of the target file.
Further, after the receiving of the deletion response sent by the remote metadata server, the method further includes: and informing other file access clients that the target file is deleted.
In a fourth aspect, an embodiment of the present invention provides a method for querying a file, where the method is applied to a file access client, and the method includes: receiving a query request sent by an application program, wherein the query request is used for querying metadata of a target file; and after the metadata is inquired in the local database, returning the metadata to the application program.
Further, after receiving the query request sent by the application program, the method further includes: after the metadata is not inquired in the local database, acquiring the metadata from a remote metadata server; and storing the metadata in the local database, and returning the metadata to the application program.
Further, the method further comprises: and when the data storage capacity of the local database exceeds a elimination threshold value, deleting the metadata with the time stamp interval larger than a preset value in the remote metadata server according to the current synchronous time stamp of the local database.
In a fifth aspect, an embodiment of the present invention provides a file access client, where the file access client includes: the system comprises a file access module, a local metadata service module and a local database; the file access module is used for receiving a write request sent by an application program, wherein the write request is used for requesting to write data to be written into a target file; the data server is also used for writing the data to be written into the corresponding data server according to the fragment position information of the target file; the local metadata service module is configured to obtain metadata of the target file from a local database, where the metadata at least includes the fragment location information; the local database is used for storing the metadata.
Further, the local metadata service module is further configured to initiate a write lock request to a remote metadata server before the file access module writes the data to be written into the data server according to the fragment location information, where the write lock request is used to request the remote metadata server to write lock the metadata; receiving a write lock response returned by the remote metadata server, wherein the write lock response carries a timestamp; and saving the time stamp to the local database.
Further, the local metadata service module is further configured to update the metadata in the local database after the file access module writes the data to be written into the data server according to the fragment location information; the updated metadata is synchronized to the remote metadata server.
Further, the local metadata service module is specifically configured to, in the process of writing the data to be written, obtain, every interval of a preset duration, the updated metadata from the local database, and synchronize the metadata to the remote metadata server; or receiving a closing request from the application program, acquiring the updated metadata from the local database, and synchronizing to the remote metadata server.
Further, the file access module is further configured to receive a file creation request sent by the application program before receiving the write request, where the file creation request is used to request that the target file be created for the application program; the local metadata service module is further configured to send an index node creation request to a directory tree server, where the index node creation request is used to request the directory tree server to create an index node for the target file; and receiving and adding the index information of the index node and the metadata from the directory tree server in the local database to complete the creation of the target file.
Further, the file access module is further configured to receive a read request sent by an application program, where the read request is used to request to read data to be read in a target file; the data reading module is also used for returning the data to be read to the application program; the local metadata service module is further configured to obtain metadata of the target file from the local database, where the metadata at least includes fragment location information of the target file; and reading the data to be read from the corresponding data server according to the fragment position information.
Further, the file access module is configured to receive a read lock request sent by an application before receiving a read request sent by the application, where the read lock request is used to request to perform read lock on the target file; when the metadata is stored in the local database, returning a lock reading success response to the application program; the system is also used for returning the reading lock success response to the application program after the metadata is stored in the local database when the metadata is not stored in the local database; the local metadata service module is further configured to, when the metadata is not stored in the local database, obtain the metadata from a remote metadata server and store the metadata in the local database.
Further, the local metadata service module is configured to send a timestamp of the target file to the remote metadata server after the file access module receives the read lock request sent by the application, where the timestamp is used by the remote metadata server to determine whether the metadata is stored in the local database.
Further, the file access module is further configured to receive a close request sent by the application program after the data to be read is returned to the application program, where the close request is used to request to close the target file; and the local metadata service module is also used for informing the remote metadata server of releasing the read lock of the target file.
Further, the file access module is further configured to receive a file deletion request sent by the application program; the local metadata service module is also used for deleting the metadata of the target file in the local database.
Further, the local metadata service module is further configured to send a first deletion request to a remote metadata server after the file access module receives a file deletion request sent by the application program, where the first deletion request is used to instruct the remote metadata server to delete the metadata; and receiving a deletion response sent by the remote metadata server.
Further, the local metadata service module is further configured to send a second delete request to the directory tree server before sending the first delete request to the remote metadata server, where the second delete request is used to instruct the directory tree server to delete the index information of the target file.
Further, the local metadata service module is further configured to notify other file access clients that the target file is deleted after receiving the deletion response sent by the remote metadata server.
Further, the file access module is further configured to receive a query request sent by an application program, where the query request is used to query metadata of a target file; and also for returning the metadata to the application; the local metadata service module is further configured to query the metadata in a local database.
Further, the local metadata service module is further configured to, after the file access module receives the query request sent by the application program and after the metadata is not queried in the local database, obtain the metadata from a remote metadata server; storing the metadata in the local database; the file access module is further configured to return the metadata to the application program.
Further, the local metadata service module is further configured to delete, when the data storage amount of the local database exceeds a culling threshold value, metadata whose time stamp interval with a time stamp in the remote metadata server is greater than a preset value according to the time stamp currently synchronized with the local database.
By adopting the method and the client for writing the file, reading the file, deleting the file and inquiring the file, the file access client stores the metadata of the application program in the local data, so that the application program can be directly obtained from the local database when needing to use the metadata without frequent interaction with a remote metadata server, thereby reducing the interaction overhead of the file access client and the metadata server and reducing the pressure of the metadata server to the maximum extent.
Drawings
Fig. 1 is a schematic structural diagram of a file access client in an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for writing a file according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a method for reading a file according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a method for deleting a file according to an embodiment of the present invention;
fig. 5 is a flowchart illustrating a method for querying a file according to an embodiment of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
The embodiment of the invention provides a method for writing a file, which is applied to a file access client. In practical application, the file access client may be a data card, a mobile terminal, or the like.
Fig. 1 is a schematic structural diagram of a file access client in an embodiment of the present invention, and referring to fig. 1, the file access client includes: a file access module 11, a local metadata service module 12 and a local database 13.
The file access module 11 is used for performing data interaction with an application program running on a file access client and a plurality of data servers; the local metadata server 12 may perform data interaction with the directory tree server, the remote metadata server, and other file access clients; the local database 13 is used for storing metadata of applications and also for metadata synchronization with remote metadata servers.
The following describes the working process of the file client in conjunction with the embodiment of the present invention.
The first embodiment is as follows:
fig. 2 is a schematic flow chart of a method for writing a file in an embodiment of the present invention, and referring to fig. 2, the method includes:
s201: receiving a write request sent by an application program, wherein the write request is used for requesting to write data to be written into a target file;
in another embodiment, in order to make the file access client have the right to write the file, before S201, the method further includes: sending a write lock request to a remote metadata server, wherein the write lock request is used for requesting the remote metadata server to write lock metadata; receiving a write lock response returned by the remote metadata server, wherein the write lock response carries a timestamp; and saving the time stamp to a local database.
Specifically, the local metadata service module initiates a write lock request to the remote metadata server, the remote metadata server responds to the request, writes the metadata of the lock target file, and returns a timestamp to the local metadata service module, and the local database stores the timestamp to update the metadata of the target file stored in the local database.
Further, before the file access client obtains the write lock, the target file needs to be created, and then the method further includes: receiving a file creating request sent by an application program, wherein the file creating request is used for requesting to create a target file for the application program; sending an index node creating request to a directory tree server, wherein the index node creating request is used for requesting the directory tree server to create an index node for a file; and receiving and adding index information and metadata of the index nodes from the directory tree server in the local database to complete the creation of the target file.
Specifically, before writing a file, an object file may be created, at this time, the application program sends a file creation request to the file access module, the file access module calls an add file (addfile) interface of the local metadata service module, requests the directory tree server to add a new index node, then, the directory tree server responds to the request, creates a new index node, and sends index information of the index node to the local metadata service module, and the local metadata service module receives the index node, stores the index information in the local database, and may also store metadata of the new file, at this time, the created file is the object file in one or more embodiments described above. Next, the local metadata service module may also acquire a write lock from the remote metadata server.
In this way, the creation of the file is basically completed at the file access client, and the interaction with the directory tree server and the remote metadata server required in the process is to ensure the uniqueness of the index node, and the directory tree server is required to allocate unique index information and a corresponding remote metadata server position.
S202: obtaining metadata of a target file from a local database, wherein the metadata at least comprises fragment position information of the target file;
specifically, after the application program sends a write request to the file access module, the file access module responds to the request to request fragment position information from the local metadata service module, and the local metadata module calls the local database interface to return the fragment position information.
In practical application, the metadata may further include information such as a file name, a file size, an access right, and a data fragment size of the target file, and the present invention is not limited specifically.
S203: and writing the data to be written into the corresponding data server according to the fragment position information.
Specifically, the application writes the write data to the file access client, and the file access client writes the data to the data server to request the write of the data to be written to the target file.
Further, after S203, since the target file has new data written therein, the local metadata service module needs to update the metadata of the target file stored in the local database, and then the method further includes: updating metadata in a local database; the updated metadata is synchronized to the remote metadata server.
Further, after receiving the updated metadata, the remote metadata server may further determine whether there are other file access clients to open the target file, and if so, broadcast the updated metadata to the file access clients while synchronizing the updated metadata; if not, only the updated metadata is synchronized.
It should be noted that, the updating of the metadata in the local database is to update the local metadata service module serial number, the remote metadata server serial number, and the fragment allocation timer. The data can be acquired only by the local metadata service module and is not acquired by the remote metadata server, and interaction with the remote metadata server is reduced.
In a specific implementation process, the time consumed for writing some files is long, and at this time, in the process of writing data to be written, the local metadata service module updates the metadata in the local database once every preset time interval, acquires the updated metadata from the local database, and synchronizes the updated metadata to the remote metadata server; of course, the updated metadata may also be obtained from the local database and synchronized to the remote metadata server after receiving a close request from the application.
At this point, the process of writing the file is completed.
Example two:
fig. 3 is a schematic flowchart of a method for reading a file according to an embodiment of the present invention, and referring to fig. 3, the method includes:
s301: receiving a reading request sent by an application program, wherein the reading request is used for requesting to read data to be read in a target file;
specifically, when the application needs to read a certain part of data in the target file, a read request may be sent to the file access module, requesting to read the data to be read in the target file.
Further, before reading the file, the file access client needs to obtain the right to read the file, so before S301, the method further includes: receiving a reading lock request sent by an application program, wherein the reading lock request is used for requesting a reading lock target file; when the metadata is stored in the local database, returning a response of successful lock reading to the application program; and when the metadata is not stored in the local database, the metadata is stored in the local database after being acquired from the remote metadata server, and a response of successful lock reading is returned to the application program.
Specifically, an application program sends a lock reading request to a file access module, the file access module responds to the request and calls an Open (Open) function of a local metadata service module, if metadata of a target file is stored in a local database, a recorded timestamp is taken out, then the local metadata service module sends the lock reading request carrying the timestamp to a remote metadata server, the remote metadata server responds to the lock reading request and judges whether the timestamp sent by the local metadata service module is equal to the timestamp recorded by the local metadata service module, if so, the local metadata server indicates that the metadata of the target file is stored in the local database, and at the moment, direct response lock reading is successful; if not, the metadata of the target file is not stored in the local database, at this time, the metadata of the target file is sent to the local metadata service module and a lock reading success response, then, the local metadata service module stores the acquired metadata in the local database for later subsequent use, and returns the lock reading success response to the application program.
S302: obtaining metadata of a target file from a local database, wherein the metadata at least comprises fragment position information of the target file;
specifically, after the application program sends a read request to the file access module, the file access module responds to the request to request fragment position information from the local metadata service module, and the local metadata module calls the local database interface to return the fragment position information.
S303: reading data to be read from a corresponding data server according to the fragment position information;
s304: and returning the data to be read to the application program.
Specifically, the file access module takes out the data to be read from the corresponding data server according to the fragment position information, and returns the data to the application program.
Further, after the application finishes reading the file, a Close request may also be sent to the file access module to request to Close the target file, and the file access module responds to the request, invokes a Close function of the local metadata service module, closes the target file, and notifies the remote metadata server to release the read lock of the target file, where the Close request is used to request to Close the target file.
At this point, the process of reading the file is completed.
Example three:
fig. 4 is a schematic flowchart of a method for deleting a file in an embodiment of the present invention, and as shown in fig. 4, the method includes:
s401: receiving a file deleting request sent by an application program;
specifically, when the application needs to delete the target file, a delete file request may be issued to the file access module.
S402: and deleting the metadata of the target file in the local database.
Specifically, the file access module responds to the request, calls the local metadata service module, and deletes the metadata of the target file in the local database.
Further, after S402, the local metadata service module may further send a first deletion request to the remote metadata server, where the first deletion request is used to instruct the remote metadata server to delete the metadata; and receiving a deletion response sent by the remote metadata server.
Specifically, after the local metadata service module sends a first deletion request to the remote metadata server, the remote metadata server responds to the request, deletes the metadata of the target file stored by the remote metadata server, and notifies the corresponding data server of deleting the fragment data, that is, notifies the data server of deleting the target file, and finally, performs a deletion response to the local metadata service module.
Further, after S402, the local metadata service module may further send a second deletion request to the directory tree server, where the second deletion request is used to instruct the directory tree server to delete the index information of the target file.
Specifically, the local metadata server transmits a second deletion request to the directory tree server, and the directory tree server deletes the index information of the target file stored by itself in response to the second deletion request.
In practical applications, after the target file is deleted, the local metadata service module may also notify other file access clients that the target file has been deleted.
At this point, the process of deleting the file is completed.
Example four:
fig. 5 is a schematic flow chart of a method for querying a file in an embodiment of the present invention, and as shown in fig. 5, the method includes:
s501: receiving a query request sent by an application program, wherein the query request is used for querying metadata of a target file;
in particular, when an application needs to query the metadata of a target file, a query request may be sent to the file access module.
S502: and after the metadata is inquired in the local database, returning the metadata to the application program.
Specifically, the file access module responds to the request, calls a local metadata service module, queries whether the metadata exists in a local database, and directly returns the metadata to the application program if the metadata exists; if the metadata does not exist, the metadata is obtained from the remote metadata server after the metadata is not inquired in the local database, then the metadata from the remote metadata server is stored in the local database, and the metadata is returned to the application program.
In practical application, because the data storage amount of the local database is limited, when the data storage amount of the local database exceeds the elimination threshold value, according to the current synchronous timestamp of the local database, the metadata with the timestamp interval greater than the preset value in the remote metadata server is deleted.
In another embodiment, after the file access client is restarted, when the local database is powered on, the local metadata service module determines metadata that needs to be reacquired by the remote metadata server, and then, when the file access client is idle, the local metadata service module may synchronize with the remote metadata server in the background, for example, when the file access client is idle, the local metadata service module traverses all metadata in the local database, packages index information and timestamps of the metadata that needs to be reacquired onto the remote metadata server for comparison, and if the timestamps are consistent, the remote metadata server returns a data consistency response; and if not, waiting for the next time of triggering update of the application program.
At this point, the process of querying the file is completed.
Therefore, the file access client stores the metadata of the application program in the local data, so that the application program can be directly acquired from the local database when the application program needs to use the metadata without frequent interaction with a remote metadata server, thereby reducing the interaction overhead of the file access client and the metadata server and reducing the pressure of the metadata server to the maximum extent.
Based on the same inventive concept, the embodiment of the present invention further provides a file access client, which is consistent with the file access client described in one or more embodiments above.
Referring to fig. 1, the file access client includes: a file access module 11, a local metadata service module 12 and a local database 13; the file access module 11 is configured to receive a write request sent by an application program, where the write request is used to request to write data to be written into a target file; the data server is also used for writing the data to be written into the corresponding data server according to the fragment position information of the target file; a local metadata service module 12, configured to obtain metadata of a target file from a local database 13, where the metadata at least includes fragment location information; a local database 13 for storing metadata.
Further, the local metadata service module 12 is further configured to send a write lock request to the remote metadata server before the file access module 11 writes the data to be written into the data server according to the fragment location information, where the write lock request is used to request the remote metadata server to write lock metadata; receiving a write lock response returned by the remote metadata server, wherein the write lock response carries a timestamp; the time stamp is saved to the local database 13.
Further, the local metadata service module 12 is further configured to update metadata in the local database 13 after the file access module 11 writes the data to be written into the data server according to the fragment location information; the updated metadata is synchronized to the remote metadata server.
Further, the local metadata service module 12 is specifically configured to, in the process of writing data to be written, obtain updated metadata from the local database 13 at preset time intervals, and synchronize the updated metadata to the remote metadata server; or, receive a close request from the application, obtain the updated metadata from the local database 13, and synchronize to the remote metadata server.
Further, the file access module 11 is further configured to receive a file creation request sent by the application program before receiving the write request, where the file creation request is used to request to create a target file for the application program;
the local metadata service module 12 is further configured to send an index node creation request to the directory tree server, where the index node creation request is used to request the directory tree server to create an index node for the target file; and receiving and adding index information and metadata of the index nodes from the directory tree server in the local database 13 to complete the creation of the target file.
Further, the file access module 11 is further configured to receive a read request sent by the application program, where the read request is used to request to read data to be read in the target file; the data reading device is also used for returning the data to be read to the application program;
the local metadata service module 12 is further configured to obtain metadata of the target file from the local database 13, where the metadata at least includes fragment location information of the target file; and reading the data to be read from the corresponding data server according to the fragment position information.
Further, the file access module 11 is configured to receive a read lock request sent by an application before receiving a read request sent by the application, where the read lock request is used to request a read lock target file; when the metadata is stored in the local database, returning a response of successful lock reading to the application program; the system is also used for returning a lock reading success response to the application program after the metadata is stored in the local database when the metadata is not stored in the local database; and the local metadata service module is also used for obtaining the metadata from the remote metadata server and storing the metadata in the local database when the metadata is not stored in the local database.
Further, the local metadata service module 12 is configured to send a timestamp of the target file to the remote metadata server after the file access module 11 receives the read lock request sent by the application, where the timestamp is used by the remote metadata server to determine whether the metadata is stored in the local database 13.
Further, the file access module 11 is further configured to receive a closing request sent by the application program after the data to be read is returned to the application program, where the closing request is used to request to close the target file; the local metadata service module 12 is further configured to notify the remote metadata server to release the read lock of the target file.
Further, the file access module 11 is further configured to receive a file deletion request sent by the application program; the local metadata service module 12 is further configured to delete the metadata of the target file in the local database 13.
Further, the local metadata service module 12 is further configured to send a first deletion request to the remote metadata server after the file access module 11 receives a file deletion request sent by the application program, where the first deletion request is used to instruct the remote metadata server to delete metadata; and receiving a deletion response sent by the remote metadata server.
Further, the local metadata service module 12 is further configured to send a second delete request to the directory tree server before sending the first delete request to the remote metadata server, where the second delete request is used to instruct the directory tree server to delete the index information of the target file.
Further, the local metadata service module 12 is further configured to notify other file access clients that the target file is deleted after receiving the deletion response sent by the remote metadata server.
Further, the file access module 11 is further configured to receive a query request sent by an application program, where the query request is used to query metadata of a target file; and also for returning metadata to the application; the local metadata service module 12 is further configured to query the local database 13 for metadata.
Further, the local metadata service module 12 is further configured to, after the file access module 11 receives the query request sent by the application program and after the metadata is not queried in the local database 13, obtain the metadata from the remote metadata server; storing the metadata in the local database 13; the file access module 11 is also used for returning the metadata to the application program.
Further, the local metadata service module 12 is further configured to delete, when the data storage amount of the local database 13 exceeds the elimination threshold value, the metadata whose time stamp interval with the time stamp in the remote metadata server is greater than the preset value according to the time stamp currently synchronized with the local database 13.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (32)
1. A method for writing a file, which is applied to a file access client, is characterized in that the method comprises the following steps:
receiving a write request sent by an application program, wherein the write request is used for requesting to write data to be written into a target file;
obtaining metadata of the target file from a local database, wherein the metadata at least comprises fragment position information of the target file;
and writing the data to be written into the corresponding data server according to the fragment position information.
2. The method according to claim 1, wherein before the writing the data to be written into the corresponding data server according to the slice location information, the method further comprises:
sending a write lock request to a remote metadata server, wherein the write lock request is used for requesting the remote metadata server to write lock the metadata;
receiving a write lock response returned by the remote metadata server, wherein the write lock response carries a timestamp;
and saving the time stamp to the local database.
3. The method according to claim 1, wherein after the writing the data to be written into the corresponding data server according to the slice location information, the method further comprises:
updating the metadata in the local database;
the updated metadata is synchronized to the remote metadata server.
4. The method of claim 3, wherein synchronizing the updated metadata to a remote metadata server comprises:
in the process of writing the data to be written, acquiring the updated metadata from the local database at intervals of preset duration, and synchronizing the metadata to the remote metadata server; or,
and receiving a closing request from the application program, acquiring the updated metadata from the local database, and synchronizing the metadata to the remote metadata server.
5. The method of claim 1, wherein prior to said receiving a write request sent by an application, the method further comprises:
receiving a file creation request sent by the application program, wherein the file creation request is used for requesting to create the target file for the application program;
sending an index node creating request to a directory tree server, wherein the index node creating request is used for requesting the directory tree server to create an index node for the target file;
and receiving and adding the index information of the index node and the metadata from the directory tree server in the local database to complete the creation of the target file.
6. A method for reading a file is applied to a file access client, and is characterized by comprising the following steps:
receiving a reading request sent by an application program, wherein the reading request is used for requesting to read data to be read in a target file;
obtaining metadata of the target file from a local database, wherein the metadata at least comprises fragment position information of the target file;
reading the data to be read from a corresponding data server according to the fragment position information;
and returning the data to be read to the application program.
7. The method of claim 6, wherein prior to the receiving the read request sent by the application, the method further comprises:
receiving a read lock request sent by the application program, wherein the read lock request is used for requesting to read lock the target file;
when the metadata is stored in the local database, returning a lock reading success response to the application program;
and when the metadata is not stored in the local database, the metadata is acquired from a remote metadata server, then is stored in the local database, and returns the response of successful lock reading to the application program.
8. The method of claim 7, wherein after the receiving the read lock request sent by the application, the method further comprises:
sending a timestamp of the target file to the remote metadata server, wherein the timestamp is used by the remote metadata server to determine whether the metadata is stored in the local database.
9. The method of claim 6, wherein after the returning the data to be read to the application, the method further comprises:
and after receiving a closing request sent by the application program, informing a remote metadata server to release the read lock of the target file, wherein the closing request is used for requesting to close the target file.
10. A method for deleting files is applied to a file access client, and is characterized in that the method comprises the following steps:
receiving a file deletion request sent by the application program;
and deleting the metadata of the target file in the local database.
11. The method of claim 10, wherein after receiving a delete file request sent by the application, the method further comprises:
sending a first deletion request to a remote metadata server, wherein the first deletion request is used for instructing the remote metadata server to delete the metadata;
and receiving a deletion response sent by the remote metadata server.
12. The method of claim 11, wherein prior to said sending the first delete request to the remote metadata server, the method further comprises:
and sending a second deletion request to a directory tree server, wherein the second deletion request is used for indicating the directory tree server to delete the index information of the target file.
13. The method of claim 11, wherein after said receiving a delete reply sent by said remote metadata server, said method further comprises:
and informing other file access clients that the target file is deleted.
14. A method for inquiring files is applied to a file access client, and is characterized in that the method comprises the following steps:
receiving a query request sent by an application program, wherein the query request is used for querying metadata of a target file;
and after the metadata is inquired in the local database, returning the metadata to the application program.
15. The method of claim 14, wherein after receiving the query request sent by the application program, the method further comprises:
after the metadata is not inquired in the local database, acquiring the metadata from a remote metadata server;
and storing the metadata in the local database, and returning the metadata to the application program.
16. The method of claim 14, further comprising:
and when the data storage capacity of the local database exceeds a elimination threshold value, deleting the metadata with the time stamp interval larger than a preset value in the remote metadata server according to the current synchronous time stamp of the local database.
17. A file access client, the file access client comprising: the system comprises a file access module, a local metadata service module and a local database; wherein,
the file access module is used for receiving a write request sent by an application program, wherein the write request is used for requesting to write data to be written into a target file; the data server is also used for writing the data to be written into the corresponding data server according to the fragment position information of the target file;
the local metadata service module is configured to obtain metadata of the target file from a local database, where the metadata at least includes the fragment location information;
the local database is used for storing the metadata.
18. The file access client according to claim 17, wherein the local metadata service module is further configured to send a write lock request to a remote metadata server before the file access module writes the data to be written into the data server according to the fragment location information, where the write lock request is used to request the remote metadata server to write lock the metadata; receiving a write lock response returned by the remote metadata server, wherein the write lock response carries a timestamp; and saving the time stamp to the local database.
19. The file access client according to claim 17, wherein the local metadata service module is further configured to update the metadata in the local database after the file access module writes the data to be written into the data server according to the fragment location information; the updated metadata is synchronized to the remote metadata server.
20. The file access client according to claim 19, wherein the local metadata service module is specifically configured to, during the process of writing the data to be written, obtain the updated metadata from the local database at every preset time interval, and synchronize the updated metadata to the remote metadata server; or receiving a closing request from the application program, acquiring the updated metadata from the local database, and synchronizing to the remote metadata server.
21. The file access client of claim 17, wherein the file access module is further configured to receive a file creation request sent by the application program before receiving the write request, wherein the file creation request is used to request that the target file be created for the application program;
the local metadata service module is further configured to send an index node creation request to a directory tree server, where the index node creation request is used to request the directory tree server to create an index node for the target file; and receiving and adding the index information of the index node and the metadata from the directory tree server in the local database to complete the creation of the target file.
22. The file access client according to claim 17, wherein the file access module is further configured to receive a read request sent by an application program, where the read request is used to request to read data to be read in a target file; the data reading module is also used for returning the data to be read to the application program;
the local metadata service module is further configured to obtain metadata of the target file from the local database, where the metadata at least includes fragment location information of the target file; and reading the data to be read from the corresponding data server according to the fragment position information.
23. The file access client of claim 22, wherein the file access module is configured to receive a read lock request sent by an application before receiving a read request sent by the application, where the read lock request is used to request to read lock the target file; when the metadata is stored in the local database, returning a lock reading success response to the application program; the system is also used for returning the reading lock success response to the application program after the metadata is stored in the local database when the metadata is not stored in the local database;
the local metadata service module is further configured to, when the metadata is not stored in the local database, obtain the metadata from a remote metadata server and store the metadata in the local database.
24. The file access client of claim 23, wherein the local metadata service module is configured to send a timestamp of the target file to the remote metadata server after the file access module receives the read lock request sent by the application, wherein the timestamp is used by the remote metadata server to determine whether the metadata is stored in the local database.
25. The file access client according to claim 22, wherein the file access module is further configured to receive a close request sent by the application program after the data to be read is returned to the application program, where the close request is used to request to close the target file;
and the local metadata service module is also used for informing the remote metadata server of releasing the read lock of the target file.
26. The file access client of claim 17, wherein the file access module is further configured to receive a delete file request sent by the application;
the local metadata service module is also used for deleting the metadata of the target file in the local database.
27. The file access client of claim 26, wherein the local metadata service module is further configured to send a first delete request to a remote metadata server after the file access module receives a delete file request sent by the application, wherein the first delete request is used to instruct the remote metadata server to delete the metadata; and receiving a deletion response sent by the remote metadata server.
28. The file access client of claim 27, wherein the local metadata service module is further configured to send a second delete request to the directory tree server before sending the first delete request to the remote metadata server, wherein the second delete request is used to instruct the directory tree server to delete the index information of the target file.
29. The file access client of claim 26, wherein the local metadata service module is further configured to notify other file access clients that the target file has been deleted after receiving the deletion response sent by the remote metadata server.
30. The file access client of claim 17, wherein the file access module is further configured to receive a query request sent by an application program, wherein the query request is used to query metadata of a target file; and also for returning the metadata to the application;
the local metadata service module is further configured to query the metadata in a local database.
31. The file access client of claim 30, wherein the local metadata service module is further configured to obtain the metadata from a remote metadata server after the file access module receives the query request sent by the application program and the metadata is not queried in the local database; storing the metadata in the local database;
the file access module is further configured to return the metadata to the application program.
32. The file access client of claim 30, wherein the local metadata service module is further configured to delete metadata that is separated from a timestamp in the remote metadata server by more than a predetermined value according to a timestamp currently synchronized with the local database when the data storage amount of the local database exceeds a de-selection threshold.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410733953.9A CN105718484A (en) | 2014-12-04 | 2014-12-04 | File writing method, file reading method, file deletion method, file query method and client |
| PCT/CN2015/074949 WO2016086552A1 (en) | 2014-12-04 | 2015-03-24 | File processing method and client |
| PCT/CN2015/082131 WO2016086649A1 (en) | 2014-12-04 | 2015-06-23 | Method, client, and storage medium for reading, writing, deleting, and querying a file |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410733953.9A CN105718484A (en) | 2014-12-04 | 2014-12-04 | File writing method, file reading method, file deletion method, file query method and client |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105718484A true CN105718484A (en) | 2016-06-29 |
Family
ID=56090902
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410733953.9A Pending CN105718484A (en) | 2014-12-04 | 2014-12-04 | File writing method, file reading method, file deletion method, file query method and client |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN105718484A (en) |
| WO (2) | WO2016086552A1 (en) |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106354830A (en) * | 2016-08-31 | 2017-01-25 | 天津南大通用数据技术股份有限公司 | Device and method for data synchronization of database cluster nodes |
| CN106960055A (en) * | 2017-04-01 | 2017-07-18 | 广东浪潮大数据研究有限公司 | A kind of file delet method and device |
| CN107133304A (en) * | 2017-04-28 | 2017-09-05 | 努比亚技术有限公司 | A kind of data manipulation method, device and computer-readable recording medium |
| CN107562757A (en) * | 2016-07-01 | 2018-01-09 | 阿里巴巴集团控股有限公司 | Inquiry, access method based on distributed file system, apparatus and system |
| CN107656939A (en) * | 2016-07-26 | 2018-02-02 | 南京中兴新软件有限责任公司 | File wiring method and device |
| CN107783776A (en) * | 2016-08-26 | 2018-03-09 | 阿里巴巴集团控股有限公司 | The processing method and processing device of firmware upgrade bag, electronic equipment |
| CN108449376A (en) * | 2018-01-31 | 2018-08-24 | 合肥和钧正策信息技术有限公司 | A kind of load-balancing method of big data calculate node that serving enterprise |
| CN109299035A (en) * | 2018-07-04 | 2019-02-01 | 中通服建设有限公司 | A kind of CHR file management method, system and computer readable storage medium |
| CN109976668A (en) * | 2019-03-14 | 2019-07-05 | 北京达佳互联信息技术有限公司 | Data-erasure method, data deletion apparatus and computer readable storage medium |
| CN110109886A (en) * | 2018-02-01 | 2019-08-09 | 中兴通讯股份有限公司 | The file memory method and distributed file system of distributed file system |
| CN110334072A (en) * | 2018-03-22 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of distributed file system, file updating method and device |
| CN110543325A (en) * | 2019-08-15 | 2019-12-06 | 中国银行股份有限公司 | data processing method and device |
| CN110830580A (en) * | 2019-11-12 | 2020-02-21 | 腾讯云计算（北京）有限责任公司 | Storage data synchronization method and device |
| CN110968563A (en) * | 2018-09-29 | 2020-04-07 | 华为技术有限公司 | Data storage method, metadata server and client |
| CN111339046A (en) * | 2020-02-26 | 2020-06-26 | 中国工商银行股份有限公司 | Data writing, reading and deleting method and device for file |
| CN111597149A (en) * | 2020-04-27 | 2020-08-28 | 五八有限公司 | Data cleaning method and device for database |
| CN111831618A (en) * | 2020-07-21 | 2020-10-27 | 北京青云科技股份有限公司 | Data writing method, data reading method, device, equipment and storage medium |
| CN112053128A (en) * | 2020-09-03 | 2020-12-08 | 国网上海电力设计有限公司 | Power grid engineering standardization family database data management system and method |
| CN112578996A (en) * | 2019-09-30 | 2021-03-30 | 华为技术有限公司 | Method for sending metadata of storage system and storage system |
| CN113032335A (en) * | 2021-05-24 | 2021-06-25 | 北京世纪好未来教育科技有限公司 | File access method, device, equipment and storage medium |
| WO2021169113A1 (en) * | 2020-02-26 | 2021-09-02 | 平安科技（深圳）有限公司 | Data management method and apparatus, and computer device and storage medium |
| CN114461146A (en) * | 2022-01-26 | 2022-05-10 | 北京百度网讯科技有限公司 | Cloud storage data processing method, device, system, equipment, medium and product |
| CN115017534A (en) * | 2021-11-05 | 2022-09-06 | 荣耀终端有限公司 | File processing authority control method and device and storage medium |
| CN115470008A (en) * | 2022-11-14 | 2022-12-13 | 杭州拓数派科技发展有限公司 | Metadata access method and device and storage medium |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111309245B (en) * | 2018-12-12 | 2024-04-09 | 阿里巴巴集团控股有限公司 | Hierarchical storage writing method and device, reading method and device and system |
| CN109885620A (en) * | 2018-12-25 | 2019-06-14 | 航天信息股份有限公司 | Metadata read method and device based on Hive data warehouse |
| CN111078643B (en) * | 2019-12-16 | 2022-07-22 | 新华三大数据技术有限公司 | Method and device for deleting files in batch and electronic equipment |
| CN111026936B (en) * | 2019-12-18 | 2023-10-20 | 鹏城实验室 | Data cognition method, device, terminal and computer storage medium of interaction theme |
| CN111125034A (en) * | 2019-12-27 | 2020-05-08 | 深信服科技股份有限公司 | Aggregation object data processing method, system and related equipment |
| CN111176898A (en) * | 2019-12-29 | 2020-05-19 | 浪潮电子信息产业股份有限公司 | Distributed file system MDS (maintenance description Server) fault switching method, device, equipment and medium |
| CN111241455B (en) * | 2020-01-22 | 2023-08-25 | 抖音视界有限公司 | Data processing apparatus, computer device, and storage medium |
| CN111198845B (en) * | 2020-02-13 | 2023-07-21 | 橙三角(广东)科技有限公司 | Data migration method, readable storage medium and computing device |
| CN111597196B (en) * | 2020-05-09 | 2024-02-13 | 北京百度网讯科技有限公司 | Data processing method and device and electronic equipment |
| CN111737223B (en) * | 2020-06-28 | 2023-02-28 | 浪潮电子信息产业股份有限公司 | File copying method, device, equipment and storage medium |
| CN112685094B (en) * | 2020-12-24 | 2023-12-22 | 北京浪潮数据技术有限公司 | Service unloading method, device, equipment and computer readable storage medium |
| CN112632008B (en) * | 2020-12-29 | 2024-07-16 | 华录光存储研究院（大连）有限公司 | Data slicing transmission method and device and computer equipment |
| CN114911759A (en) * | 2021-02-10 | 2022-08-16 | 华为技术有限公司 | File connection method, device, terminal equipment and storage medium |
| CN114356230B (en) * | 2021-12-22 | 2024-04-23 | 天津南大通用数据技术股份有限公司 | Method and system for improving read performance of column storage engine |
| CN115086328B (en) * | 2022-04-30 | 2024-10-22 | 济南浪潮数据技术有限公司 | Metadata service business processing method, device, equipment and medium |
| CN114676166B (en) * | 2022-05-26 | 2022-10-11 | 阿里巴巴（中国）有限公司 | Data processing method and device |
| CN116521744B (en) * | 2023-06-30 | 2023-09-12 | 杭州拓数派科技发展有限公司 | Full duplex metadata transmission method, device, system and computer equipment |
| CN118714147A (en) * | 2024-08-28 | 2024-09-27 | 北京仁科互动网络技术有限公司 | Metadata synchronization method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101997884A (en) * | 2009-08-18 | 2011-03-30 | 升东网络科技发展(上海)有限公司 | Distributed storage system and method |
| CN102143215A (en) * | 2011-01-20 | 2011-08-03 | 中国人民解放军理工大学 | Network-based PB level cloud storage system and processing method thereof |
| CN102546776A (en) * | 2011-12-27 | 2012-07-04 | 北京中科大洋科技发展股份有限公司 | Method for realizing off-line reading files in SAN (Storage Area Networking) shared file system |
| US20130103708A1 (en) * | 2011-10-24 | 2013-04-25 | Electronics And Telecommunications Research Institute | Apparatus and method for enabling clients to participate in data storage in distributed file system |
| CN103076994A (en) * | 2012-12-31 | 2013-05-01 | 北京中科大洋科技发展股份有限公司 | Method for realizing off-line file writing in SAN (Storage Area Network) shared-file system |
| CN103310000A (en) * | 2013-06-25 | 2013-09-18 | 曙光信息产业(北京)有限公司 | Metadata management method |
| CN104144150A (en) * | 2013-05-10 | 2014-11-12 | 中国电信股份有限公司 | Method, device and system for having access to information through metadata |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673288A (en) * | 2009-10-13 | 2010-03-17 | 中兴通讯股份有限公司 | Method and system for reading and writing files in IPTV system |
| CN102164161B (en) * | 2011-01-10 | 2013-12-04 | 清华大学 | Method and device for performing file layout extraction on parallel file system |
| CN102682012A (en) * | 2011-03-14 | 2012-09-19 | 成都市华为赛门铁克科技有限公司 | Method and device for reading and writing data in file system |
| CN102999519B (en) * | 2011-09-15 | 2017-05-17 | 上海盛付通电子商务有限公司 | Read-write method and system for database |
| CN102880658B (en) * | 2012-08-31 | 2016-11-16 | 电子科技大学 | Distributed file management system based on seismic data process |
| CN103167026B (en) * | 2013-02-06 | 2016-05-18 | 数码辰星科技发展（北京）有限公司 | A kind of cloud store environmental data processing method, system and equipment |
-
2014
- 2014-12-04 CN CN201410733953.9A patent/CN105718484A/en active Pending
-
2015
- 2015-03-24 WO PCT/CN2015/074949 patent/WO2016086552A1/en active Application Filing
- 2015-06-23 WO PCT/CN2015/082131 patent/WO2016086649A1/en active Application Filing
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101997884A (en) * | 2009-08-18 | 2011-03-30 | 升东网络科技发展(上海)有限公司 | Distributed storage system and method |
| CN102143215A (en) * | 2011-01-20 | 2011-08-03 | 中国人民解放军理工大学 | Network-based PB level cloud storage system and processing method thereof |
| US20130103708A1 (en) * | 2011-10-24 | 2013-04-25 | Electronics And Telecommunications Research Institute | Apparatus and method for enabling clients to participate in data storage in distributed file system |
| CN102546776A (en) * | 2011-12-27 | 2012-07-04 | 北京中科大洋科技发展股份有限公司 | Method for realizing off-line reading files in SAN (Storage Area Networking) shared file system |
| CN103076994A (en) * | 2012-12-31 | 2013-05-01 | 北京中科大洋科技发展股份有限公司 | Method for realizing off-line file writing in SAN (Storage Area Network) shared-file system |
| CN104144150A (en) * | 2013-05-10 | 2014-11-12 | 中国电信股份有限公司 | Method, device and system for having access to information through metadata |
| CN103310000A (en) * | 2013-06-25 | 2013-09-18 | 曙光信息产业(北京)有限公司 | Metadata management method |
Cited By (35)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107562757B (en) * | 2016-07-01 | 2021-08-10 | 阿里巴巴集团控股有限公司 | Query and access method, device and system based on distributed file system |
| CN107562757A (en) * | 2016-07-01 | 2018-01-09 | 阿里巴巴集团控股有限公司 | Inquiry, access method based on distributed file system, apparatus and system |
| CN107656939A (en) * | 2016-07-26 | 2018-02-02 | 南京中兴新软件有限责任公司 | File wiring method and device |
| CN107783776A (en) * | 2016-08-26 | 2018-03-09 | 阿里巴巴集团控股有限公司 | The processing method and processing device of firmware upgrade bag, electronic equipment |
| CN106354830A (en) * | 2016-08-31 | 2017-01-25 | 天津南大通用数据技术股份有限公司 | Device and method for data synchronization of database cluster nodes |
| CN106960055B (en) * | 2017-04-01 | 2020-08-04 | 广东浪潮大数据研究有限公司 | File deletion method and device |
| CN106960055A (en) * | 2017-04-01 | 2017-07-18 | 广东浪潮大数据研究有限公司 | A kind of file delet method and device |
| CN107133304A (en) * | 2017-04-28 | 2017-09-05 | 努比亚技术有限公司 | A kind of data manipulation method, device and computer-readable recording medium |
| CN108449376A (en) * | 2018-01-31 | 2018-08-24 | 合肥和钧正策信息技术有限公司 | A kind of load-balancing method of big data calculate node that serving enterprise |
| CN110109886A (en) * | 2018-02-01 | 2019-08-09 | 中兴通讯股份有限公司 | The file memory method and distributed file system of distributed file system |
| CN110334072A (en) * | 2018-03-22 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of distributed file system, file updating method and device |
| CN109299035A (en) * | 2018-07-04 | 2019-02-01 | 中通服建设有限公司 | A kind of CHR file management method, system and computer readable storage medium |
| CN110968563B (en) * | 2018-09-29 | 2023-11-17 | 华为技术有限公司 | Data storage method, metadata server and client |
| CN110968563A (en) * | 2018-09-29 | 2020-04-07 | 华为技术有限公司 | Data storage method, metadata server and client |
| CN109976668A (en) * | 2019-03-14 | 2019-07-05 | 北京达佳互联信息技术有限公司 | Data-erasure method, data deletion apparatus and computer readable storage medium |
| CN110543325A (en) * | 2019-08-15 | 2019-12-06 | 中国银行股份有限公司 | data processing method and device |
| CN112578996B (en) * | 2019-09-30 | 2024-06-04 | 华为云计算技术有限公司 | Metadata sending method of storage system and storage system |
| CN112578996A (en) * | 2019-09-30 | 2021-03-30 | 华为技术有限公司 | Method for sending metadata of storage system and storage system |
| WO2021063242A1 (en) * | 2019-09-30 | 2021-04-08 | 华为技术有限公司 | Metadata transmission method of storage system, and storage system |
| CN110830580A (en) * | 2019-11-12 | 2020-02-21 | 腾讯云计算（北京）有限责任公司 | Storage data synchronization method and device |
| CN111339046A (en) * | 2020-02-26 | 2020-06-26 | 中国工商银行股份有限公司 | Data writing, reading and deleting method and device for file |
| WO2021169113A1 (en) * | 2020-02-26 | 2021-09-02 | 平安科技（深圳）有限公司 | Data management method and apparatus, and computer device and storage medium |
| CN111339046B (en) * | 2020-02-26 | 2023-09-19 | 中国工商银行股份有限公司 | Method and device for writing, reading and deleting data of file |
| CN111597149A (en) * | 2020-04-27 | 2020-08-28 | 五八有限公司 | Data cleaning method and device for database |
| CN111597149B (en) * | 2020-04-27 | 2023-03-31 | 五八有限公司 | Data cleaning method and device for database |
| CN111831618A (en) * | 2020-07-21 | 2020-10-27 | 北京青云科技股份有限公司 | Data writing method, data reading method, device, equipment and storage medium |
| CN112053128A (en) * | 2020-09-03 | 2020-12-08 | 国网上海电力设计有限公司 | Power grid engineering standardization family database data management system and method |
| CN112053128B (en) * | 2020-09-03 | 2023-12-01 | 国网上海电力设计有限公司 | Power grid engineering standardized family database data management system and method |
| CN113032335A (en) * | 2021-05-24 | 2021-06-25 | 北京世纪好未来教育科技有限公司 | File access method, device, equipment and storage medium |
| CN115017534A (en) * | 2021-11-05 | 2022-09-06 | 荣耀终端有限公司 | File processing authority control method and device and storage medium |
| CN115017534B (en) * | 2021-11-05 | 2023-08-29 | 荣耀终端有限公司 | File processing authority control method, device and storage medium |
| CN114461146A (en) * | 2022-01-26 | 2022-05-10 | 北京百度网讯科技有限公司 | Cloud storage data processing method, device, system, equipment, medium and product |
| CN114461146B (en) * | 2022-01-26 | 2024-05-07 | 北京百度网讯科技有限公司 | Cloud storage data processing method, device, system, equipment, medium and product |
| CN115470008A (en) * | 2022-11-14 | 2022-12-13 | 杭州拓数派科技发展有限公司 | Metadata access method and device and storage medium |
| CN115470008B (en) * | 2022-11-14 | 2023-03-10 | 杭州拓数派科技发展有限公司 | Metadata access method and device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2016086552A1 (en) | 2016-06-09 |
| WO2016086649A1 (en) | 2016-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105718484A (en) | File writing method, file reading method, file deletion method, file query method and client | |
| US11531482B2 (en) | Data deduplication method and apparatus | |
| US11157445B2 (en) | Indexing implementing method and system in file storage | |
| EP3125501B1 (en) | File synchronization method, server, and terminal | |
| US11615000B2 (en) | Method and backup server for processing expired backups | |
| CN103473277B (en) | The Snapshot Method and device of file system | |
| CN103167026B (en) | A kind of cloud store environmental data processing method, system and equipment | |
| CN111221469B (en) | Method, device and system for synchronizing cache data | |
| CN107590207B (en) | Data synchronization method and device and electronic equipment | |
| CN109144785B (en) | Method and apparatus for backing up data | |
| CN108829720B (en) | Data processing method and device | |
| JP5686034B2 (en) | Cluster system, synchronization control method, server device, and synchronization control program | |
| TW201202959A (en) | Data synchronization methods for synchronizing data in communication systems and communication systems | |
| WO2015065861A1 (en) | Synchronizing event history for multiple clients | |
| CN112698926B (en) | Data processing method, device, equipment, storage medium and system | |
| US9990378B2 (en) | Opportunistic clearing of sync states associated with a database | |
| CN113268540B (en) | Data synchronization method and device | |
| CN109558270B (en) | Data backup method and device and data restoration method and device | |
| CN112749172A (en) | Data synchronization method and system between cache and database | |
| CN111147226B (en) | Data storage method, device and storage medium | |
| CN108712505B (en) | Data synchronization method, device, equipment, system and storage medium | |
| CN102932513B (en) | A kind of business card maintaining method, device and equipment | |
| WO2022042514A1 (en) | Metadata synchronization method and apparatus | |
| CN113625952A (en) | Object storage method, device, equipment and storage medium | |
| CN114416689A (en) | Data migration method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160629 |
|
| RJ01 | Rejection of invention patent application after publication |