CN105005611A - File management system and file management method - Google Patents
File management system and file management method Download PDFInfo
- Publication number
- CN105005611A CN105005611A CN201510401914.3A CN201510401914A CN105005611A CN 105005611 A CN105005611 A CN 105005611A CN 201510401914 A CN201510401914 A CN 201510401914A CN 105005611 A CN105005611 A CN 105005611A
- Authority
- CN
- China
- Prior art keywords
- file
- name
- data server
- main control
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1737—Details of further file system functions for reducing power consumption or coping with limited storage space, e.g. in mobile devices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of the Internet. Provided is a file management system that is composed of a client, a main control server and a data server. The three parts communicate with each other through the Internet. The client is used to send a request to the main control server and to receive data information sent by the main control server or the data server. The main control server is used to receive a request or data sent from the client and the data server, perform control processing according to received content, and meanwhile, perform management and maintenance on the data server. The data server is used to store data sent from the client or the main control server. According to the file management system provided by the invention, mass small files can be managed more effectively, a file that the user needs can be quickly and effectively found; when the number of small files increases, storage space of the system can be easily and effectively increased without the need to modify the system architecture.
Description
[technical field]
The present invention relates to Internet technical field, be specifically related to a kind of file management system in internet and file management method.
[background technology]
In existing Internet technical field, user has higher requirement to file management system, in file management system, how to store and how to be called rapidly and be emphasis wherein.User is mainly reflected in three aspects to the demand stored:
(1) all data must be all reliable, definitely can not lose.This is the tolerance bottom line of user for storage, and therefore reliability is placed in center-stage.
(2) under the prerequisite meeting reliability, operation or online storage system need to have high-performance.This is because the business datum of user almost can all be stored in data server, high-performance can facilitate user's response speed rapidly.
(3) off-line or Backup Data need high power capacity, low price.This part data amount commonly is very large, but not high to performance requirement, and especially infrequently use for those but the data that must preserve, user does not wish to bear great number cost for this reason.
Traditional network store system adopts the storage server concentrated to deposit all data, and storage server becomes the bottleneck of system performance, is also the focus of reliability and security, can not meet the needs of Mass storage application.
At present, file management system many employings distributed file system of the prior art (Distributed File System), distributed file system refers to that the physical memory resources of file system management is not necessarily connected directly between on local node, but is connected with node by computer network.Distributed file system adopts extendible system architecture, utilizes multiple stage storage server to share storage load, utilizes location server to locate storage information, and it not only increases the reliability of system, availability and access efficiency, is also easy to expansion.
Along with the development of Internet technology, cloud store concept gradually accept by user, the bottleneck of single memory node has been broken away from cloud storage, can in the past (scale-out) extending transversely, but, how to provide rational framework and effective method for designing, set up the file storage service of high-performance, high scalable, High Availabitity, be still pendulum in face of system designer, be rich in challenging task.
[summary of the invention]
In view of the above problems, propose the present invention, to provide a kind of overcoming the problems referred to above or a kind of file management system solved the problem at least in part and file management method.
As one aspect of the present invention, a kind of file management system, be made up of client, main control server and data server, three is communicated by network each other, wherein:
Described client is used for sending request to described main control server, and receives the data message that described main control server or described data server send;
Described main control server for the request that receives described client and described data server and send or data, and carries out control treatment according to the content received, and administers and maintains described data server simultaneously;
Described data server is for storing the data of being sent by described client or described main control server.
Further, described main control server adds file group name to the filename that described client intends storage file, and the data server corresponding with file group name to the file allocation intending storing stores.
Further, described file group name can be the name of the data server be assigned with, and also can be main control server according to the name of certain rule or sequential definition, described file group name adds to before described filename.
Further, during using the name of data server as file group name, file group name can represent the segment space in a database, also can be a data server, or the combination of several data server; During using the name of main control server definition as file group name, can using the file of same type as a group, dissimilar file is as different groups.
Further, described main control server described client is intended the file group name of storage file and data server name corresponding to this file group name sends to described client, described file group name is added on filename by described client, and sends to the data server of described correspondence to store this file.
Further, described client comprises interface module, memory module and security module, wherein: described interface module is responsible for the interface that user provides access data; Described memory module is responsible for the file storing client, or metamessage client being used for file access carries out buffer memory; Described security module is responsible for data to be encrypted, and ensures the security of data;
Described main control server comprises active and standby disaster tolerance module, name index module, data server administration module and service dispatch module, wherein: described active and standby disaster tolerance module ensures to take over its work when main control server node failure; Described name index module in charge sets up name index catalogue, the mapping relations of storage file name and corresponding data server; Described data server administration module, manages the state of all data servers, and performs load balancing plan; Described service dispatch module, is responsible for the request received from client and data server;
Described data server comprises data memory module, replica management module, name search module and state-maintenance module, wherein: described data memory module is responsible for storage file being carried out persistence in this locality; File is stored one or more copy on data server by described replica management module; Described name search module in charge is to the file set up name search catalogue in described data memory module; The status information of oneself is regularly periodically reported to main control server in the mode of heartbeat packet by described state-maintenance module in charge.
Further, described client also comprises name module, is added in filename for the file group name sent by described main control server.
According to a further aspect in the invention, a kind of method of file management system storage file is provided, is specially:
File is sent to main control server by client;
Main control server adds file group name on the filename of this file, and according to this file of file group star added stored in corresponding file group;
Main control server is the data server that this file component joins storage, and sets up name index catalogue;
Main control server by the total data of this file group stored in the data server be assigned with;
Data server sets up name search catalogue.
According to a further aspect in the invention, a kind of method of file management system file reading is provided, is specially:
Client sends to main control server by needing the filename read;
Main control server searches file group name corresponding to this filename and the data server name corresponding with this file group name in name index catalogue, and file read request is sent to the data server to correspondence;
Corresponding data server is retrieved containing the filename of this file group name and this file and/or the memory address of its copy in data server in its name search catalogue, and from the memory address of correspondence, transfers file send to main control server;
The file received is sent to client by main control server.
According to a further aspect in the invention, a kind of method of file management system storage file is also provided, is specially:
File storage resource request is sent to main control server by client;
File group name and the data server name that should deposit are sent to client by main control server, and set up name index catalogue;
File group name adds on filename to be stored by client, and file to be stored is sent to the data server of corresponding name;
The data server of corresponding name stores file, and sets up name search catalogue.
According to a further aspect in the invention, a kind of method of file management system file reading is also provided, is specially:
Filename containing file group name is sent to main control server by client;
Main control server, according to name index catalogue, indexes the data server name that this file group name is corresponding, and the filename containing this file group name is sent to corresponding data server;
Corresponding data server, according to name search catalogue, retrieves filename containing this file group name and this file and/or the memory address of its copy in data server, and from the memory address of correspondence, transfers file send to client.
Further, described name index catalogue comprises file group name, stores the data server name of this group file and/or stores the data server name of this group duplicate of the document; Described name search catalogue comprises filename, file and/or the memory address of its copy in data server containing file group name.
The file management system that the present invention proposes, can the small documents of more effective management magnanimity, the file needed for user can be found fast and effectively, and can when small documents quantity increases, the storage space of simple and effective increase system, and do not need to modify to system architecture.The invention solves mass small documents storage in prior art and can waste the problem of disk space and the time of searching constantly increases with the increase of quantity of documents, and then affect the problem of Consumer's Experience.
[accompanying drawing explanation]
In order to be illustrated more clearly in the technical scheme of the embodiment of the present invention, below the accompanying drawing used required in describing embodiment is briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the structural representation of the file management system of one embodiment of the invention.
Fig. 2 is the schematic flow sheet of the file memory method of one embodiment of the invention.
Fig. 3 is the schematic flow sheet of the file reading of one embodiment of the present of invention.
Fig. 4 is the schematic flow sheet of the file memory method of another embodiment of the present invention.
Fig. 5 is the schematic flow sheet of the file reading of an alternative embodiment of the invention.
[embodiment]
Describe in more detail exemplary embodiment of the present invention below with reference to accompanying drawings.Although show exemplary embodiment of the present invention in accompanying drawing, but should be appreciated that and can realize the present invention in a variety of manners, and not should limit by the embodiment set forth here.On the contrary, provide these embodiments be in order to can be more thorough understanding the present invention, and complete for scope of the present invention can be conveyed to those skilled in the art.
Fig. 1 shows file management system structural representation according to an embodiment of the invention.As shown in Figure 1, a kind of file management system, is made up of client 10, main control server 20 and data server 30, is communicated each other by network.Wherein:
Client 10 for sending request to main control server 20, and receives the data message that main control server 20 or data server 30 send.Client 10 comprises the equipment such as various desktop computer, notebook computer, intelligent terminal (such as smart mobile phone, panel computer, Intelligent bracelet, intelligent glasses).The request that client 10 sends to main control server 20 comprises data storage request, data inquiry request, data read request, data download request or data modification request etc.
Main control server 20 for the request that receives client 10 and data server 30 and send or data, and carries out control treatment according to the content received.Concrete, when client (user) needs to carry out data storage, the filename that main control server 20 is responsible for intending client 10 storage file adds file group name, and the data server corresponding with file group name to the file allocation intending storing stores, briefly, be exactly that client 10 sends the file intending storing to main control server 20, main control server 20 forwards this file to one or more data servers 30 corresponding with file group name after adding file group name to the file intending storing; As client 10(user) need inquiry, read or downloading data time, main control server 20 needs the filename inquired about, read or download according to user, retrieve its file group name and data server corresponding to file group name, and from the data server of correspondence, transfer corresponding data and fed back to client 10; Main control server 20 is also for administering and maintaining data server 30.
Concrete, main control server wants the filename of storage file to add file group name to client, and carry out data storage according to the file group name distribution data server corresponding with this file group name, file group name can be numeral, it can be letter, can also be numeral and alphabetical combination, file group name can be named at random, also the suffix can intending storage file according to client distributes the file group name of a certain class, what such as main control server judged that client will store is suffix is the picture group " 1.jpg of " .jpg " form, 2.jpg, 3.jpg ... ", it is then this group image data unified distribution file group name " 001_ ", such filename can by RNTO " 001_1.jpg, 001_2.jpg, 001_3.jpg ... ", suppose that the data of file group " 001_ " by name are all corresponding with data server A, then add this picture group sheet after file group name all stored in data server A.Store and follow-up reading speed to improve, file group name is preferably added on before the filename of file, can deposit the multiple files with group in a file group name.File group name can be the name of the data server be assigned with, and also can be that main control server is according to certain is regular or the name of sequential definition.During using the name of data server as file group name, file group name can represent the segment space in a database, also can be a data server, or the combination of several data server, and preferably a group name represents a data server.During using the name of main control server definition as file group name, can using the file of same type as a group, dissimilar file is as different groups, and what such as suffix was " .jpg " form is one group, suffix be " .doc " be other one group ...
In order to ensure the reliability of data, main control server is made up of master server and backup server, and when master server is working properly, primary responsibility stores the data received and controls, and storage can be interim storage, also can be permanent storage; Backup server is responsible for Backup Data, and substitutes it when master server fault and carry out work.
Data server 30 is for storing the data of being sent by client 10 or main control server 20.Data server 30 can be one, also can be multiple, multiple data server can leave different physical locations in respectively, but the physical location of its dispersion does not affect the United Dispatching that it accepts main control server as a whole, certainly, if place is enough large, multiple data server also can be placed in a physical location.Data server can store according to file to be needed to increase or reduce, and can not have an impact to whole system framework.
Client 10, main control server 20 is communicated by network each other with data server 30 three, carries out data transmission as by internet, 2G/3G/4G network, WIFI network, each LAN (Local Area Network) etc.
File management system of the present invention, can be used in the mass small documents environment of facing cloud storage, and guarantee the security that file stores and confidentiality.
As an alternative embodiment of the invention, client is the same with the effect of data server, and main control server can have different control principles.Concrete, when client (user) needs to carry out file storage, the filename that main control server is responsible for intending client storage file adds file group name, and the address of the name after adding file group name and data server corresponding to this file group name is sent to client together, file after rename is sent to corresponding data server by address by client, data server stores this file, and event memory is fed back to Control Server; When client (user) need inquiry, read or download file time, main control server is sent to by needing the filename downloaded, main control server finds corresponding file group name according to filename, and then find the data server that file group name is corresponding, and order this data server that corresponding data are sent to client; Main control server is also for administering and maintaining data server.
As the improvement of above-mentioned two kinds of embodiments, storage means to data and the inquiry to data, reading, method for down loading can be bonded to each other,, by client, data are directly sent to corresponding data server when such as data store, in data query, reading or when downloading, the data on data server send to client eventually through Control Server; Or when data store, the storage that the data of client are completed on corresponding data server by Control Server, in data query, reading or when downloading, the corresponding data on data server directly sends to client.
As the improvement of above-mentioned two kinds of embodiments, also can complete in client the process of file rename.Main control server sends file group name to client, and file group name is added on filename by client, is preferably added on before filename, as filename prefix.
The present invention can be used for the mass small documents management system that facing cloud stores, the distributed column storage system that it stores based on cloud, data scatter is stored in multiple stage independently on equipment.Technical scheme of the present invention makes when processing mass small documents, the Single document stored for small documents takes up room little, but the feature that quantity of documents is huge, use distributed storage technology, again small documents storage and NameSpace when searching is planned, decrease the time needed for locating file, make client not spend the more time to wait for because of the increase of quantity of documents, improve Consumer's Experience.
Fig. 1 also shows the inner structure schematic diagram of the client 10 of the file management system of one embodiment of the invention, main control server 20 and data server 30.As shown in the figure:
Client 10 comprises interface module 11, memory module 12 and security module 13, wherein:
Interface module 11 is responsible for the interface that user provides access data, as posix interface.
Memory module 12 is responsible for the file storing client, or metamessage client being used for file access carries out buffer memory, as being buffered in client this locality or disk, also can be buffered in far-end.
Security module 13 is responsible for data to be encrypted, and ensures the security of data.
Main control server 20 comprises active and standby disaster tolerance module 21, name index module 22, data server administration module 23, and service dispatch module 24, wherein:
Active and standby disaster tolerance module 21 is the single-point problems in order to avoid main control server, can be its configuration standby server usually, to ensure to take over its work when main control server node failure.Active and standby disaster tolerance module 21 is also backup server.
Name index module 22, is responsible for setting up name index catalogue, the mapping relations of storage file name and corresponding data server.Name index catalogue comprises file group name, filename containing file group name, stores the data server name of this group file and/or store the data server name of this group duplicate of the document.
Data server administration module 23, manages the state of all data servers, and performs load balancing plan.Main control server sends file group name to client, or adds file group name to filename, is all realized by data server administration module 23.Main control server is with the form of heartbeat packet centralized management data server.When receiving client storage resource request, main control server needs to select one group of data server to serve for it according to information such as the loads of each data server group (if storable data server corresponding to each file group name is more than); When main control server find that there is data server delay machine time, need to perform the file of some number of copies deficiencies to copy plan, when there being new data server to add cluster or certain data server load too high, main control server also can perform copy migration plan as required.When distributing data server, according to the information of client, data server can be sorted according to the distance with client, client first being chosen and accesses from the data server close to oneself.
Service dispatch module 24, is responsible for the request received from client and data server.Usually accepted request by independent thread, and joined in task queue, the thread in thread pool then from task queue continuous taking-up task process.The request of client comprises to be needed to store data, read, inquire about, download, amendment etc., and the request of data server comprises and sends the status information of data server and the response message to main control server to Control Server timing or not timing.
Data server 30 comprises data memory module 31, replica management module 32, name search module 33 and state-maintenance module 34.
Data memory module 31 is responsible for storage file being carried out persistence in this locality.
Replica management module 32 is the securities in order to ensure data, by the file in distributed file system, stores one or more copy on data server.
Name search module 33 is responsible for the file set up name search catalogue in data memory module 31.Name search catalogue comprises filename, file and/or the memory address of its copy in data server containing file group name.The utilization factor of storage space can be improved so greatly, and the time needed for client locating file can be reduced, promote Consumer's Experience.
State-maintenance module 34, be responsible for regularly the status information of oneself being periodically reported to main control server in the mode of heartbeat packet, main control server is known, and whether oneself normally works.Usual heartbeat packet also can comprise the current loading condition of data server, as CPU, and internal memory, disk I/O, disk storage space, process resource, network I/O etc.
Can find out according to above-described embodiment, the present invention is by the file consolidation management that nodes all in server cluster store and store.These nodes comprise main control server name index module and data server name search module, and the mass small documents management system inside stored at facing cloud provides Metadata Service; Many data servers provide stores service.The file be stored in the mass small documents management system of facing cloud storage is divided into group, and then copy in one or more data server by these groups, this and traditional RAID framework differ widely.Data server can space needed for the size determination storage file of file.The physical location that the name search module in charge storage file group of data server is real, there is the file operation in data server cluster on all nodes in the name index module monitors of main control server, such as document creation, deletion, movement, rename etc.
As another kind of embodiment of the present invention, the method for filename being added to file group name can complete in client, and therefore, client 10 also comprises name module 14, for adding the file group name distributed by main control server to filename.Main control server can intend the file allocation file group name of storage to client, and this file group name is sent to client, the file group name received is added on before filename or the optional position of filename by the name module 14 of client, realize the rename to filename, preferably say that file group name is added on before filename.
The name index module 22 of the main control server 20 of the mass small documents management system that facing cloud stores is after determining storage file to the mapping relations of data server 30, carry out dividing into groups (as divided into groups according to beginning letter according to file group star file, or divide into groups etc. according to the value of the byte of beginning), and according to group result, file and copy thereof are assigned in corresponding data server according to file group name, each group of one or more data server corresponding, or a part of storage space of corresponding individual data server.Name index module 22 only preserves the group name at the rear file place of grouping and the data server name corresponding with this group name.This group file and copy thereof will be stored in the data server corresponding with this group name respectively, file in each group stores physical location information and will be stored in the name search module 33 of corresponding data server, during client-access file, main control server only need find the group name belonging to this file, the data server name storing this group file is found again according to group name, then in the name search module 33 of this data server, search the physical storage address of respective file, and then corresponding document is read from corresponding physical storage locations, a large amount of for saving are searched the time by undoubtedly, and, as long as add new group name and the data server corresponding with this group name, the storage space of whole system just can be increased with minimum cost, system is become and is more easy to expansion.
The mass small documents management system of a facing cloud storage is made up of at least two main control servers and a large amount of data servers, and allows multiple client to access simultaneously.
As another kind of embodiment of the present invention, client 10 is encrypted the file intending storing by security module 13, and the filename intending storing is sent to the service dispatch module 24 of main control server 20 by interface module 11, the filename received is sent to data server administration module 23 by service dispatch module 24, server management module 23 judges file type according to the suffix of filename, and the file group name corresponding with the type file is added on before filename, filename after adding file group name is sent to name index module 22, the corresponding data server name that name index module 22 should store according to file group name locating file, and the data server name of the filename containing file group name and correspondence is sent to service dispatch module 24, service dispatch module 24 sends to client by what receive containing the filename of file group name and the data server name of correspondence, the file that plan containing file group name stores is sent to corresponding data server by client, the file received is stored in data memory module 31 by data server, and the copy making file is kept in replica management module 32, concrete storage address information in filename containing file group name and memory module 31 is sent to name search module 33 simultaneously, name search module 33 sets up name search catalogue to facilitate client or main control server is follow-up calls file.
The mass small documents management system that the facing cloud of above-described embodiment stores adopts the distributed column storage system stored based on cloud, exactly data scatter is stored in multiple stage independently on equipment.Traditional network store system adopts the storage server concentrated to deposit all data, and storage server becomes the bottleneck of system performance, is also the focus of reliability and security, can not meet the needs of Mass storage application.Distributed network storage system adopts extendible system architecture, utilizes multiple stage storage server to share storage load, utilizes location server to locate storage information, and it not only increases the reliability of system, availability and access efficiency, is also easy to expansion.Main control server and data server are to being transparent per family, and user only need accept file group name from main control server on filename, and are sent by file and store, the particular location of file of forgetting it storage.The mass small documents management system that facing cloud stores will guarantee safety that file stores and secret.
As can be seen from the technical scheme of the above embodiment of the present invention, the mass small documents management system that can be used for facing cloud storage described in the present invention, main control server arranges name index module, name search module is set on data server simultaneously, the file grouping that the name index module primary responsibility of main control server is submitted to client is also distributed to corresponding data server and is stored, the name search module of data server is to stored file set up index, user is facilitated to search, make storage system more can save the storage space of main control server, more flexibly according to the file size allocation storage space in data server, the small documents of more effective management magnanimity, find the file needed for user more rapidly and effectively, and can when small documents quantity increases, the storage space of simple and effective increase system, and do not need to modify to system architecture, (scale-out) extending transversely can be realized with the storage server of low cost and reach high performance requirement, when storage capacity can not meet the demands, can by constantly increasing memory node to improve, system storage ability and I/O parallel ability can seamless upgrades at any time, this is that conventional store is not accomplished.
The present invention also provides a kind of file management method, and because this file management method is based on above-mentioned file management system, its corresponding principle is all the same, therefore repeats no more.
Here is a specific embodiment of file management system of the present invention storage and file reading method.
(1) when client needs storage file, as shown in Figure 2:
Step S210, file is sent to main control server by client;
Step S220, main control server adds file group name on the filename of this file, and according to this file of file group star added stored in corresponding file group.Store and follow-up reading speed to improve, file group name is preferably added on before the filename of this file, can deposit the multiple files with group in a file group name.File group name can be the name of the data server be assigned with, and also can be that main control server is according to certain is regular or the name of sequential definition.During using the name of data server as file group name, file group name can represent the segment space in a database, also can be a data server, or the combination of several data server, and preferably a group name represents a data server.During using the name of main control server definition as file group name, can using the file of same type as a group, dissimilar file is as different groups, and what such as suffix was " .jpg " form is one group, suffix be " .doc " be other one group ...
Step S230, main control server is the data server that this file component joins storage, and sets up name index catalogue.Name index catalogue comprises file group name, filename containing file group name, stores the data server name of this group file and/or store the data server name of this group duplicate of the document.
Step S240, main control server by the total data of this file group stored in the data server be assigned with.Data server can copy the storage file received, and also stores the copy copying generation simultaneously.
Step S250, data server sets up name search catalogue.Name search catalogue comprises filename, file and/or the memory address of its copy in data server containing file group name.
(2) when client needs file reading, as shown in Figure 3:
Step S310, client sends to main control server by needing the filename read.
Step S320, main control server searches file group name corresponding to this filename and the data server name corresponding with this file group name in name index catalogue, and file read request is sent to the data server to correspondence.
Step S330, corresponding data server is retrieved containing the filename of this file group name and this file and/or the memory address of its copy in data server in its name search catalogue, and from the memory address of correspondence, transfers file send to main control server.Preferential transmission file original, when the data server cisco unity malfunction at file original place, sends to main control server by duplicate of the document.
Step S340, the file received is sent to client by main control server.
Here is another specific embodiment of file management system of the present invention storage and file reading method.
(1) when client needs storage file, as shown in Figure 4:
Step S410, file storage resource request is sent to main control server by client.
Step S420, file group name and the data server name that should deposit are sent to client by main control server, and set up name index catalogue.Name index catalogue comprises file group name, store the data server name of this group file and/or store the data server name of this group duplicate of the document.
Step S430, file group name adds on filename to be stored by client, and file to be stored is sent to the data server of corresponding name.File group name is preferably added on before the filename of file to be stored.File group name can be the name of the data server be assigned with, and also can be that main control server is according to certain is regular or the name of sequential definition.During using the name of data server as file group name, file group name can represent the segment space in a database, also can be a data server, or the combination of several data server, and preferably a group name represents a data server.During using the name of main control server definition as file group name, can using the file of same type as a group, dissimilar file is as different groups, and what such as suffix was " .jpg " form is one group, suffix be " .doc " be other one group ...
Step S440, the data server of corresponding name stores file, and sets up name search catalogue.Data server can copy the storage file received, and also stores the copy copying generation simultaneously.Name search catalogue comprises filename, file and/or the memory address of its copy in data server containing file group name.
(2) when client needs file reading, as shown in Figure 5:
Step S510, the filename containing file group name is sent to main control server by client.
Step S520, main control server, according to name index catalogue, indexes the data server name that this file group name is corresponding, and the filename containing this file group name is sent to corresponding data server.
Step S530, corresponding data server, according to name search catalogue, retrieves filename containing this file group name and this file and/or the memory address of its copy in data server, and from the memory address of correspondence, transfers file send to client.Preferential transmission file original, when the data server cisco unity malfunction at file original place, sends to client by duplicate of the document.
The mass small documents management system that the facing cloud that the present invention proposes stores, make in the storage system stored at facing cloud, the small documents of more effective management magnanimity, find the file needed for user more rapidly and effectively, and can when small documents quantity increases, the storage space of simple and effective increase system, and do not need to modify to system architecture.
As seen through the above description of the embodiments, those skilled in the art can be well understood to the mode that the present invention can add required general hardware platform by software and realizes.Based on such understanding, technical scheme of the present invention can embody with the form of software product the part that prior art contributes in essence in other words, this computer software product can be stored in storage medium, as ROM/RAM, magnetic disc, CD etc., comprising some instructions in order to make a computer equipment (can be personal computer, server, or the network equipment etc.) perform the method described in some part of each embodiment of the present invention or embodiment.
Each embodiment in this instructions all adopts the mode of going forward one by one to describe, between each embodiment identical similar part mutually see, what each embodiment stressed is the difference with other embodiments.Especially, for device or system embodiment, because it is substantially similar to embodiment of the method, so describe fairly simple, relevant part illustrates see the part of embodiment of the method.Apparatus and system embodiment described above is only schematic, the wherein said unit illustrated as separating component or can may not be and physically separates, parts as unit display can be or may not be physical location, namely can be positioned at a place, or also can be distributed in multiple network element.Some or all of module wherein can be selected according to the actual needs to realize the object of the present embodiment scheme.Those of ordinary skill in the art, when not paying creative work, are namely appreciated that and implement.
It should be noted that:
Intrinsic not relevant to any certain computer, virtual system or miscellaneous equipment with display at this algorithm provided.Various general-purpose system also can with use based on together with this teaching.According to description above, the structure constructed required by this type systematic is apparent.In addition, the present invention is not also for any specific programming language.It should be understood that and various programming language can be utilized to realize content of the present invention described here.
Those skilled in the art are appreciated that and can carry out the change of adaptivity to module each in embodiment and they are arranged in one or more equipment different from this embodiment.Unless otherwise clearly stating, each feature disclosed in this instructions can by providing identical, alternative features that is equivalent or similar object replaces.
All parts embodiment of the present invention with hardware implementing, or can realize with the software module run on one or more processor, or realizes with their combination.
The foregoing is only the preferred embodiment of the present invention, and be not used to limit claims of the present invention.Above explanation simultaneously, should be appreciated that and implement for those skilled in the technology concerned, therefore other equivalent changes completed based on disclosed content, all should be included in the covering scope of these claims.
Claims (12)
1. a file management system, is characterized in that, described file management system is made up of client, main control server and data server, and three is communicated by network each other, wherein:
Described client is used for sending request to described main control server, and receives the data message that described main control server or described data server send;
Described main control server for the request that receives described client and described data server and send or data, and carries out control treatment according to the content received, and administers and maintains described data server simultaneously;
Described data server is for storing the data of being sent by described client or described main control server.
2. a kind of file management system according to claim 1, it is characterized in that: described main control server adds file group name to the filename that described client intends storage file, and the data server corresponding with file group name to the file allocation intending storing stores.
3. a kind of file management system according to claim 2, it is characterized in that: described file group name can be the name of the data server be assigned with, also can be main control server according to the name of certain rule or sequential definition, described file group name adds to before described filename.
4. a kind of file management system according to claim 3, it is characterized in that: during using the name of data server as file group name, file group name can represent the segment space in a database, also can be a data server, or the combination of several data server; During using the name of main control server definition as file group name, can using the file of same type as a group, dissimilar file is as different groups.
5. a kind of file management system according to claim 1, it is characterized in that: described main control server described client is intended the file group name of storage file and data server name corresponding to this file group name sends to described client, described file group name is added on filename by described client, and sends to the data server of described correspondence to store this file.
6. a kind of file management system according to claim 1, is characterized in that:
Described client comprises interface module, memory module and security module, wherein: described interface module is responsible for the interface that user provides access data; Described memory module is responsible for the file storing client, or metamessage client being used for file access carries out buffer memory; Described security module is responsible for data to be encrypted, and ensures the security of data;
Described main control server comprises active and standby disaster tolerance module, name index module, data server administration module and service dispatch module, wherein: described active and standby disaster tolerance module ensures to take over its work when main control server node failure; Described name index module in charge sets up name index catalogue, the mapping relations of storage file name and corresponding data server; Described data server administration module, manages the state of all data servers, and performs load balancing plan; Described service dispatch module, is responsible for the request received from client and data server;
Described data server comprises data memory module, replica management module, name search module and state-maintenance module, wherein: described data memory module is responsible for storage file being carried out persistence in this locality; File is stored one or more copy on data server by described replica management module; Described name search module in charge is to the file set up name search catalogue in described data memory module; The status information of oneself is regularly periodically reported to main control server in the mode of heartbeat packet by described state-maintenance module in charge.
7. a kind of file management system according to claim 6, is characterized in that: described client also comprises name module, is added in filename for the file group name sent by described main control server.
8. a method for file management system storage file, is characterized in that:
File is sent to main control server by client;
Main control server adds file group name on the filename of this file, and according to this file of file group star added stored in corresponding file group;
Main control server is the data server that this file component joins storage, and sets up name index catalogue;
Main control server by the total data of this file group stored in the data server be assigned with;
Data server sets up name search catalogue.
9. a method for file management system file reading, is characterized in that:
Client sends to main control server by needing the filename read;
Main control server searches file group name corresponding to this filename and the data server name corresponding with this file group name in name index catalogue, and file read request is sent to the data server to correspondence;
Corresponding data server is retrieved containing the filename of this file group name and this file and/or the memory address of its copy in data server in its name search catalogue, and from the memory address of correspondence, transfers file send to main control server;
The file received is sent to client by main control server.
10. a method for file management system storage file, is characterized in that:
File storage resource request is sent to main control server by client;
File group name and the data server name that should deposit are sent to client by main control server, and set up name index catalogue;
File group name adds on filename to be stored by client, and file to be stored is sent to the data server of corresponding name;
The data server of corresponding name stores file, and sets up name search catalogue.
The method of 11. 1 kinds of file management system file readings, is characterized in that:
Filename containing file group name is sent to main control server by client;
Main control server, according to name index catalogue, indexes the data server name that this file group name is corresponding, and the filename containing this file group name is sent to corresponding data server;
Corresponding data server, according to name search catalogue, retrieves filename containing this file group name and this file and/or the memory address of its copy in data server, and from the memory address of correspondence, transfers file send to client.
12. file management system methods according to Claim 8 described in-11 arbitrary claims, is characterized in that: described name index catalogue comprises file group name, store the data server name of this group file and/or store the data server name of this group duplicate of the document; Described name search catalogue comprises filename, file and/or the memory address of its copy in data server containing file group name.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510401914.3A CN105005611B (en) | 2015-07-10 | 2015-07-10 | A kind of file management system and file management method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510401914.3A CN105005611B (en) | 2015-07-10 | 2015-07-10 | A kind of file management system and file management method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105005611A true CN105005611A (en) | 2015-10-28 |
| CN105005611B CN105005611B (en) | 2018-11-30 |
Family
ID=54378287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510401914.3A Active CN105005611B (en) | 2015-07-10 | 2015-07-10 | A kind of file management system and file management method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105005611B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106024039A (en) * | 2016-06-02 | 2016-10-12 | 深圳市爱思拓信息存储技术有限公司 | Magnetic, optical, and cloud integrated optical disc mass safe storage array database |
| CN106024038A (en) * | 2016-06-02 | 2016-10-12 | 深圳市爱思拓信息存储技术有限公司 | Electromagnetic, optical, and cloud integrated optical disc mass safe storage array database |
| CN106528830A (en) * | 2016-11-16 | 2017-03-22 | 华为技术有限公司 | Method and device for recovering file index catalogue |
| CN107092686A (en) * | 2017-04-24 | 2017-08-25 | 浙江宇视科技有限公司 | A kind of file management method and device based on cloud storage platform |
| CN107194271A (en) * | 2017-04-18 | 2017-09-22 | 华南农业大学 | A kind of shared private cloud storage system of weak center |
| CN107967366A (en) * | 2017-12-22 | 2018-04-27 | 深圳Tcl新技术有限公司 | File management method, USB flash disk and computer-readable recording medium |
| CN108156040A (en) * | 2018-01-30 | 2018-06-12 | 北京交通大学 | A kind of central control node in distribution cloud storage system |
| CN109756573A (en) * | 2019-01-15 | 2019-05-14 | 苏州链读文化传媒有限公司 | A kind of file system based on block chain |
| CN109871365A (en) * | 2019-01-15 | 2019-06-11 | 苏州链读文化传媒有限公司 | A kind of distributed file system |
| CN110941848A (en) * | 2018-09-21 | 2020-03-31 | 富士施乐株式会社 | Document management system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020315A (en) * | 2013-01-10 | 2013-04-03 | 中国人民解放军国防科学技术大学 | Method for storing mass of small files on basis of master-slave distributed file system |
| CN103532754A (en) * | 2013-10-12 | 2014-01-22 | 北京首信科技股份有限公司 | System and method for high-speed memory and distributed type processing of massive logs |
-
2015
- 2015-07-10 CN CN201510401914.3A patent/CN105005611B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020315A (en) * | 2013-01-10 | 2013-04-03 | 中国人民解放军国防科学技术大学 | Method for storing mass of small files on basis of master-slave distributed file system |
| CN103532754A (en) * | 2013-10-12 | 2014-01-22 | 北京首信科技股份有限公司 | System and method for high-speed memory and distributed type processing of massive logs |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106024038A (en) * | 2016-06-02 | 2016-10-12 | 深圳市爱思拓信息存储技术有限公司 | Electromagnetic, optical, and cloud integrated optical disc mass safe storage array database |
| CN106024039A (en) * | 2016-06-02 | 2016-10-12 | 深圳市爱思拓信息存储技术有限公司 | Magnetic, optical, and cloud integrated optical disc mass safe storage array database |
| CN106528830A (en) * | 2016-11-16 | 2017-03-22 | 华为技术有限公司 | Method and device for recovering file index catalogue |
| CN106528830B (en) * | 2016-11-16 | 2019-05-10 | 华为技术有限公司 | A kind of method and apparatus for restoring file index catalogue |
| CN107194271A (en) * | 2017-04-18 | 2017-09-22 | 华南农业大学 | A kind of shared private cloud storage system of weak center |
| CN107092686B (en) * | 2017-04-24 | 2020-04-10 | 浙江宇视科技有限公司 | File management method and device based on cloud storage platform |
| CN107092686A (en) * | 2017-04-24 | 2017-08-25 | 浙江宇视科技有限公司 | A kind of file management method and device based on cloud storage platform |
| CN107967366A (en) * | 2017-12-22 | 2018-04-27 | 深圳Tcl新技术有限公司 | File management method, USB flash disk and computer-readable recording medium |
| CN107967366B (en) * | 2017-12-22 | 2022-03-01 | 深圳Tcl新技术有限公司 | File management method, U disk and computer readable storage medium |
| CN108156040A (en) * | 2018-01-30 | 2018-06-12 | 北京交通大学 | A kind of central control node in distribution cloud storage system |
| CN110941848A (en) * | 2018-09-21 | 2020-03-31 | 富士施乐株式会社 | Document management system |
| CN109871365A (en) * | 2019-01-15 | 2019-06-11 | 苏州链读文化传媒有限公司 | A kind of distributed file system |
| CN109756573B (en) * | 2019-01-15 | 2022-02-08 | 苏州链读文化传媒有限公司 | File system based on block chain |
| CN109756573A (en) * | 2019-01-15 | 2019-05-14 | 苏州链读文化传媒有限公司 | A kind of file system based on block chain |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105005611B (en) | 2018-11-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105005611A (en) | File management system and file management method | |
| CN107547653B (en) | Distributed file storage system | |
| CN110213352B (en) | Method for aggregating dispersed autonomous storage resources with uniform name space | |
| Deka | A survey of cloud database systems | |
| US7076553B2 (en) | Method and apparatus for real-time parallel delivery of segments of a large payload file | |
| US10853242B2 (en) | Deduplication and garbage collection across logical databases | |
| CN102831120B (en) | A kind of data processing method and system | |
| US20150215405A1 (en) | Methods of managing and storing distributed files based on information-centric network | |
| CN104516967A (en) | Electric power system mass data management system and use method thereof | |
| CN102664914A (en) | IS/DFS-Image distributed file storage query system | |
| CN105354250A (en) | Data storage method and device for cloud storage | |
| Honnutagi | The Hadoop distributed file system | |
| KR20100048130A (en) | Distributed storage system based on metadata cluster and method thereof | |
| CN104750757A (en) | Data storage method and equipment based on HBase | |
| Chandra et al. | A study on cloud database | |
| US10387384B1 (en) | Method and system for semantic metadata compression in a two-tier storage system using copy-on-write | |
| CN112334891A (en) | Centralized storage for search servers | |
| CN114610680A (en) | Method, device and equipment for managing metadata of distributed file system and storage medium | |
| KR20130038517A (en) | System and method for managing data using distributed containers | |
| Ye | Research on the key technology of big data service in university library | |
| CN117056303B (en) | Data storage method and device suitable for military operation big data | |
| CN107547657A (en) | A kind of method, apparatus and storage medium numbered based on one point data in cloud storage system | |
| Luo et al. | LAYER: A cost-efficient mechanism to support multi-tenant database as a service in cloud | |
| EL-SAYED et al. | Impact of small files on hadoop performance: literature survey and open points | |
| Fong et al. | Toward a scale-out data-management middleware for low-latency enterprise computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |