CN105242881B - Distributed memory system and its data read-write method - Google Patents
Distributed memory system and its data read-write method Download PDFInfo
- Publication number
- CN105242881B CN105242881B CN201510657207.0A CN201510657207A CN105242881B CN 105242881 B CN105242881 B CN 105242881B CN 201510657207 A CN201510657207 A CN 201510657207A CN 105242881 B CN105242881 B CN 105242881B
- Authority
- CN
- China
- Prior art keywords
- module
- write
- data
- mapping table
- block identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of distributed memory system, the distributed memory system includes a proxy module, a metadata service module and multiple storage services modules, and each storage services module manages at least one memory node.Metadata service module is used for the one-level mapping table for safeguarding the overall situation, and storage services module is used to safeguard two level mapping table.Due to using two-stage metadata organization, when writing data, without being actually written into if existing identical data, so as to reduce the operation of data write-in, and reach the target of global data de-duplication.Disclosed herein as well is corresponding data read-write method.Application scheme can realize following technique effect：Online global data de-duplication is realized in distributed memory system, the physical memory space is reduced and takes；Because the data repeated do not need actual write operation when writing data, data interaction and the bandwidth of application system and storage system are saved, improves storage efficiency.
Description
Technical field
The application is related to computer memory technical field, more particularly to distributed memory system and its data read-write method.
Background technology
With flourishing for a variety of applications such as mobile device, social networks, Internet of Things, data are in caused by human society
Explosive growth.Traditional disk array is increasingly difficult to meet based on mass data in terms of capacity, performance and bandwidth
Data-intensive applications memory requirement.Therefore, scale-out frameworks, capacity and performance is taken linearly to increase with nodes
The distributed cluster storage system added arises at the historic moment, and can provide number (IOPS, the Input/ of higher read-write per second (IO) operation
Output Operations Per Second) performance solid state hard disc also gradually substitution traditional magnetic disk turn into I/O intensive type application
First choice.In this context, the expensive price of solid state hard disc, first wipe the application characteristic write afterwards, require in storage system
In as far as possible the physical memory space of compressed data, reduce write-in number, further to improve using distributed flash memory system
Cost performance.
Data de-duplication is one kind in data reducti techniques, is generally used for the standby system based on disk, it is intended to subtract
The memory capacity actually used in few storage system.The working method of data de-duplication is typically in some time cycle at present
It is interior, in running background data de-duplication program, the duplicate data block of diverse location in different files is searched, the data repeated
Block is substituted with designator, to reduce the occupancy to memory capacity.The data set (such as Backup Data) of high redundancy is from repeat number
It is very big according to the benefit of deleting technique；Other data de-duplication technology can allow to carry out between the different websites of user efficiently,
Economic Backup Data replicates.But for distributed cluster storage system, the duplicate data in existing single device is deleted
Except technology is unable to reach the target of global data de-duplication, data reduction ratio does not reach optimum efficiency；In addition, background process
Data de-duplication mode can not reduce the operation of data write-in, for the storage system using solid state hard disc, just up to not
To the target for reducing erasable number, extending solid-state disk service life.
The content of the invention
This application provides one kind to be applied to distributed memory system and its data read-write method, without carrying out actual weight
Complex data deletion action, you can reach the target of global data de-duplication, and the operation of data write-in can be reduced.
The embodiment of the present application provides a kind of distributed memory system, including：The distributed memory system includes one
Proxy module, a metadata service module and multiple storage services modules, each storage services module management are at least one
Memory node；
The proxy module is used to receive the write request from application system, and the data to be write are calculated by piecemeal parameter
Cryptographic Hash, obtain block identification, the write request for carrying the block identification sent to metadata service module；Receive Metadata Service
Write request, according to the nodal information, is routed to corresponding memory node by the nodal information that module returns；And it will come from
The write-in success message of metadata service module or storage services module returns to application system；
The metadata service module is used for the one-level mapping table for safeguarding the overall situation, and one-level mapping table is with containing global logic
Location and memory node, the mapping relations of block identification；The write request of the carrying block identification from proxy module is received, one-level is searched and reflects
Firing table, if the map record of existing corresponding block identification, refreshes one-level mapping table, increase a write-in initial address and corresponding
The map record of block identification, write-in success message is returned to application system by proxy module；If being not present, a storage is selected
Node, the nodal information of the memory node is returned to proxy module；Receive the write-in information from storage services module, brush
New one-level mapping table, increase write-in initial address, corresponding block identification, the map record of respective stored node newly, taken to the storage
Module of being engaged in returns to Flushing success message；
Storage services module is used to safeguard two level mapping table, and two level mapping table contains block identification and actual storage physically
The mapping relations of location；The write request for being routed to the memory node that this storage services module is managed is received, is write the data to described
The disk of memory node, refresh two level mapping table, increase corresponding block identification newly and be actually written into the record of physical address, and to first number
Write-in information is sent according to service module.
Alternatively, the proxy module is additionally operable to receive the read request from application system, and the read request is passed through into member
Data service module, and the reading data from storage services module are returned into application system；
The metadata service module is additionally operable to receive the read request that initial address is read in the carrying from proxy module, looks into
Map record corresponding to reading initial address described in one-level mapping table is looked for, obtains corresponding memory node and block identification,
Read request is routed to corresponding memory node；
The storage services module be additionally operable to receive be routed to the memory node that this storage services module is managed reading please
Ask, two level mapping table is searched according to block identification, obtains actual physical address, obtained from actual physical address and read data, will read
Data return to proxy module.
Alternatively, the metadata service module further comprises：
Load Sharing Algorithm unit, for selecting the memory node of a light load according to Load Sharing Algorithm.
Alternatively, metadata service module is individually deployed in isolated node, or distributed deployment is in all nodes
On cluster.
The embodiment of the present application additionally provides a kind of method for writing data of distributed memory system, the distributed memory system
As it was previously stated, the method for writing data includes：
Proxy module receives the write request from application system, and the Hash for the data to be write is calculated by piecemeal parameter
Value, obtains block identification, and the write request for carrying the block identification is sent to metadata service module；
Metadata service module receives the write request of the carrying block identification from proxy module, searches one-level mapping table, such as
The map record of the existing corresponding block identification of fruit, refreshes one-level mapping table, increases a write-in initial address and corresponding block identification
Map record, by proxy module to application system return write-in success message, terminate write-in flow；If being not present, selection
One memory node, the nodal information of the memory node is returned to proxy module；
Proxy module receives the nodal information that metadata service module returns, according to the nodal information, write request
It is routed to corresponding memory node；
Storage services module receives the write request for being routed to the memory node that this storage services module is managed, and data are write
Enter the disk of the memory node, refresh two level mapping table, increase corresponding block identification newly and be actually written into the record of physical address, and
Write-in information is sent to metadata service module；
Metadata service module receives the write-in information from storage services module, refreshes one-level mapping table, increases write-in newly
Initial address, corresponding block identification, the map record of respective stored node, return to Flushing success to the storage services module and disappear
Breath；
Write-in success message from metadata service module or storage services module is returned to application by storage services module
System.
The embodiment of the present application additionally provides a kind of method for reading data of distributed memory system, the distributed memory system
As it was previously stated, the method for reading data includes：
Proxy module receives the read request from application system, and the read request is passed through into metadata service module；
Metadata service module receives the read request that initial address is read in the carrying from proxy module, searches one-level mapping
Map record corresponding to reading initial address described in table, obtains corresponding memory node and block identification, read request is route
To corresponding memory node；
Storage services module receives the read request for being routed to the memory node that this storage services module is managed, according to block mark
Know and search two level mapping table, obtain actual physical address, obtained from actual physical address and read data, data will be read and return to generation
Manage module；
Reading data from storage services module are returned to application system by proxy module.
As can be seen from the above technical solutions, due to using two-stage metadata organization, when writing data, if existing
Identical data so as to reduce the operation of data write-in, and reaches the mesh of global data de-duplication then without being actually written into
Mark.Application scheme can realize following technique effect：
Online global data de-duplication is realized in distributed memory system, the physical memory space is reduced and takes；
Because the data repeated do not need actual write operation when writing data, the number of application system and storage system is saved
According to interaction and bandwidth, storage efficiency is improved；
Metadata Service and storage service can be flexibly disposed on memory node, is easy to structure distribution, big rule
The storage system of mould.
Brief description of the drawings
Fig. 1 is that the framework for the distributed memory system using two level metadata organization method that the embodiment of the present application provides shows
It is intended to；
Fig. 2 is that the data for the distributed memory system that the embodiment of the present application provides write schematic flow sheet；
Fig. 3 is the time data stream journey schematic diagram for the distributed memory system that the embodiment of the present application provides.
Embodiment
To make the technical principle of technical scheme, feature and technique effect clearer, below in conjunction with specific reality
Example is applied technical scheme is described in detail.
In application scheme, using two level metadata organization method, the service module of distributed memory system is divided into member
Data service module and storage services module, wherein metadata service module are responsible for safeguarding global one-level mapping table, and one-level is reflected
Firing table contains global logic address and memory node, the mapping relations of block identification (i.e. data block Hash (Hash) value)；Storage
Service module is responsible for safeguarding two level mapping table, and two level mapping table contains block identification and the mapping of actual storage physical address is closed
System.
The framework such as Fig. 1 institutes for the distributed memory system using two level metadata organization method that the embodiment of the present application provides
Show, distributed memory system 100 includes 101, metadata service modules 102 of a proxy module and multiple storage clothes
Business module 103.
The proxy module 101 is used to receive the write request from application system, calculates what is write by piecemeal parameter
The cryptographic Hash of data, obtains block identification, and the write request for carrying the block identification is sent to metadata service module 102；Receive member
The nodal information that data service module 102 returns, according to the nodal information, write request is routed to corresponding storage section
Point；And the write-in success message from metadata service module 102 or storage services module 103 is returned into application system；
The metadata service module 102 is used for the one-level mapping table for safeguarding the overall situation, and one-level mapping table contains the overall situation and patrolled
Collect address and memory node, the mapping relations of block identification；The write request of the carrying block identification from proxy module is received, searches one
Level mapping table, if the map record of existing corresponding block identification, refreshes one-level mapping table, increase a write-in initial address and
The map record of corresponding block identification, write-in success message is returned to application system by proxy module 101；If being not present, selection
One memory node, the nodal information of the memory node is returned to proxy module 101；Reception comes from storage services module
103 write-in information, refresh one-level mapping table, increase write-in initial address, corresponding block identification, the mapping note of respective stored node newly
Record, Flushing success message is returned to the storage services module 103；
Storage services module 103 is used to safeguard two level mapping table, and two level mapping table contains block identification and actual storage thing
Manage the mapping relations of address；The write request for being routed to the memory node that this storage services module 103 is managed is received, data are write
Enter the disk of the memory node, refresh two level mapping table, increase corresponding block identification newly and be actually written into the record of physical address, and
Write-in information is sent to metadata service module 102.
According to another embodiment of the application, the proxy module 101 is additionally operable to receive the read request from application system,
The read request is passed through into metadata service module 102, and the reading data from storage services module 103 are returned into application system
System；
The metadata service module 102 is additionally operable to receive the reading that initial address is read in the carrying from proxy module 101
Request, map record corresponding to reading initial address described in one-level mapping table is searched, obtains corresponding memory node and block
Read request, is routed to corresponding memory node by mark；
The storage services module 103, which is additionally operable to receive, is routed to the memory node that this storage services module 103 is managed
Read request, according to block identification search two level mapping table, obtain actual physical address, reading obtained from actual physical address
According to, by read data return to proxy module 101.
Alternatively, the metadata service module 103 further comprises：
Load Sharing Algorithm unit, for selecting the memory node of a light load according to Load Sharing Algorithm.
Alternatively, metadata service module is individually deployed in isolated node, or distributed deployment is in all nodes
On cluster.
The data for the distributed memory system that the embodiment of the present application provides write flow as shown in figure 1, comprising the following steps：
Step 201：The data that application system asks to write initial address LBA1, length is L to distributed memory system.
Step 202：The proxy module of distributed memory system receives request, and the cryptographic Hash of data is calculated by piecemeal parameter,
Block identification is obtained, the write request for carrying the block identification is sent to metadata service module.
Step 203：Metadata service module searches one-level mapping table, if the map record of existing corresponding block identification,
Step 204 is performed, otherwise, performs step 205.
Step 204：Metadata service module refreshes one-level mapping table, increases the mapping note of a LBA1 and corresponding block identification
Record, return and write successfully to application system, flow terminates.
Step 205：Metadata service module selects the memory node of a light load according to Load Sharing Algorithm, section
Point information returns to proxy module.
Step 206：The nodal information that proxy module returns according to metadata service module, write request is routed to accordingly
Memory node.
Step 207：Storage services module corresponding to the memory node receives write request, writes the data to memory node
Disk, refresh two level mapping table, increase corresponding block identification newly and be actually written into the record of physical address.And to metadata service module
Send write-in information.
Step 208：Metadata service module receives the write-in information that storage services module is sent, and refreshes one-level mapping table,
Newly-increased LBA1, corresponding block identification, the map record of respective stored node；Flushing success is returned to the storage services module to disappear
Breath.
Step 209：Storage services module receives the message of the Flushing success of metadata service module, is returned to application system
Write and successfully (pass through proxy module), flow terminates.
The data to be write existing identical data block in distributed memory system is can be seen that from write-in flow
When, corresponding data need not be write again in ablation process, and only need to refresh one-level mapping table, increase logical address and block mark
The map record of knowledge, equivalent to data de-duplication is realized automatically, the occupancy of the physical memory space is greatlyd save, and
Reduce data write-in flow and data transfer, improve storage efficiency.
The time data stream journey for the distributed memory system that the embodiment of the present application provides is as shown in figure 3, comprise the following steps：
Step 301：Initial address LBA1 is read in proxy module request of the application system to distributed memory system, and length is
L data.
Step 302：The read request is passed through metadata service module by proxy module.
Step 303：Metadata service module receives request, searches map record corresponding to LBA1 in one-level mapping table, obtains
To corresponding memory node and block identification.
Step 304：Read request is routed to corresponding memory node by metadata service module.
Step 305：Storage services module corresponding to the memory node receives read request, and searching two level according to block identification reflects
Firing table, actual physical address is obtained, data are read from actual physical address, return to application system (passing through proxy module).
From above-mentioned read-write flow, because technical scheme employs two-stage mapping table, except in a distributed system
It is natural to support outside online global data de-duplication, also have the advantage that, the coupling between node is more open, can be with
Flexibly use different deployment way.Such as asymmetric distribution formula framework dispositions method, metadata service module portion can be used
Administration is responsible for the management of one-level mapping table and the management of each memory node specially in independent metadata node.Storage service mould
Block is then deployed in memory node and forms cluster, is responsible for the management of two level mapping table and data actual storage；Or it can also adopt
With symmetric distributed framework dispositions method, metadata service module and storage services module are deployed in each memory node jointly,
Metadata service module is responsible for the management of global one-level mapping table, and passes through back-end network real-time synchronization to all nodes, storage
The management of the node two level mapping table and the storage of real data are then responsible in service；Or both the above mode can also be used
Mixed deployment method.
The foregoing is only the preferred embodiment of the application, not to limit the protection domain of the application, it is all
Within the spirit and principle of technical scheme, any modification, equivalent substitution and improvements done etc., this Shen should be included in
Within the scope of please protecting.
Claims (6)
- A kind of 1. distributed memory system, it is characterised in that including：The distributed memory system include a proxy module, One metadata service module and multiple storage services modules, each storage services module manage at least one memory node；The proxy module is used to receive the write request from application system, and the Kazakhstan for the data to be write is calculated by piecemeal parameter Uncommon value, obtains block identification, and the write request for carrying the block identification is sent to metadata service module；Receive metadata service module The nodal information of return, according to the nodal information, write request is routed to corresponding memory node；And metadata will be come from Service module or the write-in success message of storage services module return to application system；The metadata service module be used for safeguard the overall situation one-level mapping table, one-level mapping table contain global logic address and The mapping relations of memory node, block identification；The write request of the carrying block identification from proxy module is received, searches one-level mapping Table, if the map record of existing corresponding block identification, refreshes one-level mapping table, increase a write-in initial address and relevant block The map record of mark, write-in success message is returned to application system by proxy module；If being not present, a storage section is selected Point, the nodal information of the memory node is returned to proxy module；The write-in information from storage services module is received, is refreshed One-level mapping table, write-in initial address, corresponding block identification, the map record of respective stored node are increased newly, to the storage service Module returns to Flushing success message；Said write information includes newly-increased write-in initial address, corresponding block identification；Storage services module is used to safeguard two level mapping table, and two level mapping table contains block identification and actual storage physical address Mapping relations；The write request for being routed to the memory node that this storage services module is managed is received, writes the data to the storage The disk of node, refresh two level mapping table, increase corresponding block identification newly and be actually written into the record of physical address, and taken to metadata Module of being engaged in sends write-in information, receives the Flushing success message that the metadata service module returns, by proxy module to should Write-in success message is returned with system.
- 2. distributed memory system according to claim 1, it is characterised in that the proxy module is additionally operable to receive to come from and answered With the read request of system, the read request is passed through into metadata service module, and by the reading data from storage services module Return to application system；The metadata service module is additionally operable to receive the read request that initial address is read in the carrying from proxy module, searches one Map record corresponding to reading initial address described in level mapping table, obtains corresponding memory node and block identification, please reading Ask and be routed to corresponding memory node；The storage services module is additionally operable to receive the read request for being routed to the memory node that this storage services module is managed, root Two level mapping table is searched according to block identification, obtains actual physical address, is obtained from actual physical address and reads data, data will be read and returned Back to proxy module.
- 3. distributed memory system according to claim 1, it is characterised in that the metadata service module is further wrapped Include：Load Sharing Algorithm unit, for selecting the memory node of a light load according to Load Sharing Algorithm.
- 4. according to the distributed memory system described in any one of claims 1 to 3, it is characterised in that metadata service module list Solely it is deployed in isolated node, or distributed deployment is on all node clusters.
- A kind of 5. method for writing data of distributed memory system, it is characterised in that the distributed memory system such as claim 1 Described, the method for writing data includes：Proxy module receives the write request from application system, and the cryptographic Hash for the data to be write is calculated by piecemeal parameter, is obtained To block identification, the write request for carrying the block identification is sent to metadata service module；Metadata service module receives the write request of the carrying block identification from proxy module, one-level mapping table is searched, if In the presence of the map record of corresponding block identification, refresh one-level mapping table, increase reflecting for a write-in initial address and corresponding block identification Record is penetrated, write-in success message is returned to application system by proxy module, terminates write-in flow；If being not present, one is selected Memory node, the nodal information of the memory node is returned to proxy module；Proxy module receives the nodal information that metadata service module returns, and according to the nodal information, write request is route To corresponding memory node；Storage services module receives the write request for being routed to the memory node that this storage services module is managed, and writes the data to institute State the disk of memory node, refresh two level mapping table, increase corresponding block identification newly and be actually written into the record of physical address, and to member Data service module sends write-in information；Metadata service module receives the write-in information from storage services module, refreshes one-level mapping table, increases write-in starting newly Address, corresponding block identification, the map record of respective stored node, Flushing success message is returned to the storage services module；Write-in success message from metadata service module or storage services module is returned to application system by storage services module.
- A kind of 6. method for reading data of distributed memory system, it is characterised in that the distributed memory system such as claim 2 Described, the method for reading data includes：Proxy module receives the read request from application system, and the read request is passed through into metadata service module；Metadata service module receives the read request that initial address is read in the carrying from proxy module, searches in one-level mapping table Map record corresponding to the reading initial address, corresponding memory node and block identification are obtained, read request is routed to phase The memory node answered；Storage services module receives the read request for being routed to the memory node that this storage services module is managed, and is looked into according to block identification Two level mapping table is looked for, obtains actual physical address, is obtained from actual physical address and reads data, reading data are returned to and act on behalf of mould Block；Reading data from storage services module are returned to application system by proxy module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510657207.0A CN105242881B (en) | 2015-10-12 | 2015-10-12 | Distributed memory system and its data read-write method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510657207.0A CN105242881B (en) | 2015-10-12 | 2015-10-12 | Distributed memory system and its data read-write method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105242881A CN105242881A (en) | 2016-01-13 |
| CN105242881B true CN105242881B (en) | 2018-02-16 |
Family
ID=55040544
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510657207.0A Active CN105242881B (en) | 2015-10-12 | 2015-10-12 | Distributed memory system and its data read-write method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105242881B (en) |
Families Citing this family (35)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106843773B (en) * | 2017-02-16 | 2021-12-03 | 北京书生云科技有限公司 | Storage method and distributed storage system |
| CN107135242B (en) * | 2016-02-29 | 2020-06-05 | 阿里巴巴集团控股有限公司 | Mongodb cluster access method, device and system |
| CN107436725B (en) * | 2016-05-25 | 2019-12-20 | 杭州海康威视数字技术股份有限公司 | Data writing and reading methods and devices and distributed object storage cluster |
| CN107948233B (en) * | 2016-10-13 | 2021-01-08 | 华为技术有限公司 | Method for processing write request or read request, switch and control node |
| CN106802776A (en) * | 2017-01-19 | 2017-06-06 | 郑州云海信息技术有限公司 | A kind of method of mapped file to actual physical storage |
| KR20180121733A (en) | 2017-04-28 | 2018-11-08 | 에스케이하이닉스 주식회사 | Data storage device and operating method thereof |
| CN107193503B (en) * | 2017-05-27 | 2020-05-29 | 杭州宏杉科技股份有限公司 | Data deduplication method and storage device |
| CN109284621B (en) * | 2017-07-19 | 2022-10-25 | 阿里巴巴集团控股有限公司 | Data protection/recovery method and device and data storage/protection method |
| CN108121670B (en) * | 2017-08-07 | 2021-09-28 | 鸿秦（北京）科技有限公司 | Mapping method for reducing solid state disk metadata back-flushing frequency |
| CN108280208B (en) * | 2018-01-30 | 2022-05-13 | 深圳市茁壮网络股份有限公司 | Sample searching method and device |
| CN109327539A (en) * | 2018-11-15 | 2019-02-12 | 上海天玑数据技术有限公司 | A kind of distributed block storage system and its data routing method |
| CN109614036B (en) * | 2018-11-16 | 2022-05-10 | 新华三技术有限公司成都分公司 | Storage space deployment method and device |
| CN109597587A (en) * | 2018-12-10 | 2019-04-09 | 浪潮(北京)电子信息产业有限公司 | A kind of method for writing data, medium and Nonvolatile memory |
| CN111367461B (en) * | 2018-12-25 | 2024-02-20 | 兆易创新科技集团股份有限公司 | Storage space management method and device |
| CN109862090A (en) * | 2019-01-25 | 2019-06-07 | 深圳市太之阳网络技术有限公司 | A kind of high degree of safety distribution cloud storage system |
| CN109885536B (en) * | 2019-02-26 | 2023-06-16 | 深圳众享互联科技有限公司 | Distributed data fragment storage and fuzzy search method |
| US10908960B2 (en) * | 2019-04-16 | 2021-02-02 | Alibaba Group Holding Limited | Resource allocation based on comprehensive I/O monitoring in a distributed storage system |
| CN110751275B (en) * | 2019-08-03 | 2022-09-02 | 北京达佳互联信息技术有限公司 | Graph training system, data access method and device, electronic device and storage medium |
| CN110502507B (en) * | 2019-08-29 | 2022-02-08 | 上海达梦数据库有限公司 | Management system, method, equipment and storage medium of distributed database |
| CN110597779A (en) * | 2019-09-20 | 2019-12-20 | 浪潮电子信息产业股份有限公司 | Data reading and writing method in distributed file system and related device |
| EP4057585A4 (en) * | 2019-12-06 | 2022-12-28 | Huawei Cloud Computing Technologies Co., Ltd. | Edge system and method for processing data operation request |
| US11709609B2 (en) * | 2020-03-27 | 2023-07-25 | Via Technologies, Inc. | Data storage system and global deduplication method thereof |
| CN111538465B (en) * | 2020-07-07 | 2020-10-23 | 南京云信达科技有限公司 | Linux-based high-performance data deduplication method |
| CN114138756B (en) * | 2020-09-03 | 2023-03-24 | 金篆信科有限责任公司 | Data deduplication method, node and computer-readable storage medium |
| CN112052218B (en) * | 2020-09-25 | 2023-04-25 | 杭州宏杉科技股份有限公司 | Snapshot implementation method and distributed storage cluster |
| CN112181300B (en) * | 2020-09-25 | 2022-02-22 | 杭州宏杉科技股份有限公司 | Data processing method and distributed storage cluster |
| CN112181299B (en) * | 2020-09-25 | 2022-02-25 | 杭州宏杉科技股份有限公司 | Data restoration method and distributed storage cluster |
| CN112487089B (en) * | 2020-11-26 | 2024-07-02 | 广州海量数据库技术有限公司 | Distributed storage method and system based on data page routing |
| CN112527200A (en) * | 2020-12-07 | 2021-03-19 | 成都网信天成科技有限公司 | Jmatrix distributed cloud storage system |
| CN113312004A (en) * | 2021-06-15 | 2021-08-27 | 华录光存储研究院（大连）有限公司 | Multi-level distributed storage system and storage method |
| CN115712581A (en) * | 2021-08-23 | 2023-02-24 | 华为技术有限公司 | Data access method, storage system and storage node |
| CN113918097A (en) * | 2021-10-22 | 2022-01-11 | 云南电网有限责任公司电力科学研究院 | Distributed storage method and system |
| CN114518845B (en) * | 2022-01-06 | 2024-09-10 | 中汽创智科技有限公司 | Data storage method, device, medium and equipment |
| CN117271224B (en) * | 2023-11-14 | 2024-02-20 | 苏州元脑智能科技有限公司 | Data repeated storage processing method and device of storage system, storage medium and electronic equipment |
| CN118394802A (en) * | 2024-06-27 | 2024-07-26 | 国网山东省电力公司滨州市沾化区供电公司 | Power monitoring data storage management method, system, equipment and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915278A (en) * | 2012-09-19 | 2013-02-06 | 浪潮(北京)电子信息产业有限公司 | Data deduplication method |
| CN103455631A (en) * | 2013-09-22 | 2013-12-18 | 广州中国科学院软件应用技术研究所 | Method, device and system for processing data |
| CN103970875A (en) * | 2014-05-15 | 2014-08-06 | 华中科技大学 | Parallel repeated data deleting method |
| CN104778095A (en) * | 2015-01-20 | 2015-07-15 | 成都携恩科技有限公司 | Cloud platform data management method |
| CN104932841A (en) * | 2015-06-17 | 2015-09-23 | 南京邮电大学 | Saving type duplicated data deleting method in cloud storage system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8321648B2 (en) * | 2009-10-26 | 2012-11-27 | Netapp, Inc | Use of similarity hash to route data for improved deduplication in a storage server cluster |
| KR102127116B1 (en) * | 2014-03-12 | 2020-06-26 | 삼성전자 주식회사 | Device and method for storing data in distributed storage system |
-
2015
- 2015-10-12 CN CN201510657207.0A patent/CN105242881B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915278A (en) * | 2012-09-19 | 2013-02-06 | 浪潮(北京)电子信息产业有限公司 | Data deduplication method |
| CN103455631A (en) * | 2013-09-22 | 2013-12-18 | 广州中国科学院软件应用技术研究所 | Method, device and system for processing data |
| CN103970875A (en) * | 2014-05-15 | 2014-08-06 | 华中科技大学 | Parallel repeated data deleting method |
| CN104778095A (en) * | 2015-01-20 | 2015-07-15 | 成都携恩科技有限公司 | Cloud platform data management method |
| CN104932841A (en) * | 2015-06-17 | 2015-09-23 | 南京邮电大学 | Saving type duplicated data deleting method in cloud storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105242881A (en) | 2016-01-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105242881B (en) | Distributed memory system and its data read-write method | |
| CN102662992B (en) | Method and device for storing and accessing massive small files | |
| CN106407040B (en) | A kind of duplicating remote data method and system | |
| CN102222085B (en) | Data de-duplication method based on combination of similarity and locality | |
| CN103067433B (en) | A kind of data migration method of distributed memory system, equipment and system | |
| CN104850358B (en) | A kind of magneto-optic electricity mixing storage system and its data acquisition and storage method | |
| CN107832423B (en) | File reading and writing method for distributed file system | |
| US20160364407A1 (en) | Method and Device for Responding to Request, and Distributed File System | |
| US8930501B2 (en) | Distributed data storage system and method | |
| CN101840308A (en) | Hierarchical memory system and logical volume management method thereof | |
| CN102821138A (en) | Metadata distributed storage method applicable to cloud storage system | |
| CN104156380A (en) | Distributed memory Hash indexing method and system | |
| CN102467408A (en) | Method and device for accessing data of virtual machine | |
| CN104503703B (en) | The treating method and apparatus of caching | |
| CN106775446A (en) | Based on the distributed file system small documents access method that solid state hard disc accelerates | |
| CN104408111A (en) | Method and device for deleting duplicate data | |
| CN107329704B (en) | Cache mirroring method and controller | |
| CN103888499A (en) | Distributed object processing method and system | |
| CN103501319A (en) | Low-delay distributed storage system for small files | |
| CN104765574A (en) | Data cloud storage method | |
| CN107026876A (en) | A kind of file data accesses system and method | |
| CN102023816A (en) | Object storage policy and access method of object storage system | |
| CN109407985B (en) | Data management method and related device | |
| CN104158863A (en) | Cloud storage mechanism based on transaction-level whole-course high-speed buffer | |
| CN109862090A (en) | A kind of high degree of safety distribution cloud storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 518057 Room 501A, 9 Buildings, Shenzhen Software Park (Phase 2), No. 1 Science and Technology Middle Road, Nanshan High-tech Zone, Shenzhen, Guangdong Province Co-patentee after: Innovation Technology Co., Ltd. Patentee after: Shenzhen Innovation Software Technology Co., Ltd. Address before: 518057 Section D West, 2nd Floor, 1st # Building, Industrial Building, Great Wall Computer Building, No. 3 Kefa Road, Nanshan District, Shenzhen City, Guangdong Province Co-patentee before: Innovation and Technology Storage Technology Co., Ltd. Patentee before: Chuangxinke Software Technology (Shenzhen) Co., Ltd. |