CN104573064A - Data processing method under big-data environment - Google Patents
Data processing method under big-data environment Download PDFInfo
- Publication number
- CN104573064A CN104573064A CN201510036120.1A CN201510036120A CN104573064A CN 104573064 A CN104573064 A CN 104573064A CN 201510036120 A CN201510036120 A CN 201510036120A CN 104573064 A CN104573064 A CN 104573064A
- Authority
- CN
- China
- Prior art keywords
- file
- data
- files
- blocks
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 10
- 230000005540 biological transmission Effects 0.000 claims abstract description 19
- 238000000034 method Methods 0.000 claims description 24
- 230000015654 memory Effects 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 13
- 238000012795 verification Methods 0.000 claims description 8
- 238000009825 accumulation Methods 0.000 claims description 7
- 230000004069 differentiation Effects 0.000 claims description 7
- 230000015572 biosynthetic process Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/178—Techniques for file synchronisation in file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
- G06F16/1752—De-duplication implemented within the file system, e.g. based on file segments based on file chunks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a data processing method under big-data environment. The data processing method includes according to operating granularity of files, dividing the files into file blocks, physical files and logic files, and defining file block metadata, physical file metadata and logic file metadata respectively; comparing file metadata of a new version and an old version on a server side, and performing redundancy duplicate removal on same data through the file blocks in a storage layer so as to execute differential transmission of the data of the new version and the old version on a client side and the server side. By the data processing method, data synchronizing under the big-data environment is realized, computing pressure of the server side is reduced, and transmission efficiency and resource utilization rate are improved.
Description
Technical field
The present invention relates to data processing, particularly the large data synchronizing processing method of one.
Background technology
Along with the development of cloud computing technology, increasing user selects remote storage service to carry out management by synchronization personal document resource.Share the file synchronization transmission under scene, upload down with personal document and be loaded with a great difference.A file may be shared to the different terminals of multiple user, and the transmission path of file data is the star-like topological structure by central server and each client's side link.The now amendment of a client file, the file data synchronous operation between the user terminal that all participations can be brought to share and cluster server.When height is concurrent, the I/O pressure of server end can sharply increase.And under remote storage environment, traditional File Transmission Algorithm exists great limitation in conjunction with concrete transmission environment, there is the necessity optimized further.
Therefore, for the problems referred to above existing in correlation technique, at present effective solution is not yet proposed.
Summary of the invention
For solving the problem existing for above-mentioned prior art, the present invention proposes the data processing method under a kind of large data environment, comprising:
According to the operation granularity of file, file is divided into blocks of files, physical file and logical file, and difference defined file block metadata, physical file metadata and logical file metadata;
The file metadata of new and old two versions is compared at server end, by accumulation layer file block, redundancy duplicate removal is carried out to identical data, thus perform the differentiation transmission of new and old edition data at client and server.
Preferably, each described logical file associates a physical file, a physical file is divided into multiple blocks of files and is stored into accumulation layer, blocks of files is exercisable file data unit minimum in system, multiple logical file can associate same physical file, and multiple physical file can share a blocks of files.
Preferably, if client upload new edition file is to server, then client is as transmit leg, cluster server is as take over party, when user have modified client source document copy formation updating file when cluster server and client there being the identical copies of a source document, the updating file of client being uploaded to process cluster server being formed redaction and comprises:
Step 1. client initiates updating file upload request to cluster server;
Source document is divided into blocks of files by fixed size by step 2. cluster server, calculates Hash check code and the MD5 check code of each blocks of files, and by two of each blocks of files check code regiments to transmitting terminal.
Step is after 3. cluster server receives the set of two check codes, moving window updating file being opened to fixed size verifies, the verification of moving window with 1 byte for single distance is carried out forward, every 1 byte is carried out a Hash School Affairs MD5 and is verified, after the whole file of verification traversal, orient the position of updating file and all identical block of source document, be then transferred to the data block information of the updating file of cluster server;
4. after cluster server receives the content of client transmissions, file data blocks is associated with the piecemeal metadata of updating file, form new edition file, need the blocks of files of association to be made up of two parts, the blocks of files that updating file is exclusive and updating file and the public blocks of files of source document; The updating file of client upload is increased newly blocks of files to associate with the piecemeal metadata of updating file; Then orient common file block by existing for cluster server end source document, and it associated with updating file metadata, so far form complete updating file at cluster server, upload file process terminates.
Preferably, described step moving window 3. carries out checking procedure and comprises further:
First calculate the hashed value of data in moving window, and in check code set, search this hashed value whether exist; If there is no this hashed value, then moving window forward slip 1 byte; If there is this hashed value, then calculate the MD5 value of data in moving window further, if MD5 value exists in check code set, then determine that in current moving window, data are repeating data, if there is no MD5 value, then moving window forward slip 1 byte;
Wherein said check code only calculates byte values different between twice window at every turn, the next byte at every turn read after moving window from disk when calculating Hash check code enters internal memory, with moving window first character joint number according to replacement, upgrade the information of check code of moving window.
The present invention compared to existing technology, has the following advantages:
Present invention achieves the data syn-chronization under large data environment, alleviate server end calculating pressure, improve transfer efficiency and resource utilization.
Accompanying drawing explanation
Fig. 1 is the process flow diagram according to the data processing method under the large data environment of the embodiment of the present invention.
Embodiment
Detailed description to one or more embodiment of the present invention is hereafter provided together with the accompanying drawing of the diagram principle of the invention.Describe the present invention in conjunction with such embodiment, but the invention is not restricted to any embodiment.Scope of the present invention is only defined by the claims, and the present invention contain many substitute, amendment and equivalent.Set forth many details in the following description to provide thorough understanding of the present invention.These details are provided for exemplary purposes, and also can realize the present invention according to claims without some in these details or all details.
Therefore, the present invention proposes the method for data synchronization under a kind of large data environment, the mode of each client is given by the use of Optimized Service device end memory and Computation for apportionment pressure, reduce the CPU calculating of server end and the expense of I/O as far as possible, make the utilization factor of transfer efficiency and cluster server resource all more efficient.Fig. 1 is according to the data processing method process flow diagram under the large data environment of the embodiment of the present invention.
First defined file metadata, according to file by the granularity operated, is divided into blocks of files, physical file metadata and logical file metadata by file metadata.Each logical file associates a physical file.A physical file cutting is that multiple blocks of files is stored into accumulation layer.Blocks of files is exercisable file data unit minimum in system.Wherein, allow to occur that multiple logical file associates the situation of same physical file, allow to occur that multiple physical file shares the situation of a blocks of files.
Because the operation of logical file separates with the operation in fact descending year physical file, framework is more clear.Piecemeal is more convenient for differentiation transmission.
Multiple client forms the application layer of large data handling system; The core business functions such as file service, user management, equipment control form service layer; The bottom is the accumulation layer of cluster server, and providing can the memory interface of adaptive multiple different storage device, supports local storage and third party's remote storage.File data transfer mainly occurs when adjacent two layers carries out data interaction.Namely in the Internet Transmission in upper strata and middle layer and the transmission of the storage data between middle layer and bottom.
Pass in new edition file process on the client, definition client is transmit leg, and cluster server is take over party.Cluster server and client there is the identical copies of a file f ile.old.Subsequently when user have modified client file.old copy, form file.new.Therefore need the file.new in client to upload on cluster server to form redaction.Differentiation transmission can be divided into four steps:
1. client initiates file.new upload request to cluster server.
2. file.old is divided into blocks of files by fixed size by cluster server, calculates Hash check code and the MD5 check code of each blocks of files, and by two of each blocks of files check code regiments to transmitting terminal.
3., after cluster server receives the set of two check codes, the moving window of file.new file opening fixed size is verified.The verification of moving window is with 1 byte for single distance is carried out forward, and every 1 byte is carried out a Hash School Affairs MD5 and verified.After the whole file of verification traversal, just can orient the position of all identical block of file.new and file.old, remaining is exactly the data block information of the file.new needing to be transferred to cluster server.So last client needs the data of transmission to be data block not identical with file.old in file.new, and the piecemeal metadata information of redaction file.new.
4. after cluster server receives the content of client differentiation transmission, file data blocks is associated with the piecemeal metadata of file.new, form new edition file.The blocks of files of association is needed to be made up of two parts, the blocks of files that file.new is exclusive and the public blocks of files of file.new and file.old.Therefore need to do two step operations: 1. the file.new of client upload is increased newly blocks of files and associate with the piecemeal metadata of file.new; 2. orient common file block by existing for cluster server end file.old file, and it is associated with file.new file metadata.So far, complete file.new file is formed at cluster server.One time upload file process terminates.
Wherein, the verification mentioned is 3. time overhead main in transmitting procedure, and checking procedure is specific as follows:
1. first calculate the hashed value of data in moving window, and in check code set, search this hashed value whether exist.
2. if there is no, then moving window forward slip 1 byte.If existed, then calculate the MD5 value of data in moving window further.If MD5 value exists in check code set, then determine that in current moving window, data are repeating data, if there is no then moving window forward slip 1 byte.
Data transmission method can be applied in upload and download two processes respectively.Upload in synchronizing process aforementioned, because the identical data between the new old edition of upload file is stored in bottom accumulation layer with blocks of files form.System can carry out redundancy duplicate removal to identical block, the blocks of files of the multiplexing legacy version stored.Therefore, the content that needs upgrade is the blocks of files of new edition file metadata information and redaction knots modification.
Downloading synchronous process, is namely the extensive typical scene shared.Now, server file, than client file version updating, needs new edition file to be synchronized to each client.The transmit leg of parallel downloading is cluster server, and when large-scale concurrent is downloaded, verification also can be carried out server end is concurrent.Because the version of downloading process client file will lower than cluster server FileVersion, namely the FileVersion of client is an old version of server file.From above, all old versions of file can obtain in server end bottom stores, and new and old edition is that differentiation stores.Therefore, by being compared at server end by the file metadata of two versions, just can determine which blocks of files needs to be transferred to client.Now, the expense of server end calculation check code can be transferred to client-side program.
Different from end-to-end application, there is under remote storage environment high concurrent characteristic.And the differentiation under cluster-based storage environment is transmitted, server end is as the central point in star transmission topological structure, and need the high concurrent scene of reply, the concurrency of system is subject to the restriction of memory size, the network bandwidth, magnetic disc i/o read or write speed.
Check code only needs byte values different between calculating twice window at every turn.Therefore magnetic disc i/o only needs the data paying close attention to these two bytes of the first tail of moving window one.Therefore, the next byte at every turn read after moving window from disk when calculating Hash check code enters internal memory, with moving window first character joint number according to replacement, realizes the information of check code upgrading moving window.This mode can solve EMS memory occupation when each Hash of calculating verifies and depend on the problem of moving window size.
For byte level, other slides, and the operation of reading disk is very frequent and consuming time, and the present invention adopts buffer memory to pre-read several bytes.Concrete data cached update method is as follows:
Open up two buffer memorys, headBuffer and tailBuffer, size is all bufferSize, and wherein size is much smaller than moving window size.Moving window shifts out HeadBuffer [i] and comprises tailBuffer [i] is a window sliding.
Moving window read (bufferSize-1) secondary after, move to buffer memory the 1st position by data cached for bufferSize, read in (bufferSize-1) individual byte subsequently, complete the forward slip operation of buffer memory.
Moving window end to end data buffer memory is set effectively can optimizes disk access.Take into account the restriction of memory size and disk read-write speed, made algorithm more be applicable to concurrent actual motion environment.
In sum, the multiple terminals file synchronization of remote storage service can carry out transmission optimization, the present invention proposes the optimization to internal memory space complexity and disk access frequency, for the high concurrent feature of remote storage clothes, utilize the metadata information of server file old version, carry out download process optimization, reduce calculated amount; Redaction storing process is optimized, and realizes storage redundancy duplicate removal, promotes storage space utilization factor.
Obviously, it should be appreciated by those skilled in the art, above-mentioned of the present invention each module or each step can realize with general computing system, they can concentrate on single computing system, or be distributed on network that multiple computing system forms, alternatively, they can realize with the executable program code of computing system, thus, they can be stored in storage platform and be performed by computing system.Like this, the present invention is not restricted to any specific hardware and software combination.
Should be understood that, above-mentioned embodiment of the present invention only for exemplary illustration or explain principle of the present invention, and is not construed as limiting the invention.Therefore, any amendment made when without departing from the spirit and scope of the present invention, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.In addition, claims of the present invention be intended to contain fall into claims scope and border or this scope and border equivalents in whole change and modification.
Claims (4)
1. the data processing method under large data environment, for synchrodata between client and cluster server, is characterized in that, comprising:
According to the operation granularity of file, file is divided into blocks of files, physical file and logical file, and difference defined file block metadata, physical file metadata and logical file metadata;
The file metadata of new and old two versions is compared at server end, by accumulation layer file block, redundancy duplicate removal is carried out to identical data, thus perform the differentiation transmission of new and old edition data at client and server.
2. method according to claim 1, it is characterized in that, each described logical file associates a physical file, a physical file is divided into multiple blocks of files and is stored into accumulation layer, blocks of files is exercisable file data unit minimum in system, multiple logical file can associate same physical file, and multiple physical file can share a blocks of files.
3. method according to claim 2, it is characterized in that, if client upload new edition file is to server, then client is as transmit leg, cluster server is as take over party, when user have modified client source document copy formation updating file when cluster server and client there being the identical copies of a source document, the updating file of client being uploaded to process cluster server being formed redaction and comprises:
Step 1. client initiates updating file upload request to cluster server;
Source document is divided into blocks of files by fixed size by step 2. cluster server, calculates Hash check code and the MD5 check code of each blocks of files, and by two of each blocks of files check code regiments to transmitting terminal;
Step is after 3. cluster server receives the set of two check codes, moving window updating file being opened to fixed size verifies, the verification of moving window with 1 byte for single distance is carried out forward, every 1 byte is carried out a Hash School Affairs MD5 and is verified, after the whole file of verification traversal, orient the position of updating file and all identical block of source document, be then transferred to the data block information of the updating file of cluster server;
4. after cluster server receives the content of client transmissions, file data blocks is associated with the piecemeal metadata of updating file, form new edition file, need the blocks of files of association to be made up of two parts, the blocks of files that updating file is exclusive and updating file and the public blocks of files of source document; The updating file of client upload is increased newly blocks of files to associate with the piecemeal metadata of updating file; Then orient common file block by existing for cluster server end source document, and it associated with updating file metadata, so far form complete updating file at cluster server, upload file process terminates.
4. method according to claim 3, is characterized in that, described step moving window 3. carries out checking procedure and comprises further:
First calculate the hashed value of data in moving window, and in check code set, search this hashed value whether exist; If there is no this hashed value, then moving window forward slip 1 byte; If there is this hashed value, then calculate the MD5 value of data in moving window further, if MD5 value exists in check code set, then determine that in current moving window, data are repeating data, if there is no MD5 value, then moving window forward slip 1 byte;
Wherein said check code only calculates byte values different between twice window at every turn, the next byte at every turn read after moving window from disk when calculating Hash check code enters internal memory, with moving window first character joint number according to replacement, upgrade the information of check code of moving window.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510036120.1A CN104573064B (en) | 2015-01-23 | 2015-01-23 | A kind of data processing method under big data environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510036120.1A CN104573064B (en) | 2015-01-23 | 2015-01-23 | A kind of data processing method under big data environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104573064A true CN104573064A (en) | 2015-04-29 |
| CN104573064B CN104573064B (en) | 2017-12-08 |
Family
ID=53089126
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510036120.1A Active CN104573064B (en) | 2015-01-23 | 2015-01-23 | A kind of data processing method under big data environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104573064B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107103051A (en) * | 2017-04-05 | 2017-08-29 | 成都爱途享科技有限公司 | Set up the quick loading device in processing data |
| CN109977077A (en) * | 2019-03-25 | 2019-07-05 | 腾讯科技（深圳）有限公司 | Model file storage method, device, readable storage medium storing program for executing and computer equipment |
| CN110520857A (en) * | 2017-04-17 | 2019-11-29 | 微软技术许可有限责任公司 | Data processing performance enhancing is carried out to neural network using virtualization data iterator |
| CN110633257A (en) * | 2019-09-20 | 2019-12-31 | 中国银行股份有限公司 | Real-time synchronization method and system for bank parameter files in private cloud environment |
| CN110795499A (en) * | 2019-09-17 | 2020-02-14 | 中国平安人寿保险股份有限公司 | Cluster data synchronization method, device and equipment based on big data and storage medium |
| CN111241189A (en) * | 2018-11-29 | 2020-06-05 | 北京京东尚科信息技术有限公司 | Method and device for synchronizing data |
| CN111880956A (en) * | 2020-07-24 | 2020-11-03 | 北京达佳互联信息技术有限公司 | Data synchronization method and device |
| CN113647076A (en) * | 2019-03-26 | 2021-11-12 | 微软技术许可有限责任公司 | Sliding window buffer for minimum local resource requirements |
| CN114490625A (en) * | 2022-04-06 | 2022-05-13 | 深圳嘉麟科技有限责任公司 | Artificial intelligence public data platform capable of automatically updating and removing redundancy based on big data processing |
| CN115760023A (en) * | 2022-10-27 | 2023-03-07 | 上海美嘉林信息技术有限公司 | Intelligent alignment system and method for digital collaborative design environment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101387976A (en) * | 2008-11-07 | 2009-03-18 | 清华大学 | Logical volume rapid synchronization method based on data differentia |
| CN101770515A (en) * | 2010-01-18 | 2010-07-07 | 杭州顺网科技股份有限公司 | Data block comparison based data updating method |
| US20120179784A1 (en) * | 2009-09-21 | 2012-07-12 | Thomson Licensing | Device and method for generating confirmations of data transfers between communication equipments, by data comparison |
| CN103379160A (en) * | 2012-04-25 | 2013-10-30 | 上海咏云信息技术有限公司 | Difference synchronizing method for oversized file |
-
2015
- 2015-01-23 CN CN201510036120.1A patent/CN104573064B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101387976A (en) * | 2008-11-07 | 2009-03-18 | 清华大学 | Logical volume rapid synchronization method based on data differentia |
| US20120179784A1 (en) * | 2009-09-21 | 2012-07-12 | Thomson Licensing | Device and method for generating confirmations of data transfers between communication equipments, by data comparison |
| CN101770515A (en) * | 2010-01-18 | 2010-07-07 | 杭州顺网科技股份有限公司 | Data block comparison based data updating method |
| CN103379160A (en) * | 2012-04-25 | 2013-10-30 | 上海咏云信息技术有限公司 | Difference synchronizing method for oversized file |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107103051A (en) * | 2017-04-05 | 2017-08-29 | 成都爱途享科技有限公司 | Set up the quick loading device in processing data |
| CN110546628B (en) * | 2017-04-17 | 2023-10-20 | 微软技术许可有限责任公司 | Minimizing memory reads with directed line buffers to improve neural network environmental performance |
| CN110520857A (en) * | 2017-04-17 | 2019-11-29 | 微软技术许可有限责任公司 | Data processing performance enhancing is carried out to neural network using virtualization data iterator |
| CN110546628A (en) * | 2017-04-17 | 2019-12-06 | 微软技术许可有限责任公司 | minimizing memory reads with directed line buffers to improve neural network environmental performance |
| CN111241189A (en) * | 2018-11-29 | 2020-06-05 | 北京京东尚科信息技术有限公司 | Method and device for synchronizing data |
| CN109977077A (en) * | 2019-03-25 | 2019-07-05 | 腾讯科技（深圳）有限公司 | Model file storage method, device, readable storage medium storing program for executing and computer equipment |
| CN113647076A (en) * | 2019-03-26 | 2021-11-12 | 微软技术许可有限责任公司 | Sliding window buffer for minimum local resource requirements |
| CN113647076B (en) * | 2019-03-26 | 2023-02-10 | 微软技术许可有限责任公司 | Sliding window buffer for minimum local resource requirements |
| CN110795499A (en) * | 2019-09-17 | 2020-02-14 | 中国平安人寿保险股份有限公司 | Cluster data synchronization method, device and equipment based on big data and storage medium |
| CN110795499B (en) * | 2019-09-17 | 2024-04-16 | 中国平安人寿保险股份有限公司 | Cluster data synchronization method, device, equipment and storage medium based on big data |
| CN110633257A (en) * | 2019-09-20 | 2019-12-31 | 中国银行股份有限公司 | Real-time synchronization method and system for bank parameter files in private cloud environment |
| CN111880956A (en) * | 2020-07-24 | 2020-11-03 | 北京达佳互联信息技术有限公司 | Data synchronization method and device |
| CN111880956B (en) * | 2020-07-24 | 2023-12-05 | 北京达佳互联信息技术有限公司 | Data synchronization method and device |
| CN114490625A (en) * | 2022-04-06 | 2022-05-13 | 深圳嘉麟科技有限责任公司 | Artificial intelligence public data platform capable of automatically updating and removing redundancy based on big data processing |
| CN115760023A (en) * | 2022-10-27 | 2023-03-07 | 上海美嘉林信息技术有限公司 | Intelligent alignment system and method for digital collaborative design environment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104573064B (en) | 2017-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104573064A (en) | Data processing method under big-data environment | |
| US11985192B2 (en) | Synchronized content library | |
| CN106156359B (en) | A kind of data synchronization updating method under cloud computing platform | |
| US10831720B2 (en) | Cloud storage distributed file system | |
| US20240179212A1 (en) | Hosted file sync with stateless sync nodes | |
| US11294855B2 (en) | Cloud-aware snapshot difference determination | |
| US20190370362A1 (en) | Multi-protocol cloud storage for big data and analytics | |
| US10019460B2 (en) | Hosted file sync with direct access to hosted files | |
| US10659225B2 (en) | Encrypting existing live unencrypted data using age-based garbage collection | |
| US9460111B2 (en) | Method and apparatus for virtualization of a file system, data storage system for virtualization of a file system, and file server for use in a data storage system | |
| US20190370365A1 (en) | Distributed transactions in cloud storage with hierarchical namespace | |
| US9785646B2 (en) | Data file handling in a network environment and independent file server | |
| US9645753B2 (en) | Overlapping write detection and processing for sync replication | |
| EP2557514A1 (en) | Cloud Storage System with Distributed Metadata | |
| US10459886B2 (en) | Client-side deduplication with local chunk caching | |
| US9917894B2 (en) | Accelerating transfer protocols | |
| CN109379432A (en) | Data processing method, device, server and computer readable storage medium | |
| Uppoor et al. | Cloud-based synchronization of distributed file system hierarchies | |
| CN103875229A (en) | Asynchronous replication method, device and system | |
| US10339124B2 (en) | Data fingerprint strengthening | |
| JP6196389B2 (en) | Distributed disaster recovery file synchronization server system | |
| CN105721520B (en) | A kind of file synchronisation method and device | |
| KR20180073128A (en) | A data updating method based on data block comparison | |
| Zhang et al. | SimpleSync: A parallel delta synchronization method based on Flink | |
| CN116542668A (en) | Block chain-based data processing method, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20200113 Address after: 610000 No. 603, 604, floor 6, building 5, Jingrong International Plaza, No. 200, Tianfu 5th Street, Chengdu hi tech Zone, Chengdu, Sichuan Province Patentee after: Sichuan Aofei Air Service Technology Co.,Ltd. Address before: 610041, No. 1, No. 209, building 6, 1, No. 1, Tianfu Road, Chengdu hi tech Zone, Sichuan Patentee before: SICHUAN ZHONGKE TENGXIN TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| CP01 | Change in the name or title of a patent holder |
Address after: No. 603, 604, 6th Floor, Block B, Building 5, Jingrong International Plaza, No. 200 Tianfu Fifth Street, Chengdu High tech Zone, Chengdu, Sichuan Province, 610000 Patentee after: Sichuan Xian'en Airsuit Technology Co.,Ltd. Address before: No. 603, 604, 6th Floor, Block B, Building 5, Jingrong International Plaza, No. 200 Tianfu Fifth Street, Chengdu High tech Zone, Chengdu, Sichuan Province, 610000 Patentee before: Sichuan Aofei Air Service Technology Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231013 Address after: No. 204b, Building B, No.1 Keyuan Second Road, High tech Zone, Chengdu, Sichuan Province, 610000 (self numbered) Patentee after: Chengdu Kaiyuan Zhongzhi Information Technology Co.,Ltd. Address before: No. 603, 604, 6th Floor, Block B, Building 5, Jingrong International Plaza, No. 200 Tianfu Fifth Street, Chengdu High tech Zone, Chengdu, Sichuan Province, 610000 Patentee before: Sichuan Xian'en Airsuit Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right |