[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

Ko et al., 2013 - Google Patents

Stride static chunking algorithm for deduplication system

Ko et al., 2013

View PDF
Document ID
14921594065516597527
Author
Ko Y
Jung H
Lee W
Kim M
Yoo C
Publication year
Publication venue
IEICE TRANSACTIONS on Information and Systems

External Links

Snippet

In this paper, we propose a stride static chunking deduplication algorithm using a hybrid approach that exploits the advantages of static chunking and byte-shift chunking algorithm. The key contribution of our approach is to reduce the computation time and enhance …
Continue reading at www.jstage.jst.go.jp (PDF) (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30067—File systems; File servers
    • G06F17/30129—Details of further file system functionalities
    • G06F17/3015—Redundancy elimination performed by the file system
    • G06F17/30153—Redundancy elimination performed by the file system using compression, e.g. sparse files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30067—File systems; File servers
    • G06F17/30129—Details of further file system functionalities
    • G06F17/3015—Redundancy elimination performed by the file system
    • G06F17/30156—De-duplication implemented within the file system, e.g. based on file segments
    • G06F17/30159—De-duplication implemented within the file system, e.g. based on file segments based on file chunks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30067—File systems; File servers
    • G06F17/30129—Details of further file system functionalities
    • G06F17/30144—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30067—File systems; File servers
    • G06F17/30091—File storage and access structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30067—File systems; File servers
    • G06F17/30182—File system types
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G06F17/30312—Storage and indexing structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601—Dedicated interfaces to storage systems
    • G06F3/0628—Dedicated interfaces to storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G06F3/0641—De-duplication techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Error detection; Error correction; Monitoring responding to the occurence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1448—Management of the data involved in backup or backup restore
    • G06F11/1453—Management of the data involved in backup or backup restore using de-duplication of the data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Error detection; Error correction; Monitoring responding to the occurence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1458—Management of the backup or restore process

Similar Documents

Publication Publication Date Title
US9268783B1 (en) Preferential selection of candidates for delta compression
US9262434B1 (en) Preferential selection of candidates for delta compression
US8972672B1 (en) Method for cleaning a delta storage system
US9405764B1 (en) Method for cleaning a delta storage system
You et al. Deep Store: An archival storage system architecture
US9727573B1 (en) Out-of core similarity matching
US9400610B1 (en) Method for cleaning a delta storage system
US10339109B2 (en) Optimizing hash table structure for digest matching in a data deduplication system
US20120303595A1 (en) Data restoration method for data de-duplication
Xia et al. Combining deduplication and delta compression to achieve low-overhead data reduction on backup datasets
US9026740B1 (en) Prefetch data needed in the near future for delta compression
US8825626B1 (en) Method and system for detecting unwanted content of files
US9183218B1 (en) Method and system to improve deduplication of structured datasets using hybrid chunking and block header removal
US10191934B2 (en) De-duplication system and method thereof
Romański et al. Anchor-driven subchunk deduplication
US20140358872A1 (en) Storage system and method for performing deduplication in conjunction with host device and storage device
US10366072B2 (en) De-duplication data bank
Bo et al. Research on chunking algorithms of data de-duplication
US8756249B1 (en) Method and apparatus for efficiently searching data in a storage system
US10838923B1 (en) Poor deduplication identification
CN106611035A (en) Retrieval algorithm for deleting repetitive data in cloud storage
Bhalerao et al. A survey: On data deduplication for efficiently utilizing cloud storage for big data backups
US9116902B1 (en) Preferential selection of candidates for delta compression
Tan et al. Exploring the potential of fast delta encoding: Marching to a higher compression ratio
US10120875B1 (en) Method and system for detecting boundaries of data blocks for deduplication