Efficient Multi-Resolution Compression Algorithm for Disk-Based Backup and Recovery

0 IntroductionTraditionally,there are two kinds of compression meth-ods developedfor tape-based backup.Oneisincrementalbackup method, which can distinguish modified files , andbackup only those modifiedfilesinthe backup set . The otherone is streamcompression, which eli minate redundancy in asingle file.In a large backup system,numerous redundant data ex-ists across files .Files with the same content may be foundinother nodes due tofile reduplication and distribution;a partlymodifiedfile may …

Efficient multi-resolution compression algorithm for disk-based backup and recovery

In this paper, we deal with the problem of improving backup and recovery performance by compressing redundancies in large disk-based backup system. We analyze some general compression algorithms; evaluate their scalability and applicability. We investigate the distribution features of the redundant data in whole system range, and propose a multi-resolution distributed compression algorithm which can discern duplicated data at granularity of file level, block level or byte level to reduce the redundancy in backup environment. In order to accelerate recovery, we propose a synthetic backup solution which stores data in a recovery-oriented way and can compose the final data in back-end backup server. Experiments show that this algorithm can greatly reduce bandwidth consumption, save storage cost, and shorten the backup and recovery time. We implement these technologies in our product, called H-info backup system, which is capable of achieving over 10x compression ratio in both network utilization and data storage during backup.