Reducing storage requirements of snapshot backups based on rsync utility

Abstract

Backing up data is a necessary requirement typically enforced by government too for its redundancy and availability. Disks have already emerged as one of the front runners for fast and efficient backup, given the way storage cost has reduced with the advent of high capacity SATA drives. rsync has established itself as a reliable backup tool in small and medium scale business units. This paper discusses techniques on top of the rsync algorithm which can increase the storage capacity significantly by eliminating duplicate data in a file and avoiding duplication of information in storing file system backed up data. The technique uses the signatures generated by rsync to eliminate duplicate data in the file across backups thus helping in lesser storage costs. The technique has the potential to be extended across the entire data blocks of the file system for better efficiencies and can be made as the cheapest de-dup available in the field. The storage savings are very high for loads which do less percentage of overwrites and higher percentage of append writes.