Backup

-c the backup is stored in the local backup directory specified with the -c option or a default backup directory according to built-in rules if -c isn’t used. If a full local copy of the backup is not wanted in the backup directory, set cache-size-limit with the config command and create a dest.conf file in the backup directory. See Destinations for details and examples.

-D enable dedup, saving only 1 copy of identical data blocks. -D is followed by the amount of memory to dedicate to the dedup operation, for example: -D1g would use 1 gigabyte of RAM, -D100m would use 100mb. The default is -D100M. The more memory, the more duplicate data HashBackup can detect, though it is still effective even with limited memory. Dedup is also enabled with the dedup-mem config option. See Dedup for detailed information and recommendations. -D0 disables dedup for one backup. Unlike most options, -D uses a factor of 1024, so -D100M means 100 * 1024 * 1024 bytes.

-v control how much output is displayed, higher numbers for more:

-v0 = print version, backup directory, copy messages, error count
-v1 = include names of skipped filesystems
-v2 = include names of files backed up (this is the default)
-v3 = include names of excluded files

-X cross mount points (separate file systems). Unless -X is used, each filesystem to backup must be explicitly listed on the command line . Use the df command to see a list of mounted filesystems. To see what filesystem a directory is on, use df followed by a pathname, or use df . to see the current directory’s filesystem.

Each backup creates a new version in the backup directory containing all files modified since the previous backup. An easy way to think of this is a series of incremental backups, stacked one on top of the other. HashBackup presents the illusion that every version is a full backup, while performing the backups at incremental speeds and using incremental backup storage space and bandwidth. While full backups are an option, there is no requirement to do them. HashBackup is designed to not need full backups, yielding huge savings of time, disk space, and bandwidth compared to traditional backup methods. The retain command explains how to maintain weekly or monthly snapshots - or both!

All commands accept a -c option to specify the backup directory. HashBackup will store backup data and the encryption key in this directory. If -c is not specified on the command line, the environment variable HASHBACKUP_DIR is checked. If this environment variable doesn’t exist, the directory ~/hashbackup is used if it exists. If ~/hashbackup doesn’t exist, /var/hashbackup is used. Backup directories must be first be initialized with the hb init command. If a complete local copy of your backup is not wanted, set the cache-size-limit config option with the config command to limit the size of the local backup directory, then setup a dest.conf file to send backup data to a destination.

HashBackup saves all file properties in a database, so backups can be stored on any type of "dumb" storage: FAT, VFAT, CIFS/SMB/Samba, NFS, USB thumb drives, SSHFS, WebDAV, FTP servers, etc. All file attributes, including hard links, ACLs, and extended attributes will be accurately saved and restored, even if not supported by the storage system. An encrypted incremental backup of the backup database is automatically included with every backup.

It’s possible to backup directly to a mounted remote filesystem or USB drive with the -c option. However, the backup performance may be slower than with a local filesystem, and these may not provide robust sync and lock facilities. Sync facilities are used to ensure that the backup database doesn’t get corrupted if the computer halts unexpectedly during the backup process, for example, if there is a power outage. Lock facilities ensure that two backups don’t run at the same time, possibly overwriting each others' backup files. It’s best to specify a local backup directory with -c and let HashBackup copy the backup data to mounted remote storage or USB drive by using a Dir destination in the dest.conf file in the backup directory.

Remote destinations are optional. They are setup in the dest.conf file, located in the backup directory. See the Destinations page for more detailed info on how to configure destinations for SSH, RSYNC, and FTP servers, Amazon S3, Google Storage, Backblaze, Gmail, IMAP/emailservers, and simple directory (dir) destinations. Dir destinations are used to copy the backup to USB thumb drives, NFS, WebDAV, Samba, and any other storage that is mounted as a filesystem.

The backup command creates backup archive files in the backup directory. After arc files fill up, they can be transmitted to one or more remote destinations, configured with a dest.conf file. Transmissions run in parallel with the backup process. After the backup has completed, it waits for all files to finish transmitting. By default, archives are kept in the local backup directory after sending to remote destinations, making restores very fast because data does not have to be downloaded from a remote site. Keeping a local copy also adds redundancy and makes it easier to pack remote archives (remove deleted or expired backup data) because archives don’t have to be downloaded first. It is possible to delete archives from the local backup directory as they are sent to destinations by using the cache-size-limit config option.

If storing the backup directly to a USB or other removable drive with the -c option, and that’s the only backup copy you want, there is no need to use the dest.conf file. dest.conf is only used to copy your backup, usually to a remote site.

Important Security Note: the backup encryption key is stored in the file key.conf in the backup directory. Usually this is a local directory, but when writing a backup directly to mounted remote storage with the -c option, for example, Google Drive, DropBox, etc., or to a removable USB drive, be sure to set a passphrase when initializing the backup directory. This is done with the -p ask option to hb init or hb rekey. Both the passphrase and key are required to access the backup. When writing directly to mounted remote storage that you control, such as an NFS server, a passphrase may not be necessary. If a dest.conf Dir destination is used to copy the backup to remote storage or a removable drive and the backup directory and key are on a local drive, a passphrase is not necessary because the key is never copied to destinations.

An inex.conf file is created by the init command in the backup directory and automatically excludes a handful of system files and directories, like swapfiles, /tmp, core files, /proc, /sys, and hibernation files. The backup directory itself is also automatically excluded.

To exclude other files, edit the inex.conf file in the backup directory. The format of this file is:

Example inex.conf:

Any abbreviation of the exclude keyword at the beginning of the line is recognized. Tilde characters ~ are not expanded into user directories.

There are several other ways to exclude files from the backup:

Web browser and other caches are difficult to backup for a number of reasons (see Browser Caches note), and should always be excluded from backups, especially on large servers that store caches for many users.

The backup command has more options that may be useful:

-B blocksize the backup command splits files into blocks of data. It chooses a "good" block size based on the type of file being saved and by default uses variable-sized blocks. Variable block sizes work well for files that change in unpredictable ways, like a text file. For files that change in fixed-size units - raw database files (not text dumps!), VM images, etc. - forcing a specific, fixed block size works better. Fixed-size blocks can be any size from 128 bytes to 2GB-1 (2,147,483,647) bytes.

-B forces a fixed block size, which usually dedups less than a variable block size except for cases where a file is updated in fixed chunks; then it is usually more efficient than variable blocks. Larger block sizes are more efficient in general but dedup less; small block sizes cause more overhead but dedup better. Sometimes the extra dedup savings of a small block size is less than the extra metadata needed to track more blocks. Since this is very data dependent, experiment with actual data to determine the best block size to use.

-B1m usually speeds up backups on a single CPU system because it disables variable block sizes. -B4M or higher is useful when files do not have much common data, for example media files. For identical files, backup will only save 1 copy. A huge block size like 64M may be useful for large scientific data files that have little duplicate data. A large block size does not make small files use more backup space. The simulated-backup config option with different block sizes is helpful for determining the best block size. Unlike most options, -B uses a multiplier of 1024, so -B4K means 4096 bytes rather than 4000 bytes.

IMPORTANT: very large block sizes will use a lot of RAM. A good estimate is 8x block size with a couple of CPUs, or more if there are more CPUs. -p0 disables multi-threading and will use less RAM and CPU but will also be slower.

-F filesources read pathnames to be saved from file sources. More than one file source can be used. This is often used with very large file systems to avoid having to scan for modified files by having the processes that modify files update a file list. Each file source is a:

The backup command line can still have pathnames to backup in addition to -F, but these must come before -F.

--full the backup command usually backs up only modified files and determines this by looking at mtime, ctime, and other file attributes for each file. This option overrides that behavior and causes every file to be saved, even if it has not changed. Taking a full backup adds redundancy to the backup data as it will not dedup against previous backups. Another way to accomplish this is to create a new backup directory on both local and remote storage, but this is harder to manage and retain will not work across multiple backup directories. Full backups also limit the number of incremental archives required to do large restores, though in testing, restoring a complete OSX system from 3 years of daily incrementals took only 15% longer than restoring from a full backup.

-m maxfilesize skip files larger than maxfilesize. The size can be specified as 100m or 100mb for 100 megabytes, 3g or 3gb for 3 gigabytes, etc. This limit does not apply to fifos or devices since they are always zero size.

--maxtime time specifies the maximum time to spend saving files. When this time is exceeded, the backup stops and waits for uploads to finish using --maxwait, which is adjusted based on how long the backup took.

--maxwait time when used by itself, specifies the maximum time to wait for files to be transmitted to remote destinations after the backup has finished. When used with --maxtime, the wait time is increased if the backup finishes with time remaining. The special value off disables all destinations for this one backup. This is useful for example to avoid sending hourly backups offsite and letting the nightly backup send them.

These two options have several uses:

If a backup does not complete because of --maxtime, the next backup using the same command line will restart where the previous backup left off, without rescanning the entire filesystem. This allows backups of huge filesystems with tens of millions of files over a period of time.

Time limits may be exceeded for various technical reasons. Large arc files, slow upload rates, and many worker threads cause more variability in the timing. It’s a good idea to initially set the time limits an hour less than required to get a feel for how much variability occurs in a specific backup environment.

Examples for --maxtime and --maxwait

--maxtime 1h means backup for up to 1 hour then wait the remainder of the hour for uploads to finish, ie, total time is limited to 1 hour.

--maxwait 1h means backup everything requested, however long it takes, but only wait 1 hour after the backup for uploads to finish.

--maxtime 1h --maxwait 2h means backup for up to 1 hour, then wait 2 hours + the remaining backup time for uploads to finish, ie, the total time is limited to 3 hours.

--maxtime 1h --maxwait 1y means backup for 1 hour, then wait a year for uploads to finish, ie, only the backup time is limited.

--no-ino on Unix filesystems, every file has a unique inode number. If two paths have the same inode number, they are hard linked and both paths refer to the same file data. Some filesystems do not have real inode numbers: FUSE (sshfs, UnRaid) and Samba/SMB/CIFS for example. Instead, they invent inode numbers and cache them for a while, sometimes for many days. HashBackup normally verifies a path’s inode number during backups, and if it changes, it triggers a backup of the path. Inode numbers are also used to detect hard links. If a filesystem does not have real inode numbers, it causes unexpected and unpredictable full backups of files that haven’t changed and after enough of these mistakes, HashBackup will stop with an error. The --no-ino option prevents these unnecessary full backups.

A negative side-effect of --no-ino is that hard-link detection is disabled since it relies on stable inode numbers. With --no-ino, hard-linked files are backed up like regular files and if restored, are not hard-linked.

--no-mtime there are two timestamps on every Unix file: mtime is the last time a file’s data was modified, and ctime is the last time a file’s attributes were modified (permissions, etc). When a file’s data is changed, both mtime and ctime are updated. If only the attributes are changed, only ctime is updated. Because mtime can be set by user programs, some administrators may not trust it to indicate whether a file’s data has changed. The --no-mtime option tells the backup command to verify whether file contents have changed by computing a strong checksum and comparing that with the previous backup rather than trusting mtime. This is not recommended for normal use since the backup program has to read every unmodified file’s data to compute the checksum, then possibly read it again to backup the file.

-p procs specifies how many additional processes to use for the backup. Normally HashBackup uses 1 process on single-CPU systems, and several processes on multi-core systems with more than 1 CPU. Multiple processes speed up the backup but also put a higher load on your system. To reduce the performance impact of a backup on your system, you may want to use -p0 to force using only 1 CPU. Or, with a very fast hard drive and many CPU cores, you may want to use more than just a few processes. On a dual-core system, HashBackup will use both cores; on a 4-core or more system it will use 4. To use more than 4 cores, -p must be used. Backup also displays %CPU utilization at the end of the backup. If %CPU is 170%, it means that on average, HashBackup could only make use of 1.7 CPUs because of disk I/O, network, or other factors, so -p2 is fine. Experimenting is the best guide here. A negative value means to reserve (not use) cores, so -p-2 means to use all but 2 cores.

--sample p specifies a percentage from 1 to 100 of files to sample for inclusion in the backup. This is a percentage of the files seen, not a percentage of the amount of data in the files. For example, --sample 10 means to only backup 10% of the files seen. This can be useful with simulated backups, especially with huge filesystems, to try different backup options. Samples are not truly random, but are selected so that if 5% of files are backed up first, then 10%, the new sample includes the same files as the first 5% plus an additional 5%. This allows simulating incremental backups. All pathnames are sampled, whether they come from the command line or -F file sources.

-t tagname tag the backup with information-only text shown with hb versions

-V backup splits files into variable-sized blocks unless the -B option or block-size-ext config options override this. The default variable block size is automatically chosen when config option block-size is set to auto, the default. -V overrides the variable block size for a single backup. Values can be 16K 32K 64K 128K 256K 512K or 1M. During backup of a file the block size ranges from half to 4x this block size, with the average block size being about 1.5 times larger than specifed. Larger block sizes dedup less but have less block tracking overhead in the backup database, leading to faster restore planning, faster rm and retain operation, and a smaller backup database. Larger blocks also compress better. If most of the files being saved are on the large side, a larger block size is recommended. If the block size is changed on an existing backup, changed files will be saved with the new block size and will not dedup against previously saved versions because they have a different block size. If a file continues to change, it will dedup against the versions saved with the same block size.

-X backup saves all paths listed on the command line, even if they are on different filesystems. But while saving / (root) for example, backup will not descend into other mounted filesystems unless -X is used. Be careful: it’s easy to backup an NFS server or external drive unintentionally with -X.

-Z level sets the compression level. The compression level is 0 to 6, where 0 means no compression, 6 means highest compression, and 7-9 are reserved. The default is 6, the highest compression, so use -Z for slightly faster but less compression. Compression can be disabled by file extension with the no-compress-ext config variable. Disabling compression with -Z0 usually increases backup space, disk writes, and backup transmission time, and is almost always slower than -Z1.

If a device name is used on the backup command line, all data stored on the device will be stored. The mtime and ctime fields cannot be used to detect changes in a block device, so the whole device is read on every backup, though only modified data is saved if dedup is enabled. The block device should either be unmounted or mounted read-only before taking the backup, or, with logical volumes, a snapshot can be taken and then the snapshot backed up. To maximize dedup, a block size of 4K is good for most block device backups since this is the most common filesystem allocation unit. For very large devices, a block size of 64K or 1M will run faster, but may yield a bigger backup with less dedup. Experiment with your data to decide. When saving multiple block devices to the same backup directory, they will only dedup well against each other if the backup block size matches the filesystem allocation unit size - usually 4K. This is similar to the situation with VM image backups; read below for more information.

On Linux, logical volumes are given symbolic links that point to the raw device. When a symbolic link is listed on the backup command line, its target (the raw device) will also be saved.

If a named pipe or fifo is used on the backup command line, the fifo is opened and all data read from it is backed up. For example, this can be used for database dumps: instead of dumping to a huge text file, then backing up the text file, you can direct the dump output to a fifo and let HB back up the dump directly, without needing the intermediate text file. The commands to do a fifo backup are:

Be very careful not to start more than one process writing to the same fifo! If this happens, the data from the two processes is intermixed at random and HB cannot tell this is happening. The resulting backup is unusable.

On Mac OSX and BSD, fifo backups are about half the speed of backups from a regular file because the fifo buffers are small. With Linux, fifo backups can be faster than regular file backups.

It is often difficult to decide the best way to back up a specific set of data. There may be millions of small files, very large files, VM disk images, photos, large mail directories, large databases, or many other types of data. Different block sizes (-B option), arc file sizes (arc-size-limit config keyword), dedup table sizes (-D keyword), and arc file packing (pack-…​ config keywords) will all affect the size of the backup. The retention policy will also be a factor. Determining all of these variables can be difficult.

To make configuring backups easier, HB has simulated backups: the backup works as usual, but no backup data is written. The metadata about the backup - file names, blocks, arc files, etc. - is still all tracked. You can do daily incremental backups, remove files from the backup, and perform file retention, just like a normal backup, while using very little disk space. The hb stats command is then used to see how much space the backup would use. It’s easy to try one configuration for a week or two, put that aside, try a different configuration for a while, and then compare the hb stats output for the two backups to see which works better.

To create a simulated backup, use:

before the first backup. Then use normal HB commands as if this were a real backup and use the stats command to see the results. Once this option has been set and a backup created with it, it cannot be removed. It also cannot be set on an existing backup.

Some commands like get and mount will fail with a simulated backup since there is no real backup data. A useful option for very large simulated backups is --sample, to select only a portion of a filesystem for the simulation.

A sparse file is a somewhat unusual file where disk space for long runs of zeroes is not fully allocated. These are often used for virtual disk images and sometimes called "thin provisioning". An ls -ls command shows a number on the left, the space actually allocated to the file in 512-byte blocks, and the size of the file in bytes, which may be much larger. When supported by the OS, HB will backup a sparse file efficiently by skipping the unallocated file space, also called "holes", rather than backing up a long string of zero bytes. If the OS or filesystem does not support sparse file hole skipping, the file will be saved normally. In either case, a sparse file is always restored as a sparse file with holes. Any -B block size can be used (or none, and HB will pick a block size).

Sparse files are supported on Linux and BSD; Mac OS does not support sparse files. Here is an example of creating a 10GB sparse file, backing it up, and restoring it:

There are 2 ways to backup a VM: run HashBackup inside the VM as an ordinary backup, or run HashBackup from the VM host machine. Running HB inside the VM is just like any other backup and doesn’t have special considerations. VM images can be backed up while running HashBackup on the VM host, ie, outside the VM. What you are saving is a large file or collection of large files, often sparse, that together are the data for the virtual machine. This has 2 special considerations: consistent backups, and choosing a block size.

Consistent VM Image Backups

If a VM image is backed up while the VM is running, the disk image files being saved may change during the backup. For non-critical VMs that don’t have a lot of activity, this isn’t a big deal and is probably fine. If you restore a VM image saved this way, it is strongly recommended to run a forced fsck if you restore it, to ensure your filesystem is okay.

To get a consistent VM backup, you can: a) suspend the VM during the backup; b) take a snapshot with the VM software, or c) take a snapshot on the VM host filesystem, for example, an LVM snapshot, and backup the snapshot. With method b), you would need to revert to the snapshot after restoring the VM. Using method b), VM snapshots, has the additional advantage that you can run with this snapshot active for a week or two, and backups will be much faster because only the snapshot has to be scanned; the main VM image file will not change. Every week or so the snapshot(s) can be deleted, reincorporating them back into the main VM image, and then the entire VM image will be scanned on the next backup.

Block Size for VM Image Backups

HashBackup normally uses a variable block size that depends on the data being saved. For files that are updated in fixed-size blocks, like VM images, a fixed block size usually gives a smaller backups and faster backup performance.

Smaller block sizes yield higher dedup, though also cause more overhead because HB has to track more blocks. If HashBackup recognizes the file suffix for a VM image, for example, .vmdk, .qcow, etc., it will switch to a 4K block size. For higher performance, especially with very large VM images, a larger block size like -B1M could be used. For a "middle of the road" strategy, use -B64K.

Important note: if you expect to dedup across several VM images saved into the same backup directory, the most effective way to do that is with -B4K. The reason is that different VMs will likely have different filesystem layouts, even if they have the same contents. Since most filesystems allocate space in 4K chunks, the best way to dedup across VMs is to use the same 4K block size. ` `

When choosing a block size, here are some things to keep in mind:

Backup of CentOS /usr directory running in a Mac OSX VM. First create the backup directory:

Drop the filesystem cache to time the initial (full) backup. This backup is slower than normal because this VM is only configured for 1 core.

Drop the filesystem cache again to time an incremental backup:

Backup Intel i7 Mac Mini Server root drive (643,584 files in 89GB) to the other drive. Both drives are slower 5600 rpm hard drives. During this backup, HashBackup used 267MB of RAM,

Restore the backup - 28 minutes.

This is the daily backup of the HashBackup development server running on a 2010 Mac Mini (Intel Core 2 Duo 2.66GHz) with 2 SSD drives. There are 12 virtual machines hosted on this server so it averages about 80% idle. The backup of the main drive is written to the other SSD, copied to an external USB 2.0 drive, and copied to Backblaze B2 over a very slow 1Mbit/s (128KB/s) Internet connection.

There are many VM disk images stored on this server, and a tiny change in these disk images means the whole disk image has to be backed up. In this backup, out of 79GB total, files totalling 23GB were changed. But because the amount of actual data changed was small, HashBackup’s dedup feature compressed a 23GB backup down to 9.5MB of new backup data. The backup was also copied to a USB drive for redundancy. A full-drive restore from this backup is shown on the Get page.

First backup a 2GB Ubuntu test VM image with 64K block size. This is on a 2010 Macbook Pro, Core 2 Duo 2.5GHz, SSD system. The peak HB memory usage during this backup was 77MB. The purge command clears all cached disk data from memory.

Now touch the VM image to force an incremental backup. No data actually changed, but HashBackup still has to scan the file for changes. Peak memory usage during this backup was 77MB.

Try the same backup and incremental, this time with a variable block size. This gives 2x more dedup and a smaller backup, but 5% higher runtime. Peak memory usage again was around 78MB.

Now an incremental backup with variable block size. This shows that for VM images, variable-block incremental backup is slower than fixed-block backup because data has to be scanned byte-by-byte.

Let’s compare how fast dd can read the test VM image vs HashBackup, both with a block size of 64K. The results below show dd reads at 233 MB/sec on this system, HashBackup reads and compares to the previous backup at 133 MB/sec (if no data changes). The difference between the full backup time where all data is saved, and the incremental time where no data is saved, gives an idea of the time HashBackup needs to backup changed data. Based on this, HashBackup can backup 1% of changed data for this file in about .217 seconds, or 2.17 seconds if 10% of data changed. This is added to the incremental time with no changes to estimate the backup time with changed data. You can use similar formulas to estimate backup times of large VM files.

IMPORTANT: these tests are with a 2010 Intel 2.5GHz Core2Duo CPU. More recent CPUs will have faster backup times.

Do an incremental backup with 10% of the 2GB VM image data changed at random, to verify the expected runtime of 21.8 seconds for 10% changed data:

NetApp Filers have a built-in snapshot capability. Snapshots can be taken automatically with generated pathnames, or manually with a specified pathname. An example of an automatic name would be /nfs/snapshot.0, /nfs/snapshot.1, etc., with the higher number being the most recent snapshot. Saving the highest snapshot is a problem for HashBackup because the pathname changes on every backup, causing unreasonable metadata growth in the hb.db database file.

To make efficient backups, use a bind mount to make the snapshot directory appear at a fixed name:

$ mount --bind /nfs/snapshot.N /nfstree (Linux)

Then backup /nfstree with HashBackup.

Dirvish is an open-source backup tool that uses rsync to create hard-linked backup trees. Each tree is a complete snapshot of a backup source, but unchanged files are actually hard links. This saves disk space since file data is not replicated in each tree. The problem is that these trees often include a timestamp in the pathname. If the trees are backed up directly, every pathname is unique. This causes unreasonable metadata growth in the hb.db file, that leads to excessive RAM usage.

For more efficient backups, mount the tree directory (the directory containing user files) to a fixed pathname with a Linux bind mount. See the NetApp section above for details.

A second issue with hard-link backups is that HashBackup maintains an in-memory data structure for hard link inodes so that it can properly relate linked files. This data structure is not very memory efficient and uses ~165 bytes for each inode that is a hard link. This can be disabled with the --no-ino option, though that disables all hard-link processing.

Rsnapshot is an rsync-based backup tool that creates hard-linked backup trees. A big difference is that with rsnapshot, the most recent backup is contained in the .0 directory, ie, the most recent backup is at a fixed mount point. HashBackup can be used to backup this .0 directory every night. You should avoid backing up the other rsnapshot directories. If you need to save the other directories:

Since rsnapshot backups are hard-link trees, nearly every file is a hard link and you may want to consider using --no-ino to lower memory usage. This disables all hard-link processing in HashBackup.

HashBackup is designed as a backup solution, where a collection of files are saved on a rather frequent basis: daily, monthly, etc. An archive is a special kind of backup that is more of a one-time event, for example, saving a hard drive or server that is going to be decomissioned. It could be 5 years (or never!) before the archive is accessed again.

HashBackup uses a database to store information about a backup. The format of this database is periodically changed as new features are added, and any structural changes are automatically applied as necessary when a new release of HB accesses a backup. HB can automatically update any backup as long as it is accessed within the last year or so with the latest version. If it has been more than a year, the latest version of HB may or may not be able to automatically update a backup’s database; an older version may need to be used first to partially upgrade the database, then a later version used to fully upgrade the database.

One easy way to keep an archive’s database up to date is to create a cron job to backup a very small file every month. This file doesn’t have to be related in any way to the original archive - a dummy file in /tmp is fine. Also include an hb upgrade in the cron job. Together these will ensure that your copy of the HB program stays up-to-date and that your archive’s database is automatically upgraded as necessary.

Another way to handle archives is to always run the version of HB contained within the backup directory.

When a new destination is added to dest.conf, HashBackup will automatically sync existing backup data to the new destination. This can be somewhat challenging if it will take days or weeks to get the new destination in sync, either because upload bandwidth is limited or the backup is already very large. An easy way to handle this is to use the --maxwait and/or --maxtime options to limit the amount of time backup spends trying to get your new destination synced. For example, if backup normally takes an hour to run and starts at midnight, --maxtime 7h will stop the backup around 7am. The next day backup will continue syncing data to the new destination.

Another option is to allow backup to run most of the day. To avoid interfering with normal network activity, use workers 1 in dest.conf so only one file is uploaded at a time, and use the rate keyword to limit upload bandwidth used. Once your new destination is in sync, these restrictions can be removed since incremental backups are usually small.