Dest

Manipulates dest.conf and files stored on remote destinations.

$ hb dest [-c backupdir] subcommand args

The dest command requires a subcommand, and each subcommand may need arguments. Here are some examples:

$ hb dest -c hbbackup clear mys3 [--force]
$ hb dest -c hbbackup show
$ hb dest -c hbbackup load

Subcommands

clear - requires a list of destinations to be cleared. This command is typically used when a destination is about to be deleted from dest.conf and you want all the files removed first. The clear command requires confirmation before files are deleted unless --force is used. If any of the destinations to be cleared have the only copy of any file, an error occurs and no destinations are cleared.

For example, if you have cache-size-limit set to 0, all of your files are stored on remote destinations and none are local. If you only have one destination setup, the clear command will not do anything because this remote copy is your only copy. But if you have 2 destinations setup, clear will be able to clear one of them.

If you have cache-size-limit set to -1 (the default), then you have a local copy of all backup files in your backup directory. If you have one destination configured, the clear command will delete files from this destination because you still have your local copy. You could then setup a new destination later and the backup command would transfer all files from the local backup directory to the new destination.

After a destination is cleared, remove it from dest.conf or the next backup command will resync all files to the cleared destination.

edit - edits the dest.conf stored in the database using the EDITOR environment variable to choose an editor, or using vi if EDITOR is not set. If there is no dest.conf loaded into the database, a new one can be created with the editor. Changes must be saved before quitting the editor.

load - loads dest.conf from the file specified into the (encrypted) database to hide remote access credentials. If no file is specified, loads dest.conf from the backup directory. After dest.conf has been loaded into the database, the dest.conf text file is not used by HB; it can be stored with your backup key to a safe place and, for security purposes, deleted from disk. Keep in mind that since the key.conf file is stored alongside the database on the local system, it is theoretically possible to obtain the remote credentials from the database unless the key has a passphrase. There is nothing to be done about this because the remote credentials are needed to access the remote systems. For the best security keep an admin-passphrase set on the database (see the config command) and use a key passphrase. Unfortunately, using a key passphrase makes automated backups difficult so there is a trade-off.

ls - display a listing of all files stored at each destination.

$ hb dest -c hbb2 ls
HashBackup #2674 Copyright 2009-2022 HashBackup, LLC
Dest |   File   |  Size |    Modified Time    |     Time Stored
-----+----------+-------+---------------------+--------------------
 b2  | arc.0.0  |  9840 | 2022-01-10 12:31:20 | 2022-01-17 12:48:08
 b2  | arc.1.0  |  9840 | 2022-01-10 12:33:49 | 2022-01-17 12:48:07
 b2  | arc.2.0  |    64 | 2022-01-10 15:55:30 | 2022-01-17 12:48:09
 b2  | hb.db.16 | 11872 | 2022-01-17 12:48:04 | 2022-01-17 12:48:06

Add -v for more details:

$ hb dest -c hbb2 ls -v
HashBackup #2674 Copyright 2009-2022 HashBackup, LLC
Dest |   File   |  Size |    Modified Time    |     Time Stored
-----+----------+-------+---------------------+--------------------
 b2  | arc.0.0  |  9840 | 2022-01-10 12:31:20 | 2022-01-17 12:48:08
Part 1: b2.sha1: 6e13c3b5e87dbc532d054b73807b94afa3a2fd7a b2.fileid: 4_z782ca905d783415977e50313_f1149a59115daac3e_d20220117_m174807_c001_v0001012_t0013

 b2  | arc.1.0  |  9840 | 2022-01-10 12:33:49 | 2022-01-17 12:48:07
Part 1: b2.sha1: 6e118e0163358b059a70208ea4ed19d97e5911a1 b2.fileid: 4_z782ca905d783415977e50313_f102cd52568251c05_d20220117_m174806_c001_v0001117_t0024

 b2  | arc.2.0  |    64 | 2022-01-10 15:55:30 | 2022-01-17 12:48:09
Part 1: b2.sha1: 68ea9c4d6c459356cfb87ed34d62ea5f965f0aa9 b2.fileid: 4_z782ca905d783415977e50313_f102cd52568251c14_d20220117_m174808_c001_v0001117_t0034

 b2  | hb.db.16 | 11872 | 2022-01-17 12:48:04 | 2022-01-17 12:48:06
Part 1: b2.sha1: b933c9b8a7ea9100e911634de7ba6c06bcbebffe b2.fileid: 4_z782ca905d783415977e50313_f102cd52568251c03_d20220117_m174805_c001_v0001117_t0050

show - display the dest.conf loaded into the database

setid - forces a destination to be tied to a backup. This is normally not necessary unless the DESTID file was manually removed from a remote destination, or a backup directory was deleted manually without clearing the remote backup area.

Be very careful using this command with production backups because it allows overwriting a remote storage area that may be in use by another backup. It’s the fastest, most reliable way to corrupt backups.

sync - synchronize destinations if necessary. This happens automatically at the start of every backup. When adding a new destination to an already large backup, syncing the new destination may take quite a bit of time. See "Syncing a New Destination" on the Backup page for a tip on how to handle a large sync so that it doesn’t interfere with regular daily backups.

test - tests a single destination or all currently configured destinations. Without options It performs 3 rounds of upload, download, and delete tests for many file sizes, displaying the performance of each and an average performance for each file size. Options allow finer control of the test:

Test all destinations:      $ hb dest -c backupdir test
Test specific destinations: $ hb dest -c backupdir test <dest1> <dest2> ...
Run specific tests:         $ hb dest -c backupdir test -t up down del
Test sizes 1K and 128M:     $ hb dest -c backupdir test -s 1k 128m
Run 10 rounds instead of 3: $ hb dest -c backupdir test -r 10
Delay 5 mins and repeat:    $ hb dest -c backupdir test -d 5m (or 5s or 5h)
Repeat test 12 times:       $ hb dest -c backupdir test -n 12

Example:

$ hb dest -c hbb2 test -s 10k
HashBackup #2674 Copyright 2009-2022 HashBackup, LLC
Using destinations in dest.conf
CPU count: 2

2022-01-18 20:16:54 --- Testing b2, type b2, 2 workers, 1 part ---

  10 KiB:
    Round 1  Up   1.4s    6.9 KiB/s   Down 105.4ms  94.9 KiB/s   Del 197.1ms  50.7 KiB/s
    Round 2  Up 565.7ms  17.7 KiB/s   Down 101.6ms  98.4 KiB/s   Del 201.2ms  49.7 KiB/s
    Round 3  Up 585.7ms  17.1 KiB/s   Down 108.5ms  92.1 KiB/s   Del 189.7ms  52.7 KiB/s
  > Average  Up 867.0ms  11.5 KiB/s   Down 105.2ms  95.1 KiB/s   Del 196.0ms  51.0 KiB/s

Test complete

unload - write the dest.conf stored in the database to the file specified, or to dest.conf in the backup directory if no file is specified, and erase it from the database.

verify - does a quick check of all remote backup files, without downloading any data. This check might be a file existence test, a file size check, or a checksum verification, depending on the capabilities of the remote destination. S3, Google Storage, and B2 implement all 3 checks, most destinations implement the exist and size checks, and some only implement the exist check. If a file does not verify on a destination and there is a copy available on another destination, the verify command will transfer it to correct the file. Verify is designed to be quick, does not read any file data, and will not detect subtle changes in files. For an exhaustive check of all remote backup data, use selftest -v4. This downloads and checks every file.