Shell

The shell destination type enables the use of external storage not natively supported by HashBackup, where sending, getting, and removing files can be done via a Unix command or shell script. Shell destinations can be written in any language.

As an example, dirshell.py is a simple shell implementation of the built-in dir destination. For a much more complex shell example, see the Rclone destination.

Shell destinations report success or failure with their exit code. If a non-zero exit code is returned (failure), the command is retried with exponential backoff like all other destinations.

To make parsing easier, consider adding a --command option to the end of the run keyword value. The command line is processed by the regular Unix shell, so things like $VAR, shell functions, and quoting work fine.

HB adds a command and arguments to the end of the run command line. Below are the commands HB sends and the shell destination’s responsibilities.

noop

Exit with status 0 if the destination is available. This is used once per worker during startup.

send pathname filename

Send the local file pathname to the remote destination and store it as filename. The file may already exist on the remote, and HB expects send to overwrite the existing file. This happens for example with DESTID and dest.db. The file size may stay the same (always with DESTID), so don’t use file size as an indication of whether to copy a file; the file should be unconditionally copied.

Some protocols like rsync automatically prevent partial files on the remote by copying first into a temp file then renaming the temp file to the real filename. If your remote protocol doesn’t do this automatically (ftp doesn’t for example), it’s a good idea to send the file as filename.tmp, then rename filename.tmp to filename, overwriting filename if it already exists. If the remote cannot rename over an existing file, you can send as filename.tmp, delete filename, then rename filename.tmp to filename. This runs the slight risk of a failure after the delete, which will be corrected automatically on the next backup by resending the file.

get pathname filename

Fetch the remote file filename and store it in the local file pathname. Pathname will not exist on the first get attempt, but may exist on a retry. This can be used to do a restart, or pathname can be deleted or completely overwritten to start over.

rm filename

Remove the remote file filename. If it doesn’t exist, no error should occur. If the send command creates filename.tmp then renames it, both filename and filename.tmp should be removed on the remote.

Example