Automation
It’s important that backups are run on a regular schedule to be useful. Relying on yourself to periodically run backups is not a good strategy. It’s easy to setup automated backups so let’s get to it!
On Mac OSX systems Full Disk Access must be enabled first to grant HashBackup access to your data. Without it there will be permission errors and much of your data can’t be backed up. |
Setup Automated Backups
-
sudo sh
to get a root shell:$ sudo sh Password: #
-
as root, create a new backup directory with
hb init
, usually at the root level for a system-wide backup. This example uses/hbbackup
:$ hb init -c /hbbackup HashBackup #2677 Copyright 2009-2022 HashBackup, LLC Backup directory: /hbbackup Permissions set for owner access only Created key file /hbbackup/key.conf Key file set to read-only Setting include/exclude defaults: /hbbackup/inex.conf VERY IMPORTANT: your backup is encrypted and can only be accessed with the encryption key, stored in the file: /hbbackup/key.conf You MUST make copies of this file and store them in secure locations, separate from your computer and backup data. If your hard drive fails, you will need this key to restore your files. If you have setup remote destinations in dest.conf, that file should be copied too. Backup directory initialized
-
set the
dedup-mem
config option$ hb config -c /hbbackup dedup-mem 1g HashBackup #2677 Copyright 2009-2022 HashBackup, LLC Backup directory: /hbbackup Current config version: 0 Set dedup-mem to 1g (was 0) for future backups
-
edit
/hbbackup/inex.conf
to add files and directories to exclude -
create
/hbbackup/dest.conf
and setup remote destinations for the backup. This example uses:-
a directory destination for a USB drive named
hbusb1
mounted as/Volumes/hbusb1
-
a destination for Backblaze B2 storage service. This can be setup later if you don’t want to do it now.
-
a 2nd dir destination could be setup, hbusb2, to rotate between two USB drives, keeping one at another location
-
for SSD drives, leave out
workers 1
. It is there for spinning drives.
# NOTE: hbusb1 is mounted at /Volumes/hbusb1 mkdir /Volumes/hbusb1/hbbackup cat - >/hbbackup/dest.conf # use control d to exit destname hbusb1 type dir dir /Volumes/hbusb1/hbbackup workers 1 destname b2 type b2 accountid 0123456789ab appkey 0123456789abcdef0123456789abcdef0123456789 bucket hbbackup dir myhost1
-
-
it’s a good idea to keep a local copy of backup data in
/hbbackup
, another copy on a USB drive with aDir
destination, and a 3rd copy on a remote storage service. But if you’ve decided not to keep a full copy in/hbbackup
, setcache-size-limit
to the amount you want to keep there, or this can be done later:$ hb config -c /hbbackup cache-size-limit 50GB HashBackup #2677 Copyright 2009-2022 HashBackup, LLC Backup directory: /hbbackup Current config version: 1 Set cache-size-limit to 50GB (was -1) for future backups
-
test destinations with a small backup to make sure everything is working:
$ hb backup -c /hbbackup /sbin HashBackup #2677 Copyright 2009-2022 HashBackup, LLC Backup directory: /hbbackup Backup start: 2022-01-19 18:26:43 Using destinations in dest.conf Copied HB program to /hbbackup/hb#2677 This is backup version: 0 Dedup enabled, 0% of current size, 0% of max size / /hbbackup /hbbackup/inex.conf /sbin /sbin/apfs_hfs_convert /sbin/autodiskmount ... /sbin/rtsol /sbin/shutdown /sbin/umount Copied arc.0.0 to hbusb1 (821 KB 0s 505 MB/s) Writing hb.db.0 Copied hb.db.0 to hbusb1 (11 KB 0s 11 MB/s) Copied dest.db to hbusb1 (36 KB 0s 38 MB/s) Time: 0.2s CPU: 0.2s, 83% Mem: 75 MB Checked: 66 paths, 2435539 bytes, 2.4 MB Saved: 66 paths, 2434170 bytes, 2.4 MB Excluded: 0 Dupbytes: 44400, 44 KB, 1% Compression: 66%, 3.0:1 Efficiency: 6.00 MB reduced/cpusec Space: +821 KB, 858 KB total No errors
-
to automate nightly backups, add this root crontab entry to
/etc/crontab
, the system crontab file:cat - >>/etc/crontab # use control d to exit # min hour mday month wday user command MAILTO=me@email.com PATH=/bin:/usr/bin:/usr/local/bin 0 0 * * * root nice sh -c "cd /hbbackup && hb backup -c . / --maxtime 7h; hb retain -c . -s30d12m; hb selftest -c . -v4 --inc 1d/30d,1GB; hb upgrade; hb log -c . -e -x1" >/dev/null
OSX systems may not have an /etc/crontab
file, but if you create one, it will work. Prefix your edit command withsudo
to create the/etc/crontab
system file. -
Finally, and most importantly: make copies of your
key.conf
anddest.conf
files, to USB thumb drives, paper, phone pictures, and any other way necessary to make sure you don’t lose your backup key and remote credentials. Keep a thumb drive on your keychain, give copies to trusted relatives or friends, and make sure you have a copy of these two important files in multiple locations.Without the key your backup cannot be accessed!
What It Does
-
nice
prefix runs backups at a lower priority; remove if backups are too slow. On OSX, replace withtaskpolicy -d standard
for lower priority, ortaskpolicy -d throttle
for even lower priority -
backup root (
/
) at midnight every day. Add file systems as needed -
stop backup if not finished within 7 hours (
--maxtime 7h
) -
log all output to
/hbbackup/logs
with timestamps for every line -
retain 30 daily backups + one monthly backup for the last 12 months (
-s30d12m
) -
run selftest to check the backup
-
selftest downloads and verify all backup data (
-v4
) over 30 days (1d/30d
). Downloads are limited to1GB
, so it may take longer than 30 days to complete a cycle if your backup is larger than 30GB.selftest -v4 can expensive for large backups because of download fees (egress data transfer). Adjust --inc
accordingly, add--sample
to sample your backup, or use thedest verify
command instead ofselftest
for faster & cheaper remote verification — but not as thorough. -
check and install HashBackup upgrades. Automated upgrades ensure that you have the latest updates to the HashBackup program and your backup database stays up to date. Upgrade happens after the backup instead of before so that requests to the HashBackup upgrade server are spread out. Everyone’s backup time varies, so upgrade checks follow a somewhat random schedule. Cron sends an email with the release notes when a new version is installed.
-
summarize and archive the log files, sending an email only if errors occur (
-e
) with 1 line of context around the error (-x1
)
Adjustments
-
email address to receive release notes and error summary
-
start time in the first two fields, minute first, then hour. Avoid starting between 1-2AM because those times can occur twice in one day under Daylight Savings Time.
-
cd /hbbackup
backup directory name -
paths backed up. This example backs up everything (
/
) but does not cross mount points: other filesystems have to be specifically listed -
retention policy of older file versions (
-s30d12m
) -
how much time to verify the entire backup (
1d/30d
means 1/30th of the backup each day) -
limit on how much data to download from each destination (
1GB
); this has priority over the previous time limit -
see performance tuning tips if backing up makes your computer too slow
More Frequent Backups
For more frequent backups, the backup command and maintenance commands
can be separated into two jobs in /etc/crontab
. In a crontab, every
line is a job. In the first five time fields, a * means "every
interval", so a * on the hour field means "every hour"; on the daily
field it means "every day".
This example crontab adds an hourly backup of /Users/jim
at
the top of every hour (the first field, minute, is zero) between the
hours of 6AM and 11PM (6-23
).
cat - >>/etc/crontab # use control d to exit
# min hour mday month wday user command
MAILTO = me@email.com
0 6-23 * * * root nice /usr/local/bin/hb backup -c /hbbackup /Users/jim >/dev/null
For an online crontab time editor, check https://crontab.guru
Backups Don’t Run?
If your automated backup job does not run:
-
There may be a mistake in the cron command. To test this,
sudo sh
to become root, then copy and paste the cron command to see if it works or causes an error. -
Your computer may be sleeping — very common with laptops. On OSX, go to System Preferences → Energy Saver → Power Adapter and check the box for "Prevent computer from sleeping automatically when the display is off". Also, don’t close the lid! Closing the lid will make your laptop sleep and cron jobs do not run while sleeping.
-
On OSX, within Energy Saver, click Schedule and set a wake-up one minute before your cron job is scheduled to start.