Preview

The preview release is installed with hb upgrade -p. Using preview releases for non-critical backups helps find and solve problems early. The official release is sent to the preview area so hb upgrade -p gets the official release when it’s available. Thank you for your help!

Preview Notes

  • Sep 9 #3050: official release

  • Sep 9 #3049: get: remove experimental inex exclusion

  • Sep 7 #3048: Bugz: strip pathnames if background thread fails

  • Sep 5 #3047: mount: revert back to #3040; couldn’t fix FreeBSD FUSE

  • Sep 5 #3047: get: traceback during restore

  • Sep 5 #3046: backup, get, compare: round mtime more aggressively

  • Aug 31 #3042: mount: adjust for FreeBSD 12+

  • Aug 25 #3040: get: honor inex.conf excludes for restore

  • Aug 17 #3039: official release: no attribute 'SIGINFO' on Linux

#3050 Sep 9, 2022

Summary:

  • get: traceback during restore

  • Bugz: strip pathnames from reports if background thread fails

  • backup, get, mount: round mtime more aggressively

Details:

  • get: certain cirumstances with cache-size-limit >= 0 and selective download could cause a race condition, leading to a traceback and restore failure. The traceback is below. Thanks Ian!

    dest d1: Traceback (most recent call last):
      File "/basedest.py", line 438, in loop
      File "/basedest.py", line 579, in getrangecmd
      File "/basedest.py", line 660, in getspansfilename
    OSError: [Errno 2] No such file or directory
  • Bugz: if a background thread failed, as opposed to the main thread, the automatic bug report could contain pathnames from the backup because background threads don’t have access to the database to allow mapping pathnames to numbers as is normally done. Now, for privacy in this situation, any log line containing a slash is truncated and <path> is added. Thanks Ian!

  • backup, get, mount: in #2829, file modified times were rounded before comparisons to compensate for floating-point inaccuracies. This rounding is more aggressive now since there were still some incorrect comparisons.

History

#3039

  • Aug 17 #3039: official release: no attribute 'SIGINFO' on Linux

#3038

  • Aug 14 #3038: official release

  • Aug 7 #3037: backup: Exception: Bug: varblock in mixed threads

#3036

  • Jul 29 #3036: official release

  • Jul 24 #3035: log: forward siginfo signal

  • Jul 21 #3033: recover: arc file download rename race condition

  • Jul 14 #3032: official release

#3032 Jul 14, 2022

  • Jul 14 #3032: official release

  • Jul 14 #3032: db upgrade: remove .uporig if dbrev is correct, check disk space before upgrade

  • Jul 14 #3031: dest needput: revise conditions

  • Jul 8 #3030: selftest: enable dev testing of -v0 and -v1

  • Jul 4 #3029: remove leading slash from inline tracebacks

  • Jul 3 #3028: db upgrade: remove cached arc files to make space (held)

  • Jul 3 #3027: backup: reorder mtime tests

  • Jul 2 #3026: backup: cache was not always getting cleaned up

  • Jul 2 #3025: backup, get: handle int resolutions on mtimes

  • Jul 1 #3024: dest noparts: skip files with null info

  • Jun 30 #3023: selftest: only fix dedup table with --fix

  • Jun 27 #3022: get: rtrim → rstrip. Thanks Bugz!

  • Jun 27 #3021: recover: no such table: destfiles. Thanks Bugz!

  • Jun 25 #3020: dest.conf: better handling of keywords with missing values

  • Jun 25 #3019: dest: re-wrap verify help to match other subcommands

  • Jun 22 #3018: log: handle no such process in sig handler. Thanks Bugz!

  • Jun 22 #3017: handle -r with missing/empty versions better. Thanks Bugz!

  • Jun 21 #3016: backup: no partial backup warning for char devices

  • Jun 18 #3015: official release

#3015 Jun 18, 2022

  • Jun 18 #3015: official release

  • Jun 16 #3012: get: traceback restoring deleted files

  • Jun 16 #3011: backup: distinguish intterupts from timeouts

  • Jun 12 #3010: send ctrl-c and sigterm to commands

  • Jun 12 #3009: dest nopart: exit on ctrl-c and kill -term

  • Jun 7 #3007: destinations: ending support for maxsize

  • Jun 6 #3006: cleanup mount change in #3005

  • Jun 5 #3005: mount: reading variable-block files could crash mount

  • Jun 3 #3004: backup, mount: minor message changes

  • Jun 2 #3003: destcmd: don’t require setid after dest rename. Thanks Preston!

  • Jun 2 #3002: mount: remove mount timeout for slow destinations

  • Jun 2 #3001: mount: use mkdirp to create --cache directory. Thanks Bugz!

  • May 31 #2999: mount: remove a debug stack trace

  • May 31 #2998: recover: create testonly.conf file

  • May 30 #2997: compress inline traceback a bit more

  • May 30 #2996: dest.conf: integer keyword defaults

  • May 30 #2995: auto-run selftest in several more situations

  • May 30 #2994: mount improvements. Thanks Preston!

  • May 25 #2993: clear: no "if you continue" with --force

  • May 25 #2992: remove Rackspace Cloudfiles remnants

  • May 25 #2991: log: traceback if program completes during summarizing. Thanks Bugz!

  • May 24 #2990: security disclosure from 2022-02-21 published

  • May 24 #2990: handle rotating USB drives better

  • May 23 #2989: get: restore pathnames can start with ../

  • May 22 #2988: selftest, rm: change cache initialization

  • May 19 #2985: dir dest: was requiring too much free disk space

  • May 19 #2984: generalize reading passphrases

  • May 19 #2983: selftest: --fix corrects a few more errors

  • May 18 #2982: trim: flush stderr so 1st pathname isn’t cleared

  • May 18 #2981: trim: traceback "getting called too often"

  • May 18 #2979: log: prevent status line escape sequences in logs

  • May 17 #2978: dest rename: new command

#2976 May 17, 2022

  • May 17 #2976: official release

  • May 17 #2975: disabled destinations being selected

  • May 16 #2973: recover: use multiple destinations if necessary

  • May 15 #2970: clear progress reports on stderr write

  • May 9 #2969: versions: File Space was sometimes incorrectly blank

  • May 8 #2968: backup: traceback when rebuilding dedup table

  • May 7 #2967: selftest: upload hb.db if it was changed

  • May 7 #2966: ls: show date deleted

  • May 7 #2965: retain: use new status line code

  • May 6 #2964: db upgrade: error vs traceback for permission failures

  • May 5 #2963: hb: auto selftest may need to run 4 times, not 3

  • May 4 #2962: get: handle some fringe error situations

  • May 2 #2958: dest: remove sync after verify is complete

  • May 1 #2957: recover: show better stats during download

  • May 1 #2956: upgrade: add --- End release notes ---

  • May 1 #2955: hb: auto run selftest --fix for missing arc files

  • May 1 #2954: rm/retain: if download fails try other destinations

  • May 1 #2953: hb: bug fix for #2949: Unable to download archive arc.0.0: AttributeError("'module' object has no attribute 'report'"

  • Apr 30 #2952: log: if log disappears don’t traceback on rename

  • Apr 30 #2951: hb: forward SIGTERM so backup sees it

  • Apr 29 #2950: selftest: remove read count difference notice

  • Apr 29 #2949: add a couple more wait status lines

  • Apr 29 #2948: backup: better wait time stats

  • Apr 29 #2947: backup: show status msg while waiting for cache space

  • Apr 28 #2946: backup: set timelimit flag on SIGTERM to stop sooner

  • Apr 28 #2945: backup: moved and enabled signal (SIGTERM) handler

  • Apr 28 #2943: backup: don’t set checkpoint to last file

  • Apr 28 #2942: backup: maxtime interrupt on large files

  • Apr 27 #2941: backup: show maxwait stop goal

  • Apr 27 #2940: backup: delay maxwait after maxtime expires

  • Apr 26 #2939: use statline() and progrep() for status lines

  • Apr 26 #2937: get: early termination could hang with limited cache

  • Apr 25 #2936: selftest: more uniform -v5 progress reports

  • Apr 25 #2935: selftest: --sample fix, status line

  • Apr 25 #2934: misc.statline() and progrep()

  • Apr 24 #2933: Bugz: don’t send tracebacks in development

  • Apr 24 #2932: dest: setid arg not required if only 1 destination

  • Apr 24 #2931: set DESTID carefully to avoid a setid in edge cases

  • Apr 24 #2930: hb: avoid traceback if db can’t be opened

  • Apr 23 #2929: backup: further reduce --maxtime overruns

  • Apr 22 #2927: backup: close/unlock db during final sync

  • Apr 22 #2926: config: doesn’t need to lock the backup directory

  • Apr 22 #2924: rm: don’t run sync to delete arc files

  • Apr 21 #2923: misc.timeinterval() accepts multiple time periods

  • Apr 21 #2922: misc.holdquit() signal handler

  • Apr 21 #2921: misc.intty(), misc.outtty()

  • Apr 21 #2920: log: -X didn’t always exclude lines

  • Apr 17 #2919: backup: --maxtime/--maxwait more accurate

  • Apr 17 #2918: handle negative time intervals in show.vistime()

  • Apr 17 #2917: backup: poll destinations while waiting for cache

  • Apr 16 #2916: init: avoid traceback if -c is a file. Thanks Bugz!

  • Apr 16 #2915: hb: disable stderr redirection for automated tests

  • Apr 15 #2914: recover: HMAC signature verification failed

  • Apr 14 #2913: config: conditionally display enable/disable-commands note

  • Apr 14 #2912: config: flush stdout before reading admin-passphrase to avoid confusing output; handle EOF (ctrl-d) when entering admin-passphrase to avoid traceback - thanks Bugz!

  • Apr 14 #2911: don’t send tracebacks from automated tests

  • Apr 13 #2910: hb: don’t update exit code in tests (sync test)

#2909 Apr 12, 2022

  • Apr 12 #2909: official release

  • Apr 12 #2909: config: traceback on invalid or read-only key

  • Apr 12 #2908: hb command --help shows command help. Thanks Arthur!

  • Apr 11 #2907: close db after integrity check

  • Apr 8 #2905: hb, selftest: try to correct malformed database

  • Apr 8 #2903: trim: show message when -i isn’t used (maybe forgot it)

  • Apr 7 #2902: trim: traceback with -s

  • Apr 7 #2901: trim: % character causing help traceback

  • Apr 6 #2900: Bugz: don’t skip line if at beginning of log

  • Apr 6 #2899: Bugz: handle extremely long traceback (internal testing)

  • Apr 6 #2898: Bugz: flush stdout and delay to get end of traceback

#2897 Apr 6, 2022

  • Apr 6 #2897: official release

  • Apr 6 #2897: dest: cPickle.loads() type mismatch. Thanks Ware!

  • Apr 6 #2896: trim: -n always a percentage, better time left estimate

  • Apr 5 #2895: trim: -n % limit was getting applied twice

  • Apr 5 #2894: trim: limit versions with -n too

  • Apr 5 #2893: trim: -n defaults to top 50% of backup

  • Apr 5 #2891: log: ctrl-c does not log as _FAIL or _RUN

  • Apr 5 #2890: trim: check backup ownership for keep & delete

  • Apr 4 #2889: trim: reorganized code, dash should have been r for remove, reinstate -n w/default 1000 for -i, 100000 otherwise, add g command to go to (search for) pathname, add i command to show file details

  • Apr 4 #2886: trim: remove blank line when paging with space bar

  • Apr 4 #2885: trim: change --paths option to -p

  • Apr 4 #2884: trim: add note in show deletes about smaller files

  • Apr 3 #2883: trim: unique data too low, needs float division

  • Apr 3 #2882: trim: more accurate (but slower) scan for unique data, show time left

#2881 Apr 3, 2022

  • Apr 3 #2881: official release

  • Apr 3 #2881: db upgrade: fix AttributeError: 'NoneType' object has no attribute 'items'

  • Apr 2 #2877: ssh: handle dd unavailable on remote

#2875 Apr 1, 2022

  • Apr 1 #2875: official release

  • Mar 31 #2874: selftest: fix %% in progress string; upgrade: show current version

  • Mar 31 #2873: tune: tune paths, fix and re-enable block commits, speed up tune with many arc files

  • Mar 30 #2872: tune: disable block commits: interrupt testing found a bug

  • Mar 30 #2871: compare: not showing all changes, mostly deletes

  • Mar 29 #2870: trim: space key, fix suspend

  • Mar 29 #2869: trim: typo in #2861 added duplicate entries to trim list

  • Mar 28 #2868: export: workaround NFS creating .nfs files for large databases

  • Mar 28 #2866: backup: was not deleting old hb.db.N files

  • Mar 27 #2865: backup: sync remote-to-remote after backup vs at the start

  • Mar 26 #2864: tune: don’t delete version records; versions suppresses them

  • Mar 25 #2863: db upgrade: tune failed on old backups

  • Mar 25 #2862: arcmb didn’t work out, removed it

  • Mar 25 #2861: trim: --paths option to sort by pathname

  • Mar 25 #2860: rm: show user bytes deleted

  • Mar 22 #2858: selftest: performance in Paths II w/huge pathids

  • Mar 22 #2857: trim: can’t keep kid if parent deleted

  • Mar 22 #2856: export: delete key.conf.0

  • Mar 22 #2855: db upgrade: set arcmb, use MB not MiB

  • Mar 22 #2854: trim: save keeps in db for future trims

  • Mar 21 #2853: trim: line numbers, l and / don’t unstack

  • Mar 21 #2852: trim: <, >, digits

  • Mar 21 #2851: trim: l (el) list directory

  • Mar 21 #2850: trim: K keeps whole directories

  • Mar 21 #2849: trim: skip kids if parents deleted

  • Mar 20 #2848: backup: large file tracking, db upgrade changes

  • Mar 20 #2847: versions: simplify code

  • Mar 20 #2846: get: atexit traceback after restore. Thanks David!

  • Mar 19 #2845: trim: check files > 1MB to improve speed

  • Mar 19 #2843: trim: slashes on directories

  • Mar 18 #2842: trim: more -i options, fixed -s0

  • Mar 18 #2841: trim: flush stdout before changing tty mode

  • Mar 18 #2840: trim: can’t delete / with -i

  • Mar 18 #2839: backup: checked / saved progress off by 1

  • Mar 18 #2838: trim: more -i options

  • Mar 17 #2837: stats: bug in db library gave negative stats w/10GB db

  • Mar 16 #2834: export: close db before delete to avoid NFS errors

  • Mar 16 #2830: trim: suspend on ctrl-z

  • Mar 15 #2829: backup: round mtimes for comparisons

  • Mar 15 #2828: Bugz: map pathnames to numbers for privacy

  • Mar 14 #2827: trim: -i option added for interactive removal

  • Mar 14 #2826: trim: -d option removed; always shows directories

  • Mar 14 #2825: backup: error count 1 off if destinations halted

  • Mar 14 #2824: backup: show large files and recent errors

  • Mar 10 #2823: backup: --maxwait off disables destinations

  • Mar 9 #2822: log: show from & to dates in log summary

  • Mar 8 #2821: don’t pickle command for audit log

  • Mar 8 #2820: db upgrade changes

  • Mar 7 #2818: backup: save char block devices, tb in errors

  • Mar 7 #2817: yesno: delay writing to /dev/tty until stdout flushes

  • Mar 7 #2816: selftest: print block progress by space used

  • Mar 5 #2814: new base17 dbid

  • Mar 5 #2813: rm: always repack cblocks with --secure

  • Mar 3 #2805: trim: new command to find large files in backup

  • Mar 3 #2804: log: always disable stdout buffering for timestamps

  • Mar 3 #2803: tune: gave error if last backup was interrupted

  • Mar 2 #2802: backup: remove last version if nothing saved

  • Mar 2 #2801: recover is now logged

  • Mar 2 #2800: recover: be friendly if no destination on command line

  • Mar 2 #2799: backup: print warning if more than 4 billion blocks

  • Mar 1 #2798: backup: don’t crash if more than 4 billion blocks

  • Mar 1 #2797: Bugz: include end of log with tracebacks

  • Mar 1 #2796: db: clear journal before closing db for upgrade

  • Mar 1 #2795: versions: write errors to stderr not stdout

  • Mar 1 #2794: recover: static build requires sys.exit(), not exit()

  • Mar 1 #2793: improve readkey on message, shared import loop fix

  • Mar 1 #2792: db upgrade changes

  • Feb 28 #2791: traceback if logs summarized while program running

  • Feb 28 #2790: hb -c w/o backupdir should error, not traceback

  • Feb 28 #2789: handle stdout redirected on yes/no question

  • Feb 26 #2788: tune: divide by zero if no data saved

  • Feb 26 #2787: versions: better output, -l obsolete

  • Feb 26 #2786: compare: round mtimes

  • Feb 26 #2784: DATABASE UPGRADE

  • Feb 26 #2783: review readkey help

  • Feb 26 #2782: prep rekey for db upgrade

  • Feb 26 #2781: prep export for db upgrade

  • Feb 26 #2780: don’t try to clear journals on every open

  • Feb 22 #2779: traceback if database locked

  • Feb 22 #2775: add sparse support for APFS

  • Feb 22 #2774: default dedup is 100MB

  • Feb 22 #2773: shard exit code incorrect

#2772 Feb 21, 2022

  • Feb 21 #2772: official release

  • Feb 21 #2772: security issue, disclosure in 90 days

  • Feb 21 #2771: stats: split block and ref db space

  • Feb 21 #2770: better error for -c

  • Feb 21 #2769: handle halted destinations better

  • Feb 20 #2766: log when -c isn’t used

  • Feb 20 #2765: show source of backup directory

  • Feb 17 #2764: mount: make intermediate directories

  • Feb 17 #2762: add --iso option to selftest for tune testing

  • Feb 17 #2760: better permission error messages

  • Feb 17 #2759: compare traceback on empty directories

#2758 Feb 16, 2022

  • Feb 16 #2758: official release

  • Feb 16 #2756: symlink bug in Dir destination

  • Feb 16 #2754: remove progress bars from log files

  • Feb 15 #2752: cleanup tune arguments

  • Feb 15 #2751: log changes disabled all progress bars

  • Feb 15 #2744: add tune command w/progress

  • Feb 14 #2740: remove tune db precheck

  • Feb 14 #2739: remove unused code from tune

  • Feb 14 #2738: update release notes for #2735

  • Feb 13 #2737: rm: error message if path not found with -r

  • Feb 13 #2735: remove feature to remove entire versions

  • Feb 13 #2734: double audit logging

  • Feb 12 #2731: create log directory in init, log permission issues

  • Feb 11 #2730: rm: KeyError traceback

  • Feb 11 #2729: rm: remove db commit

  • Feb 11 #2725: separate preview notes

  • Feb 11 #2724: sort log summaries

  • Feb 11 #2723: log log → log: don’t log the log command even if users request it, to avoid a traceback when finished

#2722 Feb 10, 2022

  • Feb 10 #2722: released

  • Feb 10 #2720: log -s option is obsolete

  • Feb 9 #2716: add instructions for upgrading old dbrevs

  • Feb 9 #2715: remove hidden command handling (not used)

  • Feb 9 #2714: print copyright heading in parent, log in child

  • Feb 9 #2713: copyright heading missing for some commands

  • Feb 9 #2712: write /tmp cafile if needed for SSL w/o -c

  • Feb 9 #2711: remove support for dbrev < 32 (Feb 16, 2019)

  • Feb 9 #2710: log: pass back real exit code

  • Feb 8 #2704+: not so much fun with version control

  • Feb 6 #2703: selftest --inc underflow

  • Feb 2 #2702: hb log -s didn’t work

  • Feb 2 #2701: fix "Is a directory" in rmx #2683. Thanks Ware!

  • Feb 2 #2700: log every command

  • Feb 1 #2694: log perf, log clear, log command

  • Jan 31 #2692: ls -1 w/o -r shows last version

  • Jan 31 #2690: rm: incorrect accounting when removing blocks

  • Jan 31 #2689: selftest: don’t delete bad block w/o --fix

  • Jan 28 #2688: auto run selftest --fix for hash mismatch

  • Jan 27 #2682: minor formatting changes for stats --xmeta

  • Jan 26 #2681: fix dbstat build error on non-OSX. Thanks Evert!

  • Jan 25 #2680: release failed arc download from cache

  • Jan 22 #2679: NFS spans.tmp bug. Thanks Paul!

  • Jan 20 #2678: log -x0 and -x1