Preview
The preview release is installed with hb upgrade -p. Using preview releases for non-critical backups helps find and solve problems early. The official release is sent to the preview area so hb upgrade -p gets the official release when it’s available. Thank you for your help!
Preview Notes
-
Apr 9 #3067: official release
-
Apr 9 #3066: bump expiration date
-
Mar 6 #3065: sync: avoid overfilling cache
-
Feb 25 #3063: rm: traceback removing dangling pathname
-
Feb 15 #3062: error if rename to subdirectory
-
Jan 30 #3061: pack-percent-free > 1% instead of 10%
-
Jan 20 #3059: create large .zip files for huge logs
-
Jan 11 #3058: official release
#3067 Apr 9, 2023
Summary:
-
backup, dest sync: respect cache-size-limit
-
log: avoid traceback when summarizing huge log files
-
config: allow lower pack-percent-free >= 1% vs 10%
-
rename: can’t rename to a subdirectory
-
rm: traceback removing a dangling pathname
Details:
-
backup, dest sync: cache-size-limit was sometimes not being honored during sync operations. Thanks for the detailed notes Christian!
-
log: when hb log summarizes log files, it creates a .zip file for each month. For sites with huge logs, the .zip file might be more than 4GB, and this caused a traceback while summarizing the logs. Now .zip files larger han 4GB are created without errors. Thanks Bugz!
-
config: pack-percent-free previously had a lower limit of 10 (percent) but is now 1. Lowering pack-percent-free causes more frequent arc file packing, saving backup space. It should only be lowered if cache-size-limit is -1 (there is a local copy of all arc files), or the first destination in dest.conf does not have download fees. If you can pack files without triggering high download fees, it’s safe to pack more frequently, though with the low cost of object storage, it’s usually not worth it. Thanks David!
-
rename: if a directory was renamed to a subdirectory of itself, eg, hb rename /a /a/b, then /a would become inaccessible and selftest would either detect errors in the paths or get stuck. Rename now throws an error in this case. Thanks Rodrigo!
-
rm: trying to remove an existing pathname that had no associated data caused a traceback rather than the correct error, "Path exists but no files found; run selftest --fix: pathname". Thanks BugZ!
Traceback (most recent call last): File "/hb.py", line 303, in <module> File "/rm.py", line 1127, in main File "/rm.py", line 1024, in rmpath TypeError: raise: arg 3 must be a traceback or None
#3058 Jan 11, 2023
Summary:
-
rm, selftest: prevent & fix selftest error blockids overlap
-
ls: strip pathname pattern trailing slashes
-
trim: prevent hang when destinations configured
-
trim: Exception: No pathname for pathid 2141069
-
trim: always add excludes to inex.conf
-
log: summarize w/o log dir in read-only backup dir
-
backup: better error on bad wildcard in inex.conf
Details:
-
rm, selftest: if the last block of a backup is deleted with either rm or retain and enough data is deleted to trigger a database compression, then after the next backup, selftest could report an error like "Error: arc.1.0 blockids overlap arc.0.4: 458 ⇐ 486". Selftest was previously unable to correct this error with --fix; now it can. Database compression was changed so this error should not occur. Thanks Yannick!
-
ls: strip trailing slashes from pathname pattern to avoid "Path not in backup" errors. Thanks Neal!
-
trim: when items are deleted and deletes are committed, trim could hang if destinations are configured. It did all of its work and the backup was fine, but it got stuck closing an already-closed database. Thanks David!
-
trim: if a subdirectory and its parent were both marked for delete, trim failed with "Exception: No pathname for pathid xxx" and no trim changes occurred. Now, if /a and /a/b are both deleted, /a is deleted and /a/b gets deleted because it is a subdirectory of /a.
-
trim: if a path is marked exclude, it was only added to inex.conf if it was already excluded. Now excludes are always added, whether they were previously excluded or not.
-
log: if the backup directory is read-only, the logs directory doesn’t exist, and hb log is asked to summarize the logs, return 0 (no errors) rather than throwing a Permission denied error trying to create the logs directory. Thanks Bugz!
-
backup: a bad wildcard in inex.conf, for example [x-a], caused a traceback with no context and a bug report. Now a better error message is displayed before aborting, showing the bad wildcard.
History
#3058 Jan 11, 2023
-
Jan 11 #3058: official release
-
Jan 11 #3058: prevent and fix selftest error: blockids overlap
-
Jan 11 #3057: bump expiration date
-
Jan 5 #3056: inex.conf: better error on bad wildcard
-
Dec 14 #3055: log: summarize w/o log dir in RO backup dir
-
Dec 1 #3054: trim: double db close causing hang
-
Nov 27 #3053: trim: always add excludes to inex.conf
-
Nov 27 #3053: trim: traceback if parent and subdir deleted
-
Nov 10 #3052: ls: strip pathname pattern trailing slashes
-
Nov 1 #3051: experiment: preload database for get
#3050 Sep 9, 2022
-
Sep 9 #3050: official release
-
Sep 9 #3049: get: remove experimental inex exclusion
-
Sep 7 #3048: Bugz: strip pathnames if background thread fails
-
Sep 5 #3047: mount: revert back to #3040; couldn’t fix FreeBSD FUSE
-
Sep 5 #3047: get: traceback during restore
-
Sep 5 #3046: backup, get, compare: round mtime more aggressively
-
Aug 31 #3042: mount: adjust for FreeBSD 12+
-
Aug 25 #3040: get: honor inex.conf excludes for restore
#3038 Aug 14, 2022
-
Aug 14 #3038: official release
-
Aug 7 #3037: backup: Exception: Bug: varblock in mixed threads
#3036 Jul 29, 2022
-
Jul 29 #3036: official release
-
Jul 24 #3035: log: forward siginfo signal
-
Jul 21 #3033: recover: arc file download rename race condition
-
Jul 14 #3032: official release
#3032 Jul 14, 2022
-
Jul 14 #3032: official release
-
Jul 14 #3032: db upgrade: remove .uporig if dbrev is correct, check disk space before upgrade
-
Jul 14 #3031: dest needput: revise conditions
-
Jul 8 #3030: selftest: enable dev testing of -v0 and -v1
-
Jul 4 #3029: remove leading slash from inline tracebacks
-
Jul 3 #3028: db upgrade: remove cached arc files to make space (held)
-
Jul 3 #3027: backup: reorder mtime tests
-
Jul 2 #3026: backup: cache was not always getting cleaned up
-
Jul 2 #3025: backup, get: handle int resolutions on mtimes
-
Jul 1 #3024: dest noparts: skip files with null info
-
Jun 30 #3023: selftest: only fix dedup table with --fix
-
Jun 27 #3022: get: rtrim → rstrip. Thanks Bugz!
-
Jun 27 #3021: recover: no such table: destfiles. Thanks Bugz!
-
Jun 25 #3020: dest.conf: better handling of keywords with missing values
-
Jun 25 #3019: dest: re-wrap verify help to match other subcommands
-
Jun 22 #3018: log: handle no such process in sig handler. Thanks Bugz!
-
Jun 22 #3017: handle -r with missing/empty versions better. Thanks Bugz!
-
Jun 21 #3016: backup: no partial backup warning for char devices
-
Jun 18 #3015: official release
#3015 Jun 18, 2022
-
Jun 18 #3015: official release
-
Jun 16 #3012: get: traceback restoring deleted files
-
Jun 16 #3011: backup: distinguish interupts from timeouts
-
Jun 12 #3010: send ctrl-c and sigterm to commands
-
Jun 12 #3009: dest nopart: exit on ctrl-c and kill -term
-
Jun 7 #3007: destinations: ending support for maxsize
-
Jun 6 #3006: cleanup mount change in #3005
-
Jun 5 #3005: mount: reading variable-block files could crash mount
-
Jun 3 #3004: backup, mount: minor message changes
-
Jun 2 #3003: destcmd: don’t require setid after dest rename. Thanks Preston!
-
Jun 2 #3002: mount: remove mount timeout for slow destinations
-
Jun 2 #3001: mount: use mkdirp to create --cache directory. Thanks Bugz!
-
May 31 #2999: mount: remove a debug stack trace
-
May 31 #2998: recover: create testonly.conf file
-
May 30 #2997: compress inline traceback a bit more
-
May 30 #2996: dest.conf: integer keyword defaults
-
May 30 #2995: auto-run selftest in several more situations
-
May 30 #2994: mount improvements. Thanks Preston!
-
May 25 #2993: clear: no "if you continue" with --force
-
May 25 #2992: remove Rackspace Cloudfiles remnants
-
May 25 #2991: log: traceback if program completes during summarizing. Thanks Bugz!
-
May 24 #2990: security disclosure from 2022-02-21 published
-
May 24 #2990: handle rotating USB drives better
-
May 23 #2989: get: restore pathnames can start with ../
-
May 22 #2988: selftest, rm: change cache initialization
-
May 19 #2985: dir dest: was requiring too much free disk space
-
May 19 #2984: generalize reading passphrases
-
May 19 #2983: selftest: --fix corrects a few more errors
-
May 18 #2982: trim: flush stderr so 1st pathname isn’t cleared
-
May 18 #2981: trim: traceback "getting called too often"
-
May 18 #2979: log: prevent status line escape sequences in logs
-
May 17 #2978: dest rename: new command
#2976 May 17, 2022
-
May 17 #2976: official release
-
May 17 #2975: disabled destinations being selected
-
May 16 #2973: recover: use multiple destinations if necessary
-
May 15 #2970: clear progress reports on stderr write
-
May 9 #2969: versions: File Space was sometimes incorrectly blank
-
May 8 #2968: backup: traceback when rebuilding dedup table
-
May 7 #2967: selftest: upload hb.db if it was changed
-
May 7 #2966: ls: show date deleted
-
May 7 #2965: retain: use new status line code
-
May 6 #2964: db upgrade: error vs traceback for permission failures
-
May 5 #2963: hb: auto selftest may need to run 4 times, not 3
-
May 4 #2962: get: handle some fringe error situations
-
May 2 #2958: dest: remove sync after verify is complete
-
May 1 #2957: recover: show better stats during download
-
May 1 #2956: upgrade: add --- End release notes ---
-
May 1 #2955: hb: auto run selftest --fix for missing arc files
-
May 1 #2954: rm/retain: if download fails try other destinations
-
May 1 #2953: hb: bug fix for #2949: Unable to download archive arc.0.0: AttributeError("'module' object has no attribute 'report'"
-
Apr 30 #2952: log: if log disappears don’t traceback on rename
-
Apr 30 #2951: hb: forward SIGTERM so backup sees it
-
Apr 29 #2950: selftest: remove read count difference notice
-
Apr 29 #2949: add a couple more wait status lines
-
Apr 29 #2948: backup: better wait time stats
-
Apr 29 #2947: backup: show status msg while waiting for cache space
-
Apr 28 #2946: backup: set timelimit flag on SIGTERM to stop sooner
-
Apr 28 #2945: backup: moved and enabled signal (SIGTERM) handler
-
Apr 28 #2943: backup: don’t set checkpoint to last file
-
Apr 28 #2942: backup: maxtime interrupt on large files
-
Apr 27 #2941: backup: show maxwait stop goal
-
Apr 27 #2940: backup: delay maxwait after maxtime expires
-
Apr 26 #2939: use statline() and progrep() for status lines
-
Apr 26 #2937: get: early termination could hang with limited cache
-
Apr 25 #2936: selftest: more uniform -v5 progress reports
-
Apr 25 #2935: selftest: --sample fix, status line
-
Apr 25 #2934: misc.statline() and progrep()
-
Apr 24 #2933: Bugz: don’t send tracebacks in development
-
Apr 24 #2932: dest: setid arg not required if only 1 destination
-
Apr 24 #2931: set DESTID carefully to avoid a setid in edge cases
-
Apr 24 #2930: hb: avoid traceback if db can’t be opened
-
Apr 23 #2929: backup: further reduce --maxtime overruns
-
Apr 22 #2927: backup: close/unlock db during final sync
-
Apr 22 #2926: config: doesn’t need to lock the backup directory
-
Apr 22 #2924: rm: don’t run sync to delete arc files
-
Apr 21 #2923: misc.timeinterval() accepts multiple time periods
-
Apr 21 #2922: misc.holdquit() signal handler
-
Apr 21 #2921: misc.intty(), misc.outtty()
-
Apr 21 #2920: log: -X didn’t always exclude lines
-
Apr 17 #2919: backup: --maxtime/--maxwait more accurate
-
Apr 17 #2918: handle negative time intervals in show.vistime()
-
Apr 17 #2917: backup: poll destinations while waiting for cache
-
Apr 16 #2916: init: avoid traceback if -c is a file. Thanks Bugz!
-
Apr 16 #2915: hb: disable stderr redirection for automated tests
-
Apr 15 #2914: recover: HMAC signature verification failed
-
Apr 14 #2913: config: conditionally display enable/disable-commands note
-
Apr 14 #2912: config: flush stdout before reading admin-passphrase to avoid confusing output; handle EOF (ctrl-d) when entering admin-passphrase to avoid traceback - thanks Bugz!
-
Apr 14 #2911: don’t send tracebacks from automated tests
-
Apr 13 #2910: hb: don’t update exit code in tests (sync test)
#2909 Apr 12, 2022
-
Apr 12 #2909: official release
-
Apr 12 #2909: config: traceback on invalid or read-only key
-
Apr 12 #2908: hb command --help shows command help. Thanks Arthur!
-
Apr 11 #2907: close db after integrity check
-
Apr 8 #2905: hb, selftest: try to correct malformed database
-
Apr 8 #2903: trim: show message when -i isn’t used (maybe forgot it)
-
Apr 7 #2902: trim: traceback with -s
-
Apr 7 #2901: trim: % character causing help traceback
-
Apr 6 #2900: Bugz: don’t skip line if at beginning of log
-
Apr 6 #2899: Bugz: handle extremely long traceback (internal testing)
-
Apr 6 #2898: Bugz: flush stdout and delay to get end of traceback
#2897 Apr 6, 2022
-
Apr 6 #2897: official release
-
Apr 6 #2897: dest: cPickle.loads() type mismatch. Thanks Ware!
-
Apr 6 #2896: trim: -n always a percentage, better time left estimate
-
Apr 5 #2895: trim: -n % limit was getting applied twice
-
Apr 5 #2894: trim: limit versions with -n too
-
Apr 5 #2893: trim: -n defaults to top 50% of backup
-
Apr 5 #2891: log: ctrl-c does not log as _FAIL or _RUN
-
Apr 5 #2890: trim: check backup ownership for keep & delete
-
Apr 4 #2889: trim: reorganized code, dash should have been r for remove, reinstate -n w/default 1000 for -i, 100000 otherwise, add g command to go to (search for) pathname, add i command to show file details
-
Apr 4 #2886: trim: remove blank line when paging with space bar
-
Apr 4 #2885: trim: change --paths option to -p
-
Apr 4 #2884: trim: add note in show deletes about smaller files
-
Apr 3 #2883: trim: unique data too low, needs float division
-
Apr 3 #2882: trim: more accurate (but slower) scan for unique data, show time left
#2881 Apr 3, 2022
-
Apr 3 #2881: official release
-
Apr 3 #2881: db upgrade: fix AttributeError: 'NoneType' object has no attribute 'items'
-
Apr 2 #2877: ssh: handle dd unavailable on remote
#2875 Apr 1, 2022
-
Apr 1 #2875: official release
-
Mar 31 #2874: selftest: fix %% in progress string; upgrade: show current version
-
Mar 31 #2873: tune: tune paths, fix and re-enable block commits, speed up tune with many arc files
-
Mar 30 #2872: tune: disable block commits: interrupt testing found a bug
-
Mar 30 #2871: compare: not showing all changes, mostly deletes
-
Mar 29 #2870: trim: space key, fix suspend
-
Mar 29 #2869: trim: typo in #2861 added duplicate entries to trim list
-
Mar 28 #2868: export: workaround NFS creating .nfs files for large databases
-
Mar 28 #2866: backup: was not deleting old hb.db.N files
-
Mar 27 #2865: backup: sync remote-to-remote after backup vs at the start
-
Mar 26 #2864: tune: don’t delete version records; versions suppresses them
-
Mar 25 #2863: db upgrade: tune failed on old backups
-
Mar 25 #2862: arcmb didn’t work out, removed it
-
Mar 25 #2861: trim: --paths option to sort by pathname
-
Mar 25 #2860: rm: show user bytes deleted
-
Mar 22 #2858: selftest: performance in Paths II w/huge pathids
-
Mar 22 #2857: trim: can’t keep kid if parent deleted
-
Mar 22 #2856: export: delete key.conf.0
-
Mar 22 #2855: db upgrade: set arcmb, use MB not MiB
-
Mar 22 #2854: trim: save keeps in db for future trims
-
Mar 21 #2853: trim: line numbers, l and / don’t unstack
-
Mar 21 #2852: trim: <, >, digits
-
Mar 21 #2851: trim: l (el) list directory
-
Mar 21 #2850: trim: K keeps whole directories
-
Mar 21 #2849: trim: skip kids if parents deleted
-
Mar 20 #2848: backup: large file tracking, db upgrade changes
-
Mar 20 #2847: versions: simplify code
-
Mar 20 #2846: get: atexit traceback after restore. Thanks David!
-
Mar 19 #2845: trim: check files > 1MB to improve speed
-
Mar 19 #2843: trim: slashes on directories
-
Mar 18 #2842: trim: more -i options, fixed -s0
-
Mar 18 #2841: trim: flush stdout before changing tty mode
-
Mar 18 #2840: trim: can’t delete / with -i
-
Mar 18 #2839: backup: checked / saved progress off by 1
-
Mar 18 #2838: trim: more -i options
-
Mar 17 #2837: stats: bug in db library gave negative stats w/10GB db
-
Mar 16 #2834: export: close db before delete to avoid NFS errors
-
Mar 16 #2830: trim: suspend on ctrl-z
-
Mar 15 #2829: backup: round mtimes for comparisons
-
Mar 15 #2828: Bugz: map pathnames to numbers for privacy
-
Mar 14 #2827: trim: -i option added for interactive removal
-
Mar 14 #2826: trim: -d option removed; always shows directories
-
Mar 14 #2825: backup: error count 1 off if destinations halted
-
Mar 14 #2824: backup: show large files and recent errors
-
Mar 10 #2823: backup: --maxwait off disables destinations
-
Mar 9 #2822: log: show from & to dates in log summary
-
Mar 8 #2821: don’t pickle command for audit log
-
Mar 8 #2820: db upgrade changes
-
Mar 7 #2818: backup: save char block devices, tb in errors
-
Mar 7 #2817: yesno: delay writing to /dev/tty until stdout flushes
-
Mar 7 #2816: selftest: print block progress by space used
-
Mar 5 #2814: new base17 dbid
-
Mar 5 #2813: rm: always repack cblocks with --secure
-
Mar 3 #2805: trim: new command to find large files in backup
-
Mar 3 #2804: log: always disable stdout buffering for timestamps
-
Mar 3 #2803: tune: gave error if last backup was interrupted
-
Mar 2 #2802: backup: remove last version if nothing saved
-
Mar 2 #2801: recover is now logged
-
Mar 2 #2800: recover: be friendly if no destination on command line
-
Mar 2 #2799: backup: print warning if more than 4 billion blocks
-
Mar 1 #2798: backup: don’t crash if more than 4 billion blocks
-
Mar 1 #2797: Bugz: include end of log with tracebacks
-
Mar 1 #2796: db: clear journal before closing db for upgrade
-
Mar 1 #2795: versions: write errors to stderr not stdout
-
Mar 1 #2794: recover: static build requires sys.exit(), not exit()
-
Mar 1 #2793: improve readkey on message, shared import loop fix
-
Mar 1 #2792: db upgrade changes
-
Feb 28 #2791: traceback if logs summarized while program running
-
Feb 28 #2790: hb -c w/o backupdir should error, not traceback
-
Feb 28 #2789: handle stdout redirected on yes/no question
-
Feb 26 #2788: tune: divide by zero if no data saved
-
Feb 26 #2787: versions: better output, -l obsolete
-
Feb 26 #2786: compare: round mtimes
-
Feb 26 #2784: DATABASE UPGRADE
-
Feb 26 #2783: review readkey help
-
Feb 26 #2782: prep rekey for db upgrade
-
Feb 26 #2781: prep export for db upgrade
-
Feb 26 #2780: don’t try to clear journals on every open
-
Feb 22 #2779: traceback if database locked
-
Feb 22 #2775: add sparse support for APFS
-
Feb 22 #2774: default dedup is 100MB
-
Feb 22 #2773: shard exit code incorrect
#2772 Feb 21, 2022
-
Feb 21 #2772: official release
-
Feb 21 #2772: security issue, disclosure in 90 days
-
Feb 21 #2771: stats: split block and ref db space
-
Feb 21 #2770: better error for -c
-
Feb 21 #2769: handle halted destinations better
-
Feb 20 #2766: log when -c isn’t used
-
Feb 20 #2765: show source of backup directory
-
Feb 17 #2764: mount: make intermediate directories
-
Feb 17 #2762: add --iso option to selftest for tune testing
-
Feb 17 #2760: better permission error messages
-
Feb 17 #2759: compare traceback on empty directories
#2758 Feb 16, 2022
-
Feb 16 #2758: official release
-
Feb 16 #2756: symlink bug in Dir destination
-
Feb 16 #2754: remove progress bars from log files
-
Feb 15 #2752: cleanup tune arguments
-
Feb 15 #2751: log changes disabled all progress bars
-
Feb 15 #2744: add tune command w/progress
-
Feb 14 #2740: remove tune db precheck
-
Feb 14 #2739: remove unused code from tune
-
Feb 14 #2738: update release notes for #2735
-
Feb 13 #2737: rm: error message if path not found with -r
-
Feb 13 #2735: remove feature to remove entire versions
-
Feb 13 #2734: double audit logging
-
Feb 12 #2731: create log directory in init, log permission issues
-
Feb 11 #2730: rm: KeyError traceback
-
Feb 11 #2729: rm: remove db commit
-
Feb 11 #2725: separate preview notes
-
Feb 11 #2724: sort log summaries
-
Feb 11 #2723: log log → log: don’t log the log command even if users request it, to avoid a traceback when finished
#2722 Feb 10, 2022
-
Feb 10 #2722: released
-
Feb 10 #2720: log -s option is obsolete
-
Feb 9 #2716: add instructions for upgrading old dbrevs
-
Feb 9 #2715: remove hidden command handling (not used)
-
Feb 9 #2714: print copyright heading in parent, log in child
-
Feb 9 #2713: copyright heading missing for some commands
-
Feb 9 #2712: write /tmp cafile if needed for SSL w/o -c
-
Feb 9 #2711: remove support for dbrev < 32 (Feb 16, 2019)
-
Feb 9 #2710: log: pass back real exit code
-
Feb 8 #2704+: not so much fun with version control
-
Feb 6 #2703: selftest --inc underflow
-
Feb 2 #2702: hb log -s didn’t work
-
Feb 2 #2701: fix "Is a directory" in rmx #2683. Thanks Ware!
-
Feb 2 #2700: log every command
-
Feb 1 #2694: log perf, log clear, log command
-
Jan 31 #2692: ls -1 w/o -r shows last version
-
Jan 31 #2690: rm: incorrect accounting when removing blocks
-
Jan 31 #2689: selftest: don’t delete bad block w/o --fix
-
Jan 28 #2688: auto run selftest --fix for hash mismatch
-
Jan 27 #2682: minor formatting changes for stats --xmeta
-
Jan 26 #2681: fix dbstat build error on non-OSX. Thanks Evert!
-
Jan 25 #2680: release failed arc download from cache
-
Jan 22 #2679: NFS spans.tmp bug. Thanks Paul!
-
Jan 20 #2678: log -x0 and -x1