Commands‎ > ‎

Stats

Displays statistics about a backup.

$ hb stats [-c backupdir]


Example

This example is from the HB development server, an OSX system with 5 or 6 virtual machine images of 5GB each, 6 or 7 1GB virtual disk images, and the usual OSX system software.  This system is backed up every night since June of 2011 and retain is run every night after the backup with -s30d12m (keep the last 30 days of backups plus 1 backup each month), so there are 42 versions of all VM images.
 

$ hb stats -c /hbbackup -v

HashBackup build 893 Copyright 2009-2013 HashBackup, LLC
Backup directory: /hbbackup

                 626 completed backups

The number of backups that completed.  Incomplete backups, such as
when a control-c is pressed, are not counted.  Removed versions are
also not counted.  If you use the hb versions command to list all
versions, those with an end time are counted as completed backups.

               68 TB file bytes protected across all versions

Total bytes that have been presented to the backup program, also
sometimes called "bytes in" in other backup programs.  For example, if
a directory is saved that contains a 100K file x, protected bytes
would be 100K for version 0.  If a 200K file y is added and the
directory is backed up again, protected bytes would be 300K for
version 1.  Protected bytes across all versions is the sum of
protected bytes for each version, or 400K in this example.  Another
way to look at this is the sum of every backup if each had been a full
backup.  This is used to compute the "industry standard dedup ratio"
below.

               10 TB file bytes saved since first backup

Total bytes saved by the backup program, beginning with the first
backup.  Some of these bytes may no longer be stored if files have
been removed or retain has been run.  Another way to look at this is
the total bytes read from the filesystem for all backups.  For the
previous example, if 100K file x was backed up in version 0, then 200K
file y was added and backed up in version 1 (x was not modified), file
bytes saved since first backup would be 300K

               24 GB average file bytes saved per backup in last 5 backups

Total bytes saved by the backup program in the last 5 backups, divided
by 5.  If there are fewer than 5 backups, the number of backups is
used instead of 5.  Some of these bytes may no longer be stored if
files have been removed from the last 5 backups.  For the previous
example, file bytes saved is 100K for version 0, 200K for version 1,
and the average would be 150K

        122h 55m 20s total backup time

Time spent on all completed backups.  To be considered complete,
backups must have an end time in the version list.

             15m 25s average backup time for last 5 backups

Average backup time for the most recent 5 completed backups.

              568 GB file bytes currently stored

Total file system bytes currently stored in the backup.  Another way
to look at this is the sum of the size of all files listed by hb ls
-al, not including hard links that were previously seen in the same
version.  If 3 files are hard linked and all are saved, the file is
only counted once.

                 706 archives

The number of arc files.  Arc files may all be local, all remote, or
some combination of local and remote.  Arc files are only counted
once, whether local, remote, or on several remotes.

               98 GB archive space

Bytes used by all arc files.  Some of these bytes may be inactive
(deleted) if files have been removed or retain has been run and the
archive has not been packed.  This is also the sum of ls -l for all
archive files, both local and or remote.  It includes deleted bytes
and encryption overhead required for each block.

               95 GB active archive bytes - 97%

Active (not deleted) data bytes used by all arc files.  The percentage
show how tightly "packed" the archive files are: higher percentages
indicate better space utilization, and 100% means there are few/no
deleted blocks in the archive.  If the percentage is low, indicating
there is a lot of free space in the archives, and you want to recover
it, you could increase the pack-percent-free config variable to cause
archives to be packed sooner.  However, packing archives more often
will take more time and packed archives have to be uploaded again.  If
archives are only stored remotely, they are not packed unless the
config variable pack-remote-archives is True.  The default is False.
To pack a remote archive, it must be downloaded, packed, and then
uploaded; so there is a trade-off here between disk space savings and
download + upload bandwidth costs + processing cost.  The default for
pack-percent-free is 50: local archives are packed when they are 50%
or less full.

               692:1 industry standard dedup ratio

This "dedup" figure is displayed to allow comparing HB with other
backup programs that compute their dedup ratio as the sum(bytesin) /
sum(bytesout) for all backups.  Since compression and traditional
skipping of unmodified files is also counted here as dedup, this ratio
is not a good measure of real dedup processes.  This ratio is computed
as if every backup was a full backup, so even traditional backup
programs that only skip unmodified files and have no compression nor
dedup (plain incremental backups) could show a reasonably high dedup
ratio.

               90 MB average archive space per backup for last 5 backups

Average bytes per backup used by arc files in the last 5 backups.  It
includes active and inactive data and encryption overhead.  Another
way to look at this is the sum of the ls -l size of all archive files
in the last 5 backups.  This can be used for backup space planning.

               266:1 reduction ratio of backed up files for last 5 backups

Shows the average space reduction for modified files in the last 5
backups.  It can be used for backup space planning and is computed as
avg file bytes stored / avg arc space.  If you are planning to add N
bytes of file system space, you can divide N by the ratio to estimate
how much backup space will be required.  This assumes the new data is
similar to existing backup data, ie, a similar number of bytes are
modified before each backup, the data compresses the same, and dedups
the same.

              268 MB dedup table current size

The maximum size of the dedup table is set with either the config
variable dedup-mem or the -D backup option.  HB does not create a
maximum size dedup table until it is required.  The size shown here is
the current size of the dedup table, which is often less than the
maximum size.

           7,445,483 dedup table entries

The dedup table contains entries for every unique data block seen when
the backup program is used with the -D option or dedup-mem is set.
When -D0 is used, no entries are added to the dedup table.

                 79% dedup table utilization at current size

The dedup table has a current size and number of entries, displayed
above.  This percentage shows how full the dedup table is *at its
current size*.  When the dedup table is 100% full, it will double in
size, up to the limit of -D or dedup-mem.  Once it has reached this
limit, it will not increase further.  Instead, old dedup data is
removed as space is needed to record dedup data about new blocks.  So
even when the dedup table cannot be expanded, HB is still able to
dedup very well.  Dedup does not get disabled when the table is full,
though some blocks that might have been deduped with a larger table
would have to be saved again.  The backup program displays a better
statistic for the dedup table (% of maximum).  That cannot be
displayed here because the -D option can change on every backup.

             860,599 files

Files (non-directories) currently stored in the backup.  The same file
may be stored multiple times because there may be several versions of
the file.

             181,571 dirs

Directories currently stored in the backup.  The same directory may be
stored multiple times.

             811,477 paths

Unique pathnames stored in the backup.

         102,156,295 blocks

Data blocks stored in the backup.  If a block is referenced by more
than one file because of dedup, each reference is counted.

          11,604,602 unique blocks

Unique data blocks stored in the backup.  If a block is referenced by
more than one file because of dedup, it is only counted once.  Unique
is not quite accurate: two different blocks can contain the same data.
For example, a small file could be backed up with --full two times;
this would create two "unique" blocks, though they are both equal.

              52,961 average variable-block length (bytes)

HB supports both variable and fixed-length blocks.  This is the
average length of all variable-length blocks in the backup.  Since it
is data dependent, it may differ between backups.





Comments