Count
Scans one or more directory trees and shows statistics including file counts by file type, by size, the top N files and directories by size, how the files would be divided into shards, and how file sizes are distributed.
# hb count pathname ... [-d] [-n]
[--random P] [--sample P]
[--shards N or M-N] [-t N] [-X]At least one pathname is required. Without options, Count shows pathname statistics with counts and space usage by file type.
Options
-d shows a file size distribution table
-n disables reading file sizes and is 2-3x faster
--random P random sample a percentage P of files (by file count, not by size)
--sample P sample (like backup) a percentage P of files (by file count, not by size)
--shards N shows how files would be divided into N shards, by count and size
--shards M-N shows how files would be sharded for all values M through N
-t N shows the top N files and directories by size
-X cross filesystems
The --sample option is not a random sample and will give the same
results every time. It is designed to be compatible with backup’s
--sample option, to support incremental backup testing. For true
random sampling, using the --random option.
Example
Count output from an OSX (Mac) /Applications directory. Only the
first two sections would be displayed with no options:
$ hb count /Applications -d -t10 --shard 2-4
HashBackup #2619 Copyright 2009-2021 HashBackup, LLC
Paths: 68012
Avg path len: 93
Max path len: 224
Max pathname: /Applications/Carbon Copy Cloner.app/Contents/Library/LoginItems/CCC User Agent.app/Contents/XPCServices/CCC Stats Service.xpc/Contents/Frameworks/Paddle.framework/Versions/A/Resources/PADActivateWindowControllerYosemite.nib
Types:
dir | 7054 |
file | 60840 | 1 GiB
symlink | 118 | 2 KiB
Shard counts:
2: 30677 50% | 30281 49%
3: 20490 33% | 20199 33% | 20269 33%
4: 15380 25% | 15080 24% | 15226 24% | 15272 25%
Shard sizes:
2: 1.1 GB 58% | 806 MB 41%
3: 581 MB 29% | 579 MB 29% | 791 MB 40%
4: 444 MB 22% | 420 MB 21% | 681 MB 34% | 405 MB 20%
Top 10 files:
270 MB /Applications/Firefox.app/Contents/MacOS/XUL
44 MB /Applications/Firefox.app/Contents/Resources/browser/omni.ja
40 MB /Applications/Install Pianoteq 7.app/Contents/Resources/Install Pianoteq 7.pkg.lzma
34 MB /Applications/iTunes.app/Contents/MacOS/iTunes
29 MB /Applications/Firefox.app/Contents/Resources/omni.ja
27 MB /Applications/iTunes.app/Contents/Resources/Assets.car
25 MB /Applications/Pianoteq 7/Pianoteq 7.app/Contents/Resources/presources.dat
23 MB /Applications/PDFScanner.app/Contents/Frameworks/libopencv_imgproc.dylib
21 MB /Applications/PDFScanner.app/Contents/Resources/tesseract/fin.traineddata
20 MB /Applications/Photos.app/Contents/MacOS/Photos
Top 10 dirs:
291 MB /Applications/Firefox.app/Contents/MacOS
136 MB /Applications/Kofax Power PDF for Mac.app/Contents/PlugIns/PDFpenOCR.app/Contents/Frameworks/Nuance-OmniPage-CSDK-RunTime.framework/Versions/A/Resources
92 MB /Applications/PDFScanner.app/Contents/Frameworks
70 MB /Applications/PDFScanner.app/Contents/Resources/tesseract
61 MB /Applications/iTunes.app/Contents/Resources
50 MB /Applications/Kofax Power PDF for Mac.app/Contents/PlugIns/PDFpenOCR.app/Contents/Frameworks/Nuance-OmniPage-CSDK-RunTime.framework/Versions/A/Libraries
44 MB /Applications/Firefox.app/Contents/Resources/browser
40 MB /Applications/Install Pianoteq 7.app/Contents/Resources
34 MB /Applications/iTunes.app/Contents/MacOS
31 MB /Applications/Firefox.app/Contents/Resources
File size distribution:
Cumm Count | Pct | FILE COUNT |~Cumm Count | ~Pct | FILE SIZE | Cumm Space | Pct | SPACE USED |~Cumm Space | ~Pct
-----------+------+------------+------------+------+-----------+------------+------+------------+------------+-----
[1] 35725 | 58% | 35725 | 60958 | 100% | <1 KiB | 11 MiB | 0% | 11 MiB | 1 GiB | 100%
42029 | 68% | 6304 | 25233 | 41% | 1 KiB | 20 MiB | 1% | 8 MiB | 1 GiB | 99%
46778 | 76% | 4749 | 18929 | 31% | 2 KiB | 33 MiB | 1% | 13 MiB | 1 GiB | 98%
[3] 50426 | 82% | 3648 | 14180 | 23% | 4 KiB | 53 MiB | 2% | 20 MiB | 1 GiB | 98%
54346 | 89% | 3920 | 10532 | 17% | 8 KiB | 96 MiB | 5% | 43 MiB | 1 GiB | 97%
57155 | 93% | 2809 | 6612 | 10% | 16 KiB | 157 MiB | 8% | 61 MiB | 1 GiB | 94%
58964 | 96% | 1809 | 3803 | 6% | 32 KiB | 236 MiB | 12% | 78 MiB | 1 GiB | 91%
59547 | 97% | 583 | 1994 | 3% | 64 KiB | 285 MiB | 15% | 48 MiB | 1 GiB | 87%
60055 | 98% | 508 | 1411 | 2% | 128 KiB | 372 MiB | 20% | 87 MiB | 1 GiB | 84%
60463 | 99% | 408 | 903 | 1% | 256 KiB | 528 MiB | 28% | 155 MiB | 1 GiB | 79%
60724 | 99% | 261 | 495 | 0% | 512 KiB | 713 MiB | 38% | 184 MiB | 1 GiB | 71%
60861 | 99% | 137 | 234 | 0% | 1 MiB | 894 MiB | 48% | 181 MiB | 1 GiB | 61%
60902 | 99% | 41 | 97 | 0% | 2 MiB | 1017 MiB | 54% | 123 MiB | 967 MiB | 51%
[2] 60936 | 99% | 34 | 56 | 0% | 4 MiB | 1 GiB | 64% | 186 MiB | 843 MiB | 45%
60945 | 99% | 9 | 22 | 0% | 8 MiB | 1 GiB | 69% | 90 MiB | 657 MiB | 35%
60954 | 99% | 9 | 13 | 0% | 16 MiB | 1 GiB | 80% | 195 MiB | 567 MiB | 30%
60957 | 99% | 3 | 4 | 0% | 32 MiB | 1 GiB | 86% | 113 MiB | 371 MiB | 19%
60957 | 99% | | 1 | 0% | 64 MiB | 1 GiB | 86% | | 257 MiB | 13%
60957 | 99% | | 1 | 0% | 128 MiB | 1 GiB | 86% | | 257 MiB | 13%
[4] 60958 | 100% | 1 | 1 | 0% | 256 MiB | 1 GiB | 100% | 257 MiB | 257 MiB | 13%
68012 names in 3s, 17414 names/sThe file size distribution table is easier to understand by focusing on the 3 main columns:
-
FILE SIZE (col 6) is a range of file sizes, from this size to the next size
-
FILE COUNT (col 3) is the number of files that are in this size range
-
SPACE USED (col 9) is the total bytes used by all files in this size range
To the left and right of FILE COUNT and SPACE USED are cumulative counts and percentages. The left set of counts and percentages accumulate from the top down, while the right set, marked with a leading ~ in the title, accumulate from the bottom up.
Some observations about /Applications from this table:
[1] from line 1 col 2 (Pct), 58% of files are less than 1K; the (Cumm Space) column shows that altogether they occupy around 11MiB
[2] from line 14 col 4 (~Cumm Count), only 56 files are 4MB or larger; from the last column (~Pct), they occupy 45% of /Applications
[3] in line 4, the 1st column (Cumm Count) shows there are 50,426 files less than 8K (the FILE SIZE in the next line); from col 2 (Pct), this is 82% of the total file count, but only 2% (Pct column after FILE SIZE) of the space, at 53 MiB (Cumm Space)
[4] the last line shows there is 1 file (FILE COUNT) over 256 MiB (FILE SIZE), it uses 257 MiB (SPACE USED), and is 13% (~Pct in the last column) of the space in /Applications