Scan one or more directory trees and show statistics including file counts by file type, by size, the top N files and directories by size, how the files would be divided into shards, and how file sizes are distributed. $ hb count pathname ... [-d] [-n] [-tN] [--sample p] [--shards N or M-N] [-X] At least one pathname is required. Other options: -d shows a file size distribution table -n disables reading file sizes and is 2-3x faster --shards N shows how files would be divided into N shards, by count and size --shards M-N shows how files would be sharded for all values M through N --random p random sample a percentage p of files (by file count, not by size) --sample p sample (like backup) a percentage p of files (by file count, not by size) -tN shows the top N files and directories by size -X cross filesystems The --sample option is not a true random sample and will give the same results every time. It is designed to be compatible with backup's --sample option, which samples in this fixed way to support incremental backup testing. Sampling is based on a hash of the filename so is not always accurate, especially if many files have the same name. For random sampling, using the --random option. Example output from an OSX (Mac) /Applications directory: # hb count /Applications -d -t10 --shard 2-4 HashBackup build #2295 Copyright 2009-2019 HashBackup, LLC Paths: 214796 Avg path len: 100 Max path len: 249 Max pathname: /Applications/Adobe Photoshop Elements 11/Support Files/Adobe Photoshop Elements Editor.app/Contents/Required/LogTransportLaunch.app/Contents/Frameworks/LogTransport.framework/Versions/A/Support/LogTransport.app/Contents/_CodeSignature/CodeResources Types: dir | 51542 | file | 161483 | 5 GiB symlink | 1771 | 38 KiB Shard counts: 2: 81434 49% | 81820 50% 3: 54351 33% | 54610 33% | 54293 33% 4: 40866 25% | 40981 25% | 40736 24% | 40671 24% Shard sizes: 2: 3.0 GB 52% | 2.6 GB 47% 3: 1.9 GB 34% | 1.8 GB 33% | 1.8 GB 32% 4: 1.1 GB 19% | 1.3 GB 23% | 1.4 GB 24% | 1.8 GB 32% Top 10 files: 210 MB /Applications/ABBYY FineReader for ScanSnap 4.0/FineReader for ScanSnap Preferences.app/Contents/Frameworks/libFREngine.dylib 185 MB /Applications/VMware Fusion.app/Contents/Library/isoimages/antivirus.iso 138 MB /Applications/iPhoto.app/Contents/Resources/PointOfInterest.db 127 MB /Applications/Firefox.app/Contents/MacOS/XUL 97 MB /Applications/Mnemosyne.app/Contents/MacOS/libopenblasp-r0.3.0.dev.dylib 96 MB /Applications/Mnemosyne.app/Contents/MacOS/QtWebEngineCore 88 MB /Applications/VMware Fusion.app/Contents/Resources/naos-1.0.vmwarevm/naos.vmdk 65 MB /Applications/Adobe Photoshop Elements 11/Support Files/Adobe Photoshop Elements Editor.app/Contents/MacOS/Adobe Photoshop Elements Editor 63 MB /Applications/VMware Fusion.app/Contents/Library/isoimages/windows.iso 62 MB /Applications/VMware Fusion.app/Contents/Library/isoimages/linux.iso Top 10 dirs: 368 MB /Applications/VMware Fusion.app/Contents/Library/isoimages 299 MB /Applications/Mnemosyne.app/Contents/MacOS 210 MB /Applications/iPhoto.app/Contents/Resources 210 MB /Applications/ABBYY FineReader for ScanSnap 4.0/FineReader for ScanSnap Preferences.app/Contents/Frameworks 140 MB /Applications/ABBYY FineReader for ScanSnap 4.0/FineReader for ScanSnap Preferences.app/Contents/fre 140 MB /Applications/ABBYY FineReader for ScanSnap 4.0/FineReader for ScanSnap Preferences.app/Contents/freppc 136 MB /Applications/Firefox.app/Contents/MacOS 94 MB /Applications/VMware Fusion.app/Contents/Resources/naos-1.0.vmwarevm 86 MB /Applications/VMware Fusion.app/Contents/Library 67 MB /Applications/VMware Fusion.app/Contents/Frameworks File size distribution: Cumm Count | Pct | FILE COUNT |~Cumm Count | ~Pct | FILE SIZE | Cumm Space | Pct | SPACE USED |~Cumm Space | ~Pct -----------+------+------------+------------+------+-----------+------------+------+------------+------------+----- 35682 | 21% | 35682 | 163254 | 100% | <1 KiB | 13 MiB | 0% | 13 MiB | 5 GiB | 100% 51666 | 31% | 15984 | 127572 | 78% | 1 KiB | 36 MiB | 0% | 22 MiB | 5 GiB | 99% 81377 | 49% | 29711 | 111588 | 68% | 2 KiB | 126 MiB | 2% | 90 MiB | 5 GiB | 99% 122176 | 74% | 40799 | 81877 | 50% | 4 KiB | 348 MiB | 6% | 221 MiB | 5 GiB | 97% 141618 | 86% | 19442 | 41078 | 25% | 8 KiB | 563 MiB | 10% | 215 MiB | 4 GiB | 93% 150924 | 92% | 9306 | 21636 | 13% | 16 KiB | 769 MiB | 14% | 205 MiB | 4 GiB | 89% 157135 | 96% | 6211 | 12330 | 7% | 32 KiB | 1 GiB | 19% | 269 MiB | 4 GiB | 85% 159976 | 97% | 2841 | 6119 | 3% | 64 KiB | 1 GiB | 23% | 242 MiB | 4 GiB | 80% 161574 | 98% | 1598 | 3278 | 2% | 128 KiB | 1 GiB | 28% | 274 MiB | 4 GiB | 76% 162333 | 99% | 759 | 1680 | 1% | 256 KiB | 1 GiB | 33% | 266 MiB | 3 GiB | 71% 162736 | 99% | 403 | 921 | 0% | 512 KiB | 2 GiB | 38% | 274 MiB | 3 GiB | 66% 162967 | 99% | 231 | 518 | 0% | 1 MiB | 2 GiB | 44% | 324 MiB | 3 GiB | 61% 163088 | 99% | 121 | 287 | 0% | 2 MiB | 2 GiB | 51% | 346 MiB | 2 GiB | 55% 163176 | 99% | 88 | 166 | 0% | 4 MiB | 3 GiB | 60% | 497 MiB | 2 GiB | 48% 163223 | 99% | 47 | 78 | 0% | 8 MiB | 3 GiB | 70% | 527 MiB | 2 GiB | 39% 163241 | 99% | 18 | 31 | 0% | 16 MiB | 4 GiB | 77% | 389 MiB | 1 GiB | 29% 163247 | 99% | 6 | 13 | 0% | 32 MiB | 4 GiB | 83% | 301 MiB | 1 GiB | 22% 163251 | 99% | 4 | 7 | 0% | 64 MiB | 4 GiB | 90% | 391 MiB | 900 MiB | 16% 163254 | 100% | 3 | 3 | 0% | 128 MiB | 5 GiB | 100% | 508 MiB | 508 MiB | 9% 214796 names in 22s, 9747 names/s The file size distribution table is easier to understand by focusing on the 3 main columns:
|
Commands >