Destinations‎ > ‎

Backblaze B2

Backblaze's B2: https://www.backblaze.com/b2/cloud-storage.html.

Advantages:
  • Low cost storage: .005 cents/GB/month
  • First 10GB of storage is free every month
  • Free to upload backup data
  • Low cost to retrieve data: .02 cents/GB
  • Free 1GB per day retrieval allowance
  • Supports selective downloads to lower retrieval costs
  • No minimum storage time
  • No minimum file size
  • Each file is spread across 20 servers, any 17 can reconstruct the original
  • Data is actively "scrubbed" within B2
  • SHA1 hash verification on file transfers 


File Transfer Verification


Uploads: HB generates a SHA1 hash for every file uploaded to B2.  The B2 service verifies that the SHA1 sent by HB matches the SHA1 B2 generates for the data it received.  This verifies that the file was received correctly by B2.  If this check fails, B2 signals an error and HB retries the upload.


Downloads: B2 sends the file's SHA1 with the downloaded file. HB computes the SHA1 of the file it receives and compares it to the SHA1 B2 sent to make sure there were no transmission errors. HB also compares this to the SHA1 sent when the file was uploaded.  This verifies that the file received by HB is exactly what it uploaded.  If any tests fail, an error is signaled and HB retries the download.  


NOTE: HB often requests partial file downloads to save download costs.  The B2 file SHA1 cannot be verified with partial file downloads, but HB always verifies the SHA1 of every backup block before using it, and verifies the SHA1 of every file restored.



Versioning


B2 versions files by default.  That means if file1 is created in the bucket, then deleted, it is not really deleted.  If the same file is uploaded again, there will be 2 copies.  This behavior is not useful for HashBackup, so the first time a bucket is accessed, HB will set a lifecycle rule that means "only keep 1 version".  If there is already a lifecycle rule, HB does not change it.  HB also does an immediate delete of the extra versions B2 creates to save storage costs.



B2's dest.conf Keywords


type (required)


type b2


accountid


Your B2 Account Id.  This keyword has priority over environment variables (more below about environment variables).


appkey


Your B2 App Key.  This keyword has priority over environment variables.


Environment variables are checked if accountid and/or appkey are not specified in dest.conf.  The environment variable names are:


B2_ACCOUNT_ID

B2_APP_KEY


These environment variables are set and exported in your login script.  For example, in .profile in your home directory:


   export B2_ACCOUNT_ID=123456789012

   export B2_APP_KEY=1234567890123456789012345678901234567890


If you add these to your .profile (or .bash_profile), you should protect the file so only you can access it with: chmod 600 .profile


bucket (required)


Each B2 destination requires a bucket keyword.  B2 buckets have both a bucket name and a bucket id.  HB tries the value as a bucket name first, then as a bucket id.  HB will create a private B2 bucket if the bucket name you use doesn't exist.  Each B2 account can have many buckets.  Bucket names are globally unique, so names like "backup" are probably taken and will need something added to make them unique.


Bucket Name Restrictions:

  • must be 6-50 characters
  • must start and end with a letter or digit
  • contains only letters, digits, and dashes

B2 allows mixed case in bucket names but behaves as if all letters are the same case, so AbcDef and abcdef are the same bucket.


dir


This keyword allows many backups to be stored in the same bucket by prepending the value to the backup filename.  Without the dir keyword, a backup will create arc.0.0 in the top level of the bucket.  With the dir keyword and value abc/xyz, the first backup will create abc/xyz/arc.0.0 in the bucket.  Leading slashes are stripped because B2 does not allow leading slashes.


NOTE: if you have an existing B2 backup and want to start using dir, you will have to move the backup files already stored by hand.  Then add the dir keyword to dest.conf.


rate


Limits upload bandwidth per worker.  See Destinations for details.


workers


Backblaze B2 has somewhat higher latency than Amazon S3 or Google Storage.  To compensate, you may want to increase the number of workers for higher performance.  Add workers 2-4 at a time until there is no performance improvement.  More than 20 workers probably doesn't make sense.  See Destinations for details.


debug


If set to 1 or higher, all traffic will be logged to <destname>.<timestamp>.log in the backup directory.  All debug values generate the same log data, though this may change in the future.  With debug 99, all exceptions are fatal and display a traceback instead of doing retries as usual.  Setting workers to 1 makes debugging easier.


Example dest.conf for B2


destname b2

type b2

accountid 0123456789ab

appkey 0123456789abcdef0123456789abcdef0123456789

bucket hbbackup

dir myhost1

Comments