Keeping Backup Costs Down with B2?
from keytonowhere@lemmy.world to selfhosted@lemmy.world on 21 Jul 14:38
https://lemmy.world/post/49719185

I’m considering Restic and Backblaze B2 as my backup solution. I’m a little worried having read about the potential for unexpected restore costs. For example, I’ve read restoring lots of little files adds to the cost (reference). What are the things to look out for when backing up and restoring that can add to the cost? Are there any best practices to keep costs down?

#selfhosted

threaded - newest

femtek@lemmy.blahaj.zone on 21 Jul 14:57 next collapse

Backup as a zip? Small files warped up in one big file. Only backup what you can’t restore from somewhere else. I think B2 can ship you a HDD to restore from if I remember correctly.

keytonowhere@lemmy.world on 21 Jul 15:17 collapse

I’d suspect a zip could be problematic for incremental backups. Though that’s not a bad idea for files that won’t be changing regularly (photos, archives).

femtek@lemmy.blahaj.zone on 21 Jul 15:39 next collapse

I’ll have to look more into it but I use this. I thought it compresses before uploading but I can’t find the doc yet. docs.duplicati.com/…/set-up-a-backup-in-the-ui

atzanteol@sh.itjust.works on 21 Jul 20:53 collapse

Restic will compress as well.

BlueBockser@programming.dev on 21 Jul 15:29 next collapse

Restic doesn’t write your files 1:1 to B2, but instead chunks their content into files according to a configurable pack size (16MiB by default). If you have a very large amount of data, you could look into increasing the pack size, although that comes with its own risks (see the docs above).

ShortN0te@lemmy.ml on 21 Jul 15:48 collapse

I doubt that this applies to files smaller than the pack size or a for example bitwise unique 17MB file would result in a (overly simplified) 16 MB and a 1 MB chunk or am i wrong here?

theit8514@lemmy.world on 21 Jul 18:36 collapse

Most of these backup solutions try to reduce the amount of data backed up, so the system will process each file and determine if the contents changed and only upload new data. In this case if you modify the file you may end up with 2 16 MB chunks that have slightly different data, and only when the old backup expires and the pruning process completes would one of those chunks be deleted.

Smaller chunks sizes means you get more files written to S3, but less duplicate data when deduping without compression.

ShortN0te@lemmy.ml on 21 Jul 21:54 collapse

That is not what i have asked.

doeknius_gloek@discuss.tchncs.de on 21 Jul 15:36 next collapse

Does it have to be B2? If you want free egress and cheap storage, you may also consider Hetzner’s storage boxes.

femtek@lemmy.blahaj.zone on 21 Jul 15:44 next collapse

I’ll have to give that a look. I spun up immich to replace Google photos but I can’t fully switch over till I have them in a cloud backup. Everything else is config/docker files that are small.

doeknius_gloek@discuss.tchncs.de on 21 Jul 16:40 collapse

Feel free to DM me if you have questions or encounter issues. I’m using restic + hetzner myself.

keytonowhere@lemmy.world on 21 Jul 16:57 collapse

I’ll check out Hetzner. I kept seeing B2 come up so started with researching it.

Cyber@feddit.uk on 21 Jul 22:34 collapse

I moved from Backblaze to a Hetzner storagebox, to backup my Arch (btw) NAS.

Simpler to work with and cheaper.

ShortN0te@lemmy.ml on 21 Jul 15:37 next collapse

I use B2 for my restic Backup. Currently i have about 100GB stored there and it says i have about 100,000 files stored.

With the adjusted calculation of the blog you posted for 10TB of 50KB files, adjusted to the mean file size of my restic repository 10 TB of 1MB files:

Transaction fees: 10,000,000 / 10,000 x $0.004 = $4

My Laptop has a mean file size of about 16 MB per file on the local restic repository.

EDIT: Forgot a resume, i do not think you need to worry about it too much. But maybe verify the amount and type of data you want to backup and use calculators to calculate the real cost, also you can always test it by creating a test dataset and then do a backup and recovery scenario and then use this to derive an estimate.

irmadlad@lemmy.world on 21 Jul 17:21 next collapse

Question: What does BackBlaze B2 give you that BackBlaze’s personal account doesn’t?

keytonowhere@lemmy.world on 21 Jul 17:50 next collapse

The BackBlaze personal page only lists Windows and Mac as supported. I have both, but also multiple Linux machines that I need to back up.

irmadlad@lemmy.world on 21 Jul 20:30 collapse
ShortN0te@lemmy.ml on 21 Jul 23:21 collapse

To my knowledge:

  • File Versions only 30 days available.
  • -> Everything older and deleted than 30 days gone.
  • Slow and manual Restore Process
  • -> Web downloads max. 500 GB manual selection.

B2 is data storage, Backblaze Personal is a service. 2 Completely different things.

KairuByte@lemmy.dbzer0.com on 22 Jul 06:00 collapse

I’ve always wondered, doesn’t Backblaze cut exfiltration costs to zero if you proxy it through cloudflare? Why isn’t that an option so you’re only paying for storage costs and not bandwidth?