I have a couple of local copies of my media collection, but in case of my house burning down in a fire i would like to not have to rebuild my entire media collection. rsync.net offers some fairly reasonable storage prices (i guess there are many other good options as well).
Would you guys have any second thoughts on storing the entirety of your media collection on a remote server like that unencrypted?
I wouldn’t.
Use a proper backup tool for this, like restic. BackBlaze has reasonable rates, especially of you’re mostly write-only, and restic has built-in support for B2 and encrypts everything by default. It also supports compression, but you won’t get much out of that on media files. restic is also cross-platform and a single executable, so you can throw binaries for OSX, Linux, and Windows on a USB stick and know you can get to your backups from anywhere. It also allows you to mount a remote repository like a filesystem (on Linux, at least), and browse a backup and get at individual files without having to restore everything. It’s super handy if you screw up a single file or directory.
I wouldn’t
Is it rsync in general you wouldnt or rsync.net?
Never heard of restic so i will definitely need to check this out. I was not planning on having a solution that is continuously running but rather dumping everything there once and then sync new file maybe once a month or something.
rsync by itself provides no encryption, and i wouldn’t use just rsync by itself. That’s probably what he meant.
You should use something else with rsync that encrypts the data before uploading it to a server that you don’t own physically.
Or use restic, which takes care of the uploading part too, so you can skip using rsync altogether.
I have no opinion about rsync.net. I’d check which services restic supports; there are several, and it is it supports rsync.net and that’s what you want to use, you’re golden. Or, use another backup tool that has encryption-by-default and does support rsync.net - there are a couple of options.
I would just never store any data that wasn’t meant for public consumption unencrypted on someone else’s servers. I make an exception for my VPS, but that’s only because I’m more paranoid about exposing my LAN that putting my email on a VPS.
restic, and other backup tools, are generally not always on. You run them; they back up. If you run them only one a month, that’s how often they run. The remote mounting is just a nice feature when you want to grab a single file from one of the backups.
What you’re describing is a classic backup use-case. I’m recommending the easiest, cheapest, most reliable offsite solution I’ve used. restic has been around for years, and has a lot of users and a lot of eyeballs look at it, and it’s OSS. There are even GUIs for it, if you’re not comfortable with the CLI. B2 is generally well-regarded, is fairly easy to figure out, and has also been around for ages. Together, they make a solid combo. I also backup with restic to a local disk and use that for accessing history - B2 is just, as you say, in case of a fire, or theft, I suppose.
thanks, thats very helpful!
One (maybe stupid) question - since restic encrypts, does it do this in transfer or would i need to have additional space on my local hard drives for the encrypted archive?
edit: got an answer to this above: does not require intermediate storage! :)
Second restic or Borg with a rclone sync to storage. Restic will handle both for you though. Borg is an option if you want a local back up that then gets synced (or use restic to do multiple backups)
I use B2 storage and it’s dirt cheap compared to other offerings. You can use rclone to mount the bucket locally and only recover what you need to save on egress costs.
The advantage of restic/borg is not only encryption but snapshots, deduplication, and compression over a simple rsync.
Rsync.net can run a Borg server if you want to back up to that but B2 is much more cost effective.
Ok thanks i will look into B2 (is that just shorthand for backblaze or is it something in particular?)
I asked below here as well so no need to answer multiple times, but will restic encrypt when transferring or does it require that i can store the entire encrypted archive locally as well? or is that just borg?
B2 is Backblaze’a version of S3. It’s a cloud storage solution. https://www.backblaze.com/cloud-storage/pricing
Restic will encrypt the backup https://restic.readthedocs.io/en/latest/070_encryption.html
And it will backup directly to B2 so not intermediate storage is needed.
perfect, thanks a lot!!!
This is great additional information, much of which I didn’t know!
I’m doing the backing-up-twice thing; it’d probably be better if I backed up once and rsync’d - it’d be less computationally intensive and save disk space used by multiple restic caches. OTOH, it’d also have more moving parts and be harder to manage, and IME things that I touch rarely need to be as simple as possible because I forget how to use them in between uses.
Anyway, great response!
For me i keep a local one so if I lose a file or something gets corrupted I can restore locally without any egress costs or network lag. The sync to remote is in case of local data loss for example fire or theft.
Rclone will (should) be faster than doing a restic sync due to not having to do any deduping etc.
I have a 60TB media collection, so this would end up costing $600/month.
Instead, I back all my media up to LTO-6 tapes, and store them at a storage unit.
LTO-6 drive: $400 10x LTO-6 tapes (62.5TB): $200 Small off-site storage unit: $30/month
Thats pretty cool, never heard of those kinds of tapes. Im at somewhere between 5-10 TB myself. The cheapest storage units ive seen is about $90/month here so that would be a bit expensive, but there might be some smaller ones for item storage that i dont know about.
I use fireproof safe (good one) and annual backup to that.
My impressions is that such fireproof safes are only fireproof for so long?
i have been planning on getting on though so could perhaps store the original local copy there for additional safety. depends on the space tho
Need a fireproof data safe. Typically around $600-700
Not what your asking but, I have a backup hard disk at my workplace. I have the same size disk at home and when I need to update something on my backup, I copy it to the at home disk. I bring that to work and bring back the other disk home. This way, there is always a backup away from home. All encrypted so nobody can steal the data.
Other options can be a friend or family member.
And for other stuff I also have it on Google Drive.
ah rotating the drives can is a smart solution, that could be an option.
If you’re talking multiple Terrabytes and are located in the EU you might want to consider AWS Glacier I have like 6Tb on there and pay sub 20€ p.m. If you’re in the EU you can request one free migration download by contacting the support. Otherwise you’ll pay thousands.
ah good to know. hopefully will not need the recovery but if i do i would like to avoid paying thousands, because then the idea of spending money to save time in case of emergency doesnt really work anymore
At $12/terabyte/month it seems pretty expensive for media collection (I mean: family photos are irreplaceable but generic video?)
Other options are to use “glacier” tier S3 which is cheap to rent but ultra expensive to recover (but hopefully you won’t need that)
Or just put a pi+HDD hidden somewhere at work/parents and copy to that
At $12/terabyte/month it seems pretty expensive for media collection (I mean: family photos are irreplaceable but generic video?)
It is the value of my time i am concerned about not the files themselves. But i can see that there are cheaper options
Other options are to use “glacier” tier S3 which is cheap to rent but ultra expensive to recover (but hopefully you won’t need that)
Ill check that out :)
Or just put a pi+HDD hidden somewhere at work/parents and copy to that
This could be an option but a little cumbersome to keep updated perhaps?
This could be an option but a little cumbersome to keep updated perhaps?
Debian on auto update with minimal packages lasts for years
Another option that I’m using is this:
- I do a encrypted backup with Borg on a separate drive on my server
- On my work desktop PC, Windows, at boot it connects via ssh and syncs that Borg backups on a new HDD that I purchased and installed, one way sync, silently and without prompts (We are a small business and I am allowed to do that, if you’re not allowed to do that it could be your parents PC)
- Success syncs are pinged to healthchecks.io which emails me if after too many days (configurable) the sync hasn’t been completed
- Errors are also sent to healthchecks.io
- Company group policy settings then keeps my backup server automatically updated
