Motivation

Way too many times, I’ve found myself needing to get going on a fresh computer. This may be due to a new distro I’ve installed (I have done my fair share of distro hopping over the tears as my needs changed from school student to university student to working professional), or a new computer or even a new SSD. And almost every time, I scramble to copy my home directory on to some ragged old hard drive from under my bed, wait a few hours, nuke my computer and then copy everything back. I have the lovely idea to start doing this at 20:00 on a worknight and so I stay up into the wee hours figuring out how best to transfer things back to my new machine. Finally, at 02:00, I settle for just my main working documents going back. Catch a few hours sleep and struggle with life the next day. Over the following few weeks, my life at home is spent renaming the new backup folder on the hard drive to DS_Backup_April2020-fedora-to-neon or something ridiculous of the sort. Thousands of duplicated files exist in all the older versions of backups I have… And only then do I start painstakingly commencing the process of installing all my programs from memory. apt install this and apt install that-dev. Oh and not to mention tweaking all my settings for all those programs again! Yeah. Absolute pain.

I know, I know. I’ve read hundreds of articles about backup scripts, cool process flows to make everything link up from your phone to your computer to some cloud provider. And of course saving your dotfiles everywhere. And a plaintext list of your important apps… and tree structures etc. I know. But the point is we’re all human and we read more about what we should do and less of actual doing, because lets face it: life happens and we get lazy. Motivation justified? Read on.

Why restic?

Up until now, I knew of loads of cloud storage providers. I never used them for personal backups though, for the simple reason that I didn’t want my personal files floating on some corporate server somewhere free for anyone to mine just because I didn’t read their T&Cs thoroughly enough. Much worse, being beholden to that one service for keeping my data safe and losing everything if they collapsed. Because eventually, nothing is truly permanent either. So there’s a good starting point for me:

Assume nobody can be trusted with your information

Key point… information. Not data. I have no problems with multiple providers (for redundancy) hosting gibberish that only I can understand. Reminds me of the time I was in primary school, scribbling my schedule / points down on the desk in a language my mate and I made up. I didn’t mind that everyone could see it because the desk was convenient in that it wouldn’t get lost (like some niche artisan cloud provider suddenly shutting down), and anyone who read it wouldn’t know head or tail of what it meant. That in mind, I figure, if I can get a protocol / program to encrypt data well enough on my end and then distribute it out, heck I can store it somewhere convenient like the world’s largest storage providers.

Setting your backup repo up

Restic is fairly simple to get going. I invite you to check out their docs here. Let’s go through each step:

Spinning up a wasabi bucket

Create an account on Wasabi and create a new bucket there. E.g. muh-gr8-backup. You can configure which server location you’d like to use e.g. eu-central or america-east etc. And now your bucket is available at s3.wasabisys.com/muh-gr8-backup

Getting restic to use your wasabi credentials

Typing private keys in your shell in plaintext is probably a bad idea. Not to mention a huge pain if you want to automate this process to take place frequently. So you can just shove your credentials in a file and get restic to read them in everytime. There’s ways to do it even more securely, perhaps getting a program like keepass to pass them over but I am not that bothered at this point.

So create a restic variables file in your home folder. Let’s call it .restic.env

And in it, we have the following variables defined:

export RESTIC_REPOSITORY="s3:https://s3.wasabisys.com/BUCKETNAME"
export AWS_ACCESS_KEY_ID="YOUR_ACCESS_ID"
export AWS_SECRET_ACCESS_KEY="YOUR_SECRET_KEY"
export RESTIC_PASSWORD="YOUR REPO PASSWORD"

Note that if you chose a non-default location, such as the EU, your address would be something like https://s3.eu-central-1.wasabisys.com/BUCKETNAME. This is extremely important. I couldn’t find mention of it anywhere and without that change of address, you are able to init but not later access the bucket for some reason. It can throw you into quite the panic!

First Backup

Once you’ve set up all your environment variables, you’re all good to go with backing up. Pro tip: your first backup will always take ages so consider doing this early evening so it can run overnight and ensure your computer doesn’t disconnect Wi-Fi or kill processes if it sleeps. Not that restic running in the background should take too many resources running in the background for most users; I just find that having it run while I’m working is a massive distraction as I keep looking at it!

Apart from that, we’re good to go. Open a terminal emulator and load your environmental variables. This is easier / cleaner than copy pasting keys all the time:

source .restic.env

Then begin the backup process:

restic backup ~/home/some-directory --verbose

And let that run through.

Listing snapshots

You can list existing snapshots. Note that if this is a separate terminal session from the time you did your backup, you’ll have to load your access tokens again:

source .restic.env
restic snapshots

Restoring

You can restore from a repo by providing a target directory and specifying what snapshot you want to restore:

source .restic.env
restic restore d22fas1 --target=home/dsoni/some-place

To use the latest snapshot, you can just replace that snapshot ID with the alias ‘latest’. Verbosity doesn’t seem to be availavble at the time of this post, and it is a frequent point of pain for me especially when restoring massive backups from scratch. Seems to be taking me several nights to restore a remotely backed up repository when I installed a fresh OS, but that is probably down to my residential speeds too. I often interrupt the process and then resume it at a later time, restic should be able to incrementally restore portions, but more updates on this at a later date.