commit f07606f8bd9e6a8cb59994a8a784723925070428
parent 1edd4c10035484db2c6f0086d3094575f5bfac11
Author: JayVii <jayvii[AT]posteo[DOT]de>
Date: Sun, 28 Apr 2024 17:40:37 +0200
Mention git page; fix typos
Diffstat:
1 file changed, 42 insertions(+), 37 deletions(-)
diff --git a/content/posts/backups.md b/content/posts/backups.md
@@ -8,6 +8,11 @@ ShowToc: false
ShowBreadCrumbs: true
---
+> **Update 2024-04-28:**
+>
+> The resulting script from this blog post can be found on my git page:
+> [rsync_encrypted_backup](https://src.jayvii.de/pub/rsync_encrypted_backup/)
+
Backups have been somewhat of a pain for me for quite a while, as I could never
find a suitable, easy to manage and easy to recover option for my private
computer.
@@ -19,17 +24,17 @@ AES256-level.
I tried several options like `7z` with encryption and low (or no) compression
rate, sending a whole ZIP archive to a remote storage or even updating existing
-archives. However this of course turned out to be rather cumbersome, prone to
+archives. However, this of course turned out to be rather cumbersome, prone to
write-errors / connection issues and extremely slow.
The next approach did work reasonably well, and is what I want to present here.
I am sure there is still room for improvement, so if you have any suggestions,
-feel free to send me a DM in the fediverse or an e-mail.
+feel free to send me a DM in the Fediverse or an e-mail.
The well-known `rsync` tool is a natural candidate for atomic backups in the
Linux-world. It can sync directories with all sort of remote end-points,
-including (S)FTP, webdav, etc. It keeps ACLs, modes and ownership of files
-intact and is realtively fast, light on system resources and can do syncing both
+including (S)FTP, WebDAV, etc. It keeps ACLs, modes and ownership of files
+intact and is relatively fast, light on system resources and can do syncing both
ways (i.e. it may also be used to restore your files). However, `rsync` does not
support encryption while syncing your files.
@@ -40,18 +45,18 @@ all that much.
## File Encryption
A close to perfect solution for this task is `gocryptfs`, the spiritual
-successor of `encryptfs`. It is an encrypted overlay-filesystem, that
-(crucially) supports "reverse-mode", is extremely fast and utilises strong
+successor of `encryptfs`. It is an encrypted overlay-file-system, that
+(crucially) supports "reverse-mode", is extremely fast and utilizes strong
encryption methods.
-What this means exactly in the context of backups is, that we can mount the
-directory we want to backup (e.g. our home-directory) in an encrpyted, real-
-time updated form, and sync the encrypted versions of all files rather than
-the original unencrypted versions. The aformentioned "reverse-mode" is useful,
+What this means exactly in the context of backups is, that we can mount
+the directory we want to back up (e.g. our home-directory) in an encrypted,
+real-time updated form, and sync the encrypted versions of all files rather than
+the original unencrypted versions. The aforementioned "reverse-mode" is useful,
because it mounts a pre-existing, unencrypted directory as encrypted volume.
So first, let's start with creating a setup for the encrypted file-system. This
-has to be done only once and creates the meta data and encryption heads for the
+has to be done only once and creates the metadata and encryption heads for the
volume. Once this is done, we only need to mount the encrypted volume in the
future:
@@ -69,7 +74,7 @@ This process will ask you to set an encryption passphrase as well as provide you
with a master restore key. **BACK THIS KEY UP SOMEWHERE SAFE AND IN SEVERAL
PLACES, BOTH DIGITALLY AS WELL AS PHYSICALLY!**
-The meta data file will be stored in your unencrypted directory as
+The metadata file will be stored in your unencrypted directory as
`.gocryptfs.reverse.conf` and in the encrypted storage as `gocryptfs.conf`
(unencrypted). Make sure to store this somewhere secure too, as it is required
to decrypt the storage in case you need to restore your backups.
@@ -95,9 +100,9 @@ gocryptfs \
In your encrypted directory, you will now find your entire home-directory in
encrypted form. The reason we used `--plaintextnames` before was, that it makes
the recovery process a lot easier, if you can actually identify the files and
-folders from their names (ofc their contents are encrypted). If you do not need
+folders from their names (ofc. their contents are encrypted). If you do not need
this feature, because you'd recover the entire directory, rather than only
-partials of it, you may consider to remove that parameter when creating the
+partials of it, you may consider removing that parameter when creating the
volume.
The `--ro` parameter sets read-only permissions for the encrypted mount, meaning
@@ -107,8 +112,9 @@ encrypted directory in real-time. The parameter may protect our directory from
technical or user mistakes, however, i.e. if we by accident use the reverse
order of target and source in `rsync`...
-If we want to recover a backup later-on, of course we do need write permissions
-in the encrypted volme. This is mentioned later in this blog post again.
+If we want to recover a backup later-on, of course we do need to write
+permissions in the encrypted volume. This is mentioned later in this blog post
+again.
## File Transmission
@@ -126,24 +132,23 @@ is handy here. Since we want to see what is happening during the procedure, the
`--verbose` and `--progress` parameters are useful as well. Additionally, files
that we have deleted from our system should also disappear from the backup, next
time we sync them up. Ideally, this should happen *after* new files are
-transfered via the `--delete-delay`.
+transferred via the `--delete-delay`.
-Because I backup multiple devices to the same network storage, it is a good idea
-to name the target folder after the hostname of the device. Furthermore,
-eventhough I do atomic backups, I want to keep several versions of
-my backup-files. So I backup to different folders on the remote storage
-solutions, based on time. To be more exact, I append the current month to the
-target directory's name, such that I always have the past 12 versions of my
-backups (considering that I run backups once every month.):
-`${HOSTNAME}_$(date +%m)/`.
+Because I back up multiple devices to the same network storage, it is a good
+idea to name the target folder after the hostname of the device. Furthermore,
+even though I do atomic backups, I want to keep several versions of my
+backup-files. So I back up to different folders on the remote storage solutions,
+based on time. To be more exact, I append the current month to the target
+directory's name, such that I always have the past 12 versions of my backups
+(considering that I run backups once every month.): `${HOSTNAME}_$(date +%m)/`.
However, if you want to keep fewer past versions, there is a little trick via
the [modulo](https://en.wikipedia.org/wiki/Modulo) of the current month. Say,
you want to keep only the past 3 versions, you can do `$(($(date +%m) % 3))`,
which will divide the number of the current month (i.e. 5 for may) by 3 and give
you the remainder of 2. So over the course of a year, this calculation would
-give you 1 in january, 2 in february, 0 in march, 1 in april, 2 in may, 0 in
-june and so on. This in turn means that you'll always keep the past 3 months as
+give you 1 in January, 2 in February, 0 in March, 1 in April, 2 in May, 0 in
+June and so on. This in turn means that you'll always keep the past 3 months as
different versions of your backup. Adjust this value to your needs and the size
of your remote storage.
@@ -176,12 +181,12 @@ rsync \
For an easy, semi-automated backup routine, a few additional ease of life
improvements come in handy, such as mounting the reverse filesystem before
-backup and unmounting them afterwards.
+backup and unmounting them afterward.
Additionally, I like to send desktop notifications whenever I am using the
script in a desktop environment. In order to detect this, I use the `$DISPLAY`
environment variable for X11 desktops and the `$WAYLAND_DISPLAY` variable for
-wayland environments. I typically use `gdbus` to send notifications, wrapped in
+Wayland environments. I typically use `gdbus` to send notifications, wrapped in
a shell-function:
```{.sh}
@@ -305,17 +310,17 @@ fusermount -u "$ENCRYPTED_DIR"
Restoring the backup is relatively easy as well. For simplicity, I'll assume
that the entire backup should be restored. Take a look at `rsync`'s options if
-that is not what you want. of course you can also recover only specific
+that is not what you want. Of course, you can also recover only specific
directories or files.
First off we need the `.gocryptfs.reverse.conf` that the encryption tool created
-when we initilised the filesystem for the first time. That file contains meta
+when we initialized the file system for the first time. That file contains meta
information about the encrypted storage, but crucially not the decryption
-password. When mounting the filesystem, it has been put *un-encrypted in plain
-text* into the encrypted storage as `gocryptfs.conf` and transfered to the
+password. When mounting the file system, it has been put *unencrypted in plain
+text* into the encrypted storage as `gocryptfs.conf` and transferred to the
remote storage.
-In case you lost your entire local filesystem and want to restore it from the
+In case you lost your entire local file system and want to restore it from the
backup, we first need to fetch this configuration file:
```{.sh}
@@ -329,9 +334,9 @@ scp user[AT]my[DOT]remote.backup:${OLD_HOSTNAME}_${TARGET_VERSION}/gocryptfs.conf \
```
-Once this is done, we can mount the encrypted filestorage again, however this
+Once this is done, we can mount the encrypted file storage again, however this
time with writing permissions, so we can restore the files from the remote
-storage into the encrypted filesystem:
+storage into the encrypted file system:
```{.sh}
#!/usr/bin/env bash
@@ -349,7 +354,7 @@ gocryptfs \
echo "Mounting $HOME to $ENCRYPTED_DIR failed!"
```
-Now we can finally start to transfer the files. They will simultanously show up
+Now we can finally start to transfer the files. They will simultaneously show up
as decrypted files in the home directory:
```{.sh}
