pub / www.jayvii.de

My personal website
git clone https://src.jayvii.de/pub/www.jayvii.de.git
Home | Log | Files | Exports | Refs | Submodules | RSS

commit c6f58d6bca9fb32b3e772db5ac4c7ffcd54f8195
Author: JayVii <jayvii[AT]posteo[DOT]de>
Date:   Sat, 27 Apr 2024 20:34:23 +0200

Initial State of Website

Diffstat:
A.gitignore | 52++++++++++++++++++++++++++++++++++++++++++++++++++++
A.gitmodules | 3+++
Aconfig.yml | 125+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Acontent/posts/backups.md | 371+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Acontent/privacy.md | 49+++++++++++++++++++++++++++++++++++++++++++++++++
Astatic/favicon.ico | 0
Astatic/favicon_128.png | 0
Astatic/favicon_16.png | 0
Astatic/favicon_32.png | 0
Astatic/favicon_64.png | 0
Astatic/images/jayvii.jpg | 0
Athemes/PaperMod | 1+
12 files changed, 601 insertions(+), 0 deletions(-)

diff --git a/.gitignore b/.gitignore @@ -0,0 +1,52 @@ +# Build and Release Folders +bin/ +bin-debug/ +bin-release/ + +# Other files and folders +.settings/ + +# lock +*.lock + +# Executables +*.swf +*.air +*.ipa +*.apk +*.exe + +# Hugo +public/ +.hugo_build.lock +resources/_gen/ + +# Backup Files +*.markdown~ +*.md~ + +# Apache +.htpasswd + +# Logs +logs +*.log + +# Runtime data +pids +*.pid +*.seed + +# Build directories +public +dist + +# Dependency directory +# https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git +node_modules +package-lock.json +bower_components + +# Mac File System File +.DS_Store + diff --git a/.gitmodules b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "themes/PaperMod"] + path = themes/PaperMod + url = https://github.com/adityatelange/hugo-PaperMod/ diff --git a/config.yml b/config.yml @@ -0,0 +1,125 @@ +baseURL: "https://www.jayvii.de/" +title: JayVii +paginate: 5 +theme: PaperMod + +enableRobotsTXT: true +buildDrafts: false +buildFuture: false +buildExpired: false + +minify: + disableXML: true + minifyOutput: true + +params: + env: production + title: JayVii + description: "My Personal Home Page" + keywords: [Blog, Portfolio, PaperMod] + author: JayVii + # images: ["<link or path of image for opengraph, twitter-cards>"] + DateFormat: "January 2, 2006" + defaultTheme: auto # dark, light + disableThemeToggle: false + + ShowReadingTime: true + ShowShareButtons: false + ShowPostNavLinks: true + ShowBreadCrumbs: true + ShowCodeCopyButtons: true + ShowWordCount: true + ShowRssButtonInSectionTermList: false + UseHugoToc: true + disableSpecial1stPost: false + disableScrollToTop: false + comments: false + hidemeta: false + hideSummary: false + showtoc: false + tocopen: false + + assets: + # disableHLJS: true # to disable highlight.js + # disableFingerprinting: true + favicon: "favicon.ico" + favicon16x16: "favicon_16.png" + favicon32x32: "favicon_32.png" + apple_touch_icon: "favicon_128.png" + safari_pinned_tab: "favicon_64.png" + + label: + text: "JayVii" + #icon: /favicon_32.png + #iconHeight: 32 + + # profile-mode + profileMode: + enabled: false # needs to be explicitly set + title: Hi. + subtitle: "Data Enthusiast with a passion for Free and Open Source Software. I am an environmentalist and care about social inclusion and privacy." + imageUrl: "/images/jayvii.jpg" + imageWidth: 120 + imageHeight: 120 + imageTitle: Me + buttons: + - name: Posts + url: posts + - name: Tags + url: tags + + # home-info mode + homeInfoParams: + Title: "Hi." + Content: I am a data enthusiast with a passion for Free and Open Source Software. I am an environmentalist and care about social inclusion &amp; privacy. + + socialIcons: + - name: mastodon + url: "https://social.jayvii.de" + - name: gitea + url: "https://codeberg.org/jayvii" + - name: email + url: "mailto:jayvii [AT] posteo [DOT] de" + - name: key + url: "/downloads/FEE9ABD6B7532AF539FC6A4928464A547871C4C8.asc" + - name: rss + url: "/index.xml" + + cover: + hidden: true # hide everywhere but not in structured data + hiddenInList: true # hide on list pages and home + hiddenInSingle: true # hide on single page + + # for search + # https://fusejs.io/api/options.html + fuseOpts: + isCaseSensitive: false + shouldSort: true + location: 0 + distance: 1000 + threshold: 0.4 + minMatchCharLength: 0 + keys: ["title", "permalink", "summary", "content"] +menu: + main: + - identifier: home + name: Home + url: / + weight: 10 + - identifier: blog + name: Blog + url: /posts/ + weight: 20 + - identifier: tags + name: Tags + url: /tags/ + weight: 30 +pygmentsUseClasses: true +markup: + highlight: + noClasses: false + anchorLineNos: true + codeFences: true + guessSyntax: true + lineNos: true + style: monokai diff --git a/content/posts/backups.md b/content/posts/backups.md @@ -0,0 +1,371 @@ +--- +author: "JayVii" +title: "Encrypted Backup via rsync" +date: "2023-04-07" +description: "How I create atomic encrypted backups via gocryptfs and rsync" +tags: ["backup", "encryption", "tech"] +ShowToc: false +ShowBreadCrumbs: true +--- + +Backups have been somewhat of a pain for me for quite a while, as I could never +find a suitable, easy to manage and easy to recover option for my private +computer. + +My goal was to create a simple off-site backup routine (i.e. "the cloud"), which +would be easy to recover from, suitably fast - ideally with atomic/delta +updates - and reasonably secure, i.e. strong default encryption like +AES256-level. + +I tried several options like `7z` with encryption and low (or no) compression +rate, sending a whole ZIP archive to a remote storage or even updating existing +archives. However this of course turned out to be rather cumbersome, prone to +write-errors / connection issues and extremely slow. + +The next approach did work reasonably well, and is what I want to present here. +I am sure there is still room for improvement, so if you have any suggestions, +feel free to send me a DM in the fediverse or an e-mail. + +The well-known `rsync` tool is a natural candidate for atomic backups in the +Linux-world. It can sync directories with all sort of remote end-points, +including (S)FTP, webdav, etc. It keeps ACLs, modes and ownership of files +intact and is realtively fast, light on system resources and can do syncing both +ways (i.e. it may also be used to restore your files). However, `rsync` does not +support encryption while syncing your files. + +So in order to encrypt backups within `rsync`, they have to be encrypted +**before** transmission, ideally in real-time and without impacting read-speed +all that much. + +## File Encryption + +A close to perfect solution for this task is `gocryptfs`, the spiritual +successor of `encryptfs`. It is an encrypted overlay-filesystem, that +(crucially) supports "reverse-mode", is extremely fast and utilises strong +encryption methods. + +What this means exactly in the context of backups is, that we can mount the +directory we want to backup (e.g. our home-directory) in an encrpyted, real- +time updated form, and sync the encrypted versions of all files rather than +the original unencrypted versions. The aformentioned "reverse-mode" is useful, +because it mounts a pre-existing, unencrypted directory as encrypted volume. + +So first, let's start with creating a setup for the encrypted file-system. This +has to be done only once and creates the meta data and encryption heads for the +volume. Once this is done, we only need to mount the encrypted volume in the +future: + +```{.sh} +#!/usr/bin/env bash + +gocryptfs \ + --init \ # initilise the volume + --reverse \ # use "reverse mode" + --plaintextnames \ # do not obfuscate names of files and directories + "$HOME" # target directory. Here: our home-folder +``` + +This process will ask you to set an encryption passphrase as well as provide you +with a master restore key. **BACK THIS KEY UP SOMEWHERE SAFE AND IN SEVERAL +PLACES, BOTH DIGITALLY AS WELL AS PHYSICALLY!** + +The meta data file will be stored in your unencrypted directory as +`.gocryptfs.reverse.conf` and in the encrypted storage as `gocryptfs.conf` +(unencrypted). Make sure to store this somewhere secure too, as it is required +to decrypt the storage in case you need to restore your backups. + +From now on, we may mount the directory in its encrypted from: + +```{.sh} +#!/usr/bin/env bash + +# create temporary directory as encrypted mountpoint +ENCRYPTED_DIR=$(mktemp -d) + +# mount home-directory to newly created mountpoint +gocryptfs \ + --ro \ # mount in read-only mode + --reverse \ # use "reverse mode" + "$HOME" \ # unencrypted directory to be backed up + "$ENCRYPTED_DIR" && # temporary mount point + echo "$HOME has been mounted in encrypted form to $ENCRYPTED_DIR" || + echo "Mounting $HOME to $ENCRYPTED_DIR failed!" +``` + +In your encrypted directory, you will now find your entire home-directory in +encrypted form. The reason we used `--plaintextnames` before was, that it makes +the recovery process a lot easier, if you can actually identify the files and +folders from their names (ofc their contents are encrypted). If you do not need +this feature, because you'd recover the entire directory, rather than only +partials of it, you may consider to remove that parameter when creating the +volume. + +The `--ro` parameter sets read-only permissions for the encrypted mount, meaning +that you can not write new files to the encrypted volume. Importantly, writing +to the unencrypted directory is still possible. Doing so will also update the +encrypted directory in real-time. The parameter may protect our directory from +technical or user mistakes, however, i.e. if we by accident use the reverse +order of target and source in `rsync`... + +If we want to recover a backup later-on, of course we do need write permissions +in the encrypted volme. This is mentioned later in this blog post again. + +## File Transmission + +Next, we can finally back up our encrypted directory via `rsync`. Let's first +talk about the parameters that might be useful for backups. Personally, I want +to exclude several directories in the backup, like the "Downloads", ".cache" and +similar folders. `rsync` can even use wild-cards here, so you can exclude every +`.git` folder or specific file-types (if they have the appropriate file-ending). +This is of course not possible (or a lot harder...) if you skipped the +`--plaintextnames` when creating the encrypted volume, as all file- and +directory names are obfuscated without it. + +We want to keep file-permissions and file-owners, so the `--archive` parameter +is handy here. Since we want to see what is happening during the procedure, the +`--verbose` and `--progress` parameters are useful as well. Additionally, files +that we have deleted from our system should also disappear from the backup, next +time we sync them up. Ideally, this should happen *after* new files are +transfered via the `--delete-delay`. + +Because I backup multiple devices to the same network storage, it is a good idea +to name the target folder after the hostname of the device. Furthermore, +eventhough I do atomic backups, I want to keep several versions of +my backup-files. So I backup to different folders on the remote storage +solutions, based on time. To be more exact, I append the current month to the +target directory's name, such that I always have the past 12 versions of my +backups (considering that I run backups once every month.): +`${HOSTNAME}_$(date +%m)/`. + +However, if you want to keep fewer past versions, there is a little trick via +the [modulo](https://en.wikipedia.org/wiki/Modulo) of the current month. Say, +you want to keep only the past 3 versions, you can do `$(($(date +%m) % 3))`, +which will divide the number of the current month (i.e. 5 for may) by 3 and give +you the remainder of 2. So over the course of a year, this calculation would +give you 1 in january, 2 in february, 0 in march, 1 in april, 2 in may, 0 in +june and so on. This in turn means that you'll always keep the past 3 months as +different versions of your backup. Adjust this value to your needs and the size +of your remote storage. + +The whole transfer procedure looks like this: + +```{.sh} +#!/usr/bin/env bash + +# How many versions should be kept? +TARGET_VERSION=$(($(date +%m) % 3)) + +# define target backup storage (here: remote SFTP storage) +TARGET="user[AT]my[DOT]remote.backup:${HOSTNAME}_${TARGET_VERSION}/" + +# define directories to be excluded +EXCL="--exclude=Downloads/* --exclude=.cache/* --exclude=.var/* --exclude=.local/share/Trash/* --exclude=*.git/* --exclude=.davfs2/*" + +# back up directory +rsync \ + --archive \ # keep ownership information in tact + --verbose \ # print more information during transmission + --progress \ # show progress of the transmission + --delete-delay \ # remove deleted files from target after sync + ${EXCL} \ # set exclusion parameters from above + "$ENCRYPTED_DIR" \ # source (encrypted mount volume) + "$TARGET" \ # remote target (see above) +``` + +## Final touches + +For an easy, semi-automated backup routine, a few additional ease of life +improvements come in handy, such as mounting the reverse filesystem before +backup and unmounting them afterwards. + +Additionally, I like to send desktop notifications whenever I am using the +script in a desktop environment. In order to detect this, I use the `$DISPLAY` +environment variable for X11 desktops and the `$WAYLAND_DISPLAY` variable for +wayland environments. I typically use `gdbus` to send notifications, wrapped in +a shell-function: + +```{.sh} +#!/usr/bin/env bash + +# notification function +## Args: +## 1. Headline +## 2. Notification ID (0 for new) +## 3. icon-name +## 4. Notification text +## 5. additional information ("[]" for none) +## 6. timeout in ms +function send_notify { + gdbus call --session \ + --dest=org.freedesktop.Notifications \ + --object-path=/org/freedesktop/Notifications \ + --method=org.freedesktop.Notifications.Notify \ + "$1" $2 "$3" "$1" "$4" "$5" \ + '{"category": <"im.received">}' $6 +} +``` + +All in all the final script looks like this: + +```{.sh} +#!/usr/bin/env bash + +# configs ------------------------------ + +# How many versions should be kept? +TARGET_VERSION=$(($(date +%m) % 3)) + +# define target backup storage (here: remote SFTP storage) +TARGET="user[AT]my[DOT]remote.backup:${HOSTNAME}_${TARGET_VERSION}/" + +# define directories to be excluded +EXCL="--exclude=Downloads/* --exclude=.cache/* --exclude=.var/* --exclude=.local/share/Trash/* --exclude=*.git/* --exclude=.davfs2/*" + +# functions ---------------------------- + +# notification function +## Args: +## 1. Headline +## 2. Notification ID (0 for new) +## 3. icon-name +## 4. Notification text +## 5. additional information ("[]" for none) +## 6. timeout in ms +function send_notify { + gdbus call --session \ + --dest=org.freedesktop.Notifications \ + --object-path=/org/freedesktop/Notifications \ + --method=org.freedesktop.Notifications.Notify \ + "$1" $2 "$3" "$1" "$4" "$5" \ + '{"category": <"im.received">}' $6 +} + +# mounting encrypted filesystem -------- +echo "[INFO] Attempting to mount source as encrypted dir." + +# create temporary directory as encrypted mountpoint +ENCRYPTED_DIR=$(mktemp -d) + +# mount home-directory to newly created mountpoint +gocryptfs \ + --ro \ # mount in read-only mode + --reverse \ # use "reverse mode" + "$HOME" \ # unencrypted directory to be backed up + "$ENCRYPTED_DIR" && # temporary mount point + echo "$HOME has been mounted in encrypted form to $ENCRYPTED_DIR" || + echo "Mounting $HOME to $ENCRYPTED_DIR failed!" + +# Transfer data ------------------------ + +## send notification +if [[ ! -z $WAYLAND_DISPLAY ]] || [[ ! -z $DISPLAY ]]; then + send_notify \ + "BackUpr" \ + 0 \ + "document-send" \ + "Starting backup procedure to $TARGET" \ + "[]" \ + 3000 +else + echo "[INFO] Starting backup procedure to $TARGET" +fi + +# back up directory +rsync \ + --archive \ # keep ownership information in tact + --verbose \ # print more information during transmission + --progress \ # show progress of the transmission + --delete-delay \ # remove deleted files from target after sync + ${EXCL} \ # set exclusion parameters from above + "$ENCRYPTED_DIR" \ # source (encrypted mount volume) + "$TARGET" \ # remote target (see above) + +## send notifications about status +if [[ $? == 0 ]]; then + if [[ ! -z $WAYLAND_DISPLAY ]] || [[ ! -z $DISPLAY ]]; then + send_notify "BackUpr" 0 "document-send" "Backup finished successfully." "[]" 0 + else + echo "[INFO] Backup finished successfully" + fi +else + if [[ ! -z $WAYLAND_DISPLAY ]] || [[ ! -z $DISPLAY ]]; then + send_notify "BackUpr" 0 "document-send" "Backup failed!" "[]" + else + echo "[ERROR] Backup failed!" + fi +fi + +## Unmount encrypted file-system +fusermount -u "$ENCRYPTED_DIR" + +# EOF backup.sh +``` + +## Restoring a backup + +Restoring the backup is relatively easy as well. For simplicity, I'll assume +that the entire backup should be restored. Take a look at `rsync`'s options if +that is not what you want. of course you can also recover only specific +directories or files. + +First off we need the `.gocryptfs.reverse.conf` that the encryption tool created +when we initilised the filesystem for the first time. That file contains meta +information about the encrypted storage, but crucially not the decryption +password. When mounting the filesystem, it has been put *un-encrypted in plain +text* into the encrypted storage as `gocryptfs.conf` and transfered to the +remote storage. + +In case you lost your entire local filesystem and want to restore it from the +backup, we first need to fetch this configuration file: + +```{.sh} +#!/usr/bin/env bash + +OLD_HOSTNAME="myoldpc" +TARGET_VERSION=1 + +scp user[AT]my[DOT]remote.backup:${OLD_HOSTNAME}_${TARGET_VERSION}/gocryptfs.conf \ + ~/.gocryptfs.reverse.conf + +``` + +Once this is done, we can mount the encrypted filestorage again, however this +time with writing permissions, so we can restore the files from the remote +storage into the encrypted filesystem: + +```{.sh} +#!/usr/bin/env bash + +# create temporary directory as encrypted mountpoint +ENCRYPTED_DIR=$(mktemp -d) + +gocryptfs \ + --rw \ # mount in read-write mode + --reverse \ # use "reverse mode" + --config "$HOME/.gocryptfs.reverse.conf" \ # config file + "$HOME" \ # unencrypted directory to be backed up + "$ENCRYPTED_DIR" && # temporary mount point + echo "$HOME has been mounted in encrypted form to $ENCRYPTED_DIR" || + echo "Mounting $HOME to $ENCRYPTED_DIR failed!" +``` + +Now we can finally start to transfer the files. They will simultanously show up +as decrypted files in the home directory: + +```{.sh} +#!/usr/bin/env bash + +OLD_HOSTNAME="myoldpc" +TARGET_VERSION=1 + +# transfer files +rsync \ + --archive \ + --verbose \ + --update \ + user[AT]my[DOT]remote.backup:${OLD_HOSTNAME}_${TARGET_VERSION}/ \ + ${ENCRYPTED_DIR}/ + +# unmount the encrypted storage +fusermount -u "$ENCRYPTED_DIR" +``` diff --git a/content/privacy.md b/content/privacy.md @@ -0,0 +1,49 @@ + --- +author: " " +title: "Privacy Policy" +ShowToc: true +ShowBreadCrumbs: true +--- + +# General + +The policies may change any time without notice (especially when we start +providing a new service). + +No identifiable data is shared with third parties, unless such data transfer is +initiated by the user, required to combat abuse, or operationally necessary. + +Some services require the use of first-party cookies and/or local storage, which +are required to provide the service (such as remembering user preferences) and +are not collected by anyone. None of the services employ third-party cookies. + +# Access Logs + +Unless otherwise noted below, the access.log, containing your IP address, +timestamp, path, and User-Agent, is retained for a maximum of 14 days. The logs +are used to combat abuse, debug, and collect aggregated usage statistics. + +# www.jayvii.de + +Besides [Access Information](#access-logs), no information of site visitors are +collected. No cookies are set and no data is shared with anyone. + +# start.jayvii.de + +Besides [Access Information](#access-logs), no information of site visitors are +collected. No cookies are set and no data is shared with anyone. + +# social.jayvii.de + +Besides [Access Information](#access-logs), no information of site visitors are +collected. Unless logged in (which nobody should be able to...), no cookies are +set and no data is shared with anyone. + +# search.jayvii.de + +Besides [Access Information](#access-logs), no information of site visitors are +collected. Neither your serach terms or other search related information is +logged. Cookies may be set for your user-defined specific configuration on your +device. IP-adresses may accidentially be leaked to other search providers by the +software. Also, please consider +["Why use a private instance?"](https://docs.searxng.org/own-instance.html). diff --git a/static/favicon.ico b/static/favicon.ico Binary files differ. diff --git a/static/favicon_128.png b/static/favicon_128.png Binary files differ. diff --git a/static/favicon_16.png b/static/favicon_16.png Binary files differ. diff --git a/static/favicon_32.png b/static/favicon_32.png Binary files differ. diff --git a/static/favicon_64.png b/static/favicon_64.png Binary files differ. diff --git a/static/images/jayvii.jpg b/static/images/jayvii.jpg Binary files differ. diff --git a/themes/PaperMod b/themes/PaperMod @@ -0,0 +1 @@ +Subproject commit 25e5b4d2982aacfee9a1194d2738db284c2ac380