homelab/active/software_borg/borg.md

# Borg Backup

- [Borg Backup](#borg-backup)
  - [Install Borg](#install-borg)
  - [Set up a new client](#set-up-a-new-client)
  - [Check backup service logs](#check-backup-service-logs)
  - [Run a Manual Backup](#run-a-manual-backup)
    - [Back up and Entire System](#back-up-and-entire-system)
  - [Upgrade a Borg Repo](#upgrade-a-borg-repo)

## Install Borg

<https://borgbackup.readthedocs.io/en/stable/deployment/central-backup-server.html#user-and-group>

User: backup

Group: backup

Shell: /bin/bash (or other capable to run the borg serve command)

Home: /home/backup

```bash
dnf install borgbackup

useradd backup
mkdir /home/backup/.ssh
touch /home/backup/.ssh/authorized_keys
chown -R backup:backup /home/backup/.ssh
```

## Set up a new client

Clients will either use a backup app (Pika) or the `backup.sh.j2` script in this
directory.

We'll be using ssh authorized keys to control where the repo gets created. This
keeps client configuration simple, since clients will just need to reference
their repo at `.` rather than an absolute path.

For pika, you can set the backup location to something like `borg-backup:.`
(assuming you have an entry matching `borg-backup` in your `.ssh/config`)

Backups will be run as the root user. Generate them an SSH key to connect to the borg server.

1. On the client server, install borg. On Fedora this is `dnf install borgbackup`.
2. On the client server as root (or as the laptop user, for pika), create an SSH key for borg

    ```bash
    export BACKUP_HOST="borg.reeselink.com"

    ssh-keygen -C ${USER}@${HOSTNAME} -f ~/.ssh/id_${BACKUP_HOST}

    cat <<EOF >> ~/.ssh/config
    Host ${BACKUP_HOST}
        Hostname ${BACKUP_HOST}
        IdentityFile ~/.ssh/id_${BACKUP_HOST}
        User backup
        Port 22
    EOF
    ```

3. On borg.reeselink.com as the `backup` user, allow that SSH key to access the repo

    ```bash
    # Fill these out
    export CLIENT_HOSTNAME="fqdn.reeseapps.com"
    export SSH_PUBKEY="ssh-rsa abcd1234 backup@fqdn.something.com"

    # Just copy and paste these
    export BORG_COMMAND="cd /home/backup/repos/${CLIENT_HOSTNAME}; borg serve --restrict-to-path /home/backup/repos/${CLIENT_HOSTNAME}"
    export AUTHKEY_ENTRY="command=\"${BORG_COMMAND}\",restrict ${SSH_PUBKEY}"

    # Inspect the authkey entry
    echo $AUTHKEY_ENTRY

    # add the authkey entry to authorized_keys and create the repo dir
    echo $AUTHKEY_ENTRY >> ~/.ssh/authorized_keys
    mkdir /home/backup/repos/${CLIENT_HOSTNAME}
    ```

4. On the client server, `ssh borg.reeselink.com` to accept the SSH key
5. Create the vars file for your client in `secrets/host_vars.yaml` like the following

    ```yaml
    repo_name: my_repo
    borg_user: backup
    borg_host: borg.reeselink.com
    borg_passphrase: ""
    backup_dirs:
    - /home/foobar
    exclude_dirs: []
    keep_daily: 7
    keep_weekly: 4
    keep_monthly: 1
    stop_services: []
    stop_user_services:
    - foobar
    docker_compose_down:
    - /root/release-compose.yaml
    ```

6. Create the backup task with ansible

    ```bash
    ansible-playbook \
    -i ansible/inventory.yaml \
    -l deskwork-root \
    active/software_borg/install_backup.yaml \
    -e "@active/software_borg/secrets/ai.deskwork_vars.yaml"
    ```

## Check backup service logs

```bash
export SERVER_SSH_NAME=
ssh $SERVER_SSH_NAME journalctl -u 'backup-*' -f
```

## Run a Manual Backup

```bash
borg list borg.reeselink.com:home

# Do not include the first / in the path
export PATH_TO_BACKUP=home/ducoterra
export BORG_REPO=borg.reeselink.com:home

# If not initialized, do that now
borg init --encryption none $BORG_REPO

# Run backup and timestamp it
borg create \
--verbose \
--filter AME \
--list \
--stats \
--progress \
--show-rc \
--compression lz4 \
--exclude-caches \
-e "pp:/${PATH_TO_BACKUP}/.cache" \
-e "re:^${PATH_TO_BACKUP}/\.var/app/[^/]+/cache/" \
-e "CACHEDIR.TAG" \
-e "pp:/${PATH_TO_BACKUP}/.local/share/gnome-boxes" \
-e "pp:/${PATH_TO_BACKUP}/.var/app/org.gnome.Boxes" \
-e "pp:/${PATH_TO_BACKUP}/.var/app/org.gnome.BoxesDevel" \
-e "pp:/${PATH_TO_BACKUP}/.local/share/bottles" \
-e "pp:/${PATH_TO_BACKUP}/.var/app/com.usebottles.bottles" \
-e "pp:/${PATH_TO_BACKUP}/.local/share/libvirt" \
-e "pp:/${PATH_TO_BACKUP}/.config/libvirt" \
-e "pp:/${PATH_TO_BACKUP}/.local/share/containers" \
-e "pp:/${PATH_TO_BACKUP}/.local/share/docker" \
-e "pp:/${PATH_TO_BACKUP}/.npm" \
-e "pp:/${PATH_TO_BACKUP}/.ollama" \
-e "pp:/${PATH_TO_BACKUP}/Downloads" \
-e "pp:/${PATH_TO_BACKUP}/Nextcloud" \
${BORG_REPO}::$(date +"%F-%H-%M-%S") \
/${PATH_TO_BACKUP}

# Mount a borg archive
borg mount $BORG_REPO::2025-05-14-00-44-05 /mnt/

# Restore a borg archive to a location (dry run)
borg extract --dry-run --list --strip-components 1 $BORG_REPO::my-files home/USERNAME
```

### Back up and Entire System

```bash
export BORG_REPO=borg.reeselink.com:root

borg create \
--verbose \
--filter AME \
--list \
--stats \
--progress \
--show-rc \
--compression lz4 \
--exclude root/.cache \
--exclude var/lib/docker \
--exclude var/lib/containers \
--exclude usr/share/ollama \
--exclude home \
--exclude proc \
--exclude dev \
--exclude sys \
--exclude tmp \
--exclude .snapshots \
${BORG_REPO}::$(date +"%F-%H-%M-%S") \
/
```

## Upgrade a Borg Repo

<https://borgbackup.readthedocs.io/en/stable/usage/upgrade.html>