add btrfs notes from server setup

active/software_btrfs/btrfs.md

@@ -0,0 +1,217 @@
+# BTRFS
+
+- [BTRFS](#btrfs)
+  - [Creating an Array](#creating-an-array)
+  - [Mounting the Array](#mounting-the-array)
+  - [Adding Disks](#adding-disks)
+  - [Replacing a Disk](#replacing-a-disk)
+  - [Scrubbing the Array](#scrubbing-the-array)
+  - [Creating Subvolumes](#creating-subvolumes)
+  - [Monitoring Usage](#monitoring-usage)
+  - [Encrypting BTRFS with LUKS](#encrypting-btrfs-with-luks)
+  - [Monitoring Disk Health](#monitoring-disk-health)
+  - [Defragmenting and Compressing](#defragmenting-and-compressing)
+
+Oracle [has decent docs here](https://docs.oracle.com/en/operating-systems/oracle-linux/8/btrfs/btrfs-ResizingaBtrfsFileSystem.html)
+
+You'll also want to [read about btrfs compression](https://thelinuxcode.com/enable-btrfs-filesystem-compression/)
+
+## Creating an Array
+
+```bash
+# At any point you can check the status of an array by referencing any member
+btrfs filesystem show /dev/vdb
+```
+
+```bash
+# Raid0
+mkfs.btrfs --data raid0 --metadata raid0 /dev/vdb /dev/vdc
+btrfs device scan
+
+# Raid1
+mkfs.btrfs --data raid1 --metadata raid1 /dev/vdb /dev/vdc
+btrfs device scan
+
+# Raid1c3
+mkfs.btrfs --data raid1c3 --metadata raid1c3 /dev/vdb /dev/vdc /dev/vdd
+btrfs device scan
+
+# Raid10
+mkfs.btrfs --data raid10 --metadata raid10 /dev/vdb /dev/vdc /dev/vdd /dev/vde
+btrfs device scan
+
+# Convert to raid1
+# -dconvert == "data convert"
+# -mconvert == "metadata convert"
+btrfs balance start -dconvert=raid1 -mconvert=raid1 /btrfs
+btrfs balance status
+```
+
+## Mounting the Array
+
+One off
+
+```bash
+# Create a mount point
+mkdir /btrfs
+
+# Mount the top level subvolume
+mount /dev/vdb /btrfs -o subvolid=5
+
+# Mount with better SSD support
+mount /dev/vdb /btrfs -o subvolid=5,ssd
+
+# Mount with auto defragmentation for HDD support
+mount /dev/vdb /btrfs -o subvolid=5,autodefrag
+
+# Mount a subvolume
+mount /dev/vdb /btrfs -o subvol=home
+
+# Inspect
+btrfs filesystem show /btrfs
+```
+
+In fstab
+
+```conf
+UUID=btrfs_uuid  /btrfs  btrfs  defaults 0 0
+```
+
+## Adding Disks
+
+```bash
+# Add a disk
+btrfs device add /dev/vdd /btrfs
+
+# Watch the expansion
+btrfs filesystem usage /btrfs
+```
+
+## Replacing a Disk
+
+```bash
+# Remove a disk from the array
+btrfs device delete /dev/vdb /btrfs
+
+# Add the new device
+btrfs device add /dev/vdg /btrfs
+```
+
+## Scrubbing the Array
+
+```bash
+# Start a scrub to check for errors
+# -B prevents the process from going to the background
+# -d prints stats for each device
+btrfs scrub start -Bd /btrfs
+
+# Check the status of a scrub
+btrfs scrub status /btrfs
+
+# Watch for disk failures
+dmesg | grep btrfs
+```
+
+## Creating Subvolumes
+
+```bash
+# Create a new subvolume (make sure to mount /btrfs as subvolid=5)
+btrfs subvolume create /btrfs/foo
+
+# List all subvolumes under a path
+btrfs subvolume list -t /btrfs
+
+# Delete a subvolume
+btrfs subvolume delete /btrfs/foo
+```
+
+## Monitoring Usage
+
+```bash
+# Quick info for all btrfs arrays
+btrfs filesystem show
+
+# Show usage for a specific array
+btrfs filesystem usage /btrfs
+
+# Quick command to filter for data used
+btrfs filesystem usage /btrfs | grep 'Data.*Used'
+```
+
+## Encrypting BTRFS with LUKS
+
+```bash
+export KEYFILE_PATH=/root/btrfs.keyfile
+export LUKS_DEVS="sdb sdc sdd sde sdf sdg sdh"
+
+# Create a key file
+dd if=/dev/urandom of=${KEYFILE_PATH} bs=128 count=1
+chmod 400 ${KEYFILE_PATH}
+
+# Create partitions
+for luks_dev in $LUKS_DEVS; do
+echo Creating partition for /dev/$luks_dev
+parted -s -a optimal -- /dev/$luks_dev mklabel gpt mkpart primary 1MiB 100%
+done
+
+# Check that your list is good
+for luks_dev in $LUKS_DEVS; do 
+echo will encrypt /dev/${luks_dev}1 and create /dev/mapper/luks-$(lsblk -n -o PARTUUID /dev/${luks_dev}1)
+done
+
+# Create the luks partitions
+# Note that --iter-time 10000 is how long, in milliseconds, to decrypt the key
+# -v is verbose
+# -q is "batch mode", don't ask for confirmation
+# Longer makes it harder to brute-force
+for luks_dev in $LUKS_DEVS; do \
+    LUKS_UUID=$(lsblk -n -o PARTUUID /dev/${luks_dev}1)
+    LUKS_NAME=luks-${LUKS_UUID}
+    echo "Encrypting /dev/${luks_dev}1"; \
+    cryptsetup luksFormat -v -q --key-file ${KEYFILE_PATH} /dev/${luks_dev}1
+    echo "Unlocking /dev/${luks_dev}1 as ${LUKS_NAME}"
+    cryptsetup open /dev/${luks_dev}1 ${LUKS_NAME} --key-file=${KEYFILE_PATH}
+    echo "Adding ${LUKS_NAME} UUID=${LUKS_UUID} ${KEYFILE_PATH} discard to crypttab"
+    echo "${LUKS_NAME} UUID=${LUKS_UUID} none discard" >> /etc/crypttab
+done
+
+# List filesystems with UUID
+lsblk --fs
+
+# Now create the array using the /dev/mapper entries from above
+mkfs.btrfs --data raid1 --metadata raid1 /dev/mapper/crypt-btrfs-vdb /dev/mapper/crypt-btrfs-vdc...
+btrfs device scan
+```
+
+## Monitoring Disk Health
+
+<https://www.dotlinux.net/blog/how-to-configure-smartd-and-be-notified-of-hard-disk-problems-via-email/#installing-smartd-smartmontools>
+
+```bash
+# btrfs device stats shows any errors
+# Grep for any line not ending in "0"
+btrfs device stats /mnt | grep -vE ' 0$'
+
+# Show the device IDs for the mounted filesystem
+btrfs filesystem show /mnt
+
+# Delete a device (with ID 8, for example)
+btrfs device delete 8 /mnt
+
+# Add a device to the array
+btrfs device add /dev/vdi1 /mnt
+
+# Rebalance the array
+btrfs balance start /mnt
+```
+
+## Defragmenting and Compressing
+
+```bash
+# Defrag a filesystem
+btrfs filesystem defragment /mnt
+
+# Defrag and apply compression
+# zstd:20 is currently the best compression algorithm
+btrfs filesystem defragment -c zstd:20 /mnt
+```