homelab/active/software_btrfs/btrfs.md

BTRFS

• BTRFS
  • Creating an Array
  • Mounting the Array
  • Adding Disks
  • Replacing a Disk
  • Scrubbing the Array
  • Creating Subvolumes
  • Monitoring Usage
  • Encrypting BTRFS with LUKS
  • Monitoring Disk Health
  • Defragmenting and Compressing

Oracle has decent docs here

You'll also want to read about btrfs compression

Creating an Array

# At any point you can check the status of an array by referencing any member
btrfs filesystem show /dev/vdb

# Raid0
mkfs.btrfs --data raid0 --metadata raid0 /dev/vdb /dev/vdc
btrfs device scan

# Raid1
mkfs.btrfs --data raid1 --metadata raid1 /dev/vdb /dev/vdc
btrfs device scan

# Raid1c3
mkfs.btrfs --data raid1c3 --metadata raid1c3 /dev/vdb /dev/vdc /dev/vdd
btrfs device scan

# Raid10
mkfs.btrfs --data raid10 --metadata raid10 /dev/vdb /dev/vdc /dev/vdd /dev/vde
btrfs device scan

# Convert to raid1
# -dconvert == "data convert"
# -mconvert == "metadata convert"
btrfs balance start -dconvert=raid1 -mconvert=raid1 /btrfs
btrfs balance status

Mounting the Array

One off

# Create a mount point
mkdir /btrfs

# Mount the top level subvolume
mount /dev/vdb /btrfs -o subvolid=5

# Mount with better SSD support
mount /dev/vdb /btrfs -o subvolid=5,ssd

# Mount with auto defragmentation for HDD support
mount /dev/vdb /btrfs -o subvolid=5,autodefrag

# Mount a subvolume
mount /dev/vdb /btrfs -o subvol=home

# Inspect
btrfs filesystem show /btrfs

In fstab

UUID=btrfs_uuid  /btrfs  btrfs  defaults 0 0

Adding Disks

# Add a disk
btrfs device add /dev/vdd /btrfs

# Watch the expansion
btrfs filesystem usage /btrfs

Replacing a Disk

# Remove a disk from the array
btrfs device delete /dev/vdb /btrfs

# Add the new device
btrfs device add /dev/vdg /btrfs

Scrubbing the Array

# Start a scrub to check for errors
# -B prevents the process from going to the background
# -d prints stats for each device
btrfs scrub start -Bd /btrfs

# Check the status of a scrub
btrfs scrub status /btrfs

# Watch for disk failures
dmesg | grep btrfs

Creating Subvolumes

# Create a new subvolume (make sure to mount /btrfs as subvolid=5)
btrfs subvolume create /btrfs/foo

# List all subvolumes under a path
btrfs subvolume list -t /btrfs

# Delete a subvolume
btrfs subvolume delete /btrfs/foo

Monitoring Usage

# Quick info for all btrfs arrays
btrfs filesystem show

# Show usage for a specific array
btrfs filesystem usage /btrfs

# Quick command to filter for data used
btrfs filesystem usage /btrfs | grep 'Data.*Used'

Encrypting BTRFS with LUKS

export KEYFILE_PATH=/root/btrfs.keyfile
export LUKS_DEVS="sdb sdc sdd sde sdf sdg sdh"

# Create a key file
dd if=/dev/urandom of=${KEYFILE_PATH} bs=128 count=1
chmod 400 ${KEYFILE_PATH}

# Create partitions
for luks_dev in $LUKS_DEVS; do
echo Creating partition for /dev/$luks_dev
parted -s -a optimal -- /dev/$luks_dev mklabel gpt mkpart primary 1MiB 100%
done

# Check that your list is good
for luks_dev in $LUKS_DEVS; do 
echo will encrypt /dev/${luks_dev}1 and create /dev/mapper/luks-$(lsblk -n -o PARTUUID /dev/${luks_dev}1)
done

# Create the luks partitions
# Note that --iter-time 10000 is how long, in milliseconds, to decrypt the key
# -v is verbose
# -q is "batch mode", don't ask for confirmation
# Longer makes it harder to brute-force
for luks_dev in $LUKS_DEVS; do \
    LUKS_UUID=$(lsblk -n -o PARTUUID /dev/${luks_dev}1)
    LUKS_NAME=luks-${LUKS_UUID}
    echo "Encrypting /dev/${luks_dev}1"; \
    cryptsetup luksFormat -v -q --key-file ${KEYFILE_PATH} /dev/${luks_dev}1
    echo "Unlocking /dev/${luks_dev}1 as ${LUKS_NAME}"
    cryptsetup open /dev/${luks_dev}1 ${LUKS_NAME} --key-file=${KEYFILE_PATH}
    echo "Adding ${LUKS_NAME} UUID=${LUKS_UUID} ${KEYFILE_PATH} discard to crypttab"
    echo "${LUKS_NAME} UUID=${LUKS_UUID} none discard" >> /etc/crypttab
done

# List filesystems with UUID
lsblk --fs

# Now create the array using the /dev/mapper entries from above
mkfs.btrfs --data raid1 --metadata raid1 /dev/mapper/crypt-btrfs-vdb /dev/mapper/crypt-btrfs-vdc...
btrfs device scan

Monitoring Disk Health

https://www.dotlinux.net/blog/how-to-configure-smartd-and-be-notified-of-hard-disk-problems-via-email/#installing-smartd-smartmontools

# btrfs device stats shows any errors
# Grep for any line not ending in "0"
btrfs device stats /mnt | grep -vE ' 0$'

# Show the device IDs for the mounted filesystem
btrfs filesystem show /mnt

# Delete a device (with ID 8, for example)
btrfs device delete 8 /mnt

# Add a device to the array
btrfs device add /dev/vdi1 /mnt

# Rebalance the array
btrfs balance start /mnt

Defragmenting and Compressing

# Defrag a filesystem
btrfs filesystem defragment /mnt

# Defrag and apply compression
# zstd:20 is currently the best compression algorithm
btrfs filesystem defragment -c zstd:20 /mnt