finalize for kube class

.gitignore

@@ -11,4 +11,5 @@ TODO.md
 eicar.com
 *.pp
 *.mod
-*.log
+*.log
+scratch/

README.md

@@ -41,6 +41,7 @@ or give me access to your servers.
     - [tmux](#tmux)
     - [bash](#bash)
     - [Bulk File/Folder Renaming](#bulk-filefolder-renaming)
+    - [Escaping a Stuck SSH Terminal](#escaping-a-stuck-ssh-terminal)
   - [SSH Setup](#ssh-setup)
   - [Git GPG Commit Signing](#git-gpg-commit-signing)
   - [Important Dates and Times](#important-dates-and-times)
@@ -128,6 +129,10 @@ for change_dir in $(ls | grep  'podman_*'); do
 done
 ```
 
+### Escaping a Stuck SSH Terminal
+
+Press the following keys: enter + ~ + .
+
 ## SSH Setup
 
 Generate a key (password protect it!)

active/software_k3s/k3s.md

@@ -1,7 +1,6 @@
 # K3S
 
 - [K3S](#k3s)
-  - [Guide](#guide)
   - [Firewalld](#firewalld)
   - [SELinux](#selinux)
   - [Install Single Node K3S](#install-single-node-k3s)
@@ -16,24 +15,14 @@
   - [External DNS](#external-dns)
     - [Credentials](#credentials)
     - [Annotation](#annotation)
-  - [Nginx Ingress](#nginx-ingress)
   - [Cert Manager](#cert-manager)
+  - [Traefik Gateway](#traefik-gateway)
+  - [Longhorn Storage](#longhorn-storage)
   - [Test Minecraft Server](#test-minecraft-server)
   - [Automatic Updates](#automatic-updates)
   - [Database Backups](#database-backups)
   - [Uninstall](#uninstall)
 
-## Guide
-
-1. Configure Host
-2. Install CoreDNS for inter-container discovery
-3. Install Metal LB for load balancer IP address assignment
-4. install External DNS for laod balancer IP and ingress DNS records
-5. Install Nginx Ingress for http services
-6. Install Cert Manager for automatic Let's Encrypt certificates for Ingress nginx
-7. Install longhorn storage for automatic PVC creation and management
-8. Set up automatic database backups
-
 ## Firewalld
 
 ```bash
@@ -104,8 +93,12 @@ curl -sfL https://get.k3s.io | sh -s - \
 "traefik" \
 "--disable" \
 "servicelb" \
-"--tls-san" \
-"k3s.reeselink.com" \
+"--disable" \
+"local-storage" \
+"--cluster-cidr" \
+"10.42.0.0/16" \
+"--service-cidr" \
+"10.43.0.0/16" \
 --selinux
 ```
 
@@ -127,6 +120,8 @@ curl -sfL https://get.k3s.io | K3S_TOKEN=$(cat ~/.k3s-token) sh -s - \
 "traefik" \
 "--disable" \
 "servicelb" \
+"--disable" \
+"local-storage" \
 "--cluster-cidr" \
 "10.42.0.0/16" \
 "--service-cidr" \
@@ -142,6 +137,8 @@ curl -sfL https://get.k3s.io | K3S_TOKEN=$(cat ~/.k3s-token) sh -s - \
 "traefik" \
 "--disable" \
 "servicelb" \
+"--disable" \
+"local-storage" \
 "--cluster-cidr" \
 "10.42.0.0/16" \
 "--service-cidr" \
@@ -176,10 +173,6 @@ export KUBECONFIG=~/.kube/admin-kube-config
 
 ### VLAN Setup
 
-I would remove firewalld to get this working. VLAN IPv6 traffic doesn't work for some
-reason and there aren't good docs yet. Your router firewall will suffice, just be sure
-to configure those rules correctly.
-
 Before working with Metallb you'll need at least one available VLAN. On Unifi equipment
 this is accomplished by creating a new network. Don't assign it to anything.
 
@@ -219,20 +212,10 @@ address, effectively moving the IP to another node. This isn't really "load bala
 
 [Install MetalLB](/active/kubernetes_metallb/metallb.md)
 
-MetalLB doesn't know what IP addresses are available for it to allocate so
-we'll have to provide it with a list. The
-[metallb-addresspool.yaml](/active/kubernetes_metallb/addresspool.yaml) has
-the configuration for our available pools. Note these should match the VLAN you
-created above.
-
-```bash
-# create the metallb allocation pool
-kubectl apply -f active/kubernetes_metallb/addresspool.yaml
-```
-
 You'll need to annotate your service as follows if you want an external IP:
 
 ```yaml
+# Dual Stack
 metadata:
   annotations:
     metallb.universe.tf/address-pool: "unifi-pool"
@@ -241,6 +224,15 @@ spec:
   ipFamilies:
   - IPv6
   - IPv4
+
+# Single Stack
+metadata:
+  annotations:
+    metallb.universe.tf/address-pool: "unifi-pool"
+spec:
+  ipFamilyPolicy: PreferDualStack
+  ipFamilies:
+  - IPv4
 ```
 
 Then test with
@@ -259,24 +251,27 @@ kubectl apply -f active/systemd_k3s/tests/metallb-test.yaml
 
 ```bash
 aws iam create-user --user-name "externaldns"
-aws iam attach-user-policy --user-name "externaldns" --policy-arn arn:aws:iam::892236928704:policy/update-reeseapps
 aws iam attach-user-policy --user-name "externaldns" --policy-arn arn:aws:iam::892236928704:policy/update-reeselink
 
+# [OPTIONAL] Delete old access keys if you have too many
+aws iam delete-access-key --user-name externaldns --access-key-id
+
 GENERATED_ACCESS_KEY=$(aws iam create-access-key --user-name "externaldns")
 ACCESS_KEY_ID=$(echo $GENERATED_ACCESS_KEY | jq -r '.AccessKey.AccessKeyId')
 SECRET_ACCESS_KEY=$(echo $GENERATED_ACCESS_KEY | jq -r '.AccessKey.SecretAccessKey')
 
-cat <<-EOF > secrets/externaldns-credentials
-
+cat <<-EOF > active/kubernetes_external-dns/secrets/externaldns-credentials
 [default]
 aws_access_key_id = $ACCESS_KEY_ID
 aws_secret_access_key = $SECRET_ACCESS_KEY
 EOF
 
 kubectl create secret generic external-dns \
-  --namespace kube-system --from-file secrets/externaldns-credentials
+--namespace kube-system \
+--from-file active/kubernetes_external-dns/secrets/externaldns-credentials
 
 helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/
+helm repo update
 helm upgrade --install external-dns external-dns/external-dns \
 --values active/kubernetes_external-dns/values.yaml \
 --namespace kube-system
@@ -290,22 +285,6 @@ metadata:
     external-dns.alpha.kubernetes.io/hostname: example.com
 ```
 
-## Nginx Ingress
-
-Now we need an ingress solution (preferably with certs for https). We'll be using nginx since
-it's a little bit more configurable than traefik (though don't sell traefik short, it's really
-good. Just finnicky when you have use cases they haven't explicitly coded for).
-
-```bash
-helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
-helm repo update
-helm upgrade --install \
-    ingress-nginx \
-    ingress-nginx/ingress-nginx \
-    --values active/kubernetes_ingress-nginx/values.yaml \
-    --namespace kube-system
-```
-
 ## Cert Manager
 
 Install cert-manager
@@ -377,6 +356,60 @@ kubectl apply -f active/infrastructure_k3s/tests/ingress-nginx-test.yaml
 kubectl delete -f active/infrastructure_k3s/tests/ingress-nginx-test.yaml
 ```
 
+## Traefik Gateway
+
+We'll use traefik gateway to provide ingress.
+
+```bash
+# Add the repo
+helm repo add traefik https://traefik.github.io/charts
+helm repo update
+
+# Create the traefik namespace
+kubectl create namespace traefik
+
+# Generate a self‑signed certificate valid for *.reeselink.com
+openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
+-keyout active/kubernetes_traefik/secrets/tls.key -out active/kubernetes_traefik/secrets/tls.crt \
+-subj "/CN=*.reeselink.com"
+
+# Create the TLS secret in the traefik namespace
+kubectl create secret tls local-selfsigned-tls \
+--cert=active/kubernetes_traefik/secrets/tls.crt --key=active/kubernetes_traefik/secrets/tls.key \
+--namespace traefik
+
+# Install the chart into the 'traefik' namespace
+helm install traefik traefik/traefik \
+--namespace traefik \
+--values active/kubernetes_traefik/values.yaml
+
+# Deploy a demo
+kubectl apply -f active/kubernetes_traefik/demo-app.yaml
+kubectl apply -f active/kubernetes_traefik/demo-route.yaml
+```
+
+## Longhorn Storage
+
+Longhorn provides replicated block storage via raw files on the nodes.
+
+```bash
+helm repo add longhorn https://charts.longhorn.io
+helm repo update
+helm upgrade --install longhorn longhorn/longhorn \
+--namespace longhorn-system \
+--create-namespace \
+--set "httproute.enabled=true" \
+--set "httproute.parentRefs[0].name=traefik-gateway" \
+--set "httproute.parentRefs[0].namespace=traefik" \
+--set "httproute.hostnames[0]=longhorn.reeselink.com"
+
+# Check that the route was created
+kubectl get httproute longhorn-httproute -n longhorn-system -o jsonpath='{.status.parents[*].conditions}'
+
+# Create a demo app to test storage
+kubectl apply -f active/kubernetes_longhorn/demo-app.yaml
+```
+
 ## Test Minecraft Server
 
 ```bash

ansible/inventory.yaml

@@ -10,6 +10,9 @@ fedora:
     borg-root:
     elk:
     toybox-root:
+    kube1-root:
+    kube2-root:
+    kube3-root:
 
 hardware:
   hosts:

presentations/local_ai_discord_bots.md

@@ -0,0 +1,45 @@
+# Bio
+
+Reese is DIY technology enthusiast with a passion for projects that make things
+easy. He's been working in development since 2017 with experience in risk,
+compliance, scripting automation, full stack web development, container
+infrastructure, homelab server hardware, ESP Home and home automation. Reese
+has a passion for mentoring, but even more of a passion for sharing the new
+tech he found last week with anyone who will listen. Reese wants tech to be
+fun and approachable for anyone at any skill level.
+
+## Credentials
+
+Reese has spoken at multiple company conferences about building websites and
+automating with Python. He's taught multi-day intro-to-python classes both
+online and in person. He has 8 years of industry experience, 3 of which have
+been spent growing the development team at a Nimbis Services. Reese has,
+professionally and personally, written and distributed Python pip packages,
+designed and hosted websites, built and deployed a version control system, led
+AI development teams, taught Python classes, mentored high school students in
+tech, annoyed his friends with discord bots, and automated his bathroom fans.
+He's accustomed to speaking in front of large and small audiences and relishes
+the opportunity to share his excitement with a crowd.
+
+## Abstract
+
+This talk will walk through the process of putting your local LLM to good* use.
+Through the medium of a Discord bot, we will explore how to leverage llama.cpp
+to give your friends the ability to create custom bots with custom
+personalities, have those personalities talk with each other, generate images,
+edit images, and set yourself up to leverage tool calling so your bots can
+interact with the real world.
+
+We will cover the state of hosting offline LLMs and discuss some strategies for
+hosting them safely with Podman, Bifrost, and Caddy. We will also discuss the
+current state of LLM hardware and give some realistic examples with AMD, Intel,
+Nvidia, and CPU based solutions. We will not be using cloud examples, as this
+talk will focus on avoiding cloud solutions in general. We will poke fun at
+leveraging Discord as our example if our goal is to self-host.
+
+Ultimately, I want this talk's participants to leave with some functional code
+and good ideas to get them thinking about ways they can integrate LLMs into
+their communities while maintaining control and privacy (and avoiding a hefty
+bill). This talk will emphasize audience participation to generate ideas for a
+prebuilt demo of the custom bot service, but will not build anything live
+during the presentation.

presentations/self_hosting_presentation.md

@@ -0,0 +1,109 @@
+# Idea
+
+## Abstract
+
+Ever find yourself Googling "how do I build a home server" only to get
+overwhelmed by enterprise-grade documentation? Welcome to our journey from
+confused beginner to building a home server that's actually useful.
+
+Join us as we walk through real-world projects that solved actual problems:
+backing up family photos, hosting private Git repositories, running local AI
+models, managing home media, and yes—even running multiple Minecraft servers on
+a single box. We'll explore hardware choices, operating systems,
+containerization vs VMs, and the pain points that motivated each decision.
+
+This isn't a theory-heavy presentation—it's a story-driven exploration of
+building infrastructure for real people with real needs. When you're done,
+you'll leave with a roadmap for your own server that balances automation,
+redundancy, and the "I just want this to work" factor.
+
+Some prior sysadmin knowledge required. All projects are from personal
+experience, with stories about what went wrong and how we fixed it.
+
+## Structure
+
+"I'm lazy, I don't want my family to kill me, and it works"
+
+1. I want this in my house
+2. I want to connect outside my house
+3. I want my friends to connect
+4. I don't want this to go down
+5. I want to recover if there's an error
+6. My house burned down, what now?
+
+## Thoughts
+
+Give 2 ideas per section. First for "I can't let this break my family will kill
+me". Second for "I have an understanding partner who is my cat and won't care."
+
+Story driven presentation
+
+I have decided to make a strong home server. Where do I even start?
+
+Hardware: you find a box (old laptop, rpi) you're set.
+
+- Operating system (proxmox, truenas, fedora, arch linux)
+  - Alex: truenas apps
+  - Reese: Fedora, osbuild
+
+1. Install native app (npm, pip, apt, dnf, etc)
+2. Containerized (kube, docker, podman)
+3. VM (vm, pick one or two)
+
+- Ingress (nginx, caddy, haproxy)
+- Backups (rsync, borg, btrfs send, zfs send)
+
+1. I want to install a new app while I'm at friend's house
+   1. Truenas web portal (app page, both official and community)
+   2. VPN and I need access to my computer
+2. I want to check my server status on my phone (updates, disks, memory pressure, error logs, services running)
+   1. Truenas web interface
+   2. Cockpit web interface
+3. I want to add more storage
+   1. Truenas ZFS storage pools
+   2. BTRFS pools
+4. I want to install a new alpha app without much support
+   1. Truenas custom docker compose images
+   2. Fedora clone and run (in a VM for style)
+5. I want to backup my photos
+   1. Google Photos: don't use git, images aren't meant for git
+   2. **Immich, with backups (tell stories about losing my image data)**
+6. I want a local copy of my code
+   1. Github
+   2. Gitea/Gitlab (talk about that transition)
+7. I want private document editing
+   1. Google drive, Obsidian (forces use of markdown as my standard)
+   2. VSCode + pandoc (commit markdown files as your documents)
+   3. Nextcloud (Collabora)
+8. I want a local, offline LLMs
+   1. llama.cpp, stable diffusion cpp, bifrost
+   2. Ollama is switching to cloud based models
+9. I want to watch media I own
+    1. Plex boi - I know that ruffles some jimmies. Give example: add letterbox support into Plex.
+    2. Jellyfin if you're cheap
+10. I want to know when something goes wrong
+    1. Uptime Kuma!
+    2. Truenas sending emails if there's an error
+    3. Fedora requires a custom solution.
+11. I want "reasonable availability"
+    1. Truenas hits 90%+ availability. Updates take it down for reboot (5-10
+       minutes). Disk failure requires full shutdown, disk swap, and rebuild.
+       This could be half a day.
+    2. Fedora hits 90%+ availability. Updates take it down for reboot (<1 min).
+       Disk failures can be ignored by rebalancing. Disk failures still require
+       full shutdown and resilver. This can take half a day.
+12. I want to host multiple minecraft servers (SRV records)
+    1. AWS Route53 for automating SRV records.
+    2. Pihole is in the territory of making your family mad
+13. I want to automate my house
+    1. Home Assistant (raspberry pi or green)
+14. I want backups of all my data
+    1. No backups is an option
+    2. Local weekly backups to usb drives via Truenas data replication
+    3. Borg backup via CLI or Pika.
+    4. Full disk backups, app directory backups, hybrid model
+    5. Backblaze and S3 integration for Truenas
+    6. 3 copies of your data, 2 different media,1 off site.
+15. I want a private VPN
+    1. Tailscale, moved from wifiman, also moved from pivpn
+    2. unifi wireguard server, rawdog wireguard on a pi