diff --git a/active/software_k3s/k3s.md b/active/software_k3s/k3s.md index 9f9854a..5d11957 100644 --- a/active/software_k3s/k3s.md +++ b/active/software_k3s/k3s.md @@ -1,7 +1,6 @@ # K3S - [K3S](#k3s) - - [Guide](#guide) - [Firewalld](#firewalld) - [SELinux](#selinux) - [Install Single Node K3S](#install-single-node-k3s) @@ -16,24 +15,14 @@ - [External DNS](#external-dns) - [Credentials](#credentials) - [Annotation](#annotation) - - [Nginx Ingress](#nginx-ingress) - [Cert Manager](#cert-manager) + - [Traefik Gateway](#traefik-gateway) + - [Longhorn Storage](#longhorn-storage) - [Test Minecraft Server](#test-minecraft-server) - [Automatic Updates](#automatic-updates) - [Database Backups](#database-backups) - [Uninstall](#uninstall) -## Guide - -1. Configure Host -2. Install CoreDNS for inter-container discovery -3. Install Metal LB for load balancer IP address assignment -4. install External DNS for laod balancer IP and ingress DNS records -5. Install Nginx Ingress for http services -6. Install Cert Manager for automatic Let's Encrypt certificates for Ingress nginx -7. Install longhorn storage for automatic PVC creation and management -8. Set up automatic database backups - ## Firewalld ```bash @@ -104,8 +93,12 @@ curl -sfL https://get.k3s.io | sh -s - \ "traefik" \ "--disable" \ "servicelb" \ -"--tls-san" \ -"k3s.reeselink.com" \ +"--disable" \ +"local-storage" \ +"--cluster-cidr" \ +"10.42.0.0/16" \ +"--service-cidr" \ +"10.43.0.0/16" \ --selinux ``` @@ -127,6 +120,8 @@ curl -sfL https://get.k3s.io | K3S_TOKEN=$(cat ~/.k3s-token) sh -s - \ "traefik" \ "--disable" \ "servicelb" \ +"--disable" \ +"local-storage" \ "--cluster-cidr" \ "10.42.0.0/16" \ "--service-cidr" \ @@ -142,6 +137,8 @@ curl -sfL https://get.k3s.io | K3S_TOKEN=$(cat ~/.k3s-token) sh -s - \ "traefik" \ "--disable" \ "servicelb" \ +"--disable" \ +"local-storage" \ "--cluster-cidr" \ "10.42.0.0/16" \ "--service-cidr" \ @@ -176,10 +173,6 @@ export KUBECONFIG=~/.kube/admin-kube-config ### VLAN Setup -I would remove firewalld to get this working. VLAN IPv6 traffic doesn't work for some -reason and there aren't good docs yet. Your router firewall will suffice, just be sure -to configure those rules correctly. - Before working with Metallb you'll need at least one available VLAN. On Unifi equipment this is accomplished by creating a new network. Don't assign it to anything. @@ -219,20 +212,10 @@ address, effectively moving the IP to another node. This isn't really "load bala [Install MetalLB](/active/kubernetes_metallb/metallb.md) -MetalLB doesn't know what IP addresses are available for it to allocate so -we'll have to provide it with a list. The -[metallb-addresspool.yaml](/active/kubernetes_metallb/addresspool.yaml) has -the configuration for our available pools. Note these should match the VLAN you -created above. - -```bash -# create the metallb allocation pool -kubectl apply -f active/kubernetes_metallb/addresspool.yaml -``` - You'll need to annotate your service as follows if you want an external IP: ```yaml +# Dual Stack metadata: annotations: metallb.universe.tf/address-pool: "unifi-pool" @@ -241,6 +224,15 @@ spec: ipFamilies: - IPv6 - IPv4 + +# Single Stack +metadata: + annotations: + metallb.universe.tf/address-pool: "unifi-pool" +spec: + ipFamilyPolicy: PreferDualStack + ipFamilies: + - IPv4 ``` Then test with @@ -259,24 +251,27 @@ kubectl apply -f active/systemd_k3s/tests/metallb-test.yaml ```bash aws iam create-user --user-name "externaldns" -aws iam attach-user-policy --user-name "externaldns" --policy-arn arn:aws:iam::892236928704:policy/update-reeseapps aws iam attach-user-policy --user-name "externaldns" --policy-arn arn:aws:iam::892236928704:policy/update-reeselink +# [OPTIONAL] Delete old access keys if you have too many +aws iam delete-access-key --user-name externaldns --access-key-id + GENERATED_ACCESS_KEY=$(aws iam create-access-key --user-name "externaldns") ACCESS_KEY_ID=$(echo $GENERATED_ACCESS_KEY | jq -r '.AccessKey.AccessKeyId') SECRET_ACCESS_KEY=$(echo $GENERATED_ACCESS_KEY | jq -r '.AccessKey.SecretAccessKey') -cat <<-EOF > secrets/externaldns-credentials - +cat <<-EOF > active/kubernetes_external-dns/secrets/externaldns-credentials [default] aws_access_key_id = $ACCESS_KEY_ID aws_secret_access_key = $SECRET_ACCESS_KEY EOF kubectl create secret generic external-dns \ - --namespace kube-system --from-file secrets/externaldns-credentials +--namespace kube-system \ +--from-file active/kubernetes_external-dns/secrets/externaldns-credentials helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/ +helm repo update helm upgrade --install external-dns external-dns/external-dns \ --values active/kubernetes_external-dns/values.yaml \ --namespace kube-system @@ -290,22 +285,6 @@ metadata: external-dns.alpha.kubernetes.io/hostname: example.com ``` -## Nginx Ingress - -Now we need an ingress solution (preferably with certs for https). We'll be using nginx since -it's a little bit more configurable than traefik (though don't sell traefik short, it's really -good. Just finnicky when you have use cases they haven't explicitly coded for). - -```bash -helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx -helm repo update -helm upgrade --install \ - ingress-nginx \ - ingress-nginx/ingress-nginx \ - --values active/kubernetes_ingress-nginx/values.yaml \ - --namespace kube-system -``` - ## Cert Manager Install cert-manager @@ -377,6 +356,60 @@ kubectl apply -f active/infrastructure_k3s/tests/ingress-nginx-test.yaml kubectl delete -f active/infrastructure_k3s/tests/ingress-nginx-test.yaml ``` +## Traefik Gateway + +We'll use traefik gateway to provide ingress. + +```bash +# Add the repo +helm repo add traefik https://traefik.github.io/charts +helm repo update + +# Create the traefik namespace +kubectl create namespace traefik + +# Generate a self‑signed certificate valid for *.reeselink.com +openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \ +-keyout active/kubernetes_traefik/secrets/tls.key -out active/kubernetes_traefik/secrets/tls.crt \ +-subj "/CN=*.reeselink.com" + +# Create the TLS secret in the traefik namespace +kubectl create secret tls local-selfsigned-tls \ +--cert=active/kubernetes_traefik/secrets/tls.crt --key=active/kubernetes_traefik/secrets/tls.key \ +--namespace traefik + +# Install the chart into the 'traefik' namespace +helm install traefik traefik/traefik \ +--namespace traefik \ +--values active/kubernetes_traefik/values.yaml + +# Deploy a demo +kubectl apply -f active/kubernetes_traefik/demo-app.yaml +kubectl apply -f active/kubernetes_traefik/demo-route.yaml +``` + +## Longhorn Storage + +Longhorn provides replicated block storage via raw files on the nodes. + +```bash +helm repo add longhorn https://charts.longhorn.io +helm repo update +helm upgrade --install longhorn longhorn/longhorn \ +--namespace longhorn-system \ +--create-namespace \ +--set "httproute.enabled=true" \ +--set "httproute.parentRefs[0].name=traefik-gateway" \ +--set "httproute.parentRefs[0].namespace=traefik" \ +--set "httproute.hostnames[0]=longhorn.reeselink.com" + +# Check that the route was created +kubectl get httproute longhorn-httproute -n longhorn-system -o jsonpath='{.status.parents[*].conditions}' + +# Create a demo app to test storage +kubectl apply -f active/kubernetes_longhorn/demo-app.yaml +``` + ## Test Minecraft Server ```bash