finalize for kube class
All checks were successful
Podman DDNS Image / build-and-push-ddns (push) Successful in 1m14s
All checks were successful
Podman DDNS Image / build-and-push-ddns (push) Successful in 1m14s
This commit is contained in:
@@ -1,7 +1,6 @@
|
|||||||
# K3S
|
# K3S
|
||||||
|
|
||||||
- [K3S](#k3s)
|
- [K3S](#k3s)
|
||||||
- [Guide](#guide)
|
|
||||||
- [Firewalld](#firewalld)
|
- [Firewalld](#firewalld)
|
||||||
- [SELinux](#selinux)
|
- [SELinux](#selinux)
|
||||||
- [Install Single Node K3S](#install-single-node-k3s)
|
- [Install Single Node K3S](#install-single-node-k3s)
|
||||||
@@ -16,24 +15,14 @@
|
|||||||
- [External DNS](#external-dns)
|
- [External DNS](#external-dns)
|
||||||
- [Credentials](#credentials)
|
- [Credentials](#credentials)
|
||||||
- [Annotation](#annotation)
|
- [Annotation](#annotation)
|
||||||
- [Nginx Ingress](#nginx-ingress)
|
|
||||||
- [Cert Manager](#cert-manager)
|
- [Cert Manager](#cert-manager)
|
||||||
|
- [Traefik Gateway](#traefik-gateway)
|
||||||
|
- [Longhorn Storage](#longhorn-storage)
|
||||||
- [Test Minecraft Server](#test-minecraft-server)
|
- [Test Minecraft Server](#test-minecraft-server)
|
||||||
- [Automatic Updates](#automatic-updates)
|
- [Automatic Updates](#automatic-updates)
|
||||||
- [Database Backups](#database-backups)
|
- [Database Backups](#database-backups)
|
||||||
- [Uninstall](#uninstall)
|
- [Uninstall](#uninstall)
|
||||||
|
|
||||||
## Guide
|
|
||||||
|
|
||||||
1. Configure Host
|
|
||||||
2. Install CoreDNS for inter-container discovery
|
|
||||||
3. Install Metal LB for load balancer IP address assignment
|
|
||||||
4. install External DNS for laod balancer IP and ingress DNS records
|
|
||||||
5. Install Nginx Ingress for http services
|
|
||||||
6. Install Cert Manager for automatic Let's Encrypt certificates for Ingress nginx
|
|
||||||
7. Install longhorn storage for automatic PVC creation and management
|
|
||||||
8. Set up automatic database backups
|
|
||||||
|
|
||||||
## Firewalld
|
## Firewalld
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -104,8 +93,12 @@ curl -sfL https://get.k3s.io | sh -s - \
|
|||||||
"traefik" \
|
"traefik" \
|
||||||
"--disable" \
|
"--disable" \
|
||||||
"servicelb" \
|
"servicelb" \
|
||||||
"--tls-san" \
|
"--disable" \
|
||||||
"k3s.reeselink.com" \
|
"local-storage" \
|
||||||
|
"--cluster-cidr" \
|
||||||
|
"10.42.0.0/16" \
|
||||||
|
"--service-cidr" \
|
||||||
|
"10.43.0.0/16" \
|
||||||
--selinux
|
--selinux
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -127,6 +120,8 @@ curl -sfL https://get.k3s.io | K3S_TOKEN=$(cat ~/.k3s-token) sh -s - \
|
|||||||
"traefik" \
|
"traefik" \
|
||||||
"--disable" \
|
"--disable" \
|
||||||
"servicelb" \
|
"servicelb" \
|
||||||
|
"--disable" \
|
||||||
|
"local-storage" \
|
||||||
"--cluster-cidr" \
|
"--cluster-cidr" \
|
||||||
"10.42.0.0/16" \
|
"10.42.0.0/16" \
|
||||||
"--service-cidr" \
|
"--service-cidr" \
|
||||||
@@ -142,6 +137,8 @@ curl -sfL https://get.k3s.io | K3S_TOKEN=$(cat ~/.k3s-token) sh -s - \
|
|||||||
"traefik" \
|
"traefik" \
|
||||||
"--disable" \
|
"--disable" \
|
||||||
"servicelb" \
|
"servicelb" \
|
||||||
|
"--disable" \
|
||||||
|
"local-storage" \
|
||||||
"--cluster-cidr" \
|
"--cluster-cidr" \
|
||||||
"10.42.0.0/16" \
|
"10.42.0.0/16" \
|
||||||
"--service-cidr" \
|
"--service-cidr" \
|
||||||
@@ -176,10 +173,6 @@ export KUBECONFIG=~/.kube/admin-kube-config
|
|||||||
|
|
||||||
### VLAN Setup
|
### VLAN Setup
|
||||||
|
|
||||||
I would remove firewalld to get this working. VLAN IPv6 traffic doesn't work for some
|
|
||||||
reason and there aren't good docs yet. Your router firewall will suffice, just be sure
|
|
||||||
to configure those rules correctly.
|
|
||||||
|
|
||||||
Before working with Metallb you'll need at least one available VLAN. On Unifi equipment
|
Before working with Metallb you'll need at least one available VLAN. On Unifi equipment
|
||||||
this is accomplished by creating a new network. Don't assign it to anything.
|
this is accomplished by creating a new network. Don't assign it to anything.
|
||||||
|
|
||||||
@@ -219,20 +212,10 @@ address, effectively moving the IP to another node. This isn't really "load bala
|
|||||||
|
|
||||||
[Install MetalLB](/active/kubernetes_metallb/metallb.md)
|
[Install MetalLB](/active/kubernetes_metallb/metallb.md)
|
||||||
|
|
||||||
MetalLB doesn't know what IP addresses are available for it to allocate so
|
|
||||||
we'll have to provide it with a list. The
|
|
||||||
[metallb-addresspool.yaml](/active/kubernetes_metallb/addresspool.yaml) has
|
|
||||||
the configuration for our available pools. Note these should match the VLAN you
|
|
||||||
created above.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# create the metallb allocation pool
|
|
||||||
kubectl apply -f active/kubernetes_metallb/addresspool.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
You'll need to annotate your service as follows if you want an external IP:
|
You'll need to annotate your service as follows if you want an external IP:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
|
# Dual Stack
|
||||||
metadata:
|
metadata:
|
||||||
annotations:
|
annotations:
|
||||||
metallb.universe.tf/address-pool: "unifi-pool"
|
metallb.universe.tf/address-pool: "unifi-pool"
|
||||||
@@ -241,6 +224,15 @@ spec:
|
|||||||
ipFamilies:
|
ipFamilies:
|
||||||
- IPv6
|
- IPv6
|
||||||
- IPv4
|
- IPv4
|
||||||
|
|
||||||
|
# Single Stack
|
||||||
|
metadata:
|
||||||
|
annotations:
|
||||||
|
metallb.universe.tf/address-pool: "unifi-pool"
|
||||||
|
spec:
|
||||||
|
ipFamilyPolicy: PreferDualStack
|
||||||
|
ipFamilies:
|
||||||
|
- IPv4
|
||||||
```
|
```
|
||||||
|
|
||||||
Then test with
|
Then test with
|
||||||
@@ -259,24 +251,27 @@ kubectl apply -f active/systemd_k3s/tests/metallb-test.yaml
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
aws iam create-user --user-name "externaldns"
|
aws iam create-user --user-name "externaldns"
|
||||||
aws iam attach-user-policy --user-name "externaldns" --policy-arn arn:aws:iam::892236928704:policy/update-reeseapps
|
|
||||||
aws iam attach-user-policy --user-name "externaldns" --policy-arn arn:aws:iam::892236928704:policy/update-reeselink
|
aws iam attach-user-policy --user-name "externaldns" --policy-arn arn:aws:iam::892236928704:policy/update-reeselink
|
||||||
|
|
||||||
|
# [OPTIONAL] Delete old access keys if you have too many
|
||||||
|
aws iam delete-access-key --user-name externaldns --access-key-id
|
||||||
|
|
||||||
GENERATED_ACCESS_KEY=$(aws iam create-access-key --user-name "externaldns")
|
GENERATED_ACCESS_KEY=$(aws iam create-access-key --user-name "externaldns")
|
||||||
ACCESS_KEY_ID=$(echo $GENERATED_ACCESS_KEY | jq -r '.AccessKey.AccessKeyId')
|
ACCESS_KEY_ID=$(echo $GENERATED_ACCESS_KEY | jq -r '.AccessKey.AccessKeyId')
|
||||||
SECRET_ACCESS_KEY=$(echo $GENERATED_ACCESS_KEY | jq -r '.AccessKey.SecretAccessKey')
|
SECRET_ACCESS_KEY=$(echo $GENERATED_ACCESS_KEY | jq -r '.AccessKey.SecretAccessKey')
|
||||||
|
|
||||||
cat <<-EOF > secrets/externaldns-credentials
|
cat <<-EOF > active/kubernetes_external-dns/secrets/externaldns-credentials
|
||||||
|
|
||||||
[default]
|
[default]
|
||||||
aws_access_key_id = $ACCESS_KEY_ID
|
aws_access_key_id = $ACCESS_KEY_ID
|
||||||
aws_secret_access_key = $SECRET_ACCESS_KEY
|
aws_secret_access_key = $SECRET_ACCESS_KEY
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
kubectl create secret generic external-dns \
|
kubectl create secret generic external-dns \
|
||||||
--namespace kube-system --from-file secrets/externaldns-credentials
|
--namespace kube-system \
|
||||||
|
--from-file active/kubernetes_external-dns/secrets/externaldns-credentials
|
||||||
|
|
||||||
helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/
|
helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/
|
||||||
|
helm repo update
|
||||||
helm upgrade --install external-dns external-dns/external-dns \
|
helm upgrade --install external-dns external-dns/external-dns \
|
||||||
--values active/kubernetes_external-dns/values.yaml \
|
--values active/kubernetes_external-dns/values.yaml \
|
||||||
--namespace kube-system
|
--namespace kube-system
|
||||||
@@ -290,22 +285,6 @@ metadata:
|
|||||||
external-dns.alpha.kubernetes.io/hostname: example.com
|
external-dns.alpha.kubernetes.io/hostname: example.com
|
||||||
```
|
```
|
||||||
|
|
||||||
## Nginx Ingress
|
|
||||||
|
|
||||||
Now we need an ingress solution (preferably with certs for https). We'll be using nginx since
|
|
||||||
it's a little bit more configurable than traefik (though don't sell traefik short, it's really
|
|
||||||
good. Just finnicky when you have use cases they haven't explicitly coded for).
|
|
||||||
|
|
||||||
```bash
|
|
||||||
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
|
|
||||||
helm repo update
|
|
||||||
helm upgrade --install \
|
|
||||||
ingress-nginx \
|
|
||||||
ingress-nginx/ingress-nginx \
|
|
||||||
--values active/kubernetes_ingress-nginx/values.yaml \
|
|
||||||
--namespace kube-system
|
|
||||||
```
|
|
||||||
|
|
||||||
## Cert Manager
|
## Cert Manager
|
||||||
|
|
||||||
Install cert-manager
|
Install cert-manager
|
||||||
@@ -377,6 +356,60 @@ kubectl apply -f active/infrastructure_k3s/tests/ingress-nginx-test.yaml
|
|||||||
kubectl delete -f active/infrastructure_k3s/tests/ingress-nginx-test.yaml
|
kubectl delete -f active/infrastructure_k3s/tests/ingress-nginx-test.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Traefik Gateway
|
||||||
|
|
||||||
|
We'll use traefik gateway to provide ingress.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Add the repo
|
||||||
|
helm repo add traefik https://traefik.github.io/charts
|
||||||
|
helm repo update
|
||||||
|
|
||||||
|
# Create the traefik namespace
|
||||||
|
kubectl create namespace traefik
|
||||||
|
|
||||||
|
# Generate a self‑signed certificate valid for *.reeselink.com
|
||||||
|
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
|
||||||
|
-keyout active/kubernetes_traefik/secrets/tls.key -out active/kubernetes_traefik/secrets/tls.crt \
|
||||||
|
-subj "/CN=*.reeselink.com"
|
||||||
|
|
||||||
|
# Create the TLS secret in the traefik namespace
|
||||||
|
kubectl create secret tls local-selfsigned-tls \
|
||||||
|
--cert=active/kubernetes_traefik/secrets/tls.crt --key=active/kubernetes_traefik/secrets/tls.key \
|
||||||
|
--namespace traefik
|
||||||
|
|
||||||
|
# Install the chart into the 'traefik' namespace
|
||||||
|
helm install traefik traefik/traefik \
|
||||||
|
--namespace traefik \
|
||||||
|
--values active/kubernetes_traefik/values.yaml
|
||||||
|
|
||||||
|
# Deploy a demo
|
||||||
|
kubectl apply -f active/kubernetes_traefik/demo-app.yaml
|
||||||
|
kubectl apply -f active/kubernetes_traefik/demo-route.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Longhorn Storage
|
||||||
|
|
||||||
|
Longhorn provides replicated block storage via raw files on the nodes.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
helm repo add longhorn https://charts.longhorn.io
|
||||||
|
helm repo update
|
||||||
|
helm upgrade --install longhorn longhorn/longhorn \
|
||||||
|
--namespace longhorn-system \
|
||||||
|
--create-namespace \
|
||||||
|
--set "httproute.enabled=true" \
|
||||||
|
--set "httproute.parentRefs[0].name=traefik-gateway" \
|
||||||
|
--set "httproute.parentRefs[0].namespace=traefik" \
|
||||||
|
--set "httproute.hostnames[0]=longhorn.reeselink.com"
|
||||||
|
|
||||||
|
# Check that the route was created
|
||||||
|
kubectl get httproute longhorn-httproute -n longhorn-system -o jsonpath='{.status.parents[*].conditions}'
|
||||||
|
|
||||||
|
# Create a demo app to test storage
|
||||||
|
kubectl apply -f active/kubernetes_longhorn/demo-app.yaml
|
||||||
|
```
|
||||||
|
|
||||||
## Test Minecraft Server
|
## Test Minecraft Server
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
Reference in New Issue
Block a user