From 60eb1e26a0afde8f0913b8ec461d35e401cce50d Mon Sep 17 00:00:00 2001 From: ducoterra Date: Sun, 21 Jul 2024 10:37:55 -0400 Subject: [PATCH] distribute READMEs --- README.md | 187 +--------------------- ansible/README.md | 4 + cloud/incubating/aws_ses/README.md | 1 + hardware/graduated/shelly/README.md | 6 + infrastructure/graduated/k3s/README.md | 87 ++-------- kubernetes/graduated/gitea/README.md | 69 ++++++++ kubernetes/graduated/iperf3/README.md | 13 ++ kubernetes/graduated/jellyfin/README.md | 11 ++ kubernetes/graduated/minecraft/README.md | 36 +++++ kubernetes/graduated/snapdrop/README.md | 13 ++ kubernetes/incubating/grafana/README.md | 4 +- kubernetes/incubating/wordpress/README.md | 18 +++ systemd/graduated/ddns/README.md | 2 +- systemd/graduated/ipv4-proxy/README.md | 2 +- systemd/graduated/unifi-server/README.md | 4 + 15 files changed, 196 insertions(+), 261 deletions(-) create mode 100644 cloud/incubating/aws_ses/README.md create mode 100644 kubernetes/graduated/gitea/README.md create mode 100644 kubernetes/graduated/iperf3/README.md create mode 100644 kubernetes/graduated/jellyfin/README.md create mode 100644 kubernetes/graduated/minecraft/README.md create mode 100644 kubernetes/graduated/snapdrop/README.md create mode 100644 kubernetes/incubating/wordpress/README.md diff --git a/README.md b/README.md index 42a369e..7276395 100644 --- a/README.md +++ b/README.md @@ -10,19 +10,6 @@ A project to store homelab stuff. - [Supported Projects](#supported-projects) - [Graduation Requirements](#graduation-requirements) - [Retirement Requirements](#retirement-requirements) - - [Apps (TODO: Move these into their respective folders)](#apps-todo-move-these-into-their-respective-folders) - - [Gitea](#gitea) - - [Staging](#staging) - - [Install](#install) - - [Minecraft](#minecraft) - - [Testing](#testing) - - [Nimcraft](#nimcraft) - - [Courtnie](#courtnie) - - [Snapdrop](#snapdrop) - - [Jellyfin](#jellyfin) - - [Iperf3](#iperf3) - - [Wordpress](#wordpress) - - [Grafana](#grafana) ## Project Lifecycle @@ -55,7 +42,9 @@ All projects will fall into one of the following categories: - kubernetes Hardware will contain projects that relate to specific machines or equipment. 3D printers, Raspberry -Pis, and other IOT devices qualify as specialized hardware that needs documentation and configuration. +Pis, and other IOT devices qualify as specialized hardware that needs documentation and +configuration. This is not limited to computer equipment. The furnace is an important part of the +home lab. the Air Conditioner is integral to the homelab's function. These projects will also be documented. Infrastructure will contain projects that set up the environments for the remaining listed project types. For example, infrastructure will contain "how to set up a linux box with docker" or "how to @@ -87,173 +76,3 @@ Kubernetes projects are helm, kustomize, kubectl, or some other kubernetes compl - [ ] A reason for retirement is documented - [ ] If applicable, a replacement has been identified and documented - [ ] If applicable, backup data locations are documented - -## Apps (TODO: Move these into their respective folders) - -### Gitea - -Gitea provides a helm chart [here](https://gitea.com/gitea/helm-chart/). We're not -going to modify much, but we are going to solidify some of the default values in case -they decide to change things. This is the first chart (besides ingress-nginx) where -we need to pay attention to the MetalLB annotation. This has been set in the values.yaml -file. - -#### Staging - -There is a `gitea-staging.yaml` file with staging values. This should be installed in -the `gitea-staging` namespace. Follow the instructions below, but replace the `gitea` -namespace with `gitea-staging`. Staging is useful for testing major release upgrades, -especially since Gitea tends to change how `values.yaml` is structured. - -#### Install - -First we need to create the gitea admin secret - -```bash -kubectl create namespace gitea -kubectl create secret generic gitea-admin-secret \ - -n gitea \ - --from-literal=username='gitea-admin' \ - --from-literal=password="$(pwgen -c -s 64 | head -n 1)" \ - --from-literal=email='' -``` - -```bash -helm repo add gitea-charts https://dl.gitea.io/charts/ -helm repo update -helm upgrade --install \ - gitea \ - gitea-charts/gitea \ - --values gitea/gitea-values.yaml \ - --namespace gitea \ - --create-namespace -``` - -If you need to backup your database you can run: - -```bash -# Backup -kubectl exec -it -n gitea gitea-postgresql-0 -- \ - pg_dump \ - --no-owner \ - --dbname=postgresql://gitea:gitea@localhost:5432 > gitea_backup.db - -# Take gitea down to zero pods -kubectl scale statefulset gitea --replicas 0 - -# Drop the existing database -kubectl exec -it -n gitea gitea-postgresql-0 -- psql -U gitea - -\c postgres; -drop database gitea; -CREATE DATABASE gitea WITH OWNER gitea TEMPLATE template0 ENCODING UTF8 LC_COLLATE 'en_US.UTF-8' LC_CTYPE 'en_US.UTF-8'; -exit - -# restore from backup -kubectl exec -it -n gitea gitea-postgresql-0 -- \ - psql \ - postgresql://gitea:gitea@localhost:5432 gitea < gitea_backup.db - -# Restore gitea to 1 pod -kubectl scale statefulset gitea --replicas 1 -``` - -### Minecraft - -Minecraft is available through the custom helm chart (including a server downloader). The example -below installs nimcraft. For each installation you'll want to create your own values.yaml -with a new port. The server-downloader is called "minecraft_get_server" and is available on -[Github](https://github.com/ducoterra/minecraft_get_server). - -#### Testing - -```bash -helm upgrade --install \ - testcraft \ - ./minecraft \ - --namespace minecraft \ - --create-namespace -``` - -#### Nimcraft - -```bash -helm upgrade --install \ - nimcraft \ - ./minecraft \ - --namespace minecraft \ - --create-namespace -``` - -#### Courtnie - -```bash -helm upgrade --install \ - courtniecraft \ - ./minecraft \ - --namespace minecraft \ - --create-namespace -``` - -### Snapdrop - -Snapdrop is a file sharing app that allows airdrop-like functionality over the web - -```bash -helm upgrade --install \ - snapdrop \ - ./snapdrop \ - --namespace snapdrop \ - --create-namespace -``` - -### Jellyfin - -This assumes you have a media NFS share. - -```bash -helm upgrade --install \ - jellyfin \ - ./jellyfin \ - --namespace jellyfin \ - --create-namespace -``` - -### Iperf3 - -This creates a basic iperf3 server. - -```bash -helm upgrade --install \ - iperf3 \ - ./iperf3 \ - --namespace iperf3 \ - --create-namespace -``` - -### Wordpress - -The bitnami wordpress chart allows enough customization to -work on a custom K3S server. With some tweaks it's quite -good. Use the values in `bitnami/wordpress.yaml` as a starting -point. - -```bash -helm upgrade --install \ - wordpress \ - -f bitnami/wordpress.yaml \ - --set wordpressUsername=admin \ - --set wordpressPassword=password \ - --set mariadb.auth.rootPassword=secretpassword \ - oci://registry-1.docker.io/bitnamicharts/wordpress -``` - -### Grafana - - - -Grafana has a kubernetes yaml they prefer you use. See `kubectl/grafana.yaml`. - -```bash -kubectl apply -f kubectl/grafana.yaml -``` diff --git a/ansible/README.md b/ansible/README.md index 8bbaf0d..4a52fac 100644 --- a/ansible/README.md +++ b/ansible/README.md @@ -1,3 +1,7 @@ +# Ansible Inventory + +This project lives outside the typical project structures as a means of tracking inventory. + ## Install ```bash diff --git a/cloud/incubating/aws_ses/README.md b/cloud/incubating/aws_ses/README.md new file mode 100644 index 0000000..b457461 --- /dev/null +++ b/cloud/incubating/aws_ses/README.md @@ -0,0 +1 @@ +# AWS SES \ No newline at end of file diff --git a/hardware/graduated/shelly/README.md b/hardware/graduated/shelly/README.md index b11c63f..dc0869e 100644 --- a/hardware/graduated/shelly/README.md +++ b/hardware/graduated/shelly/README.md @@ -10,3 +10,9 @@ 6. Set Device Name 7. Enable Bluetooth Gateway 8. Update Firmware + +### Reset + +Resetting is super finnicky. You'll need to plug it in, press and hold the power button until the +red light flashes quickly (not slowly, that's a reboot). You'll probably have to do it multiple +times because they seem to reboot halfway through the reset process. \ No newline at end of file diff --git a/infrastructure/graduated/k3s/README.md b/infrastructure/graduated/k3s/README.md index be3eb52..82065ba 100644 --- a/infrastructure/graduated/k3s/README.md +++ b/infrastructure/graduated/k3s/README.md @@ -19,13 +19,6 @@ - [Test Minecraft Server](#test-minecraft-server) - [Automatic Updates](#automatic-updates) - [Database Backups](#database-backups) - - [Quickstart](#quickstart) - - [Help](#help) - - [Troubleshooting](#troubleshooting) - - [Deleting a stuck namespace](#deleting-a-stuck-namespace) - - [Fixing a bad volume](#fixing-a-bad-volume) - - [Mounting an ix-application volume from truenas](#mounting-an-ix-application-volume-from-truenas) - - [Mounting a volume](#mounting-a-volume) - [Uninstall](#uninstall) ## Guide @@ -104,24 +97,24 @@ scp kube:/etc/rancher/k3s/k3s.yaml ~/.kube/admin-kube-config ```bash # Download the updated template from github -kubectl kustomize "github.com/rancher/local-path-provisioner/deploy?ref=v0.0.28" > local-path-provisioner/local-path-storage.yaml +kubectl kustomize "github.com/rancher/local-path-provisioner/deploy?ref=v0.0.28" > kubernetes/incubating/local-path-provisioner/local-path-storage.yaml # Apply customizations (ssd/hdd storage, read write many support) -kubectl kustomize local-path-provisioner | kubectl apply -f - +kubectl kustomize kubernetes/incubating/local-path-provisioner/local-path-provisioner | kubectl apply -f - # Create test pod -kubectl apply -f k3s/tests/local-storage-test.yaml +kubectl apply -f infrastructure/graduated/k3s/tests/local-storage-test.yaml ``` ## Coredns -1. Edit `coredns/values.yaml` to ensure the forward nameserver is correct. +1. Edit `kubernetes/graduated/coredns/values.yaml` to ensure the forward nameserver is correct. ```bash # Install CoreDNS helm upgrade --install \ --namespace=kube-system \ - --values coredns/values.yaml \ + --values kubernetes/graduated/coredns/values.yaml \ coredns coredns/coredns # Test DNS works @@ -188,7 +181,7 @@ you can list them. We're only going to use one because we want to port forward f ```bash # create the metallb allocation pool -kubectl apply -f metallb/addresspool.yaml +kubectl apply -f kubernetes/graduated/metallb/addresspool.yaml ``` You'll need to annotate your service as follows if you want an external IP: @@ -231,9 +224,9 @@ EOF kubectl create secret generic external-dns \ --namespace kube-system --from-file secrets/externaldns-credentials -kubectl apply -f external-dns/sa.yaml +kubectl apply -f kubernetes/graduated/external-dns/sa.yaml -kubectl apply -f external-dns/deploy.yaml +kubectl apply -f kubernetes/graduated/external-dns/deploy.yaml ``` ### Annotation @@ -256,7 +249,7 @@ helm repo update helm upgrade --install \ ingress-nginx \ ingress-nginx/ingress-nginx \ - --values ingress-nginx/values.yaml \ + --values kubernetes/graduated/ingress-nginx/values.yaml \ --namespace kube-system ``` @@ -325,16 +318,16 @@ You can test if your ingress is working with: ```bash # Navigate to demo.reeseapps.com -kubectl apply -f k3s/tests/ingress-nginx-test.yaml +kubectl apply -f infrastructure/graduated/k3s/tests/ingress-nginx-test.yaml # Cleanup -kubectl delete -f k3s/tests/ingress-nginx-test.yaml +kubectl delete -f infrastructure/graduated/k3s/tests/ingress-nginx-test.yaml ``` ## Test Minecraft Server ```bash -helm upgrade --install minecraft ./minecraft -n minecraft --create-namespace +helm upgrade --install minecraft kubernetes/graduated/minecraft -n minecraft --create-namespace ``` ## Automatic Updates @@ -345,7 +338,7 @@ helm upgrade --install minecraft ./minecraft -n minecraft --create-namespace kubectl create namespace system-upgrade kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/system-upgrade-controller.yaml kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/crd.yaml -kubectl apply -f k3s/upgrade-plan.yaml +kubectl apply -f infrastructure/graduated/k3s/upgrade-plan.yaml # Check plan kubectl get plan -n system-upgrade @@ -369,60 +362,8 @@ k3s server \ --cluster-reset-restore-path=/var/lib/rancher/k3s/server/db/snapshots/on-demand-kube-1720459685 ``` -### Quickstart - -```bash -# Create certsigner pod for all other operations -./setup.sh - -# Create a user, use "admin" to create an admin user -./upsertuser.sh - -# Remove a user, their namespace, and their access -./removeuserspace -``` - -## Help - -### Troubleshooting - -#### Deleting a stuck namespace - -```bash -NAMESPACE=nginx -kubectl proxy & -kubectl get namespace $NAMESPACE -o json |jq '.spec = {"finalizers":[]}' >temp.json -curl -k -H "Content-Type: application/json" -X PUT --data-binary @temp.json 127.0.0.1:8001/api/v1/namespaces/$NAMESPACE/finalize -``` - -#### Fixing a bad volume - -```bash -xfs_repair -L /dev/sdg -``` - -#### Mounting an ix-application volume from truenas - -```bash -# set the mountpoint -zfs set mountpoint=/ix_pvc enc1/ix-applications/releases/gitea/volumes/pvc-40e27277-71e3-4469-88a3-a39f53435a8b - -#"unset" the mountpoint (back to legacy) -zfs set mountpoint=legacy enc1/ix-applications/releases/gitea/volumes/pvc-40e27277-71e3-4469-88a3-a39f53435a8b -``` - -#### Mounting a volume - -```bash -# mount -mount -t xfs /dev/zvol/enc0/dcsi/apps/pvc-d5090258-cf20-4f2e-a5cf-330ac00d0049 /mnt/dcsi_pvc - -# unmount -umount /mnt/dcsi_pvc -``` - ## Uninstall ```bash /usr/local/bin/k3s-uninstall.sh -``` \ No newline at end of file +``` diff --git a/kubernetes/graduated/gitea/README.md b/kubernetes/graduated/gitea/README.md new file mode 100644 index 0000000..9080299 --- /dev/null +++ b/kubernetes/graduated/gitea/README.md @@ -0,0 +1,69 @@ +# Gitea + +Gitea provides a helm chart [here](https://gitea.com/gitea/helm-chart/). We're not +going to modify much, but we are going to solidify some of the default values in case +they decide to change things. This is the first chart (besides ingress-nginx) where +we need to pay attention to the MetalLB annotation. This has been set in the values.yaml +file. + +## Staging + +There is a `gitea-staging.yaml` file with staging values. This should be installed in +the `gitea-staging` namespace. Follow the instructions below, but replace the `gitea` +namespace with `gitea-staging`. Staging is useful for testing major release upgrades, +especially since Gitea tends to change how `values.yaml` is structured. + +## Install + +First we need to create the gitea admin secret + +```bash +kubectl create namespace gitea +kubectl create secret generic gitea-admin-secret \ + -n gitea \ + --from-literal=username='gitea-admin' \ + --from-literal=password="$(pwgen -c -s 64 | head -n 1)" \ + --from-literal=email='' +``` + +```bash +helm repo add gitea-charts https://dl.gitea.io/charts/ +helm repo update +helm upgrade --install \ + gitea \ + gitea-charts/gitea \ + --values kubernetes/graduated/gitea/gitea-values.yaml \ + --namespace gitea \ + --create-namespace +``` + +## Backup and Restore + +If you need to backup your database you can run: + +```bash +# Backup +kubectl exec -it -n gitea gitea-postgresql-0 -- \ + pg_dump \ + --no-owner \ + --dbname=postgresql://gitea:gitea@localhost:5432 > gitea_backup.db + +# Take gitea down to zero pods +kubectl scale statefulset gitea --replicas 0 + +# Drop the existing database +kubectl exec -it -n gitea gitea-postgresql-0 -- psql -U gitea + +\c postgres; +drop database gitea; +CREATE DATABASE gitea WITH OWNER gitea TEMPLATE template0 ENCODING UTF8 LC_COLLATE 'en_US.UTF-8' LC_CTYPE 'en_US.UTF-8'; +exit + +# restore from backup +kubectl exec -it -n gitea gitea-postgresql-0 -- \ + psql \ + postgresql://gitea:gitea@localhost:5432 gitea < gitea_backup.db + +# Restore gitea to 1 pod +kubectl scale statefulset gitea --replicas 1 +``` diff --git a/kubernetes/graduated/iperf3/README.md b/kubernetes/graduated/iperf3/README.md new file mode 100644 index 0000000..f18fff4 --- /dev/null +++ b/kubernetes/graduated/iperf3/README.md @@ -0,0 +1,13 @@ +# Iperf3 + +This creates a basic iperf3 server. + +## Install + +```bash +helm upgrade --install \ + iperf3 \ + ./kubernetes/graduated/iperf3 \ + --namespace iperf3 \ + --create-namespace +``` \ No newline at end of file diff --git a/kubernetes/graduated/jellyfin/README.md b/kubernetes/graduated/jellyfin/README.md new file mode 100644 index 0000000..c381ec1 --- /dev/null +++ b/kubernetes/graduated/jellyfin/README.md @@ -0,0 +1,11 @@ +# Jellyfin + +## Install + +```bash +helm upgrade --install \ + jellyfin \ + ./kubernetes/graduated/jellyfin \ + --namespace jellyfin \ + --create-namespace +``` diff --git a/kubernetes/graduated/minecraft/README.md b/kubernetes/graduated/minecraft/README.md new file mode 100644 index 0000000..09efaa2 --- /dev/null +++ b/kubernetes/graduated/minecraft/README.md @@ -0,0 +1,36 @@ +# Minecraft + +Minecraft is available through the custom helm chart (including a server downloader). The example +below installs nimcraft. For each installation you'll want to create your own values.yaml +with a new port. The server-downloader is called "minecraft_get_server" and is available on +[Github](https://github.com/ducoterra/minecraft_get_server). + +## Testing + +```bash +helm upgrade --install \ + testcraft \ + ./kubernetes/graduated/minecraft \ + --namespace minecraft \ + --create-namespace +``` + +## Nimcraft + +```bash +helm upgrade --install \ + nimcraft \ + ./kubernetes/graduated/minecraft \ + --namespace minecraft \ + --create-namespace +``` + +## Courtnie + +```bash +helm upgrade --install \ + courtniecraft \ + ./kubernetes/graduated/minecraft \ + --namespace minecraft \ + --create-namespace +``` diff --git a/kubernetes/graduated/snapdrop/README.md b/kubernetes/graduated/snapdrop/README.md new file mode 100644 index 0000000..970ed4f --- /dev/null +++ b/kubernetes/graduated/snapdrop/README.md @@ -0,0 +1,13 @@ +# Snapdrop + +Snapdrop is a file sharing app that allows airdrop-like functionality over the web + +## Install + +```bash +helm upgrade --install \ + snapdrop \ + ./kubernetes/graduated/snapdrop \ + --namespace snapdrop \ + --create-namespace +``` diff --git a/kubernetes/incubating/grafana/README.md b/kubernetes/incubating/grafana/README.md index b965e56..e2db844 100644 --- a/kubernetes/incubating/grafana/README.md +++ b/kubernetes/incubating/grafana/README.md @@ -8,7 +8,7 @@ helm repo update helm upgrade --install my-grafana grafana/grafana \ --namespace monitoring \ --create-namespace \ - --values grafana/values.yaml + --values kubernetes/incubating/grafana/values.yaml kubectl get secret --namespace monitoring my-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo ``` @@ -25,5 +25,5 @@ helm upgrade --install \ kube-prometheus-stack \ prometheus-community/kube-prometheus-stack \ --namespace kube-system \ - --values grafana/helm-prom-stack-values.yaml + --values kubernetes/incubating/grafana/helm-prom-stack-values.yaml ``` diff --git a/kubernetes/incubating/wordpress/README.md b/kubernetes/incubating/wordpress/README.md new file mode 100644 index 0000000..47e0601 --- /dev/null +++ b/kubernetes/incubating/wordpress/README.md @@ -0,0 +1,18 @@ +# Wordpress + +The bitnami wordpress chart allows enough customization to +work on a custom K3S server. With some tweaks it's quite +good. Use the values in `bitnami/wordpress.yaml` as a starting +point. + +## Install + +```bash +helm upgrade --install \ + wordpress \ + -f kubernetes/incubating/bitnami/wordpress.yaml \ + --set wordpressUsername=admin \ + --set wordpressPassword=password \ + --set mariadb.auth.rootPassword=secretpassword \ + oci://registry-1.docker.io/bitnamicharts/wordpress +``` \ No newline at end of file diff --git a/systemd/graduated/ddns/README.md b/systemd/graduated/ddns/README.md index d167616..c79d6bf 100644 --- a/systemd/graduated/ddns/README.md +++ b/systemd/graduated/ddns/README.md @@ -10,5 +10,5 @@ records in route53. ```bash -ansible-playbook -i ansible/inventory.yaml ddns/install_ddns.yaml +ansible-playbook -i ansible/inventory.yaml systemd/graduated/ddns/install_ddns.yaml ``` diff --git a/systemd/graduated/ipv4-proxy/README.md b/systemd/graduated/ipv4-proxy/README.md index 384fd09..4814439 100644 --- a/systemd/graduated/ipv4-proxy/README.md +++ b/systemd/graduated/ipv4-proxy/README.md @@ -16,7 +16,7 @@ This project pairs with the ddns service. Set that up first! 4. Run the following ansible script: ```bash -ansible-playbook -i ansible/inventory.yaml ipv4-proxy/nginx.yaml +ansible-playbook -i ansible/inventory.yaml systemd/graduated/ipv4-proxy/nginx.yaml ``` ## Logging diff --git a/systemd/graduated/unifi-server/README.md b/systemd/graduated/unifi-server/README.md index 0c9e646..7b7c89f 100644 --- a/systemd/graduated/unifi-server/README.md +++ b/systemd/graduated/unifi-server/README.md @@ -2,6 +2,10 @@ +## Prereqs + +This runs best on Ubuntu 20.04. See `infrastructure/graduated/ubuntu`. + ## Install ```bash