diff --git a/.gitignore b/.gitignore index dc16064..c262f25 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ secrets/ venv/ +tmp/ \ No newline at end of file diff --git a/README.md b/README.md index 579f624..c9f70b3 100644 --- a/README.md +++ b/README.md @@ -7,15 +7,12 @@ A project to store homelab stuff. - [Homelab](#homelab) - [Table of Contents](#table-of-contents) - [Apps](#apps) - - [Dashboard](#dashboard) - - [Nextcloud](#nextcloud) - - [Test Deploy](#test-deploy) - [Gitea](#gitea) - [Staging](#staging) - [Install](#install) - [Minecraft](#minecraft) - - [Nimcraft](#nimcraft) - [Testing](#testing) + - [Nimcraft](#nimcraft) - [Courtnie](#courtnie) - [Snapdrop](#snapdrop) - [Jellyfin](#jellyfin) @@ -25,95 +22,6 @@ A project to store homelab stuff. ## Apps -### Dashboard - -The kubernetes dashboard isn't all that useful but it can sometimes give you a good -visual breakdown when things are going wrong. It's sometimes faster than running -`kubectl get` commands over and over. - -Create the dashboard and an admin user with: - -```bash -helm upgrade \ ---install \ ---namespace kubernetes-dashboard \ ---create-namespace \ -dashboard-user ./helm/dashboard-user - -kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml -``` - -Then login with the following: - -```bash -kubectl -n kubernetes-dashboard create token admin-user -kubectl proxy -``` - -### Nextcloud - -The first chart we'll deploy is nextcloud. This is a custom chart because Nextcloud -doesn't support helm installation natively (yet). There is a native Docker image and -really detailed installation instructions so we can pretty easily piece together what's -required. - -This image runs the nextcloud cron job automatically and creates random secrets for all -infrastructure - very helpful for a secure deployment, not very helpful for migrating -clusters. You'll want to export the secrets and save them in a secure location. - -```bash -helm upgrade --install \ - nextcloud \ - ./helm/nextcloud \ - --namespace nextcloud \ - --create-namespace -``` - -Need to add lots of files? Copy them to the user data dir and then run - -```bash -./occ files:scan --all -``` - -Set up SES with the following links: - - - -To upgrade you'll need to: - -1. Apply the new image in values.yaml -2. Exec into the container and run the following: - - ```bash - su -s /bin/bash www-data - ./occ upgrade - ./occ maintenance:mode --off - ``` - -See -for more information. - -#### Test Deploy - -You can create a test deployment with the following: - -```bash -helm upgrade --install nextcloud ./helm/nextcloud \ - --namespace nextcloud-test \ - --create-namespace \ - --set nextcloud.domain=nextcloud-test.reeseapps.com \ - --set nextcloud.html.storageClassName=zfs-nfs-enc1 \ - --set nextcloud.html.storage=8Gi \ - --set nextcloud.data.storageClassName=zfs-nfs-enc1 \ - --set nextcloud.data.storage=8Gi \ - --set postgres.storageClassName=zfs-nfs-enc1 \ - --set postgres.storage=8Gi \ - --set redis.storageClassName=zfs-nfs-enc1 \ - --set redis.storage=8Gi \ - --set show_passwords=true \ - --dry-run -``` - ### Gitea Gitea provides a helm chart [here](https://gitea.com/gitea/helm-chart/). We're not @@ -189,25 +97,24 @@ below installs nimcraft. For each installation you'll want to create your own va with a new port. The server-downloader is called "minecraft_get_server" and is available on [Github](https://github.com/ducoterra/minecraft_get_server). -#### Nimcraft - -```bash -helm upgrade --install \ - nimcraft \ - ./helm/minecraft \ - --namespace nimcraft \ - --create-namespace -``` - #### Testing ```bash helm upgrade --install \ testcraft \ - ./helm/minecraft \ - --namespace testcraft \ - --create-namespace \ - --set port=25566 + ./minecraft \ + --namespace minecraft \ + --create-namespace +``` + +#### Nimcraft + +```bash +helm upgrade --install \ + nimcraft \ + ./minecraft \ + --namespace minecraft \ + --create-namespace ``` #### Courtnie @@ -215,10 +122,9 @@ helm upgrade --install \ ```bash helm upgrade --install \ courtniecraft \ - ./helm/minecraft \ - --namespace courtniecraft \ - --create-namespace \ - --set port=25568 + ./minecraft \ + --namespace minecraft \ + --create-namespace ``` ### Snapdrop @@ -228,7 +134,7 @@ Snapdrop is a file sharing app that allows airdrop-like functionality over the w ```bash helm upgrade --install \ snapdrop \ - ./helm/snapdrop \ + ./snapdrop \ --namespace snapdrop \ --create-namespace ``` @@ -240,7 +146,7 @@ This assumes you have a media NFS share. ```bash helm upgrade --install \ jellyfin \ - ./helm/jellyfin \ + ./jellyfin \ --namespace jellyfin \ --create-namespace ``` @@ -252,7 +158,7 @@ This creates a basic iperf3 server. ```bash helm upgrade --install \ iperf3 \ - ./helm/iperf3 \ + ./iperf3 \ --namespace iperf3 \ --create-namespace ``` diff --git a/aws/README.md b/aws/README.md index ee86d7e..b432c77 100644 --- a/aws/README.md +++ b/aws/README.md @@ -1,6 +1,6 @@ # AWS Credentials -## Aws Policies +## Aws Certbot Route53 Policies Example Policy: diff --git a/dns/README.md b/dns/README.md index 66f4f59..f5fe7d2 100644 --- a/dns/README.md +++ b/dns/README.md @@ -2,10 +2,8 @@ - [Network Management](#network-management) - [Reeseapps vs Reeselink](#reeseapps-vs-reeselink) - - [DNS Caching](#dns-caching) - [Reeselink Addresses](#reeselink-addresses) - [Reeseapps Addresses](#reeseapps-addresses) - - [Duconet WG Addresses](#duconet-wg-addresses) ## Reeseapps vs Reeselink @@ -17,69 +15,14 @@ domains. and other machine to machine connections. They can be public or private and are mostly for convenience. -## DNS Caching - -Use unifi to cache important DNS records. The following are critical: - -- `driveripper-wg.reeselink.com` `Host (AAAA)` `fd00:fd41:d0f1:1010::6` -- `democratic-csi-server.reeselink.com` `Host (A)` `fd00:fd41:d0f1:1010::6` -- `driveripper.reeseapps.com` `Host (AAAA)` `2600:1700:1e6c:a81f:153e:9c35:8ff3:fa3` -- `driveripper.reeseapps.com` `Host (AAAA)` `2600:1700:1e6c:a81f:793d:7abf:e94d:9bc4` - - ## Reeselink Addresses ```bash aws route53 change-resource-record-sets --hosted-zone-id Z0092652G7L97DSINN18 --change-batch file://dns/reeselink.json ``` -You can extract these addresses into a text file with: - -```bash -# IPV6 -cat dns/reeselink.json | \ - jq -c -r '[ .Changes.[] | - select( .ResourceRecordSet.Type | . == "AAAA") ] - | .[] - | .ResourceRecordSet - | .Name,.ResourceRecords.[].Value' > dns/ipv6.txt - -# IPV4 -cat dns/reeselink.json | \ - jq -c -r '[ .Changes.[] | - select( .ResourceRecordSet.Type | . == "A") ] - | .[] - | .ResourceRecordSet - | .Name,.ResourceRecords.[].Value' > dns/ipv4.txt -``` - ## Reeseapps Addresses ```bash aws route53 change-resource-record-sets --hosted-zone-id Z012820733346FJ0U4FUF --change-batch file://dns/reeseapps.json ``` - -## Duconet WG Addresses - -After generating new addresses from wireguard's vars.yaml. Use find and replace regex -with the following: - -```regex -(.*.reeselink.com)\n(.*)$ -``` - -```regex - { - "Action": "UPSERT", - "ResourceRecordSet": { - "Name": "$1", - "Type": "AAAA", - "TTL": 300, - "ResourceRecords": [ - { - "Value": "$2" - } - ] - } - }, -``` diff --git a/dns/reeseapps.json b/dns/reeseapps.json index a02efe4..1c61bda 100644 --- a/dns/reeseapps.json +++ b/dns/reeseapps.json @@ -26,6 +26,19 @@ } ] } + }, + { + "Action": "UPSERT", + "ResourceRecordSet": { + "Name": "unifi-external.reeseapps.com", + "Type": "AAAA", + "TTL": 300, + "ResourceRecords": [ + { + "Value": "2600:1700:1e6c:a81f:2a0:98ff:fe5e:edc3" + } + ] + } } ] } diff --git a/dns/reeselink.json b/dns/reeselink.json index dbae9bd..cef4bd3 100644 --- a/dns/reeselink.json +++ b/dns/reeselink.json @@ -39,6 +39,19 @@ } ] } + }, + { + "Action": "UPSERT", + "ResourceRecordSet": { + "Name": "unifi-external.reeselink.com", + "Type": "AAAA", + "TTL": 300, + "ResourceRecords": [ + { + "Value": "2600:1700:1e6c:a81f:2a0:98ff:fe5e:edc3" + } + ] + } } ] } diff --git a/fedora/README.md b/fedora/README.md index d3e1c93..25164d5 100644 --- a/fedora/README.md +++ b/fedora/README.md @@ -130,7 +130,7 @@ dnf install -y vim-default-editor --allowerasing dnf install -y glances # Install zsh with autocomplete and suggestions -dnf install zsh zsh-autosuggestions zsh-syntax-highlighting +dnf install -y zsh zsh-autosuggestions zsh-syntax-highlighting cat < ~/.zshrc # Basic settings diff --git a/kubectl/grafana.yaml b/grafana/grafana.yaml similarity index 100% rename from kubectl/grafana.yaml rename to grafana/grafana.yaml diff --git a/helm/dashboard-user/Chart.yaml b/helm/dashboard-user/Chart.yaml deleted file mode 100644 index a94ef1e..0000000 --- a/helm/dashboard-user/Chart.yaml +++ /dev/null @@ -1,24 +0,0 @@ -apiVersion: v2 -name: dashboard -description: A Kubernetes Dashboard User Deployment - -# A chart can be either an 'application' or a 'library' chart. -# -# Application charts are a collection of templates that can be packaged into versioned archives -# to be deployed. -# -# Library charts provide useful utilities or functions for the chart developer. They're included as -# a dependency of application charts to inject those utilities and functions into the rendering -# pipeline. Library charts do not define any templates and therefore cannot be deployed. -type: application - -# This is the chart version. This version number should be incremented each time you make changes -# to the chart and its templates, including the app version. -# Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 0.1.0 - -# This is the version number of the application being deployed. This version number should be -# incremented each time you make changes to the application. Versions are not expected to -# follow Semantic Versioning. They should reflect the version the application is using. -# It is recommended to use it with quotes. -appVersion: "1.16.0" diff --git a/helm/dashboard-user/templates/dashboard.yaml b/helm/dashboard-user/templates/dashboard.yaml deleted file mode 100644 index 4c4384a..0000000 --- a/helm/dashboard-user/templates/dashboard.yaml +++ /dev/null @@ -1,20 +0,0 @@ -apiVersion: v1 -kind: ServiceAccount -metadata: - name: admin-user - namespace: kubernetes-dashboard - ---- - -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: admin-user -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: cluster-admin -subjects: -- kind: ServiceAccount - name: admin-user - namespace: kubernetes-dashboard diff --git a/helm/iperf3/values.yaml b/helm/iperf3/values.yaml deleted file mode 100755 index e69de29..0000000 diff --git a/helm/nextcloud/.helmignore b/helm/nextcloud/.helmignore deleted file mode 100755 index 0e8a0eb..0000000 --- a/helm/nextcloud/.helmignore +++ /dev/null @@ -1,23 +0,0 @@ -# Patterns to ignore when building packages. -# This supports shell glob matching, relative path matching, and -# negation (prefixed with !). Only one pattern per line. -.DS_Store -# Common VCS dirs -.git/ -.gitignore -.bzr/ -.bzrignore -.hg/ -.hgignore -.svn/ -# Common backup files -*.swp -*.bak -*.tmp -*.orig -*~ -# Various IDEs -.project -.idea/ -*.tmproj -.vscode/ diff --git a/helm/nextcloud/templates/NOTES.txt b/helm/nextcloud/templates/NOTES.txt deleted file mode 100644 index 7aa2da9..0000000 --- a/helm/nextcloud/templates/NOTES.txt +++ /dev/null @@ -1,11 +0,0 @@ -Nextcloud has been installed! - -{{ if .Values.show_passwords -}} -`show_passwords` is true. Here are the generated (or retrieved) passwords: - -NEXTCLOUD_ADMIN_PASSWORD: {{ include "NEXTCLOUD_ADMIN_PASSWORD" . | quote }} -POSTGRES_PASSWORD: {{ include "POSTGRES_PASSWORD" . | quote }} -REDIS_HOST_PASSWORD: {{ include "REDIS_PASSWORD" . | quote }} -{{ else }} -Run with `--set show_passwords=true` to output the generated passwords. -{{- end }} diff --git a/helm/nextcloud/templates/_helpers.tpl b/helm/nextcloud/templates/_helpers.tpl deleted file mode 100644 index 9d48111..0000000 --- a/helm/nextcloud/templates/_helpers.tpl +++ /dev/null @@ -1,47 +0,0 @@ -{{- define "helm_keep_annotation" -}} -"helm.sh/resource-policy": keep -{{- end -}} - -{{/* Generated Postgres Config */}} -{{ define "POSTGRES_NAME" }}{{ printf "%s-postgres" .Release.Name | lower }}{{ end }} -{{ define "POSTGRES_DB" }}nextcloud{{ end }} -{{ define "DATABASE_HOST" }}{{ .Release.Name }}-postgres{{ end }} -{{ define "POSTGRES_USER" }}postgres{{ end }} - -{{/* Generated Nextcloud Config */}} -{{ define "NEXTCLOUD_NAME" }}{{ printf "%s-nextcloud" .Release.Name | lower }}{{ end }} -{{ define "ADMIN_USER" }}admin{{ end }} - -{{/* Generated Redis Config */}} -{{ define "REDIS_NAME" }}{{ printf "%s-redis" .Release.Name | lower }}{{ end }} -{{ define "REDIS_HOST" }}{{ .Release.Name }}-redis{{ end }} - -{{/* Postgres password lookup - uses existing password if possible */}} -{{ define "POSTGRES_PASSWORD" -}} -{{- $POSTGRES_SECRET := (lookup "v1" "Secret" .Release.Namespace ( include "POSTGRES_NAME" . )).data -}} -{{- if $POSTGRES_SECRET -}} - {{- printf $POSTGRES_SECRET.POSTGRES_PASSWORD | b64enc -}} -{{- else -}} - {{- printf (required ".Values.postgres.password is required" .Values.postgres.password) | b64enc -}} -{{- end -}} -{{- end }} - -{{/* Nextcloud admin password lookup - uses existing password if possible */}} -{{ define "NEXTCLOUD_ADMIN_PASSWORD" -}} -{{- $NEXTCLOUD_SECRETS := (lookup "v1" "Secret" .Release.Namespace ( include "NEXTCLOUD_NAME" . )).data -}} -{{- if $NEXTCLOUD_SECRETS -}} - {{- printf $NEXTCLOUD_SECRETS.NEXTCLOUD_ADMIN_PASSWORD | b64enc -}} -{{- else -}} - {{- printf (required ".Values.nextcloud.admin.password is required" .Values.nextcloud.admin.password) | b64enc -}} -{{- end -}} -{{- end }} - -{{/* Redis password lookup - uses existing password if possible */}} -{{ define "REDIS_PASSWORD" -}} -{{- $REDIS_SECRETS := (lookup "v1" "Secret" .Release.Namespace ( include "REDIS_NAME" . )).data -}} -{{- if $REDIS_SECRETS -}} - {{- printf $REDIS_SECRETS.REDIS_PASSWORD | b64enc -}} -{{- else -}} - {{- printf (required ".Values.redis.password is required" .Values.redis.password) | b64enc -}} -{{- end -}} -{{- end }} \ No newline at end of file diff --git a/helm/nextcloud/templates/deployment.yaml b/helm/nextcloud/templates/deployment.yaml deleted file mode 100644 index 5381954..0000000 --- a/helm/nextcloud/templates/deployment.yaml +++ /dev/null @@ -1,110 +0,0 @@ -apiVersion: apps/v1 -kind: Deployment -metadata: - name: {{ .Release.Name }} -spec: - selector: - matchLabels: - app.kubernetes.io/name: nextcloud - strategy: - type: Recreate - template: - metadata: - labels: - app.kubernetes.io/name: nextcloud - spec: - tolerations: - - key: "node.kubernetes.io/unreachable" - operator: "Exists" - effect: "NoExecute" - tolerationSeconds: 1 - - key: "node.kubernetes.io/not-ready" - operator: "Exists" - effect: "NoExecute" - tolerationSeconds: 1 - containers: - - name: nextcloud - image: {{ .Values.nextcloud.image }} - ports: - - containerPort: 80 - name: http - envFrom: - - configMapRef: - name: {{ .Release.Name }}-nextcloud - - secretRef: - name: {{ .Release.Name }}-nextcloud - volumeMounts: - - mountPath: /var/www/html - name: html - - mountPath: /var/www/html/data - name: data - resources: - requests: - memory: "1Gi" - cpu: "1m" - limits: - memory: "4Gi" - cpu: "4" - - name: postgres - image: postgres:15 - envFrom: - - configMapRef: - name: {{ .Release.Name }}-postgres - - secretRef: - name: {{ .Release.Name }}-postgres - volumeMounts: - - name: postgres - mountPath: /var/lib/postgresql/data - - name: postgres-init - mountPath: /docker-entrypoint-initdb.d/init-user-db.sh - subPath: init-user-db.sh - ports: - - containerPort: 5432 - name: postgres - resources: - requests: - memory: "1Gi" - cpu: "1m" - limits: - memory: "4Gi" - cpu: "4" - - name: redis - image: redis:7 - ports: - - containerPort: 6379 - name: redis - volumeMounts: - - mountPath: /data - name: redis - command: - - redis-server - - --save - - "60" - - "1" - - --loglevel - - warning - - --requirepass - - {{ include "REDIS_PASSWORD" . | b64dec | quote }} - resources: - requests: - memory: "1Gi" - cpu: "1m" - limits: - memory: "4Gi" - cpu: "4" - volumes: - - name: html - persistentVolumeClaim: - claimName: {{ .Release.Name }}-html-iops - - name: data - persistentVolumeClaim: - claimName: {{ .Release.Name }}-data - - name: postgres - persistentVolumeClaim: - claimName: {{ .Release.Name }}-postgres-iops - - name: redis - emptyDir: - sizeLimit: 500Mi - - name: postgres-init - secret: - secretName: {{ .Release.Name }}-postgres-init diff --git a/helm/nextcloud/templates/ingress.yaml b/helm/nextcloud/templates/ingress.yaml deleted file mode 100644 index e9036ee..0000000 --- a/helm/nextcloud/templates/ingress.yaml +++ /dev/null @@ -1,25 +0,0 @@ -apiVersion: networking.k8s.io/v1 -kind: Ingress -metadata: - name: {{ .Release.Name }} - annotations: - cert-manager.io/cluster-issuer: letsencrypt - kubernetes.io/ingress.class: nginx - nginx.ingress.kubernetes.io/proxy-body-size: "0" - nginx.org/client-max-body-size: "0" -spec: - rules: - - host: {{ .Values.nextcloud.domain }} - http: - paths: - - path: / - pathType: Prefix - backend: - service: - name: nextcloud - port: - name: http - tls: - - hosts: - - {{ .Values.nextcloud.domain }} - secretName: nextcloud-tls-cert diff --git a/helm/nextcloud/templates/nextcloud-configmap.yaml b/helm/nextcloud/templates/nextcloud-configmap.yaml deleted file mode 100644 index 864a28d..0000000 --- a/helm/nextcloud/templates/nextcloud-configmap.yaml +++ /dev/null @@ -1,19 +0,0 @@ -apiVersion: v1 -kind: ConfigMap -metadata: - name: {{ .Release.Name }}-nextcloud - annotations: - {{- include "helm_keep_annotation" . | nindent 4 }} -data: - NEXTCLOUD_TRUSTED_DOMAINS: {{ .Values.nextcloud.domain }} - OVERWRITEPROTOCOL: https - OVERWRITECLIURL: https://{{ .Values.nextcloud.domain }} - NEXTCLOUD_ADMIN_USER: admin - POSTGRES_USER: nextcloud - POSTGRES_HOST: {{ .Release.Name }} - POSTGRES_DB: nextcloud - REDIS_HOST: {{ .Release.Name }} - PHP_UPLOAD_LIMIT: 1000000M - PHP_MEMORY_LIMIT: 2048M - TRUSTED_PROXIES: 10.42.0.1/24 - APACHE_DISABLE_REWRITE_IP: "1" diff --git a/helm/nextcloud/templates/nextcloud-cronjob.yaml b/helm/nextcloud/templates/nextcloud-cronjob.yaml deleted file mode 100644 index 1cb5da2..0000000 --- a/helm/nextcloud/templates/nextcloud-cronjob.yaml +++ /dev/null @@ -1,62 +0,0 @@ -apiVersion: batch/v1 -kind: CronJob -metadata: - name: {{ .Release.Name }}-cron -spec: - schedule: "*/5 * * * *" - failedJobsHistoryLimit: 1 - successfulJobsHistoryLimit: 0 - jobTemplate: - spec: - template: - spec: - affinity: - podAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchExpressions: - - key: app.kubernetes.io/name - operator: In - values: - - nextcloud - # This is the selector for the node - # So when you find a valid node - schedule the pod to the - # node that matches hostname - # needed for iscsi mounts - topologyKey: kubernetes.io/hostname - - securityContext: - runAsUser: 33 - runAsGroup: 33 - containers: - - name: nextcloud - image: {{ .Values.nextcloud.image }} - command: - - php - - -f - - cron.php - volumeMounts: - - mountPath: /var/www/html - name: html - - mountPath: /var/www/html/data - name: data - envFrom: - - configMapRef: - name: {{ .Release.Name }}-nextcloud - - secretRef: - name: {{ .Release.Name }}-nextcloud - resources: - requests: - memory: "1Gi" - cpu: "1m" - limits: - memory: "4Gi" - cpu: "4" - volumes: - - name: html - persistentVolumeClaim: - claimName: {{ .Release.Name }}-html-iops - - name: data - persistentVolumeClaim: - claimName: {{ .Release.Name }}-data - restartPolicy: OnFailure diff --git a/helm/nextcloud/templates/nextcloud-data-pvc.yaml b/helm/nextcloud/templates/nextcloud-data-pvc.yaml deleted file mode 100644 index a53a0b3..0000000 --- a/helm/nextcloud/templates/nextcloud-data-pvc.yaml +++ /dev/null @@ -1,13 +0,0 @@ -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: {{ .Release.Name }}-data - annotations: - "helm.sh/resource-policy": keep -spec: - storageClassName: {{ .Values.nextcloud.data.storageClassName }} - accessModes: - - ReadWriteOnce - resources: - requests: - storage: {{ .Values.nextcloud.data.storage }} diff --git a/helm/nextcloud/templates/nextcloud-html-pvc.yaml b/helm/nextcloud/templates/nextcloud-html-pvc.yaml deleted file mode 100644 index 4a69d60..0000000 --- a/helm/nextcloud/templates/nextcloud-html-pvc.yaml +++ /dev/null @@ -1,13 +0,0 @@ -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: {{ .Release.Name }}-html-iops - annotations: - "helm.sh/resource-policy": keep -spec: - storageClassName: {{ .Values.nextcloud.html.storageClassName }} - accessModes: - - ReadWriteOnce - resources: - requests: - storage: {{ .Values.nextcloud.html.storage }} diff --git a/helm/nextcloud/templates/nextcloud-secret.yaml b/helm/nextcloud/templates/nextcloud-secret.yaml deleted file mode 100644 index 9ae87d8..0000000 --- a/helm/nextcloud/templates/nextcloud-secret.yaml +++ /dev/null @@ -1,11 +0,0 @@ -apiVersion: v1 -kind: Secret -metadata: - name: {{ .Release.Name }}-nextcloud - annotations: - {{- include "helm_keep_annotation" . | nindent 4 }} -type: generic -data: - NEXTCLOUD_ADMIN_PASSWORD: {{ include "NEXTCLOUD_ADMIN_PASSWORD" . | quote }} - POSTGRES_PASSWORD: {{ include "POSTGRES_PASSWORD" . | quote }} - REDIS_HOST_PASSWORD: {{ include "REDIS_PASSWORD" . | quote }} diff --git a/helm/nextcloud/templates/postgres-configmap.yaml b/helm/nextcloud/templates/postgres-configmap.yaml deleted file mode 100644 index 7f88e52..0000000 --- a/helm/nextcloud/templates/postgres-configmap.yaml +++ /dev/null @@ -1,9 +0,0 @@ -apiVersion: v1 -kind: ConfigMap -metadata: - name: {{ .Release.Name }}-postgres - annotations: - {{- include "helm_keep_annotation" . | nindent 4 }} -data: - POSTGRES_USER: postgres - POSTGRES_DB: nextcloud diff --git a/helm/nextcloud/templates/postgres-init-secret.yaml b/helm/nextcloud/templates/postgres-init-secret.yaml deleted file mode 100644 index 756386b..0000000 --- a/helm/nextcloud/templates/postgres-init-secret.yaml +++ /dev/null @@ -1,16 +0,0 @@ -apiVersion: v1 -kind: Secret -metadata: - name: {{ .Release.Name }}-postgres-init - annotations: - {{- include "helm_keep_annotation" . | nindent 4 }} -stringData: - init-user-db.sh: | - #!/bin/bash - set -e - - psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL - CREATE USER nextcloud PASSWORD '{{ include "POSTGRES_PASSWORD" . | b64dec }}'; - GRANT ALL PRIVILEGES ON DATABASE nextcloud TO nextcloud; - GRANT USAGE, CREATE ON SCHEMA public TO nextcloud; - EOSQL diff --git a/helm/nextcloud/templates/postgres-pvc.yaml b/helm/nextcloud/templates/postgres-pvc.yaml deleted file mode 100644 index f41837a..0000000 --- a/helm/nextcloud/templates/postgres-pvc.yaml +++ /dev/null @@ -1,13 +0,0 @@ -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: {{ .Release.Name }}-postgres-iops - annotations: - "helm.sh/resource-policy": keep -spec: - storageClassName: {{ .Values.postgres.storageClassName }} - accessModes: - - ReadWriteOnce - resources: - requests: - storage: {{ .Values.postgres.storage }} diff --git a/helm/nextcloud/templates/postgres-secret.yaml b/helm/nextcloud/templates/postgres-secret.yaml deleted file mode 100644 index feb200f..0000000 --- a/helm/nextcloud/templates/postgres-secret.yaml +++ /dev/null @@ -1,9 +0,0 @@ -apiVersion: v1 -kind: Secret -metadata: - name: {{ .Release.Name }}-postgres - annotations: - {{- include "helm_keep_annotation" . | nindent 4 }} -type: generic -data: - POSTGRES_PASSWORD: {{ include "POSTGRES_PASSWORD" . | quote }} diff --git a/helm/nextcloud/templates/redis-pvc.yaml b/helm/nextcloud/templates/redis-pvc.yaml deleted file mode 100644 index be7c22f..0000000 --- a/helm/nextcloud/templates/redis-pvc.yaml +++ /dev/null @@ -1,13 +0,0 @@ -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: {{ .Release.Name }}-redis-iops - annotations: - "helm.sh/resource-policy": keep -spec: - storageClassName: {{ .Values.redis.storageClassName }} - accessModes: - - ReadWriteOnce - resources: - requests: - storage: {{ .Values.redis.storage }} diff --git a/helm/nextcloud/templates/redis-secret.yaml b/helm/nextcloud/templates/redis-secret.yaml deleted file mode 100644 index aff4222..0000000 --- a/helm/nextcloud/templates/redis-secret.yaml +++ /dev/null @@ -1,9 +0,0 @@ -apiVersion: v1 -kind: Secret -metadata: - name: {{ .Release.Name }}-redis - annotations: - {{- include "helm_keep_annotation" . | nindent 4 }} -type: generic -data: - REDIS_PASSWORD: {{ include "REDIS_PASSWORD" . | quote }} diff --git a/helm/nextcloud/templates/service.yaml b/helm/nextcloud/templates/service.yaml deleted file mode 100644 index e3fe5be..0000000 --- a/helm/nextcloud/templates/service.yaml +++ /dev/null @@ -1,21 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: {{ .Release.Name }} -spec: - type: ClusterIP - selector: - app.kubernetes.io/name: nextcloud - ports: - - name: http - protocol: TCP - port: 80 - targetPort: http - - name: postgres - protocol: TCP - port: 5432 - targetPort: postgres - - name: redis - protocol: TCP - port: 6379 - targetPort: redis diff --git a/helm/nextcloud/values.yaml b/helm/nextcloud/values.yaml deleted file mode 100755 index ea1e99d..0000000 --- a/helm/nextcloud/values.yaml +++ /dev/null @@ -1,21 +0,0 @@ -nextcloud: - image: nextcloud:27 - domain: nextcloud.reeseapps.com - html: - storageClassName: zfs-iscsi-enc1 - storage: 16Gi - data: - storageClassName: zfs-iscsi-enc0 - storage: 2Ti - admin: - password: "" - -postgres: - storageClassName: zfs-iscsi-enc1 - storage: 32Gi - password: "" - -redis: - storageClassName: zfs-nfs-enc1 - storage: 32Gi - password: "" diff --git a/helm/snapdrop/.helmignore b/helm/snapdrop/.helmignore deleted file mode 100755 index 0e8a0eb..0000000 --- a/helm/snapdrop/.helmignore +++ /dev/null @@ -1,23 +0,0 @@ -# Patterns to ignore when building packages. -# This supports shell glob matching, relative path matching, and -# negation (prefixed with !). Only one pattern per line. -.DS_Store -# Common VCS dirs -.git/ -.gitignore -.bzr/ -.bzrignore -.hg/ -.hgignore -.svn/ -# Common backup files -*.swp -*.bak -*.tmp -*.orig -*~ -# Various IDEs -.project -.idea/ -*.tmproj -.vscode/ diff --git a/helm/snapdrop/Chart.yaml b/helm/snapdrop/Chart.yaml deleted file mode 100755 index ce04c44..0000000 --- a/helm/snapdrop/Chart.yaml +++ /dev/null @@ -1,23 +0,0 @@ -apiVersion: v2 -name: Nextcloud -description: A Simple Nextcloud Chart - -# A chart can be either an 'application' or a 'library' chart. -# -# Application charts are a collection of templates that can be packaged into versioned archives -# to be deployed. -# -# Library charts provide useful utilities or functions for the chart developer. They're included as -# a dependency of application charts to inject those utilities and functions into the rendering -# pipeline. Library charts do not define any templates and therefore cannot be deployed. -type: application - -# This is the chart version. This version number should be incremented each time you make changes -# to the chart and its templates, including the app version. -# Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 0.1.0 - -# This is the version number of the application being deployed. This version number should be -# incremented each time you make changes to the application. Versions are not expected to -# follow Semantic Versioning. They should reflect the version the application is using. -appVersion: 1.16.0 diff --git a/home-assistant/README.md b/home-assistant/README.md index 9d384b4..b4d0269 100644 --- a/home-assistant/README.md +++ b/home-assistant/README.md @@ -2,34 +2,12 @@ - [Home Assistant](#home-assistant) - [Setup and Configuration](#setup-and-configuration) - - [Cert](#cert) - [Door Lock](#door-lock) - [Philips Hue Lights](#philips-hue-lights) - [Shelly](#shelly) - - [Trackers](#trackers) - - [Looping Over Entities with Labels](#looping-over-entities-with-labels) - - [Get All Entity Attributes](#get-all-entity-attributes) ## Setup and Configuration -### Cert - -```bash -openssl req -sha256 -addext "subjectAltName = DNS:homeassistant.reeselink.com" -newkey rsa:4096 -nodes -keyout secrets/ha-privkey.pem -x509 -days 3650 -out secrets/ha-fullchain.pem -scp secrets/ha-* root@homeassistant.reeselink.com:~/ssl/ -``` - -configuration.yaml - -```yaml -http: - ssl_certificate: "/ssl/ha-fullchain.pem" - ssl_key: "/ssl/ha-privkey.pem" - use_x_forwarded_for: true - trusted_proxies: - - 10.1.0.0/16 -``` - ### Door Lock 1. Install Z-wave @@ -59,25 +37,3 @@ the range of your home assistant's bluetooth capabilities. Active scanning uses is quicker to pick up and transmit device information. Note that "gateway mode" is not required, just enable bluetooth and rpc or select "active" from the configuration menu for the shelly device. - -### Trackers - -See `hass_trackers/` - -### Looping Over Entities with Labels - - - -```yaml -{% for item in label_entities("Battery Level") -%} -- {{ item }} -{% endfor %} -``` - -### Get All Entity Attributes - -```yaml -{% for item in label_entities("Battery Level") -%} -- {{ states[item].attributes }} -{% endfor %} -``` diff --git a/helm/dashboard-user/.helmignore b/iperf3/.helmignore old mode 100644 new mode 100755 similarity index 100% rename from helm/dashboard-user/.helmignore rename to iperf3/.helmignore diff --git a/helm/iperf3/Chart.yaml b/iperf3/Chart.yaml similarity index 100% rename from helm/iperf3/Chart.yaml rename to iperf3/Chart.yaml diff --git a/helm/iperf3/templates/deployment.yaml b/iperf3/templates/deployment.yaml similarity index 100% rename from helm/iperf3/templates/deployment.yaml rename to iperf3/templates/deployment.yaml diff --git a/helm/iperf3/templates/service.yaml b/iperf3/templates/service.yaml similarity index 100% rename from helm/iperf3/templates/service.yaml rename to iperf3/templates/service.yaml diff --git a/helm/dashboard-user/values.yaml b/iperf3/values.yaml old mode 100644 new mode 100755 similarity index 100% rename from helm/dashboard-user/values.yaml rename to iperf3/values.yaml diff --git a/helm/iperf3/.helmignore b/jellyfin/.helmignore similarity index 100% rename from helm/iperf3/.helmignore rename to jellyfin/.helmignore diff --git a/helm/jellyfin/Chart.yaml b/jellyfin/Chart.yaml similarity index 100% rename from helm/jellyfin/Chart.yaml rename to jellyfin/Chart.yaml diff --git a/helm/jellyfin/templates/jellyfin.yaml b/jellyfin/templates/jellyfin.yaml similarity index 100% rename from helm/jellyfin/templates/jellyfin.yaml rename to jellyfin/templates/jellyfin.yaml diff --git a/helm/jellyfin/values.yaml b/jellyfin/values.yaml similarity index 100% rename from helm/jellyfin/values.yaml rename to jellyfin/values.yaml diff --git a/k3s/README.md b/k3s/README.md index d3383b3..a2801d5 100644 --- a/k3s/README.md +++ b/k3s/README.md @@ -16,30 +16,14 @@ - [Cert Manager](#cert-manager) - [Test Minecraft Server](#test-minecraft-server) - [Automatic Updates](#automatic-updates) - - [Manual Updates](#manual-updates) - - [Create a Userspace](#create-a-userspace) + - [Database Backups](#database-backups) - [Quickstart](#quickstart) - - [Userspace](#userspace) - - [Namespace](#namespace) - - [Roles](#roles) - - [Rolebinding](#rolebinding) - - [Manual Steps](#manual-steps) - - [Create a kubernetes certsigner pod](#create-a-kubernetes-certsigner-pod) - - [Create the certsigner secret](#create-the-certsigner-secret) - - [Set up the certsigner pod](#set-up-the-certsigner-pod) - - [Generate a cert](#generate-a-cert) - - [Create a new Userspace](#create-a-new-userspace) - - [Sign the cert](#sign-the-cert) - - [Add to the config](#add-to-the-config) - - [Delete](#delete) - - [Signing a user cert - detailed notes](#signing-a-user-cert---detailed-notes) - [Help](#help) - [Troubleshooting](#troubleshooting) - [Deleting a stuck namespace](#deleting-a-stuck-namespace) - [Fixing a bad volume](#fixing-a-bad-volume) - [Mounting an ix-application volume from truenas](#mounting-an-ix-application-volume-from-truenas) - [Mounting a volume](#mounting-a-volume) - - [Database Backups](#database-backups) - [Uninstall](#uninstall) ## Guide @@ -76,6 +60,7 @@ Set SELinux to permissive by editing `/etc/selinux/config` ```bash curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.30.2+k3s2 sh -s - \ + "--cluster-init" \ "--flannel-ipv6-masq" \ "--disable" \ "traefik" \ @@ -83,6 +68,8 @@ curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.30.2+k3s2 sh -s - \ "servicelb" \ "--disable" \ "coredns" \ + "--disable" \ + "local-storage" \ "--tls-san" \ "kube.reeselink.com" \ "--cluster-cidr" \ @@ -111,6 +98,19 @@ scp kube:/etc/rancher/k3s/k3s.yaml ~/.kube/admin-kube-config 3. `systemctl daemon-reload` 4. `mount -a` + + +```bash +# Download the updated template from github +kubectl kustomize "github.com/rancher/local-path-provisioner/deploy?ref=v0.0.28" > local-path-provisioner/local-path-storage.yaml + +# Apply customizations (ssd/hdd storage, read write many support) +kubectl kustomize local-path-provisioner | kubectl apply -f - + +# Create test pod +kubectl apply -f k3s/tests/local-storage-test.yaml +``` + ## Coredns 1. Edit `coredns/values.yaml` to ensure the forward nameserver is correct. @@ -270,8 +270,7 @@ kubectl delete -f k3s/tests/ingress-nginx-test.yaml ## Test Minecraft Server ```bash -helm upgrade --install minecraft ./helm/minecraft -n minecraft --create-namespace -helm upgrade --install minecraft1 ./helm/minecraft -n minecraft --create-namespace +helm upgrade --install minecraft ./minecraft -n minecraft --create-namespace ``` ## Automatic Updates @@ -279,28 +278,33 @@ helm upgrade --install minecraft1 ./helm/minecraft -n minecraft --create-namespa ```bash +kubectl create namespace system-upgrade kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/system-upgrade-controller.yaml kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/crd.yaml kubectl apply -f k3s/upgrade-plan.yaml + +# Check plan +kubectl get plan -n system-upgrade ``` -## Manual Updates +## Database Backups - + + +Note, you must backup `/var/lib/rancher/k3s/server/token` +and use the contents as the toklisten when restoring the backup as data is encrypted with that token. + +Backups are saved to `/var/lib/rancher/k3s/server/db/snapshots/` by default. ```bash -sudo su - -wget https://github.com/k3s-io/k3s/releases/download/v1.28.3%2Bk3s1/k3s -systemctl stop k3s -chmod +x k3s -mv k3s /usr/local/bin/k3s -systemctl start k3s +k3s etcd-snapshot save +k3s etcd-snapshot list + +k3s server \ + --cluster-reset \ + --cluster-reset-restore-path=/var/lib/rancher/k3s/server/db/snapshots/on-demand-kube-1720459685 ``` -## Create a Userspace - -This creates a user, namespace, and permissions with a simple script. - ### Quickstart ```bash @@ -314,194 +318,6 @@ This creates a user, namespace, and permissions with a simple script. ./removeuserspace ``` -### Userspace - -#### Namespace - -```yaml -apiVersion: v1 -kind: Namespace -metadata: - name: {{ .Release.Name }} -``` - -#### Roles - -```yaml -kind: Role -apiVersion: rbac.authorization.k8s.io/v1beta1 -metadata: - name: namespace-manager - namespace: {{ .Release.Name }} -rules: -- apiGroups: - - "" - - extensions - - apps - - batch - - autoscaling - - networking.k8s.io - - traefik.containo.us - - rbac.authorization.k8s.io - - metrics.k8s.io - resources: - - deployments - - replicasets - - pods - - pods/exec - - pods/log - - pods/attach - - daemonsets - - statefulsets - - replicationcontrollers - - horizontalpodautoscalers - - services - - ingresses - - persistentvolumeclaims - - jobs - - cronjobs - - secrets - - configmaps - - serviceaccounts - - rolebindings - - ingressroutes - - middlewares - - endpoints - verbs: - - "*" -- apiGroups: - - "" - - metrics.k8s.io - - rbac.authorization.k8s.io - resources: - - resourcequotas - - roles - verbs: - - list -``` - -#### Rolebinding - -```yaml -kind: RoleBinding -apiVersion: rbac.authorization.k8s.io/v1beta1 -metadata: - namespace: {{ .Release.Name }} - name: namespace-manager -subjects: -- kind: User - name: {{ .Release.Name }} - apiGroup: "" -roleRef: - kind: ClusterRole - name: namespace-manager - apiGroup: "" -``` - -### Manual Steps - -#### Create a kubernetes certsigner pod - -This keeps the client-ca crt and key secret and allows the cert to be signed and stored on the pod - -#### Create the certsigner secret - -```bash -kubectl -n kube-system create secret generic certsigner --from-file /var/lib/rancher/k3s/server/tls/client-ca.crt --from-file /var/lib/rancher/k3s/server/tls/client-ca.key -``` - -#### Set up the certsigner pod - -```bash -scp certsigner.yaml :~/certsigner.yaml -kubectl apply -f certsigner.yaml -``` - -#### Generate a cert - -```bash -export USER= -docker run -it -v $(pwd)/users/$USER:/$USER python:latest openssl genrsa -out /$USER/$USER.key 2048 -docker run -it -v $(pwd)/users/$USER:/$USER python:latest openssl req -new -key /$USER/$USER.key -out /$USER/$USER.csr -subj "/CN=$USER/O=user" -``` - -#### Create a new Userspace - -```bash -helm template $USER ./namespace | kubectl --context admin apply -f - -``` - -#### Sign the cert - -```bash -export USER= -kubectl --context admin cp $(pwd)/users/$USER/$USER.csr certsigner:/certs/$USER.csr -kubectl --context admin exec -it --context admin certsigner -- openssl x509 -in /certs/$USER.csr -req -CA /keys/client-ca.crt -CAkey /keys/client-ca.key -CAcreateserial -out /certs/$USER.crt -days 5000 -kubectl --context admin cp certsigner:/certs/$USER.crt $(pwd)/users/$USER/$USER.crt -``` - -#### Add to the config - -```bash -kubectl config set-credentials $USER --client-certificate=$USER.crt --client-key=$USER.key -kubectl config set-context $USER --cluster=mainframe --namespace=$USER --user=$USER -``` - -#### Delete - -```bash -kubectl config delete-context $USER -helm template $USER ./namespace | kubectl --context admin delete -f - -``` - -### Signing a user cert - detailed notes - -NOTE: ca.crt and ca.key are in /var/lib/rancher/k3s/server/tls/client-ca.* - -```bash -# First we create the credentials -# /CN= - the user -# /O= - the group - -# Navigate to the user directory -export USER= -cd $USER - -# Generate a private key -openssl genrsa -out $USER.key 2048 -# Check the key -# openssl pkey -in ca.key -noout -text -# Generate and send me the CSR -# The "user" group is my default group -openssl req -new -key $USER.key -out $USER.csr -subj "/CN=$USER/O=user" - -# Check the CSR -# openssl req -in $USER.csr -noout -text -# If satisfactory, sign the CSR -# Copy from /var/lib/rancher/k3s/server/tls/client-ca.crt and client-ca.key -openssl x509 -req -in $USER.csr -CA ../client-ca.crt -CAkey ../client-ca.key -CAcreateserial -out $USER.crt -days 5000 -# Review the certificate -# openssl x509 -in $USER.crt -text -noout - -# Send back the crt -# cp $USER.crt $USER.key ../server-ca.crt ~/.kube/ -kubectl config set-credentials $USER --client-certificate=$USER.crt --client-key=$USER.key -kubectl config set-context $USER --cluster=mainframe --namespace=$USER --user=$USER - -# Now we create the namespace, rolebindings, and resource quotas -# kubectl apply -f k8s/ - -# Add the cluster -# CA file can be found at https://3.14.3.100:6443/cacerts -- cluster: - certificate-authority: server-ca.crt - server: https://3.14.3.100:6443 - name: mainframe - -# Test if everything worked -kubectl --context=$USER-context get pods -``` - ## Help ### Troubleshooting @@ -541,13 +357,6 @@ mount -t xfs /dev/zvol/enc0/dcsi/apps/pvc-d5090258-cf20-4f2e-a5cf-330ac00d0049 / umount /mnt/dcsi_pvc ``` -## Database Backups - - - -Note, you must backup `/var/lib/rancher/k3s/server/token` -and use the contents as the token when restoring the backup as data is encrypted with that token. - ## Uninstall ```bash diff --git a/k3s/cluster-readonly.yaml b/k3s/cluster-readonly.yaml deleted file mode 100755 index 8e04fc6..0000000 --- a/k3s/cluster-readonly.yaml +++ /dev/null @@ -1,30 +0,0 @@ -kind: ClusterRole -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: cluster-readonly -rules: -- apiGroups: - - "" - - rbac.authorization.k8s.io - - storage.k8s.io - - networking.k8s.io - - traefik.containo.us - resources: - - storageclasses - verbs: - - list - ---- - -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: cluster-readonly -subjects: -- kind: Group - name: user - apiGroup: "" -roleRef: - kind: ClusterRole - name: cluster-readonly - apiGroup: "" diff --git a/k3s/clusterrole.yaml b/k3s/clusterrole.yaml deleted file mode 100755 index 8607392..0000000 --- a/k3s/clusterrole.yaml +++ /dev/null @@ -1,16 +0,0 @@ -kind: ClusterRole -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: user-readonly -rules: -- apiGroups: - - rbac.authorization.k8s.io - - storage.k8s.io - - networking.k8s.io - resources: - - clusterroles - - storageclasses - - ingressclasses - verbs: - - list - - watch diff --git a/k3s/scripts/setup.sh b/k3s/scripts/setup.sh deleted file mode 100755 index b5f7f9f..0000000 --- a/k3s/scripts/setup.sh +++ /dev/null @@ -1,12 +0,0 @@ -#!/bin/bash - -# Use -# ./setup.sh - -export SERVER=$1 - -ssh -t $SERVER sudo kubectl -n kube-system create secret generic certsigner --from-file /var/lib/rancher/k3s/server/tls/client-ca.crt --from-file /var/lib/rancher/k3s/server/tls/client-ca.key -scp certsigner.yaml $SERVER:~/certsigner.yaml -ssh $SERVER kubectl apply -f certsigner.yaml -scp clusterrole.yaml $SERVER:~/clusterrole.yaml -ssh $SERVER kubectl apply -f clusterrole.yaml diff --git a/k3s/tests/local-storage-test.yaml b/k3s/tests/local-storage-test.yaml new file mode 100644 index 0000000..626fc64 --- /dev/null +++ b/k3s/tests/local-storage-test.yaml @@ -0,0 +1,71 @@ +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: ssd-test + namespace: default +spec: + storageClassName: ssd + accessModes: + - ReadWriteMany + resources: + requests: + storage: 8Gi + +--- + +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: hdd-test + namespace: default +spec: + storageClassName: hdd + accessModes: + - ReadWriteMany + resources: + requests: + storage: 8Gi + +--- + +apiVersion: apps/v1 +kind: Deployment +metadata: + name: local-storage-test + namespace: default +spec: + selector: + matchLabels: + app: local-storage-test + template: + metadata: + labels: + app: local-storage-test + spec: + containers: + - image: debian + command: + - bash + - -c + - 'sleep infinity' + name: local-storage-test + volumeMounts: + - mountPath: /ssd + name: ssd + - mountPath: /hdd + name: hdd + resources: + limits: + memory: "4Gi" + cpu: "2" + requests: + memory: "1Mi" + cpu: "1m" + restartPolicy: Always + volumes: + - name: hdd + persistentVolumeClaim: + claimName: hdd-test + - name: ssd + persistentVolumeClaim: + claimName: ssd-test diff --git a/k3s/upgrade-plan.yaml b/k3s/upgrade-plan.yaml index 10ceca2..2f91277 100644 --- a/k3s/upgrade-plan.yaml +++ b/k3s/upgrade-plan.yaml @@ -16,4 +16,4 @@ spec: serviceAccountName: system-upgrade upgrade: image: rancher/k3s-upgrade - channel: https://update.k3s.io/v1-release/channels/latest \ No newline at end of file + channel: https://update.k3s.io/v1-release/channels/stable \ No newline at end of file diff --git a/kubectl/upgrade-plan.yaml b/kubectl/upgrade-plan.yaml deleted file mode 100644 index 98998fa..0000000 --- a/kubectl/upgrade-plan.yaml +++ /dev/null @@ -1,42 +0,0 @@ -# Server plan -apiVersion: upgrade.cattle.io/v1 -kind: Plan -metadata: - name: server-plan - namespace: system-upgrade -spec: - concurrency: 1 - cordon: true - nodeSelector: - matchExpressions: - - key: node-role.kubernetes.io/control-plane - operator: In - values: - - "true" - serviceAccountName: system-upgrade - upgrade: - image: rancher/k3s-upgrade - channel: https://update.k3s.io/v1-release/channels/latest ---- -# Agent plan -apiVersion: upgrade.cattle.io/v1 -kind: Plan -metadata: - name: agent-plan - namespace: system-upgrade -spec: - concurrency: 1 - cordon: true - nodeSelector: - matchExpressions: - - key: node-role.kubernetes.io/control-plane - operator: DoesNotExist - prepare: - args: - - prepare - - server-plan - image: rancher/k3s-upgrade - serviceAccountName: system-upgrade - upgrade: - image: rancher/k3s-upgrade - channel: https://update.k3s.io/v1-release/channels/latest diff --git a/local-path-provisioner/ConfigMap-patch.yaml b/local-path-provisioner/ConfigMap-patch.yaml new file mode 100644 index 0000000..40a4dfa --- /dev/null +++ b/local-path-provisioner/ConfigMap-patch.yaml @@ -0,0 +1,13 @@ +- op: replace # action + path: /data/config.json # resource we want to change + value: |- + { + "storageClassConfigs": { + "ssd": { + "sharedFileSystemPath": "/opt/local-path-provisioner/ssd" + }, + "hdd": { + "sharedFileSystemPath": "/opt/local-path-provisioner/hdd" + } + } + } \ No newline at end of file diff --git a/local-path-provisioner/StorageClass-hdd-patch.yaml b/local-path-provisioner/StorageClass-hdd-patch.yaml new file mode 100644 index 0000000..b6f2997 --- /dev/null +++ b/local-path-provisioner/StorageClass-hdd-patch.yaml @@ -0,0 +1,3 @@ +- op: replace # action + path: /metadata/name # resource we want to change + value: hdd # value we want to use for patching \ No newline at end of file diff --git a/local-path-provisioner/kustomization.yaml b/local-path-provisioner/kustomization.yaml new file mode 100644 index 0000000..a931167 --- /dev/null +++ b/local-path-provisioner/kustomization.yaml @@ -0,0 +1,19 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - local-path-storage.yaml + - ssd-storage.yaml +patches: +- target: + group: storage.k8s.io + version: v1 + kind: StorageClass + name: local-path + path: StorageClass-hdd-patch.yaml +- target: + group: "" + version: v1 + kind: ConfigMap + path: ConfigMap-patch.yaml +- target: {} + path: namespace-patch.yaml \ No newline at end of file diff --git a/local-path-provisioner/local-path-storage.yaml b/local-path-provisioner/local-path-storage.yaml new file mode 100644 index 0000000..bb32bb7 --- /dev/null +++ b/local-path-provisioner/local-path-storage.yaml @@ -0,0 +1,189 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: local-path-storage +--- +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: local-path +provisioner: rancher.io/local-path +reclaimPolicy: Delete +volumeBindingMode: WaitForFirstConsumer +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: local-path-provisioner-service-account + namespace: local-path-storage +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: local-path-provisioner-role + namespace: local-path-storage +rules: +- apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch + - create + - patch + - update + - delete +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: local-path-provisioner-role +rules: +- apiGroups: + - "" + resources: + - nodes + - persistentvolumeclaims + - configmaps + - pods + - pods/log + verbs: + - get + - list + - watch +- apiGroups: + - "" + resources: + - persistentvolumes + verbs: + - get + - list + - watch + - create + - patch + - update + - delete +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch +- apiGroups: + - storage.k8s.io + resources: + - storageclasses + verbs: + - get + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: local-path-provisioner-bind + namespace: local-path-storage +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: local-path-provisioner-role +subjects: +- kind: ServiceAccount + name: local-path-provisioner-service-account + namespace: local-path-storage +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: local-path-provisioner-bind +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: local-path-provisioner-role +subjects: +- kind: ServiceAccount + name: local-path-provisioner-service-account + namespace: local-path-storage +--- +apiVersion: v1 +data: + config.json: |- + { + "nodePathMap":[ + { + "node":"DEFAULT_PATH_FOR_NON_LISTED_NODES", + "paths":["/opt/local-path-provisioner"] + } + ] + } + helperPod.yaml: |- + apiVersion: v1 + kind: Pod + metadata: + name: helper-pod + spec: + priorityClassName: system-node-critical + tolerations: + - key: node.kubernetes.io/disk-pressure + operator: Exists + effect: NoSchedule + containers: + - name: helper-pod + image: busybox + imagePullPolicy: IfNotPresent + setup: |- + #!/bin/sh + set -eu + mkdir -m 0777 -p "$VOL_DIR" + teardown: |- + #!/bin/sh + set -eu + rm -rf "$VOL_DIR" +kind: ConfigMap +metadata: + name: local-path-config + namespace: local-path-storage +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: local-path-provisioner + namespace: local-path-storage +spec: + replicas: 1 + selector: + matchLabels: + app: local-path-provisioner + template: + metadata: + labels: + app: local-path-provisioner + spec: + containers: + - command: + - local-path-provisioner + - --debug + - start + - --config + - /etc/config/config.json + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: CONFIG_MOUNT_PATH + value: /etc/config/ + image: rancher/local-path-provisioner:v0.0.28 + imagePullPolicy: IfNotPresent + name: local-path-provisioner + volumeMounts: + - mountPath: /etc/config/ + name: config-volume + serviceAccountName: local-path-provisioner-service-account + volumes: + - configMap: + name: local-path-config + name: config-volume diff --git a/local-path-provisioner/namespace-patch.yaml b/local-path-provisioner/namespace-patch.yaml new file mode 100644 index 0000000..c4d98c2 --- /dev/null +++ b/local-path-provisioner/namespace-patch.yaml @@ -0,0 +1,3 @@ +- op: replace # action + path: /metadata/namespace # resource we want to change + value: kube-system \ No newline at end of file diff --git a/local-path-provisioner/ssd-storage.yaml b/local-path-provisioner/ssd-storage.yaml new file mode 100644 index 0000000..82031ed --- /dev/null +++ b/local-path-provisioner/ssd-storage.yaml @@ -0,0 +1,9 @@ +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: ssd + annotations: + storageclass.kubernetes.io/is-default-class: "true" +provisioner: rancher.io/local-path +reclaimPolicy: Delete +volumeBindingMode: WaitForFirstConsumer \ No newline at end of file diff --git a/helm/jellyfin/.helmignore b/minecraft/.helmignore similarity index 100% rename from helm/jellyfin/.helmignore rename to minecraft/.helmignore diff --git a/helm/minecraft/Chart.yaml b/minecraft/Chart.yaml similarity index 100% rename from helm/minecraft/Chart.yaml rename to minecraft/Chart.yaml diff --git a/helm/minecraft/templates/configmap.yaml b/minecraft/templates/configmap.yaml similarity index 100% rename from helm/minecraft/templates/configmap.yaml rename to minecraft/templates/configmap.yaml diff --git a/helm/minecraft/templates/deployment.yaml b/minecraft/templates/deployment.yaml similarity index 100% rename from helm/minecraft/templates/deployment.yaml rename to minecraft/templates/deployment.yaml diff --git a/helm/minecraft/templates/pvc.yaml b/minecraft/templates/pvc.yaml similarity index 90% rename from helm/minecraft/templates/pvc.yaml rename to minecraft/templates/pvc.yaml index e083ce0..ab09174 100644 --- a/helm/minecraft/templates/pvc.yaml +++ b/minecraft/templates/pvc.yaml @@ -5,6 +5,7 @@ metadata: annotations: "helm.sh/resource-policy": keep spec: + storageClassName: ssd accessModes: - ReadWriteOnce resources: diff --git a/helm/minecraft/templates/service.yaml b/minecraft/templates/service.yaml similarity index 100% rename from helm/minecraft/templates/service.yaml rename to minecraft/templates/service.yaml diff --git a/helm/minecraft/values.yaml b/minecraft/values.yaml similarity index 100% rename from helm/minecraft/values.yaml rename to minecraft/values.yaml diff --git a/helm/minecraft/.helmignore b/snapdrop/.helmignore similarity index 100% rename from helm/minecraft/.helmignore rename to snapdrop/.helmignore diff --git a/helm/nextcloud/Chart.yaml b/snapdrop/Chart.yaml similarity index 100% rename from helm/nextcloud/Chart.yaml rename to snapdrop/Chart.yaml diff --git a/helm/snapdrop/templates/configmap.yaml b/snapdrop/templates/configmap.yaml similarity index 100% rename from helm/snapdrop/templates/configmap.yaml rename to snapdrop/templates/configmap.yaml diff --git a/helm/snapdrop/templates/deployment.yaml b/snapdrop/templates/deployment.yaml similarity index 100% rename from helm/snapdrop/templates/deployment.yaml rename to snapdrop/templates/deployment.yaml diff --git a/helm/snapdrop/templates/ingress.yaml b/snapdrop/templates/ingress.yaml similarity index 100% rename from helm/snapdrop/templates/ingress.yaml rename to snapdrop/templates/ingress.yaml diff --git a/helm/snapdrop/templates/pvc.yaml b/snapdrop/templates/pvc.yaml similarity index 100% rename from helm/snapdrop/templates/pvc.yaml rename to snapdrop/templates/pvc.yaml diff --git a/helm/snapdrop/templates/service.yaml b/snapdrop/templates/service.yaml similarity index 100% rename from helm/snapdrop/templates/service.yaml rename to snapdrop/templates/service.yaml diff --git a/helm/snapdrop/values.yaml b/snapdrop/values.yaml similarity index 100% rename from helm/snapdrop/values.yaml rename to snapdrop/values.yaml diff --git a/ubuntu/README.md b/ubuntu/README.md new file mode 100644 index 0000000..8b697db --- /dev/null +++ b/ubuntu/README.md @@ -0,0 +1,147 @@ +# Ubuntu Server + +- [Ubuntu Server](#ubuntu-server) + - [Setup SSH](#setup-ssh) + - [Fail2Ban](#fail2ban) + - [Automatic Updates](#automatic-updates) + - [Disable Swap](#disable-swap) + - [Extras](#extras) + +Note these instructions differentiate between an `operator` and a `server`. The operator can be +any machine that configure the server. A pipeline, laptop, dedicated server, etc. are all options. +The server can be its own operator, though that's not recommended since servers should be ephemeral +and the operator will store information about each server. + +## Setup SSH + +On the operator: + +```bash +export SSH_HOST=kube +ssh-keygen -t rsa -b 4096 -C ducoterra@"$SSH_HOST".reeselink.com -f ~/.ssh/id_"$SSH_HOST"_rsa + +# Note: If you get "too many authentication failures" it's likely because you have too many private +# keys in your ~/.ssh directory. Use `-o PubkeyAuthentication` to fix it. +ssh-copy-id -o PubkeyAuthentication=no -i ~/.ssh/id_$SSH_HOST_rsa.pub ducoterra@"$SSH_HOST".reeselink.com + +cat <> ~/.ssh/config + +Host $SSH_HOST + Hostname $SSH_HOST.reeselink.com + User root + ProxyCommand none + ForwardAgent no + ForwardX11 no + Port 22 + KeepAlive yes + IdentityFile ~/.ssh/id_"$SSH_HOST"_rsa +EOF + +ssh -o PubkeyAuthentication=no ducoterra@"$SSH_HOST".reeselink.com +``` + +On the server: + +```bash +# Copy authorized_keys to root +sudo cp ~/.ssh/authorized_keys /root/.ssh/authorized_keys + +# Change your password +passwd + +sudo su - +echo "PasswordAuthentication no" > /etc/ssh/sshd_config.d/01-prohibit-password.conf +echo '%sudo ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/01-nopasswd-sudo +``` + +On the operator: + +```bash +# Test if you can SSH with a password +ssh -o PubkeyAuthentication=no ducoterra@"$SSH_HOST".reeselink.com + +# Test that you can log into the server with ssh config +ssh $SSH_HOST +``` + +## Fail2Ban + +On the server: + +```bash +apt update +apt install -y fail2ban + +# Setup initial rules +cat < /etc/fail2ban/jail.local +# Jail configuration additions for local installation + +# Adjust the default configuration's default values +[DEFAULT] +# Optional enter an trusted IP never to ban +ignoreip = 2600:1700:1e6c:a81f::0/64 +bantime = 6600 +backend = auto + +# The main configuration file defines all services but +# deactivates them by default. We have to activate those neeeded +[sshd] +enabled = true +EOF + +systemctl enable fail2ban --now +tail -f /var/log/fail2ban.log +``` + +## Automatic Updates + +On the server: + +```bash +apt install -y unattended-upgrades + +systemctl enable --now unattended-upgrades.service +``` + +## Disable Swap + +```bash +swapoff -a +``` + +## Extras + +On the server: + +```bash +# Install glances for system monitoring +apt install -y glances + +# Install zsh with autocomplete and suggestions +apt install -y zsh zsh-autosuggestions zsh-syntax-highlighting + +cat < ~/.zshrc +# Basic settings +autoload bashcompinit && bashcompinit +autoload -U compinit; compinit +zstyle ':completion:*' menu select + +# Prompt settings +autoload -Uz promptinit +promptinit +prompt redhat +PROMPT_EOL_MARK= + +# Syntax Highlighting +source /usr/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh +source /usr/share/zsh-autosuggestions/zsh-autosuggestions.zsh + +### Custom Commands and Aliases ### +EOF + +chsh -s $(which zsh) && chsh -s $(which zsh) ducoterra + +# Cockpit +apt install -y cockpit +systemctl enable --now cockpit +``` diff --git a/unifi/README.md b/unifi/README.md new file mode 100644 index 0000000..0c9e646 --- /dev/null +++ b/unifi/README.md @@ -0,0 +1,15 @@ +# Unifi Server + + + +## Install + +```bash +apt-get update && apt-get install ca-certificates apt-transport-https +echo 'deb [ arch=amd64,arm64 ] https://www.ui.com/downloads/unifi/debian stable ubiquiti' | tee /etc/apt/sources.list.d/100-ubnt-unifi.list +wget -O /etc/apt/trusted.gpg.d/unifi-repo.gpg https://dl.ui.com/unifi/unifi-repo.gpg +wget -qO - https://www.mongodb.org/static/pgp/server-3.6.asc | apt-key add - +echo "deb [trusted=yes] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/3.6 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-3.6.list +apt-get update +apt-get update && apt-get install unifi -y +``` diff --git a/updates/README.md b/updates/README.md index 8ea9f44..ee1b5c5 100644 --- a/updates/README.md +++ b/updates/README.md @@ -3,7 +3,7 @@ ## Updates ```bash -ansible-playbook -i ansible/inventory.yaml updates/upgrade-kubernetes-nodes.yaml -ansible-playbook -i ansible/inventory.yaml updates/upgrade-colors.yaml +ansible-playbook -i ansible/inventory.yaml updates/upgrade-dnf.yaml ansible-playbook -i ansible/inventory.yaml updates/upgrade-apt.yaml +ansible-playbook -i ansible/inventory.yaml updates/upgrade-pacman.yaml ``` diff --git a/updates/upgrade-apt.yaml b/updates/upgrade-apt.yaml index 73186e3..444da38 100644 --- a/updates/upgrade-apt.yaml +++ b/updates/upgrade-apt.yaml @@ -1,5 +1,7 @@ - name: Upgrade - hosts: apt + hosts: + - ubuntu + - raspbian tasks: - name: Update all packages to their latest version become: true diff --git a/updates/upgrade-colors.yaml b/updates/upgrade-dnf.yaml similarity index 100% rename from updates/upgrade-colors.yaml rename to updates/upgrade-dnf.yaml diff --git a/updates/upgrade-kubernetes-nodes.yaml b/updates/upgrade-pacman.yaml similarity index 100% rename from updates/upgrade-kubernetes-nodes.yaml rename to updates/upgrade-pacman.yaml diff --git a/helm/namespace/.helmignore b/userspace/.helmignore similarity index 100% rename from helm/namespace/.helmignore rename to userspace/.helmignore diff --git a/helm/namespace/Chart.yaml b/userspace/Chart.yaml similarity index 100% rename from helm/namespace/Chart.yaml rename to userspace/Chart.yaml diff --git a/helm/namespace/README.md b/userspace/README.md similarity index 97% rename from helm/namespace/README.md rename to userspace/README.md index 35a6f6e..7d267a8 100755 --- a/helm/namespace/README.md +++ b/userspace/README.md @@ -4,11 +4,10 @@ ### Quickstart -1. Start Docker -2. Run createprojectspace.sh - ```bash -./createprojectspace.sh +./userspace/scripts/setup.sh +./userspace/scripts/upsertuser.sh +./userspace/scripts/removeuser.sh ``` ### Update a user diff --git a/userspace/certsigner.yaml b/userspace/certsigner.yaml new file mode 100644 index 0000000..9c554c5 --- /dev/null +++ b/userspace/certsigner.yaml @@ -0,0 +1,40 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: certsigner + namespace: kube-system +spec: + replicas: 1 + selector: + matchLabels: + app: certsigner + template: + metadata: + labels: + app: certsigner + spec: + containers: + - name: certsigner + image: python:latest + command: ["cat"] + tty: true + resources: + requests: + memory: 1Mi + cpu: 1m + limits: + memory: 100Mi + cpu: 100m + volumeMounts: + - mountPath: /keys + name: keys + - mountPath: /certs + name: certs + volumes: + - name: keys + secret: + secretName: certsigner + - name: certs + emptyDir: + sizeLimit: 500Mi + restartPolicy: Always \ No newline at end of file diff --git a/k3s/scripts/removeuser.sh b/userspace/scripts/removeuser.sh similarity index 100% rename from k3s/scripts/removeuser.sh rename to userspace/scripts/removeuser.sh diff --git a/userspace/scripts/setup.sh b/userspace/scripts/setup.sh new file mode 100755 index 0000000..a3a9151 --- /dev/null +++ b/userspace/scripts/setup.sh @@ -0,0 +1,10 @@ +#!/bin/bash + +# Use +# ./setup.sh + +export SERVER=$1 + +ssh -t $SERVER kubectl -n kube-system create secret generic certsigner --from-file /var/lib/rancher/k3s/server/tls/client-ca.crt --from-file /var/lib/rancher/k3s/server/tls/client-ca.key +scp ./userspace/certsigner.yaml $SERVER:~/certsigner.yaml +ssh $SERVER kubectl apply -f certsigner.yaml diff --git a/k3s/scripts/upsertuser.sh b/userspace/scripts/upsertuser.sh similarity index 97% rename from k3s/scripts/upsertuser.sh rename to userspace/scripts/upsertuser.sh index 9051dd8..7d2d0f2 100755 --- a/k3s/scripts/upsertuser.sh +++ b/userspace/scripts/upsertuser.sh @@ -119,7 +119,7 @@ exit 0 fi echo "Templating namespace with helm and copying to server" -helm template $KUBE_USER --set user=$KUBE_USER ./helm/namespace | ssh $SERVER "cat - > $SERVER_USER_DIR/namespace.yaml" +helm template $KUBE_USER --set user=$KUBE_USER ./userspace | ssh $SERVER "cat - > $SERVER_USER_DIR/namespace.yaml" if [ $? -ne 0 ]; then echo "Failed to template namespace. Is helm installed?" diff --git a/helm/namespace/templates/limitrange.yaml b/userspace/templates/limitrange.yaml similarity index 100% rename from helm/namespace/templates/limitrange.yaml rename to userspace/templates/limitrange.yaml diff --git a/helm/namespace/templates/namespace-manager-role.yaml b/userspace/templates/namespace-manager-role.yaml similarity index 100% rename from helm/namespace/templates/namespace-manager-role.yaml rename to userspace/templates/namespace-manager-role.yaml diff --git a/helm/namespace/templates/namespace-manager-rolebinding.yaml b/userspace/templates/namespace-manager-rolebinding.yaml similarity index 100% rename from helm/namespace/templates/namespace-manager-rolebinding.yaml rename to userspace/templates/namespace-manager-rolebinding.yaml diff --git a/helm/namespace/templates/namespace-readonly-role.yaml b/userspace/templates/namespace-readonly-role.yaml similarity index 100% rename from helm/namespace/templates/namespace-readonly-role.yaml rename to userspace/templates/namespace-readonly-role.yaml diff --git a/helm/namespace/templates/namespace-readonly-rolebinding.yaml b/userspace/templates/namespace-readonly-rolebinding.yaml similarity index 100% rename from helm/namespace/templates/namespace-readonly-rolebinding.yaml rename to userspace/templates/namespace-readonly-rolebinding.yaml diff --git a/helm/namespace/templates/namespace.yaml b/userspace/templates/namespace.yaml similarity index 100% rename from helm/namespace/templates/namespace.yaml rename to userspace/templates/namespace.yaml diff --git a/helm/namespace/templates/resourcequota.yaml b/userspace/templates/resourcequota.yaml similarity index 100% rename from helm/namespace/templates/resourcequota.yaml rename to userspace/templates/resourcequota.yaml diff --git a/helm/namespace/values.yaml b/userspace/values.yaml similarity index 100% rename from helm/namespace/values.yaml rename to userspace/values.yaml