services/homelab
1
1
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases Wiki Activity

coredns updates with AWS user stuff

Browse Source
This commit is contained in:
ducoterra
2023-03-31 14:53:45 -04:00
parent 3c0b8ab008
commit 4afa2c621d
11 changed files with 755 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

83
FedoraServer.md
View File

@@ -18,12 +18,95 @@ sudo systemctl restart sshd
## Certbot for Cockpit ## Certbot for Cockpit
### AWS User
Create an AWS user which will have route53 access. This is required for certbot's route53
validation.
```bash
aws iam create-user --user-name replicator
```
You'll also need a policy which allows the user to modify the selected hosted zone:
(list with `aws route53 list-hosted-zones`)
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"route53:ListHostedZones",
"route53:GetChange"
],
"Resource": [
"*"
]
},
{
"Effect" : "Allow",
"Action" : [
"route53:ChangeResourceRecordSets"
],
"Resource" : [
"arn:aws:route53:::hostedzone/Z012820733346FJ0U4FUF"
]
}
]
}
```
Attach the policy to the user:
```bash
aws iam attach-user-policy \
--user-name replicator \
--policy-arn arn:aws:iam::892236928704:policy/certbot-route53-reeseapps
```
Generate credentials:
```bash
aws iam create-access-key --user-name replicator
```
On the host machine:
```bash
mkdir ~/.aws
vim ~/.aws/config
```
```conf
[profile default]
region=us-east-2
```
```bash
vim ~/.aws/credentials
```
```conf
[default]
aws_access_key_id=
aws_secret_access_key=
```
### Initial Setup ### Initial Setup
1. Create a "containers" user in AWS. Copy the permissions from Freenas 1. Create a "containers" user in AWS. Copy the permissions from Freenas
2. Create credentials 2. Create credentials
3. Add your credentials to root 3. Add your credentials to root
4. Install the aws cli v2 4. Install the aws cli v2
```bash
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
```
5. Test your credentials with `aws route53 list-hosted-zones` 5. Test your credentials with `aws route53 list-hosted-zones`
```bash ```bash

78
README.md
View File

@@ -21,7 +21,11 @@ curl -sfL https://get.k3s.io | sh -s - \
"--disable" \ "--disable" \
"traefik" \ "traefik" \
"--disable" \ "--disable" \
"local-storage" "local-storage" \
"--disable" \
"coredns" \
"--cluster-dns" \
"10.43.0.10"
``` ```
Now you can change the ownership of (and copy) the k3s.yaml file: Now you can change the ownership of (and copy) the k3s.yaml file:
@@ -41,6 +45,27 @@ We're using SQLite (because it's all we really need). The db is stored at
and restore it by copying it back. Note, you must also copy`/var/lib/rancher/k3s/server/token` and restore it by copying it back. Note, you must also copy`/var/lib/rancher/k3s/server/token`
and use the contents as the token when restoring the backup as data is encrypted with that token. and use the contents as the token when restoring the backup as data is encrypted with that token.
### CoreDNS
We'll use our own coredns server so we can add custom hosts. This prevents the server from collapsing
if the internet drops out (something that apparently happens quite frequently)
```bash
helm repo add coredns https://coredns.github.io/helm
helm repo update
helm upgrade --install \
--namespace=kube-system \
--values coredns-values.yaml \
coredns \
coredns/coredns
```
You can test your dns config with
```bash
kubectl run -it --rm --restart=Never --image=infoblox/dnstools:latest dnstools
```
### Nginx Ingress ### Nginx Ingress
Now we need an ingress solution (preferably with certs for https). We'll be using nginx since Now we need an ingress solution (preferably with certs for https). We'll be using nginx since
@@ -135,22 +160,6 @@ And now you can install the drivers:
helm repo add democratic-csi https://democratic-csi.github.io/charts/ helm repo add democratic-csi https://democratic-csi.github.io/charts/
helm repo update helm repo update
# enc0 bulk storage (nfs)
helm upgrade \
--install \
--values truenas-nfs-enc0.yaml \
--namespace democratic-csi \
--create-namespace \
zfs-nfs-enc0 democratic-csi/democratic-csi
# enc1 fast storage (nfs)
helm upgrade \
--install \
--values truenas-nfs-enc1.yaml \
--namespace democratic-csi \
--create-namespace \
zfs-nfs-enc1 democratic-csi/democratic-csi
# enc0 bulk storage (iscsi) # enc0 bulk storage (iscsi)
helm upgrade \ helm upgrade \
--install \ --install \
@@ -159,7 +168,7 @@ helm upgrade \
--create-namespace \ --create-namespace \
zfs-iscsi-enc0 democratic-csi/democratic-csi zfs-iscsi-enc0 democratic-csi/democratic-csi
# enc1 bulk storage (iscsi) # enc1 fast storage (iscsi)
helm upgrade \ helm upgrade \
--install \ --install \
--values truenas-iscsi-enc1.yaml \ --values truenas-iscsi-enc1.yaml \
@@ -174,6 +183,27 @@ You can test that things worked with:
kubectl apply -f democratic-csi-pvc-test.yaml kubectl apply -f democratic-csi-pvc-test.yaml
``` ```
### Dashboard
Create the dashboard and an admin user with:
```bash
helm upgrade \
--install \
--namespace kubernetes-dashboard \
--create-namespace \
dashboard-user ./helm/dashboard-user
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml
```
Then login with the following:
```bash
kubectl -n kubernetes-dashboard create token admin-user
kubectl proxy
```
### Nextcloud ### Nextcloud
The first chart we'll deploy is nextcloud. The first chart we'll deploy is nextcloud.
@@ -208,6 +238,18 @@ helm upgrade --install \
--create-namespace --create-namespace
``` ```
### Minecraft
Minecraft is available through the custom helm chart (including a server downloader)
```bash
helm upgrade --install \
nimcraft \
./helm/minecraft \
--namespace nimcraft \
--create-namespace
```
### Troubleshooting ### Troubleshooting
Deleting a stuck namespace Deleting a stuck namespace

347
coredns-values.yaml Normal file
View File

@@ -0,0 +1,347 @@
# Default values for coredns.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
image:
repository: coredns/coredns
tag: "1.9.4"
pullPolicy: IfNotPresent
## Optionally specify an array of imagePullSecrets.
## Secrets must be manually created in the namespace.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
##
# pullSecrets:
# - myRegistryKeySecretName
replicaCount: 1
resources:
limits:
cpu: 100m
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
## Create HorizontalPodAutoscaler object.
##
# autoscaling:
# minReplicas: 1
# maxReplicas: 10
# metrics:
# - type: Resource
# resource:
# name: cpu
# targetAverageUtilization: 60
# - type: Resource
# resource:
# name: memory
# targetAverageUtilization: 60
rollingUpdate:
maxUnavailable: 1
maxSurge: 25%
# Under heavy load it takes more that standard time to remove Pod endpoint from a cluster.
# This will delay termination of our pod by `preStopSleep`. To make sure kube-proxy has
# enough time to catch up.
# preStopSleep: 5
terminationGracePeriodSeconds: 30
podAnnotations: {}
# cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
serviceType: "ClusterIP"
prometheus:
service:
enabled: false
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "9153"
monitor:
enabled: false
additionalLabels: {}
namespace: ""
interval: ""
service:
clusterIP: "10.43.0.10"
# loadBalancerIP: ""
# externalIPs: []
# externalTrafficPolicy: ""
# The name of the Service
# If not set, a name is generated using the fullname template
name: ""
annotations: {}
serviceAccount:
create: false
# The name of the ServiceAccount to use
# If not set and create is true, a name is generated using the fullname template
name: ""
annotations: {}
rbac:
# If true, create & use RBAC resources
create: true
# If true, create and use PodSecurityPolicy
pspEnable: false
# The name of the ServiceAccount to use.
# If not set and create is true, a name is generated using the fullname template
# name:
# isClusterService specifies whether chart should be deployed as cluster-service or normal k8s app.
isClusterService: true
# Optional priority class to be used for the coredns pods. Used for autoscaler if autoscaler.priorityClassName not set.
priorityClassName: ""
# Default zone is what Kubernetes recommends:
# https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options
servers:
- zones:
- zone: .
port: 53
# If serviceType is nodePort you can specify nodePort here
# nodePort: 30053
# hostPort: 53
plugins:
- name: errors
# Serves a /health endpoint on :8080, required for livenessProbe
- name: health
configBlock: |-
lameduck 5s
# Serves a /ready endpoint on :8181, required for readinessProbe
- name: ready
# Required to query kubernetes API for data
- name: kubernetes
parameters: cluster.local in-addr.arpa ip6.arpa
configBlock: |-
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
# Serves a /metrics endpoint on :9153, required for serviceMonitor
- name: prometheus
parameters: 0.0.0.0:9153
- name: file
parameters: /etc/coredns/democratic-csi-server.reeselink.db democratic-csi-server.reeselink.com
- name: file
parameters: /etc/coredns/democratic-csi-client.reeselink.db democratic-csi-client.reeselink.com
- name: forward
parameters: . /etc/resolv.conf
- name: cache
parameters: 30
- name: loop
- name: reload
- name: loadbalance
# Complete example with all the options:
# - zones: # the `zones` block can be left out entirely, defaults to "."
# - zone: hello.world. # optional, defaults to "."
# scheme: tls:// # optional, defaults to "" (which equals "dns://" in CoreDNS)
# - zone: foo.bar.
# scheme: dns://
# use_tcp: true # set this parameter to optionally expose the port on tcp as well as udp for the DNS protocol
# # Note that this will not work if you are also exposing tls or grpc on the same server
# port: 12345 # optional, defaults to "" (which equals 53 in CoreDNS)
# plugins: # the plugins to use for this server block
# - name: kubernetes # name of plugin, if used multiple times ensure that the plugin supports it!
# parameters: foo bar # list of parameters after the plugin
# configBlock: |- # if the plugin supports extra block style config, supply it here
# hello world
# foo bar
# Extra configuration that is applied outside of the default zone block.
# Example to include additional config files, which may come from extraVolumes:
# extraConfig:
# import:
# parameters: /opt/coredns/*.conf
extraConfig: {}
# To use the livenessProbe, the health plugin needs to be enabled in CoreDNS' server config
livenessProbe:
enabled: true
initialDelaySeconds: 60
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
successThreshold: 1
# To use the readinessProbe, the ready plugin needs to be enabled in CoreDNS' server config
readinessProbe:
enabled: true
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
successThreshold: 1
# expects input structure as per specification https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#affinity-v1-core
# for example:
# affinity:
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: foo.bar.com/role
# operator: In
# values:
# - master
affinity: {}
# expects input structure as per specification https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#topologyspreadconstraint-v1-core
# for example:
# topologySpreadConstraints:
# - maxSkew: 1
# topologyKey: zone
# whenUnsatisfiable: DoNotSchedule
# labelSelector:
# matchLabels:
# foo: bar
topologySpreadConstraints: []
# Node labels for pod assignment
# Ref: https://kubernetes.io/docs/user-guide/node-selection/
nodeSelector: {}
# expects input structure as per specification https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#toleration-v1-core
# for example:
# tolerations:
# - key: foo.bar.com/role
# operator: Equal
# value: master
# effect: NoSchedule
tolerations: []
# https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
podDisruptionBudget: {}
# configure custom zone files as per https://coredns.io/2017/05/08/custom-dns-entries-for-kubernetes/
zoneFiles:
- filename: democratic-csi-server.reeselink.db
domain: democratic-csi-server.reeselink.com
contents: |
democratic-csi-server.reeselink.com. IN SOA sns.dns.icann.org. noc.dns.icann.org. 2015082541 7200 3600 1209600 3600
democratic-csi-server.reeselink.com. IN NS b.iana-servers.net.
democratic-csi-server.reeselink.com. IN NS a.iana-servers.net.
democratic-csi-server.reeselink.com. IN A 172.20.0.1
- filename: democratic-csi-client.reeselink.db
domain: democratic-csi-client.reeselink.com
contents: |
democratic-csi-client.reeselink.com. IN SOA sns.dns.icann.org. noc.dns.icann.org. 2015082541 7200 3600 1209600 3600
democratic-csi-client.reeselink.com. IN NS b.iana-servers.net.
democratic-csi-client.reeselink.com. IN NS a.iana-servers.net.
democratic-csi-client.reeselink.com. IN A 127.0.0.1
# optional array of extra volumes to create
extraVolumes: []
# - name: some-volume-name
# emptyDir: {}
# optional array of mount points for extraVolumes
extraVolumeMounts: []
# - name: some-volume-name
# mountPath: /etc/wherever
# optional array of secrets to mount inside coredns container
# possible usecase: need for secure connection with etcd backend
extraSecrets: []
# - name: etcd-client-certs
# mountPath: /etc/coredns/tls/etcd
# - name: some-fancy-secret
# mountPath: /etc/wherever
# To support legacy deployments using CoreDNS with the "k8s-app: kube-dns" label selectors.
# See https://github.com/coredns/helm/blob/master/charts/coredns/README.md#adopting-existing-coredns-resources
# k8sAppLabelOverride: "kube-dns"
# Custom labels to apply to Deployment, Pod, Configmap, Service, ServiceMonitor. Including autoscaler if enabled.
customLabels: {}
# Custom annotations to apply to Deployment, Pod, Configmap, Service, ServiceMonitor. Including autoscaler if enabled.
customAnnotations: {}
## Alternative configuration for HPA deployment if wanted
#
hpa:
enabled: false
minReplicas: 1
maxReplicas: 2
metrics: []
## Configue a cluster-proportional-autoscaler for coredns
# See https://github.com/kubernetes-incubator/cluster-proportional-autoscaler
autoscaler:
# Enabled the cluster-proportional-autoscaler
enabled: false
# Number of cores in the cluster per coredns replica
coresPerReplica: 256
# Number of nodes in the cluster per coredns replica
nodesPerReplica: 16
# Min size of replicaCount
min: 0
# Max size of replicaCount (default of 0 is no max)
max: 0
# Whether to include unschedulable nodes in the nodes/cores calculations - this requires version 1.8.0+ of the autoscaler
includeUnschedulableNodes: false
# If true does not allow single points of failure to form
preventSinglePointFailure: true
## Optionally specify some extra flags to pass to cluster-proprtional-autoscaler.
## Useful for e.g. the nodelabels flag.
# customFlags:
# - --nodelabels=topology.kubernetes.io/zone=us-east-1a
image:
repository: registry.k8s.io/cpa/cluster-proportional-autoscaler
tag: "1.8.5"
pullPolicy: IfNotPresent
## Optionally specify an array of imagePullSecrets.
## Secrets must be manually created in the namespace.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
##
# pullSecrets:
# - myRegistryKeySecretName
# Optional priority class to be used for the autoscaler pods. priorityClassName used if not set.
priorityClassName: ""
# expects input structure as per specification https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#affinity-v1-core
affinity: {}
# Node labels for pod assignment
# Ref: https://kubernetes.io/docs/user-guide/node-selection/
nodeSelector: {}
# expects input structure as per specification https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#toleration-v1-core
tolerations: []
# resources for autoscaler pod
resources:
requests:
cpu: "20m"
memory: "10Mi"
limits:
cpu: "20m"
memory: "10Mi"
# Options for autoscaler configmap
configmap:
## Annotations for the coredns-autoscaler configmap
# i.e. strategy.spinnaker.io/versioned: "false" to ensure configmap isn't renamed
annotations: {}
# Enables the livenessProbe for cluster-proportional-autoscaler - this requires version 1.8.0+ of the autoscaler
livenessProbe:
enabled: true
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 3
successThreshold: 1
deployment:
enabled: true
name: ""
## Annotations for the coredns deployment
annotations: {}

23
helm/dashboard-user/.helmignore Normal file
View File

@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/

24
helm/dashboard-user/Chart.yaml Normal file
View File

@@ -0,0 +1,24 @@
apiVersion: v2
name: dashboard
description: A Kubernetes Dashboard User Deployment
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "1.16.0"

20
helm/dashboard-user/templates/dashboard.yaml Normal file
View File

@@ -0,0 +1,20 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard

0
helm/dashboard-user/values.yaml Normal file
View File

23
helm/minecraft/.helmignore Executable file
View File

@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/

23
helm/minecraft/Chart.yaml Executable file
View File

@@ -0,0 +1,23 @@
apiVersion: v2
name: Minecraft
description: A Simple Minecraft Chart
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
appVersion: 1.16.0

107
helm/minecraft/templates/minecraft.yaml Normal file
View File

@@ -0,0 +1,107 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}
data:
eula.txt: |
eula=true
server.properties: {{ toYaml .Values.server_props | indent 2 }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}
spec:
selector:
matchLabels:
app: {{ .Release.Name }}
strategy:
type: Recreate
template:
metadata:
labels:
app: {{ .Release.Name }}
spec:
initContainers:
- name: get-version
image: {{ .Values.get_server.image }}
imagePullPolicy: Always
env:
- name: SERVER_VERSION
value: {{ .Values.server_version | quote }}
volumeMounts:
- mountPath: /downloads
name: data
containers:
- name: {{ .Release.Name }}
image: {{ .Values.image }}
imagePullPolicy: Always
ports:
- containerPort: 25565
volumeMounts:
- mountPath: /mc_data
name: data
- name: properties
mountPath: /mc_data/server.properties
subPath: server.properties
- name: properties
mountPath: /mc_data/eula.txt
subPath: eula.txt
tty: true
stdin: true
env:
- name: MAX_RAM
value: {{ .Values.max_ram | quote }}
- name: MIN_RAM
value: "1"
resources:
requests:
memory: {{ div .Values.max_ram 2 }}Gi
cpu: 1m
limits:
memory: {{ add 1 .Values.max_ram }}Gi
cpu: {{ .Values.max_cpu | quote }}
volumes:
- name: data
persistentVolumeClaim:
claimName: {{ .Release.Name }}
# emptyDir:
# sizeLimit: 2Gi
- name: properties
configMap:
name: {{ .Release.Name }}
securityContext:
fsGroup: 2000
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}
annotations:
"helm.sh/resource-policy": keep
spec:
storageClassName: zfs-iscsi-enc1
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 32Gi
---
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}
spec:
selector:
app: {{ .Release.Name }}
ports:
- port: {{ .Values.port }}
targetPort: 25565
name: {{ .Release.Name }}
type: LoadBalancer

45
helm/minecraft/values.yaml Executable file
View File

@@ -0,0 +1,45 @@
image: ducoterra/minecraft:latest
get_server:
image: ducoterra/get-minecraft:2.0.0
server_version: "1.19.4"
port: 25565
max_cpu: 4
max_ram: 8
server_props: |
max-tick-time=60000
generator-settings=
force-gamemode=false
allow-nether=true
gamemode=0
broadcast-console-to-ops=true
enable-query=false
player-idle-timeout=0
difficulty=2
spawn-monsters=true
op-permission-level=4
pvp=true
snooper-enabled=true
level-type=default
hardcore=false
enable-command-block=false
max-players=20
network-compression-threshold=256
resource-pack-sha1=
max-world-size=29999984
server-port=25565
server-ip=
spawn-npcs=true
allow-flight=true
level-name=world
view-distance=32
resource-pack=
spawn-animals=true
white-list=true
generate-structures=true
online-mode=true
max-build-height=512
level-seed=
prevent-proxy-connections=false
use-native-transport=true
motd=Welcome to Nimcraft!
enable-rcon=false