K3OS

Install

Install as normal with k3os.yaml as your base cloud config file

Config

Copy and edit the config file

cp /k3os/system/config.yaml /var/lib/rancher/k3os/config.yaml

Create a folder to hold the certs

mkdir -p /var/lib/rancher/certs
chown rancher:rancher /var/lib/rancher/certs

scp -r certs/* k3os1:/var/lib/rancher/certs

Grab the certs for etcd and copy them to /certs

    - "--datastore-endpoint=https://etcd1:2379,https://etcd2:2379,https://etcd3:2379"
    - "--datastore-cafile=/var/lib/rancher/certs/ca.pem"
    - "--datastore-certfile=/var/lib/rancher/certs/client.pem"
    - "--datastore-keyfile=/var/lib/rancher/certs/client-key.pem"

Add the token

cat /var/lib/rancher/k3s/server/token

--token $token

Install traefik

See traefik project

Users

Generate certs for a new user

export USER=

openssl genrsa -out $USER.key 2048
openssl req -new -key $USER.key -out $USER.csr -subj "/CN=$USER/O=user"
openssl x509 -req -in $USER.csr -CA ../kube/client-ca.crt -CAkey ../kube/client-ca.key -CAcreateserial -out $USER.crt -days 5000

Create namespace

export USER=

kubectl create namespace $USER
kubectl -n $USER create role $USER --verb=get,list,create,update,patch,watch,delete,deletecollection --resource=deployments,daemonsets,pods,pods/exec,pods/log,pods/attach,services,secrets,configmaps,persistentvolumeclaims,endpoints,ingresses.extensions,ingresses.networking.k8s.io,ingressroutes.traefik.containo.us
kubectl -n $USER create rolebinding -n $USER $USER --role=$USER --user=$USER

Copy certs

export SERVER=
export USER=
scp $SERVER:~/$USER/$USER.crt  $SERVER:~/$USER/$USER.csr  $SERVER:~/$USER/$USER.key certs/

Test a bunch of deploys

for i in {1..10}; do kubectl create deploy test$i --image=nginx; done;
for i in {1..10}; do kubectl delete deploy test$i; done;

NFS Storage

helm upgrade --install --kube-context k3os-admin \
--set nfs.server=freenas \
--set nfs.path=/mnt/nvme/kube \
--set storageClass.name=nvme \
--set storageClass.defaultClass=true \
--set storageClass.allowVolumeExpansion=true \
--set storageClass.archiveOnDelete=true \
nvme stable/nfs-client-provisioner

2.2 KiB Raw Permalink Blame History