7.1 KiB
Fedora Server
Fedora server is an awesome container hosting OS. It has a lot built in, and setup is pretty quick.
Initialization
dnf install vim pwgen wireguard-toolshostnamectl hostname node1- Set a static IP through the web interface
Disable swap
```bash
swapoff -a
dnf remove zram-generator-defaults
```
mask <systemd-zram-setup@zram0.service>
Enable ISCSI
# Install the following system packages
dnf install -y lsscsi iscsi-initiator-utils sg3_utils device-mapper-multipath
# Enable multipathing
mpathconf --enable --with_multipathd y
# Ensure that iscsid and multipathd are running
systemctl enable --now iscsid multipathd
# Test that discovery works
iscsiadm -m discovery -t st -p democratic-csi-server.reeselink.com
# Remove them - democratic-csi will populate this
rm -rf /var/lib/iscsi/nodes/
# Start and enable iscsi
systemctl enable --now iscsi
Disable Firewalld
https://docs.k3s.io/advanced#red-hat-enterprise-linux--centos--fedora
Disable firewalld. You could add rules for each service but every time you open a port from a container you'd need to run a firewalld rule.
You can disable firewalld from the web interface.
Set SELinux to Permissive
K3S is more than capable of running with SELinux set to enforcing. We won't be doing that, however. We'll set it to permissive and you can reenable it once you've added all the rules you need to keep your services running.
Set SELinux to permissive by editing /etc/selinux/config
SELINUX=permissive
Install K3S
https://docs.k3s.io/installation/requirements
We're going to be tweaking some installation parameters so if you already have k3s installed you can either uninstall it or skip these steps.
This installation disables Traefik, local-storage, and Klipper. We'll replace them with our own components.
-
Generate a secure token for each node to use when connecting
umask 077 echo -n $(pwgen 16 4) | sed 's/ /-/g' > token.txt
-
Create the cluster
export SECRET=$(cat token.txt)
curl -sfL https://get.k3s.io | K3S_TOKEN=$SECRET sh -s -
"--cluster-init"
"--flannel-backend=wireguard-native"
"--disable"
"traefik"
"--disable"
"local-storage"
"--disable"
"coredns"
"--disable"
"servicelb"
"--cluster-dns"
"10.43.0.10" -
Join each server node
export SECRET=$(cat token.txt)
curl -sfL https://get.k3s.io | K3S_TOKEN=$SECRET sh -s - server
--server https://node1.reeselink.com:6443
--flannel-backend=wireguard-native
"--disable"
"traefik"
"--disable"
"local-storage"
"--disable"
"coredns"
"--disable"
"servicelb"
"--cluster-dns"
"10.43.0.10"
Now you can change the ownership of (and copy) the k3s.yaml file:
chown ducoterra /etc/rancher/k3s/k3s.yaml
scp /etc/rancher/k3s/k3s.yaml ~/.kube/config
Edit ~/.kube/config and change 127.0.0.1 to containers.reeselink.com
Database Backups
https://docs.k3s.io/cli/etcd-snapshot
Note, you must backup /var/lib/rancher/k3s/server/token
and use the contents as the token when restoring the backup as data is encrypted with that token.
Expanding Root Partition
dnf install cloud-utils-growpart
growpart /dev/sda 3
lvextend -l +100%FREE fedora
xfs_growfs /dev/mapper/fedora-root
Optional Steps
Certbot for Cockpit
During this process you'll pick one node to act as your manager for your other nodes. You'll only need to cert a single node and then it will connect via ssh over your local network to the other nodes.
Create an AWS user which will have route53 access. This is required for certbot's route53 validation.
aws iam create-user --user-name replicator
You'll also need a policy which allows the user to modify the selected hosted zone:
(list with aws route53 list-hosted-zones)
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"route53:ListHostedZones",
"route53:GetChange"
],
"Resource": [
"*"
]
},
{
"Effect" : "Allow",
"Action" : [
"route53:ChangeResourceRecordSets"
],
"Resource" : [
"arn:aws:route53:::hostedzone/Z012820733346FJ0U4FUF"
]
}
]
}
Attach the policy to the user:
aws iam attach-user-policy \
--user-name replicator \
--policy-arn arn:aws:iam::892236928704:policy/certbot-route53-reeseapps
Generate credentials:
aws iam create-access-key --user-name replicator
On the host machine:
mkdir ~/.aws
vim ~/.aws/config
[profile default]
region=us-east-2
vim ~/.aws/credentials
[default]
aws_access_key_id=<key>
aws_secret_access_key=<key>
Install the aws cli v2 on the manager node:
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
Test your credentials with aws route53 list-hosted-zones. You should see as list of your
hosted zones.
Now install certbot and acquire a cert using those credentials:
sudo dnf install certbot python3-certbot-dns-route53
sudo certbot certonly --dns-route53 -d containers.reeselink.com
sudo cp /etc/letsencrypt/live/containers.reeselink.com/fullchain.pem /etc/cockpit/ws-certs.d/50-letsencrypt.cert
sudo cp /etc/letsencrypt/live/containers.reeselink.com/privkey.pem /etc/cockpit/ws-certs.d/50-letsencrypt.key
Test the renewal process with:
sudo certbot renew --cert-name containers.reeselink.com --dry-run
Create a renewal script in /usr/lib/scripts/certbot-renew.sh
/usr/lib/scripts/certbot-renew.sh (chmod +x)
#!/bin/bash
/usr/bin/certbot renew --cert-name containers.reeselink.com
/usr/bin/cp -f /etc/letsencrypt/live/containers.reeselink.com/fullchain.pem /etc/cockpit/ws-certs.d/50-letsencrypt.cert
/usr/bin/cp -f /etc/letsencrypt/live/containers.reeselink.com/privkey.pem /etc/cockpit/ws-certs.d/50-letsencrypt.key
Now create a systemd oneshot service to run the script
/etc/systemd/system/certbot-renew.service
[Unit]
Description=Certbot Renewal
[Service]
Type=oneshot
ExecStart=/usr/lib/scripts/certbot-renew.sh
/etc/systemd/system/certbot-renew.timer
[Unit]
Description=Timer for Certbot Renewal
[Timer]
OnBootSec=300
OnUnitActiveSec=1w
[Install]
WantedBy=multi-user.target
Enable the service
systemctl enable --now certbot-renew.timer
Cockpit now has a valid TLS certificate that auto-renews!