homelab/README.md

# Homelab

Welcome to my homelab!

This repo is an in-flux collection of my personal notes, docs, and tutorials of
things I find interesting and self-host.

Take a look around!

- "Active" projects (/active) are in use today and generally fall into these
  categories:
  - `aws_` is for aws notes
  - `device_` is for hardware
  - `kubernetes_` is for helm charts or other kubernetes hosted software
  - `os_` is for operating system setup guides and notes
  - `podman_` is for containerized projects
  - `software_` is for cli tools, projects without a specific way to host them,
    or other misfits

All active projects will have a markdown file named after the project. This is
for quick access via shortcuts like `ctrl + p` in vscode. For example, I want
to check my notes for `virsh` so I would type `ctrl + p` "virsh" to open
"virsh.md".

"Retired" projects (/retired) is a graveyard of things I didn't want to delete.

"Template" projects (/templates) are quick templates for creating new active
projects with sane defaults.

I keep my GPG and SSH keys in `keys` if you want to add those to your keyring
or give me access to your servers.

## Table of Contents

- [Homelab](#homelab)
  - [Table of Contents](#table-of-contents)
  - [Fun Facts](#fun-facts)
    - [Keyboard Shortcuts](#keyboard-shortcuts)
    - [inputrc](#inputrc)
    - ["find ." shortcuts](#find--shortcuts)
    - [tmux](#tmux)
    - [bash](#bash)
  - [SSH Setup](#ssh-setup)
  - [Git GPG Commit Signing](#git-gpg-commit-signing)
  - [Important Dates and Times](#important-dates-and-times)
  - [Project Lifecycle](#project-lifecycle)
  - [Project Types](#project-types)
  - [Active Project Requirements](#active-project-requirements)
  - [Retirement Requirements](#retirement-requirements)
  - [Project Structure](#project-structure)
  - [Creating a Project](#creating-a-project)
  - [Order of Operations](#order-of-operations)

## Fun Facts

### Keyboard Shortcuts

On linux, <kbd>ctrl</kbd>+<kbd>shift</kbd>+<kbd>u</kbd>, then, while holding
<kbd>ctrl</kbd>+<kbd>shift</kbd>, typing <kbd>b</kbd>+<kbd>0</kbd> will type a
° (degree) symbol. Also you can enter any unicode symbol this way.

In vim: `esc + o` will take you to the end of a file and insert a new line.

### inputrc

Add this to your `~/.inputrc` to allow ctrl + backspace to delete whole words.

```bash
"\C-h": backward-kill-word
```

### "find ." shortcuts

```bash
# Change file mode for a bunch of directories
find . -type d -exec chmod 755 {} \;
```

### tmux

- Vertical: ctrl + b + "
- Horizontal: ctrl + b + %
- Event Horizontal Distribution: ctrl + b + alt + 1
- Even Vertical Distribution: ctrl + b + alt + 2
- Swap pane order: ctrl + b + : -> swap-pane -t 0

### bash

<https://tecadmin.net/bash-special-variables/>

Here are some handy references for default bash variables

```text
$0 – The name of the script being executed.
$1-$9 – The first nine command-line arguments.
$# – The number of command-line arguments.
$* – All command-line arguments as a single string.
$@ – All command-line arguments as an array.
$? – The exit status of the last executed command.
$$ – The process ID of the current shell.
$! – The process ID of the last background command.
$- – Shows the current shell options or flags. 
```

And here are the meanings of the shell options

```text
h – Remember the location of commands as they are looked up
i – Interactive shell
m – Job control is enabled
B – Brace expansion is enabled
H – History substitution is enabled
```

So to check if you are in an interactive shell:

```bash
[ $- == *i* ]] && Some command here
```

## SSH Setup

Generate a key (password protect it!)

```bash
# Pick one of the below key types
# ed25519
ssh-keygen -C ssh@ducoterra.net -t ed25519
# rsa 4096
ssh-keygen -C ssh@ducoterra.net -t rsa -b 4096

# Inspect a key
ssh-keygen -l -f ~/.ssh/id_rsa

# Change the password
ssh-keygen -p -f ~/.ssh/id_rsa
```

In your ~/.ssh/config, add the following line to set the default key

```conf
IdentityFile ~/.foo/identity
```

Then add a host to your local computer

```bash
Host <hostname>
    Hostname <host.something.com or IP address>
    User <remote user>
    Port <remote port>
```

And copy the key to a remote computer

```bash
# Copy the generated key to the server using password auth. Assumes password auth enabled.
ssh-copy-id -f -i ~/.ssh/id_ed25519 ${REMOTE_USER}@${REMOTE_HOST}

# Log into the server with your key
ssh -i ${KEY_NAME} ${REMOTE_HOST}
# Copy authorized_keys to root
sudo mkdir -p /root/.ssh
sudo cp ~/.ssh/authorized_keys /root/.ssh/authorized_keys
exit

# login and disable password auth
ssh ${REMOTE_HOST}
mkdir -p /etc/ssh/sshd_config.d
echo "PasswordAuthentication no" > /etc/ssh/sshd_config.d/01-prohibit-password.conf
systemctl restart sshd

# OPTIONAL: Disable sudo password
echo '%wheel    ALL=(ALL)   NOPASSWD: ALL' > /etc/sudoers.d/01-nopasswd-wheel

exit

# Test if you can SSH with a password
ssh -o PubkeyAuthentication=no ducoterra@${SSH_HOST}.reeselink.com

# Test that you can log into the server with ssh config
ssh $SSH_HOST
```

## Git GPG Commit Signing

1. Use `gpg --list-key 'git@ducoterra.net'` to find your key
2. Use `git config --global user.signingkey 0A46826A...` to set the signing key
3. Use `gpg --export -a 'git@ducoterra.net'` to export the key to copy into Gitea/Github/Gitlab

Now you can sign commits with `git commit -S`.

Alternatively, you can sign every commit by default with `git config --global commit.gpgsign true`.

You can verify a commit with `git verify-commit e1e551c`. If the commit is
signed you'll see an output. If not, nothing will show.

## Important Dates and Times

| Time  | Day      | Description                        |
| ----- | -------- | ---------------------------------- |
| 00:00 | All      | Automated builds                   |
| 00:00 | All      | NAS Snapshots                      |
| 02:00 | All      | Backups                            |
| 04:00 | All      | Bare Metal Server Security Updates |
| 05:00 | All      | VM Server Security Updates         |
| 05:00 | All      | Unifi Protect Firmware Updates     |
| 06:00 | All      | Unifi Network Firmware Updates     |
| 06:00 | Saturday | Truenas Disk Scrub                 |

## Project Lifecycle

Projects will either be `active` or `retired`.

Active projects are being actively developed. They are in-use, stable, and
production ready. Active projects should meet and track the [active project
requirements](#active-project-requirements)

Retired projects are no longer in use or recommended. They are kept for
reference. Retired projects must meet the [retirement
requirements](#retirement-requirements)

You'll notice that most of the active projects have scripts or examples that
use the `active` path as part of their install process. When moved outside the
`active` directory their scripts and examples break. This is intentional. If
you want a retired project to work again, bring it back to the active
directory.

## Project Types

All projects will be prefixed with one of the following categories:

- `device_`
- `os_`
- `software_`
- `podman_`
- `docker_`
- `kubernetes_`

Note, some projects will be named with just the prefix. These are projects for
configuring the underlying technology. The `podman` project, for example, will
tell you how to configure and install podman so it works correctly.

`device_` will prefix projects that relate to specific machines or equipment.
3D printers, Raspberry Pis, and other IOT devices qualify as specialized
hardware that needs documentation and configuration. This is not limited to
computer equipment. The furnace is an important part of the homelab. the Air
Conditioner is integral to the homelab's function. These projects will also be
documented.

`os_` will contain projects that set up operating systems. These include best
practices, backups, updates, default software, etc.

`cloud_` projects are for specific cloud providers. This will contain
documentation and errata for things like AWS IAM, Route53, etc. Note these will
be prefixed with the cloud's name, not the word "cloud". So AWS services will
be prefixed with `aws_` and azure would be `azure_`. This should make them more
searchable.

`software_` projects record configuration for common software agnostic to
operating system or linux flavor.

`podman_` projects are either designed to be run as quadlets or as podman
containers outright.

`kubernetes_` projects are helm, kustomize, kubectl, or some other kubernetes
compliant deployment.

## Active Project Requirements

- [ ] Installation is documented
- [ ] Installation configuration examples are provided
- [ ] Hardening guidelines are documented
- [ ] Upgrade procedures are documented
- [ ] Maintenance procedures are documented
- [ ] Uninstall procedures are documented
- [ ] Backup and restore procedures are documented and tested

## Retirement Requirements

- [ ] A reason for retirement is documented
- [ ] If applicable, a replacement has been identified and documented
- [ ] If applicable, backup data locations are documented

## Project Structure

All projects will have, at minimum.

1. A README named `project-name.md`
2. A directory called `secrets` which will be gitignored.

## Creating a Project

Assuming your project name is `my-project` and it runs on `podman`

1. Create a new directory called `podman_my-project` under the `active`
   directory
2. Copy the readme template: `cp project_readme_template.md
   active/podman_my-project/my-project.md`
3. Populate `my-project.md` as you work through the install process
4. Create a directory called `secrets` in `podman_my-project`. This will be
   automatically gitignored. Put all secrets here.
5. Push the changes when you have a working product

## Order of Operations

1. Configure cloud providers. These usually have no dependencies and typically
   provide critical services to other projects (DNS, email notifications, etc.)
2. Install infrastructure projects. Usually these only have dependencies on
   cloud services.
3. Install systemd, kubernetes, docker, podman, and other services.