add bifrost docs

2026-03-16 09:52:57 -04:00
parent 88ecb458e1
commit d51560f979
4 changed files with 260 additions and 0 deletions
--- a/active/container_bifrost/bifrost.md
+++ b/active/container_bifrost/bifrost.md
@@ -0,0 +1,208 @@
+# Podman bifrost
+
+- [Podman bifrost](#podman-bifrost)
+  - [Setup bifrost Project](#setup-bifrost-project)
+  - [Install bifrost](#install-bifrost)
+    - [Create the ai user](#create-the-ai-user)
+    - [Write the bifrost compose spec](#write-the-bifrost-compose-spec)
+      - [A Note on Volumes](#a-note-on-volumes)
+    - [Convert bifrost compose spec to quadlets](#convert-bifrost-compose-spec-to-quadlets)
+    - [Start and enable your systemd quadlet](#start-and-enable-your-systemd-quadlet)
+    - [Expose bifrost](#expose-bifrost)
+  - [Using bifrost](#using-bifrost)
+    - [Adding Models](#adding-models)
+    - [Testing Models](#testing-models)
+  - [Backup bifrost](#backup-bifrost)
+  - [Upgrade bifrost](#upgrade-bifrost)
+    - [Upgrade Quadlets](#upgrade-quadlets)
+  - [Uninstall](#uninstall)
+  - [Notes](#notes)
+    - [SELinux](#selinux)
+
+## Setup bifrost Project
+
+- [ ] Copy and rename this folder to active/container_bifrost
+- [ ] Find and replace bifrost with the name of the service.
+- [ ] Create the rootless user to run the podman containers
+- [ ] Write the compose.yaml spec for your service
+- [ ] Convert the compose.yaml spec to a quadlet
+- [ ] Install the quadlet on the podman server
+- [ ] Expose the quadlet service
+- [ ] Install a backup service and timer
+
+## Install bifrost
+
+### Create the ai user
+
+```bash
+# SSH into your podman server as root
+useradd ai
+loginctl enable-linger $(id -u ai)
+systemctl --user --machine=ai@.host enable podman-restart
+systemctl --user --machine=ai@.host enable --now podman.socket
+mkdir -p /home/ai/.config/containers/systemd
+```
+
+### Write the bifrost compose spec
+
+Edit the compose.yaml at active/container_bifrost/compose/compose.yaml
+
+#### A Note on Volumes
+
+Named volumes are stored at `/home/bifrost/.local/share/containers/storage/volumes/`.
+
+### Convert bifrost compose spec to quadlets
+
+Run the following to convert a compose.yaml into the various `.container` files for systemd:
+
+```bash
+# Generate the systemd service
+podman run \
+--security-opt label=disable \
+--rm \
+-v $(pwd)/active/container_bifrost/compose:/compose \
+-v $(pwd)/active/container_bifrost/quadlets:/quadlets \
+quay.io/k9withabone/podlet \
+-f /quadlets \
+-i \
+--overwrite \
+compose /compose/compose.yaml
+
+# Copy the files to the server
+export PODMAN_SERVER=ai-ai
+scp -r active/container_bifrost/quadlets/. $PODMAN_SERVER:/home/ai/.config/containers/systemd/
+```
+
+### Start and enable your systemd quadlet
+
+SSH into your podman server as root:
+
+```bash
+systemctl --user daemon-reload
+systemctl --user restart bifrost
+journalctl --user -u bifrost -f
+# Enable auto-update service which will pull new container images automatically every day
+systemctl --user enable --now podman-auto-update.timer
+```
+
+### Expose bifrost
+
+1. If you need a domain, follow the [DDNS instructions](/active/container_ddns/ddns.md#install-a-new-ddns-service)
+2. For a web service, follow the [Caddy instructions](/active/container_caddy/caddy.md#adding-a-new-caddy-record)
+3. Finally, follow your OS's guide for opening ports via its firewall service.
+
+## Using bifrost
+
+### Adding Models
+
+```json
+// qwen3.5-35b-a3b-thinking
+{
+    "temperature": 1,
+    "top_p": 0.95,
+    "presence_penalty": 1.5,
+    "extra_body": {
+        "top_k": 20,
+        "min_p": 0,
+        "repetition_penalty": 1,
+        "chat_template_kwargs": {
+            "enable_thinking": true
+        }
+    }
+}
+
+// qwen3.5-35b-a3b-coding
+{
+    "temperature": 0.6,
+    "top_p": 0.95,
+    "presence_penalty": 0,
+    "extra_body": {
+        "top_k": 20,
+        "min_p": 0,
+        "repetition_penalty": 1,
+        "chat_template_kwargs": {
+            "enable_thinking": true
+        }
+    }
+}
+
+// qwen3.5-35b-a3b-instruct
+{
+    "temperature": 0.7,
+    "top_p": 0.8,
+    "presence_penalty": 1.5,
+    "extra_body": {
+        "top_k": 20,
+        "min_p": 0,
+        "repetition_penalty": 1,
+        "chat_template_kwargs": {
+            "enable_thinking": false
+        }
+    }
+}
+```
+
+### Testing Models
+
+```bash
+# List models
+curl -L -X GET 'https://aipi.reeseapps.com/v1/models' \
+-H 'Content-Type: application/json' \
+-H 'Authorization: Bearer sk-1234'
+
+curl -L -X POST 'https://aipi.reeseapps.com/v1/chat/completions' \
+-H 'Content-Type: application/json' \
+-H 'Authorization: Bearer sk-1234' \
+-d '{
+    "model": "gpt-4o-mini", # 👈 REPLACE with 'public model name' for any db-model
+    "messages": [
+        {
+            "content": "Hey, how's it going",
+            "role": "user"
+        }
+    ],
+}'
+```
+
+## Backup bifrost
+
+Follow the [Borg Backup instructions](/active/systemd_borg/borg.md#set-up-a-client-for-backup)
+
+## Upgrade bifrost
+
+### Upgrade Quadlets
+
+Upgrades should be a repeat of [writing the compose spec](#convert-bifrost-compose-spec-to-quadlets) and [installing the quadlets](#start-and-enable-your-systemd-quadlet)
+
+```bash
+export PODMAN_SERVER=
+scp -r quadlets/. $PODMAN_SERVER$:/home/bifrost/.config/containers/systemd/
+ssh bifrost systemctl --user daemon-reload
+ssh bifrost systemctl --user restart bifrost
+```
+
+## Uninstall
+
+```bash
+# Stop the user's services
+systemctl --user disable podman-restart
+podman container stop --all
+systemctl --user disable --now podman.socket
+systemctl --user disable --now podman-auto-update.timer
+
+# Delete the user (this won't delete their home directory)
+# userdel might spit out an error like:
+# userdel: user bifrost is currently used by process 591255
+# kill those processes and try again
+userdel bifrost
+```
+
+## Notes
+
+### SELinux
+
+<https://blog.christophersmart.com/2021/01/31/podman-volumes-and-selinux/>
+
+:z allows a container to share a mounted volume with all other containers.
+
+:Z allows a container to reserve a mounted volume and prevents any other container from accessing.
--- a/active/container_bifrost/compose/README.md
+++ b/active/container_bifrost/compose/README.md
@@ -0,0 +1,3 @@
+# Compose
+
+Put your compose.yaml here.
--- a/active/container_bifrost/compose/compose.yaml
+++ b/active/container_bifrost/compose/compose.yaml
@@ -0,0 +1,32 @@
+services:
+  bifrost:
+    image: docker.io/maximhq/bifrost:latest
+    container_name: bifrost
+    ports:
+      - "8000:8000"
+    volumes:
+      - bifrost-data:/app/data
+    environment:
+      - APP_PORT=8000
+      - APP_HOST=0.0.0.0
+      - LOG_LEVEL=info
+      - LOG_STYLE=json
+    ulimits:
+      nofile:
+        soft: 65536
+        hard: 65536
+    healthcheck:
+      test:
+        [
+          "CMD",
+          "wget",
+          "--no-verbose",
+          "--tries=1",
+          "-O",
+          "/dev/null",
+          "http://localhost:8080/health",
+        ]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+    restart: unless-stopped
--- a/active/container_bifrost/quadlets/bifrost.container
+++ b/active/container_bifrost/quadlets/bifrost.container
@@ -0,0 +1,17 @@
+[Container]
+ContainerName=bifrost
+Environment=APP_PORT=8000 APP_HOST=0.0.0.0 LOG_LEVEL=info LOG_STYLE=json
+HealthCmd=["wget", "--no-verbose", "--tries=1", "-O", "/dev/null", "http://localhost:8080/health"]
+HealthInterval=30s
+HealthRetries=3
+HealthTimeout=10s
+Image=docker.io/maximhq/bifrost:latest
+PublishPort=8000:8000
+Ulimit=nofile=65536:65536
+Volume=bifrost-data:/app/data
+
+[Service]
+Restart=always
+
+[Install]
+WantedBy=default.target