dashboards/README.md

Homelab Dashboards

Three dashboards, one domain, one process, one SQLite file:

Page	What it answers	Source
/	Is anything wrong, and which dashboard do I need?	all of the below, one request
/system	What are my machines doing — CPU, memory, disk, GPU, network, and the history of all of it	agent/host_agent.py, our own stdlib agent per host
/llm	Tokens/sec, active sessions, queue depth, KV-cache pressure, latency percentiles	llama.cpp-family (gufo) + vLLM /metrics, read-only
/ups	Battery health, runtime left, power events, and battery-test buttons	NUT upsd on 3493 (its own wire protocol client)

No Prometheus, no Grafana, no CDN, no build step. Nothing is fetched from the internet at runtime; every asset is served by this app.

Quickstart

uv sync --python 3.14                       # deps + the package, editable
cp .env.example .env                        # then edit
uv run dashboards                           # http://127.0.0.1:8090
uv run uvicorn dashboards.app:app --port 8090   # equivalent
uv run pytest                               # unit tests, no network needed

Developing against the real fleet from a laptop (agents stay unexposed — ssh forwards them to localhost):

cp .env.dev.example .env.dev                # your ssh aliases + which ports
python3 - <<'PY'                            # push the agent to each host (once)
import shlex, subprocess, pathlib
for host in ("driveripper-ops", "deskwork-ops", "gateway-ops"):
    subprocess.run(["ssh", host, "mkdir -p ~/lab-agent"], check=True)
    subprocess.run(["scp", "agent/host_agent.py", f"{host}:~/lab-agent/"], check=True)
PY
./scripts/dev-tunnels.sh start              # 9121/9122/9123 -> each host's agent
set -a; . ./.env.dev; set +a
uv run dashboards

The host agent

agent/host_agent.py is one file, stdlib only, Python 3.10+. It reads /proc, /sys/block, /sys/class/drm and statvfs and answers GET /snapshot as JSON.

python3 host_agent.py --once                                # print one snapshot, exit
python3 host_agent.py --host 0.0.0.0 --port 9120 --token …  # serve, token required
deploy/install-agent.sh                                     # systemd unit (run with sudo)

It is read-only by construction: there is no code path that writes, and the HTTP server only answers GET. Counters are reported as counters — the dashboard computes rates over a labelled window, so the agent never has to remember who polled it last.

Configuration

Everything comes from the environment (see .env.example for the annotated list). At startup a .env next to the checkout is loaded first if it exists — as a seed: a variable the container, systemd unit, or your shell already set always wins. Point DASH_ENV_FILE somewhere else to keep several installs on one box. .env.example is documentation and is never loaded, and scripts/dev-server.sh exports .env.dev itself (see the quickstart) because dev settings must not leak into a production process.

Variable	Meaning
HOST_TARGETS	name=url per host running the agent; a bearer token may ride in the URL userinfo (http://TOKEN@host:9120)
HOST_INTERVAL_SECONDS	poll interval for hosts (default 10)
GUFO_BASE_URL / VLLM_BASE_URL	LLM engines; unset means not deployed here, and its half of the page disappears
RATE_WINDOW_SECONDS	the Δ window behind every measured token rate (default 10)
NUT_HOST / NUT_PORT	upsd address. Reads work unauthenticated in stock NUT
NUT_USER / NUT_PASSWORD	only needed for instant commands (battery tests)
NUT_ALLOW_ACTIONS	1 enables the one write endpoint in the app
NUT_SIMULATE	1 replaces upsd with a labelled simulator (development only)
DASH_READ_ONLY	1 disables every mutating endpoint regardless of other flags
DASH_DB	SQLite path (default data/dashboards.sqlite3)
RETAIN_RAW_HOURS / RETAIN_MINUTE_DAYS / RETAIN_HOUR_DAYS	history rollup and pruning

A source that is not configured answers 404 {"configured": false} without dialling, and the page says "not configured" instead of showing a red panel for something you never installed.

Architecture

agent/host_agent.py (each host)      gufo / vLLM            upsd (NUT)
        │ GET /snapshot                    │ GET /metrics           │ LIST UPS / LIST VAR
        ▼                                  ▼                        ▼
src/dashboards/collectors.py ────────────────────────────────────────────┐
        │  parse + Δcounter/Δt (rates.py)                                │
        ▼                                                                ▼
sources/host.py · sources/llm.py · sources/nut.py          poller.py (one task per source)
        │        samples                    events                      │
        ▼                                                                 ▼
store.py  ── SQLite (WAL): sample(source,host,metric,res,ts) + event ──  api.py  → web/

• poller.py — every source gets its own asyncio task, its own health record, and its own error handling. A dead host keeps its last good values and is labelled stale; it never blanks or blocks another source.
• store.py — raw samples roll up to 1-minute and 1-hour averages and the raw rows are pruned, so a year of history is a few hundred MB rather than a disk emergency.
• web/ — four static pages, one stylesheet, hand-rolled SVG charts (charts.js). No framework, no CDN, no bundler.

Testing

uv run pytest                                  # unit + API tests, fully mocked
uv run pytest --cov                            # coverage gate
uv run pytest -m live --live                   # optional: hit the real fleet/engines
uv run ruff check src tests agent && uv run pyright            # lint + types
./scripts/capture-fixtures.sh                  # re-record ground-truth captures

Test philosophy: real captures as fixtures, mocks for behaviour. Anything that must match an upstream (/proc layouts, vLLM metric names, NUT protocol framing) is asserted against a verbatim capture in tests/fixtures/; behaviour that needs a failure — upsd refusing a command, an engine returning 500, a host rebooting — is a fake. -m live opt-in tests are the bridge: they run only when you point them at real hardware.

Battery-test buttons

Instant commands are the only mutating endpoint in the application (POST /api/ups/command) and they are gated four times: NUT_ALLOW_ACTIONS=1, not DASH_READ_ONLY, an upsd.users account that actually has actions, and a confirmation in the UI — destructive ones (deep test, calibration, load.off) require typing DANGER. The server re-checks every gate; hiding a button is not a safety control. Commands that would cut power are refused outright while the UPS is already on battery.

Deployment

Containerfile builds a slim image (the agent is copied in too, so one image can serve a host if you want). In the deployment repo the pattern is the same as every other service: quadlet container + env template + nginx/SNI on the single domain.

podman build -t dashboards .
podman run --rm -p 8090:8090 --env-file dashboards.env dashboards

Keep the SQLite file on persistent storage (-v dashboards-data:/var/lib/dashboards).