Experimental/llm-monitor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
48bc92b9af67ce9fedfe1e94d7eb9aebfa4386cc
llm-monitor/README.md
T
ducoterra 48bc92b9af
Build and Push Container / build-and-push (push) Successful in 38s
Details
initial import: read-only gufo + vLLM dashboard 
Single-file FastAPI proxy plus a no-CDN vanilla-JS page for watching a gufo
server and a vLLM server side by side: one Prometheus parser (class Prom) with
per-engine field maps, measured Δcounter/Δt rates for engines that export no
rate gauge, and strict read-only GET-only routing. Engines are opt-in via
*_BASE_URL — unset means no panel, no poll, and a 404 {"configured": false}
without dialling.

Includes the container image and CI so the dashboard can be deployed as an
image rather than a checkout:

- Containerfile: uv-locked two-stage build on python:3.14-slim, non-root
  (uid 10001), binds 8090 (8000 is vLLM's port in this fleet), healthcheck on
  /api/config so a dead engine never marks the dashboard unhealthy. No frontend
  build stage — static/index.html is bundler-free by design and is copied as-is.
- .gitea/workflows/build-and-push.yml: publishes
  gitea.reeseapps.com/Experimental/llm-monitor on main + release tags.

114 tests, no network and no real engines required.

LLM-Generated: true
2026-10-16 12:49:45 -04:00

433 lines
28 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# llm_monitor — gufo + vLLM real-time dashboard
One page, two engines, live. Watches a **gufo** (llama.cpp-family) server and a **vLLM**
server side by side: sessions/slots, queueing, KV/GPU cache pressure, throughput, and
latency histograms — refreshed every second without a page reload.
- **Read-only by construction.** The backend only ever calls `client.get()`. There is no
POST/PUT/PATCH/DELETE route anywhere (`curl -X POST /api/gufo/slots` → `405`), nothing is
written to either engine, and upstream bodies are parsed and re-emitted as JSON instead of
echoed.
- **Per-engine health.** Each engine is polled and tracked independently. One engine being
down never blocks or blanks the other.
- **Opt-in engines.** A server only appears if its `*_BASE_URL` env var is set. Send this to
someone who runs just vLLM and they see one clean vLLM panel — no gufo half stuck in red.
- **No build step, no CDN, no browser dependencies.** 2 app files (`app.py`,
`static/index.html`). Sparklines are hand-rolled inline SVG — the engines live on an
internet-disconnected podman network, so a CDN chart library would just be a dead request.
## Endpoints and ports (source of truth)
Taken from `~/Deployments/reeseapps/ai/deployments`:
| Engine | Deployment | Host | Reachable as | Port | Upstream endpoints used |
| -------------------- | ---------------------------- | ---------------- | -------------------------------- | ---- | ------------------------------------------------ |
| gufo | `deployments/gufo` | `deskwork-ai` | `deskwork.reeselink.com` | 8731 | `GET /slots` (alias `/v1/slots`), `GET /metrics` |
| vLLM (radiance) | `deployments/vllm-radiance` | `driveripper-ai` | `driveripper.reeselink.com` | 8000 | `GET /metrics` |
| **this dashboard** | — | wherever you run it | — | **8090** | serves the page + `/api/*` |
> Note the port: `8000` is already vLLM/juggernaut in that fleet and `8731` is gufo, so the
> dashboard listens on **8090** by default. The `8080`/`8000` defaults you may expect for the
> *upstreams* are wrong for this fleet — `8731` is gufo here.
## Install & run (uv)
This is a [uv](https://docs.astral.sh/uv/) project: `pyproject.toml` + `uv.lock`, no build
backend (`package = false` — it is an app, not a library). uv manages the venv; you never
activate it manually.
```bash
uv sync # create .venv from uv.lock (incl. dev group)
# Point at the engines you actually run (these are the fleet's real endpoints), then start:
export GUFO_BASE_URL=http://deskwork.reeselink.com:8731 # gufo — deskwork-ai:8731
export VLLM_BASE_URL=http://driveripper.reeselink.com:8000 # vLLM — driveripper-ai:8000
uv run uvicorn app:app --host 127.0.0.1 --port 8090
```
Set only what exists. With just `VLLM_BASE_URL` the page is a single-engine vLLM dashboard
and `/api/gufo/*` answers `404 {"configured": false}` without ever dialling a port; with
neither set it shows a "no engines configured" card instead of two permanently red panels.
Open <http://127.0.0.1:8090>.
```bash
uv run python app.py # same thing, reads DASHBOARD_HOST/DASHBOARD_PORT
uv run uvicorn app:app --port 8090 --reload # dev loop (watches app.py + static/)
uv sync --no-dev # production install: skips the pytest group
```
> Keep the `--port 8090`. Bare `uvicorn app:app` binds `:8000`, which is where vLLM already
> lives on `driveripper-ai` — the dashboard would refuse to start there (and if it started on
> a machine without vLLM, `VLLM_BASE_URL` would point at itself).
Can't reach those hosts directly (VPN, firewall, different network)? Tunnel them to loopback
and point the env vars at the tunnel:
```bash
ssh -N -L 8731:127.0.0.1:8731 -L 8000:127.0.0.1:8000 deskwork-ai
export GUFO_BASE_URL=http://127.0.0.1:8731 VLLM_BASE_URL=http://127.0.0.1:8000
```
After editing dependencies: `uv lock` (updates `uv.lock`), then regenerate the pip shim:
`uv export --no-dev --no-hashes > requirements.txt`.
### Without uv
`requirements.txt` is a pinned export of `uv.lock`, so a host that has no uv still works:
```bash
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn app:app --host 127.0.0.1 --port 8090
```
### In a container
The `Containerfile` builds a slim, non-root image (uv-locked deps, no build tools, no
frontend build step — the page is copied verbatim). CI publishes it to
`gitea.reeseapps.com/Experimental/llm-monitor` on every push to `main` and on release tags
(`.gitea/workflows/build-and-push.yml`).
```bash
podman build -t localhost/llm-monitor:main -f Containerfile .
podman run -d --name llm-monitor -p 8090:8090 \
-e GUFO_BASE_URL=http://deskwork.reeselink.com:8731 \
-e VLLM_BASE_URL=http://driveripper.reeselink.com:8000 \
localhost/llm-monitor:main
```
Nothing is baked in: no host guesses and no API keys live in the image, so the same image
serves a two-engine deployment, a vLLM-only deployment, or a gufo-only one — set (or omit)
`GUFO_BASE_URL` / `VLLM_API_KEY` at run time. Inside the container the dashboard binds
`0.0.0.0:8090`; reach the engines by whatever name is resolvable from that network (or
publish the container onto the engines' podman network and use the quadlet hostnames,
`deskwork-ai` / `driveripper-ai`). The `HEALTHCHECK` probes `/api/config`, which never dials
an engine — a dead engine must never mark the dashboard itself unhealthy.
## Tests
```bash
uv run pytest # 114 tests, no network, no real engines
uv run pytest --cov # with coverage (app.py 98%)
uv run pytest -k parsing -vv # just the Prometheus parser / field maps
uv run -p 3.10 pytest # check the requires-python floor
```
Supported Python: **>=3.10** (CI-worthy runs verified on 3.10 / 3.13 / 3.14). There is no
`.python-version` pin on purpose — uv uses whatever interpreter is present, since this gets
dropped onto whatever box hosts the engines.
`tests/conftest.py` swaps `app.state.client` for a `FakeUpstream` that records the URLs it
was asked for and can raise `httpx.ConnectError` / `TimeoutException` on demand — so engine
outages are asserted, not hand-tested. Suite covers: parser edge cases (aliases, absent
series, label escaping, histogram interpolation), passthrough fidelity, 405 on every
non-GET verb, 502/504 translation, per-engine auth, and the no-third-party-assets rule.
## Configuration (env vars only)
| Var | Default | Meaning |
| ------------------------- | ------------------------ | -------------------------------------------------------------- |
| `GUFO_BASE_URL` | **_unset_ → gufo hidden** | gufo origin (scheme://host[:port]); unset/blank means this deployment has no gufo |
| `VLLM_BASE_URL` | **_unset_ → vLLM hidden** | vLLM origin; unset/blank means this deployment has no vLLM |
| `GUFO_API_KEY` | _unset_ | if set, sent upstream as `Authorization: Bearer <key>` |
| `VLLM_API_KEY` | _unset_ | if set, sent upstream as `Authorization: Bearer <key>` |
| `UPSTREAM_TIMEOUT_SECONDS`| `3` | per-request upstream timeout (short: the UI polls every 1s) |
| `RATE_WINDOW_SECONDS` | `10` | rolling window for vLLM Δcounter rates (see “Where prompt tok/s comes from”) |
| `POLL_MS` | `1000` | browser poll interval |
| `HISTORY_LEN` | `60` | sparkline samples kept in the browser |
| `DASHBOARD_HOST`/`DASHBOARD_PORT` | `127.0.0.1` / `8090` | only used by `python app.py` |
There is deliberately **no default host** for the two engines. A built-in `127.0.0.1:8731`
looked convenient until a vLLM-only user got a red gufo panel for a server they never
installed; "not configured" and "configured but down" are different statements, so only the
env var decides. `http://host:port/` is accepted (the trailing slash is trimmed) and a
whitespace-only value counts as unset. `/api/config` exposes `gufo_enabled` / `vllm_enabled`,
and the page builds its layout from those.
No auth on the dashboard itself by default — bind it to `127.0.0.1` (default) or put it
behind your reverse proxy. The keys are never returned by any endpoint; `/api/config`
reports only `gufo_auth: true/false`.
## Upstream start commands
These are the commands the dashboard expects to be able to scrape. Full worked examples are
in each deployment's `README.md` in the deployments repo.
**gufo** — `/slots` and `/metrics` are built in; `--sessions N` is what sizes the slot grid:
```bash
gufo serve llm --sessions 4 \
--model /models/qwen3.8-flash-next/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf \
--context 262144 --prefill-chunk 4096 --served-model-name turbo \
--host 0.0.0.0 --port 8731
# quadlet form: gufo serve --host 0.0.0.0 --port 8731 llm --sessions 2 ...
```
**vLLM** — metrics are on by default at `/metrics` (no flag needed; `--disable-metrics` would
break the vLLM panel):
```bash
vllm serve amd/Qwen3.8-27B-Quark-AWQ-MXFP4 \
--served-model-name=juggernaut \
--max-model-len=131072 --max-num-seqs=4 --max-num-batched-tokens=4096 \
--kv-cache-dtype=fp8 --enable-prefix-caching --gpu-memory-utilization=0.95 \
--host=0.0.0.0 --port=8000
```
## Backend routes
| Route | Upstream | Returns |
| -------------------- | --------------------------- | ----------------------------------------------------------------------- |
| `GET /` | — | the dashboard (`static/index.html`) |
| `GET /api/config` | — | base URLs, `*_enabled` (which engines exist), poll/history settings, `*_auth` booleans (never the keys) |
| `GET /api/gufo/slots`| `GET {GUFO_BASE_URL}/slots` | upstream JSON **passed through byte-for-byte**; falls back to `/v1/slots` |
| `GET /api/gufo/metrics` | `GET {GUFO_BASE_URL}/metrics` | Prometheus text → JSON, gufo field map |
| `GET /api/vllm/metrics` | `GET {VLLM_BASE_URL}/metrics` | Prometheus text → JSON, vLLM field map |
Envelope for the JSON routes:
```jsonc
{ "ok": true, "engine": "vllm", "upstream": "http://127.0.0.1:8000/metrics",
"scraped_at": 1760000000.123, "series": 21, "metrics": { "...": {} } }
```
If an engine's `*_BASE_URL` was unset, its two routes answer
`404 {"ok": false, "engine": "gufo", "configured": false, "detail": "gufo is not configured —
set GUFO_BASE_URL and restart"}` and **no upstream request is made** — a server that is not
installed is not an outage, so it must not look like one.
Every metric cell is `{"value": <number|null>, "source": "<series or computed:...>|null",
"derived": <bool>}`. A missing series is `value: null`, which the page renders as **`n/a`** —
it never 500s because a metric name changed. Upstream failures return `502` (unreachable /
HTTP error) or `504` (timeout) with `{"ok": false, "detail": "..."}`, and that `detail` is
what the panel shows.
## One Prometheus parser, two field maps
`app.py` contains a single Prometheus text parser (`class Prom`: tolerant line matcher,
label unescaping, `sum`/`max`/`avg` aggregation, by-label grouping, histogram
`_sum`/`_count`/`_bucket` → mean + interpolated p95). The engines differ only in their field
maps:
The tables below are **what the deployed engines actually export** (captured live; see
`tests/fixtures/*_real.txt`). Each field is an alias list, so the names from the original
spec and the legacy `llama:` prefix still work — but the deployed name comes first, and the
live fixtures are the regression test that keeps them honest.
**gufo** — note `tokens_predicted_*` (not `generated`), and the speed gauges are named
`*_tokens_seconds` while their unit is tokens/second:
| UI field | Series (deployed name first) | Agg |
| ------------------- | ------------------------------------------------------------------- | ---- |
| Running | `llamacpp:requests_processing` | sum |
| Waiting | `llamacpp:requests_deferred` | sum |
| KV cache | `llamacpp:kv_cache_usage_ratio` | max |
| prompt tok·s⁻¹ (all slots) | **`computed:Δllamacpp:prompt_tokens_total/Δt`** (`RateEstimator`), else `llamacpp:prompt_tokens_seconds` | Δ / avg |
| prompt tok·s⁻¹ (last pass) | `llamacpp:prompt_tokens_seconds` — **one request's last prefill pass**; exposed as `prompt_tokens_per_second_native` | avg |
| decode tok·s⁻¹ (all slots) | **`computed:Δllamacpp:tokens_predicted_total/Δt`** (`RateEstimator`), else `llamacpp:predicted_tokens_seconds` | Δ / avg |
| decode tok·s⁻¹ (last pass) | `llamacpp:predicted_tokens_seconds` — **one request's last decode pass**, holds that value while the box idles; exposed as `decode_tokens_per_second_native` | avg |
| prompt tokens | `llamacpp:prompt_tokens_total` — **excludes cache hits** | sum |
| cache-hit tokens | `llamacpp:prompt_tokens_cached_total` | sum |
| generated tokens | `llamacpp:tokens_predicted_total` | sum |
| total tokens | `computed:prompt+cached+generated` | — |
| prompt cache | `computed:cached/(prompt+cached)` | — |
| peak ctx | `llamacpp:n_tokens_max` | max |
| gpu losses | `gufo_device_lost_total` (non-zero = incident, shown red) | sum |
| spec accept | `computed:accepted/draft_tokens` from `llamacpp:spec_decode_num_*` | — |
**vLLM** — `kv_cache_usage_perc` (not `gpu_cache_usage`), `generation_tokens_total` (not
`generation_tokens_total`), TPOT as `inter_token_latency_seconds`, and every series tagged
`engine=` / `model_name=`:
| UI field | Series (deployed name first) | Notes |
| --------------------- | ---------------------------------------------------------------- | ------------------------------------- |
| Running | `vllm:num_requests_running` | sum across `engine=` label sets |
| Waiting | `vllm:num_requests_waiting` | sum |
| Waiting · capacity | `vllm:num_requests_waiting_by_reason{reason="capacity"}` | live reasons: `capacity`, `deferred` |
| GPU cache | `vllm:kv_cache_usage_perc`, `vllm:gpu_cache_usage` | max (never sums past 1.0) |
| TTFT | `vllm:time_to_first_token_seconds` | histogram → mean, p95 |
| TPOT | `vllm:inter_token_latency_seconds`, `vllm:time_per_output_token_seconds`, `vllm:request_time_per_output_token_seconds` | histogram → mean, p95 |
| Queue time | `vllm:request_queue_time_seconds` | histogram → mean, p95 |
| prompt / generation tokens | `vllm:prompt_tokens_total`, `vllm:generation_tokens_total` | sum |
| prefix cache hit | `computed:hits/queries` from `vllm:prefix_cache_{hits,queries}_total` | — |
| preemptions | `vllm:num_preemptions_total` (non-zero = KV pressure, shown red) | sum |
| prompt tok·s⁻¹ | `vllm:prompt_throughput`, `vllm:avg_prompt_throughput_toks_per_s` — **neither exists on this build** — else **`computed:Δprompt_tokens_total/Δt`**, else **`computed:mean(request_prompt_tokens)/mean(request_prefill_time_seconds)`** | see below |
| prefill avg | `computed:mean(request_prompt_tokens)/mean(request_prefill_time_seconds)` | lifetime average, ~4.9k tok/s live |
| decode tok·s⁻¹ | `vllm:generation_throughput` if present, else **`computed:Δgeneration_tokens_total/Δt`**, else **`computed:running/tpot`** | `derived: true`; the UI tags the source |
| per-seq tok·s⁻¹ | `computed:1/tpot` | from TPOT mean |
### Where vLLM's prompt tok/s comes from
That vLLM exports **no rate gauge at all** — no `*_throughput`, no `*_toks_per_s`, nothing
with a rate-ish name (grep its `/metrics` and you get zero hits). Counters only. So the
number is computed, in this order, and the UI labels which one you are looking at:
1. **`vllm:prompt_throughput`** if a future build provides it — the only real scraped gauge.
2. **Measured Δ** — `Δprompt_tokens_total / Δt` over a rolling `RATE_WINDOW_SECONDS` window
(default 10s). This is *actual* current throughput, tagged `Δ10.0s`. A bare 1-second delta
jitters between 0 and spikes because prompts arrive in bursts, hence the window. Counter
resets (engine restart) yield `n/a` for one scrape rather than a huge negative rate.
3. **Lifetime average prefill speed** — `mean(request_prompt_tokens) / mean(request_prefill_time_seconds)`,
i.e. tokens-per-second this engine has achieved on prefill since it started (~4.9k tok/s
live). Used only until there are two scrapes to diff, and shown on its own as **prefill avg**.
Decode tok/s follows the same precedence, and the measured Δ replaced the old
`running/tpot` estimate as step 2 — the Δ counts real output, the estimate assumed every
running request was decoding at that instant. The per-request figure is still `1/TPOT`.
An `n/a` now means the counters themselves are gone, not that a gauge name changed.
### Where gufo's decode tok/s comes from
gufo's **decode tok/s is the aggregate over every slot generating at once**, and it is
measured, not scraped. `llamacpp:predicted_tokens_seconds` is a **single unlabeled gauge**
holding the last decode pass of **one** request: with two slots decoding at 35 tok/s it still
reads 35, and when the box goes idle it keeps serving that stale number indefinitely (live:
stuck at 38.0 with `requests_processing == 0`). There are no label sets to `sum`, so no
aggregation rule could ever fix it.
`llamacpp:tokens_predicted_total` advances once per token from every slot, so the same
`RateEstimator` used for vLLM answers the question directly — two slots at 35 tok/s advance
the counter 70 tokens per second and the cell reads ~70:
1. **Measured Δ** — `Δtokens_predicted_total / Δt` over the `RATE_WINDOW_SECONDS` window,
tagged `Δ10.0s`. Goes to a genuine `0.0` when nothing is generating.
2. **The scraped gauge** — only until there are two scrapes to diff, or on a build missing
the counter; rendered with no tag (it is a real scrape) and still exposed separately as
`decode_tokens_per_second_native`.
This is the one place where measured **outranks** a native gauge, deliberately inverting the
usual precedence: vLLM's native throughput gauge really is an aggregate, gufo's is not. Both
engines share the single `RATES` estimator and cannot cross-read, because it keys off the
*resolved* series name — which also means a counter rename starts a fresh window instead of
diffing two different series and inventing throughput.
gufo's `/slots` is passed through untouched and returns a **bare JSON array** (older builds
wrap it as `{"slots": [...]}`; the page accepts both). Slot fields read: `id`,
`is_processing` (authoritative; `state` 0/1 is the fallback), `id_task`/`task_id`,
`n_prompt_tokens`, `n_prompt_tokens_cache`, `n_prompt_tokens_processed`, `n_ctx`, `model`,
`speculative`, and `next_token[0].{n_decoded,n_remain}` (`-1` means "nothing pending" → `n/a`).
## Dashboard layout
- **Fleet total** (very top): one number per metric summed across both engines — running,
waiting, prompt tok·s⁻¹, decode tok·s⁻¹, total tokens. Hidden entirely when fewer than two
engines are configured (a cross-engine sum of one engine just duplicates its own card).
Rules:
- **Only engines answering now are counted.** A dead engine keeps its last good sample in
its own panel (labelled stale, correct there), but adding a frozen number to a fleet total
would present hours-old data as current — so it is dropped, the header drops to `1 of 2
engines`, and the accent/dot go amber. Zero answering → every cell `n/a`, dot red.
Liveness is keyed on each engine's **`/metrics` job only**: gufo's `/slots` failing must
not remove it from a total that came from `/metrics`.
- **Summed in the browser**, not on the backend. The page already holds both parsed payloads,
so the API stays one-scrape-pure with no second polling path hammering the engines.
- Hover any cell for the itemised breakdown (`gufo 45 + vLLM 37 · excluded: …`), with each
part marked `(measured)` when it is a derived rate.
- Sums are computed on the raw values then rounded once, so a cell can differ from the two
rounded panel numbers above it by 1 (`308 + 177` shown, `486` total).
- **The prompt sum is the one imperfect addition**: gufo's prompt counter *excludes* cache
hits (so it measures newly-computed prefill work) while vLLM's *includes* cached prefix
tokens. Both are per-engine aggregates and additive within an engine, but the fleet number
leans slightly low for gufo. The `prompt tok/s` label carries the caveat on hover.
- **Header line** names only the engines that exist (`vLLM http://host:8000 · poll 1000ms ·
read-only`), and the tab title follows (`LLM Monitor · vLLM`).
- **Shared stats strip** (top): per engine — cache-usage bar (blue → amber ≥75% → red ≥90%),
prompt tok·s⁻¹, decode tok·s⁻¹, total tokens, plus slots-busy (gufo) / per-seq tok·s⁻¹
(vLLM). Strip labels stay terse (they share one dense grid), so gufo's cell just reads
**decode tok/s** with its all-slots meaning on hover; the gufo panel's sparkline caption
spells it out as **decode tok/s · all slots**. Every derived rate carries its provenance tag
(`Δ10.0s`, `from TPOT`, `lifetime avg`) with the exact `computed:` source on hover.
- **gufo panel**: vLLM-style banner `Running: N reqs, Waiting: M`, sparklines for the
all-slots decode tok·s⁻¹ and KV usage, then a grid with **one card per `/slots` entry**: slot id + served
model (`+spec` when speculative decoding is on), status pill (green `processing` / grey
`idle`), request id, prompt tokens, cache hits (with % of prompt), generated so far
(`next_token[0].n_decoded`), remaining budget, and slot context size. Queued requests are
not in `/slots` — that is what `Waiting` (deferred) covers.
- **Alarm colours**: `gpu losses` (gufo `gufo_device_lost_total`) and `preemptions`
(`vllm:num_preemptions_total`) turn red above zero — both are cumulative counters, so a
moving number there means something is wrong, not busy.
- **vLLM panel**: banner `Running: N reqs, Waiting: M`, sparklines for the all-slots decode
tok·s⁻¹ and GPU cache usage — deliberately the **same pair of charts as the gufo panel**, both
fed by `decode_tokens_per_second` (`Δvllm:generation_tokens_total/Δt` here) so the two engines
read alike. The running count is no longer charted; it lives in the banner and the fleet
total. Then metric cards for waiting-by-reason, GPU cache bar, TTFT / TPOT / queue
time (mean + p95), prompt + generation token totals, and series scraped.
- **Sparkline y-axis**: each chart labels the top, middle and bottom of the scale it actually
plotted against (`49 / 43 / 37`, or `100% / 50% / 0%` for the cache charts, which are pinned
to 0–1 rather than autoscaled), with faint rules at those levels. The plots autoscale, so
without the labels a 1% drift and a 2× spike are the same squiggle. Labels are rendered as
HTML beside the SVG, never inside it — `preserveAspectRatio="none"` stretches the viewBox to
the card width and would smear the text. Precision adapts so the three never collide (a 1–2
request axis reads `2.0 / 1.5 / 1.0`, not `2 / 2 / 1`), and a **flat** window labels all three
levels with the value itself — an idle box says `0 / 0 / 0`, not the `±0.001` the plot's
zero-range padding happens to produce. Fewer than two points → axis `n/a`.
- **Every second**: all four `/api/*` endpoints are polled independently
(`setInterval` + fire-and-forget, with an in-flight guard so a slow upstream can't pile up
requests). DOM is updated in place — slot cards are keyed by slot id and reused, so the
poll is invisible. Per-engine green/red dot, "updated HH:MM:SS" stamp, and a
`stale Ns` counter that only appears when that engine stopped answering. `Space` or the
**pause** button freezes polling for reading a snapshot.
## Failure behaviour
| Situation | Result |
| ---------------------------------------- | ----------------------------------------------------------------- |
| Engine's `*_BASE_URL` unset | that engine is **not on the page at all** (no strip card, no panel, no polling, no fleet share) and its routes answer `404 configured:false`; neither configured → a setup card naming the env vars |
| Engine stopped / unreachable | that panel's dot goes red, red banner with the upstream error, last good values kept and greyed, `stale Ns` grows; **the other engine keeps polling normally** |
| Engine comes back | dot goes green and the banner clears on the next successful poll — no reload needed |
| A metric series is absent | that one cell shows `n/a` (and `series absent → n/a` in its foot), everything else renders |
| Upstream returns non-Prometheus garbage | parsed as 0 series → all cells `n/a`, page still healthy |
| Sparkline has < 2 points | chart stays faded and its y-axis reads `n/a` rather than inventing a scale |
| Upstream slower than `UPSTREAM_TIMEOUT_SECONDS` | `504`, dot red, and the request is abandoned rather than queued |
## Smoke test without live engines
Start the app first (`uv run uvicorn app:app --host 127.0.0.1 --port 8090`), then:
```bash
curl -s localhost:8090/api/config | python3 -m json.tool # what it is pointed at
curl -s localhost:8090/api/gufo/slots | head -c 200 # proxy passthrough
curl -s localhost:8090/api/gufo/metrics | python3 -m json.tool | head -30
curl -s localhost:8090/api/vllm/metrics | python3 -m json.tool | head -30
curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:8090/api/gufo/slots # -> 405 (read-only)
```
One-engine deployment (what a vLLM-only recipient sees) — gufo answers `404 configured:false`
and the page has no gufo section at all:
```bash
env -u GUFO_BASE_URL VLLM_BASE_URL=http://127.0.0.1:8000 \
uv run uvicorn app:app --host 127.0.0.1 --port 8090
curl -s localhost:8090/api/config | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d["gufo_enabled"], d["vllm_enabled"])' # False True
curl -s -o /dev/null -w '%{http_code}\n' localhost:8090/api/gufo/metrics # -> 404, nothing dialled
```
`n/a` + red-dot path described above.
To see what an engine really exposes (do this before trusting any metric name, including the
ones written in this README):
```bash
curl -s "$GUFO_BASE_URL/metrics" | grep -v '^#' | sed -E 's/[{ ].*//' | sort -u
curl -s "$VLLM_BASE_URL/metrics" | grep -v '^#' | sed -E 's/[{ ].*//' | sort -u
curl -s "$GUFO_BASE_URL/slots" | python3 -m json.tool | head # a bare array
```
## Files
```
pyproject.toml uv project: deps, dev group, package=false, pytest/coverage config
uv.lock pinned resolution (commit it)
app.py FastAPI app: config, Prometheus parser, 2 field maps, 5 GET routes
static/index.html the dashboard: CSS + vanilla JS + inline-SVG sparklines
tests/ conftest.py (FakeUpstream), test_parsing.py, test_proxy.py,
test_real_contracts.py + fixtures/ (verbatim captures from the live engines)
requirements.txt pip shim, generated from uv.lock
AGENTS.md working rules for humans and agents on this repo
README.md
```
There are no JS dependencies and nothing to build: `static/index.html` is served verbatim.
Reference in New Issue View Git Blame Copy Permalink