phase: 03_system_per_host_panels
**Phase 03 final verification pass — all 4 tasks already in `complete/`; verified criteria, fixed 3 pre-existing defects.**
- **Verified live** (headless Chromium + curl against `:8090`, all 6 sources `ok`): 3 panels × six heroes in order `cpu,mem,temp,disk,gpus,net`; charts per panel `util,net[,gpu]` (gateway: no GPU chart/hole); legends name metrics only — no legend names a foreign host; every `<svg>` `role="img"` + host-scoped `aria-label`; 2 sections, 0 `<select>`; `grep -rn "renderCores|core_usage" src/dashboards/web/` → nothing; console error-free; new shot `data/logs/final-system-panels.png`.
- **Defect fixed (credential hygiene, invariant 6)**: `collectors.py` put the *dialed* URL into every `UpstreamError`, and that string lands in the health record (`/api/health`) and the `event` table (`/api/events` → page); `/api/config` also echoed `GUFO_BASE_URL`/`VLLM_BASE_URL` verbatim. Added `config.strip_userinfo()` (now what `Target.display_url` uses) and redacted all message/config URLs. Tests: `test_a_failing_host_never_echoes_its_token_in_the_message` (×3 paths), `test_a_failing_engine_never_echoes_credentials_in_its_url`, `test_a_dead_host_never_writes_its_token_into_the_feed`, `test_engine_urls_are_redacted_the_same_way_hosts_are`, `test_strip_userinfo_is_the_one_redaction_every_url_leaves_by`. (`parse_targets` already moves `HOST_TARGETS` userinfo into a header, so this is a backstop for hosts and a real fix for engine URLs.)
- **Defect fixed (invariant 2)**: `/system` footnote printed `Δ{HOST_INTERVAL}` as the measurement; it now prints each host's measured `window_seconds` (range when they disagree) — `rates over Δ10.0s · poll 10s`. Pinned by `test_system_footnote_prints_the_window_it_actually_measured`. Also fixed the page's stale "window in each header" note and the empty-state's `--token` (argv) advice → `--token-file`.
- `bash .agents/validate.sh` → **EXIT=0**: **472 passed, 7 skipped**, coverage **96.87%** (floor 94), `ruff check src tests agent` clean, `pyright` 0 errors, `node --check` clean · `uv run pytest -m live --live` → **7 passed** · `uv pip list --outdated` empty.
- Criteria: no multi-host legend ✅ · no per-core visual ✅ · six heroes/panel incl. no-GPU `n/a` uncoloured ✅ · per-core history still queryable ✅ (354 pts) · ≥3 shape screenshots ✅ (`system-driveripper/deskwork/gateway.png` + dark/light/stale) · pytest+coverage+lint ✅ · phases 01/02 unchanged (their pin tests + `/api/summary` shape tests pass; live UPS gap in AGENTS.md re-confirmed read-only against real upsd) ✅.
- **Docs**: AGENTS.md — invariant 6 rewritten to cover message/event redaction, `collectors.py` line, new "Δ is a measurement" rule, "hosts answering total" → "nine-figure fleet-totals panel" (the subtitle still counts hosts), test count 452→472, coverage 96.8→96.9, rewrapped an over-long line. README.md — same fleet-totals wording, `RATE_WINDOW_SECONDS` now covers every measured rate (not just tokens), agent route list completed (`/`, `/schema`, `?top=N`), `LIST CMD` in the diagram, new paragraph on URL redaction.
- **Deviation**: the criterion's literal `curl …/api/system/history?host=…&metric=cpu.usage_pct@core7` uses singular params that endpoint has never accepted (it takes `hosts=`/`metrics=` and ignores unknown ones). Verified 354 points via `…system/history?hosts=driveripper&metrics=cpu.usage_pct@core7&range=1h` and `/api/history?source=host&host=…&metric=…`; I did **not** add singular param aliases (unauthorized API surface). No live infra changes made — reads only.
- **Next pending phase**: `04_llm_chart_readability`.
This commit is contained in:
+10
-4
@@ -5,7 +5,8 @@ hot path) plus SQLite for anything older than a few minutes. Two rules from llm_
|
||||
are worth restating here because they are easy to break by accident:
|
||||
|
||||
* `/api/config` never leaks a token or password — target URLs are passed through
|
||||
`Target.display_url`, which strips userinfo.
|
||||
`Target.display_url`, which strips userinfo, and so does every URL that reaches an error
|
||||
message, because those travel into `/api/health` and the event feed too.
|
||||
* an unconfigured source answers 404 `{"configured": false}` without dialling anything, so
|
||||
the page can tell "not deployed here" apart from "deployed and broken".
|
||||
"""
|
||||
@@ -20,7 +21,7 @@ from fastapi import APIRouter, Body, Depends, HTTPException, Query, Request
|
||||
|
||||
from . import alerts as alert_rules
|
||||
from .collectors import NutMonitor
|
||||
from .config import Settings
|
||||
from .config import Settings, strip_userinfo
|
||||
from .poller import Hub
|
||||
from .severity import classify
|
||||
from .sources import nut
|
||||
@@ -133,9 +134,14 @@ def common_router() -> APIRouter:
|
||||
"target_problems": list(settings.host_target_problems),
|
||||
},
|
||||
"llm": {
|
||||
"gufo": {"enabled": bool(settings.gufo_url), "base_url": settings.gufo_url,
|
||||
# `strip_userinfo` on the engine URLs too: `GUFO_API_KEY` is the documented
|
||||
# way to authenticate, but a URL that carries credentials must never be echoed
|
||||
# to a browser whatever the config says (invariant 6).
|
||||
"gufo": {"enabled": bool(settings.gufo_url),
|
||||
"base_url": strip_userinfo(settings.gufo_url),
|
||||
"auth": bool(settings.gufo_api_key)},
|
||||
"vllm": {"enabled": bool(settings.vllm_url), "base_url": settings.vllm_url,
|
||||
"vllm": {"enabled": bool(settings.vllm_url),
|
||||
"base_url": strip_userinfo(settings.vllm_url),
|
||||
"auth": bool(settings.vllm_api_key)},
|
||||
"interval_seconds": settings.llm_interval,
|
||||
"poll_ms": settings.poll_ms_llm,
|
||||
|
||||
@@ -13,7 +13,7 @@ from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from .config import Target
|
||||
from .config import Target, strip_userinfo
|
||||
from .poller import Job, Result
|
||||
from .rates import RateEstimator
|
||||
from .sources import llm as llm_source
|
||||
@@ -37,17 +37,21 @@ class UpstreamError(Exception):
|
||||
async def fetch_host_snapshot(client: httpx.AsyncClient, target: Target,
|
||||
host_collector: HostCollector) -> Result:
|
||||
url = f"{target.url}/snapshot"
|
||||
# Every message below uses the redacted URL on purpose: an UpstreamError string is kept
|
||||
# in the job's health record and written to the event table, both of which the browser
|
||||
# reads, and `target.url` may carry the agent's token as userinfo.
|
||||
shown = f"{target.display_url}/snapshot"
|
||||
try:
|
||||
response = await client.get(url, headers=target.headers)
|
||||
except httpx.TimeoutException as exc:
|
||||
raise UpstreamError(f"{target.name}: {url} timed out", status=504) from exc
|
||||
raise UpstreamError(f"{target.name}: {shown} timed out", status=504) from exc
|
||||
except httpx.HTTPError as exc:
|
||||
raise UpstreamError(f"{target.name}: {url} unreachable ({type(exc).__name__})") from exc
|
||||
raise UpstreamError(f"{target.name}: {shown} unreachable ({type(exc).__name__})") from exc
|
||||
if response.status_code == 401:
|
||||
raise UpstreamError(f"{target.name}: agent rejected the token (401) — check the "
|
||||
f"userinfo token in HOST_TARGETS")
|
||||
if response.status_code >= 400:
|
||||
raise UpstreamError(f"{target.name}: {url} -> HTTP {response.status_code}")
|
||||
raise UpstreamError(f"{target.name}: {shown} -> HTTP {response.status_code}")
|
||||
try:
|
||||
body = response.json()
|
||||
except ValueError as exc:
|
||||
@@ -71,14 +75,15 @@ async def fetch_engine(client: httpx.AsyncClient, engine: str, base_url: str,
|
||||
with_slots: bool = False) -> Result:
|
||||
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
|
||||
url = f"{base_url}/metrics"
|
||||
shown = strip_userinfo(url) # same rule as the host fetch above: no credential in a message
|
||||
try:
|
||||
response = await client.get(url, headers=headers)
|
||||
except httpx.TimeoutException as exc:
|
||||
raise UpstreamError(f"{engine}: {url} timed out", status=504) from exc
|
||||
raise UpstreamError(f"{engine}: {shown} timed out", status=504) from exc
|
||||
except httpx.HTTPError as exc:
|
||||
raise UpstreamError(f"{engine}: {url} unreachable ({type(exc).__name__})") from exc
|
||||
raise UpstreamError(f"{engine}: {shown} unreachable ({type(exc).__name__})") from exc
|
||||
if response.status_code >= 400:
|
||||
raise UpstreamError(f"{engine}: {url} -> HTTP {response.status_code} "
|
||||
raise UpstreamError(f"{engine}: {shown} -> HTTP {response.status_code} "
|
||||
f"{response.reason_phrase}")
|
||||
text = response.text
|
||||
|
||||
|
||||
@@ -75,6 +75,24 @@ def _base_url(name: str) -> str:
|
||||
# variable; the token never leaves the backend, it is stripped from every /api payload.
|
||||
_TARGET_SPLIT = re.compile(r"[,\s]+")
|
||||
_NAME_RE = re.compile(r"^[a-z0-9][a-z0-9_.-]*$")
|
||||
_USERINFO_RE = re.compile(r"^(https?://)[^/@]+@", re.IGNORECASE)
|
||||
|
||||
|
||||
def strip_userinfo(url: str) -> str:
|
||||
"""`url` with any userinfo (`http://TOKEN@host` -> `http://host`) removed.
|
||||
|
||||
The only form a URL may take on its way out of the process. Two things make this needed
|
||||
rather than belt-and-braces: a URL does not travel only in the fetch — an `UpstreamError`
|
||||
message repeats the URL it was dialling, and that string lands in the job's health record
|
||||
(`/api/health`) and in the `event` table (`/api/events`), both of which a browser reads —
|
||||
and the engine URLs (`GUFO_BASE_URL`, `VLLM_BASE_URL`) are used verbatim, with no
|
||||
`parse_targets`-style move of userinfo into a header, so a credential written into one of
|
||||
them would otherwise be served by `/api/config` and echoed by every failed poll.
|
||||
`HOST_TARGETS` strips its token into a header, so this is the backstop for a `Target` that
|
||||
still carries one. One rule, stated once: AGENTS.md invariant 6, no credential in anything
|
||||
a browser can read.
|
||||
"""
|
||||
return _USERINFO_RE.sub(r"\1", url)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -88,7 +106,7 @@ class Target:
|
||||
@property
|
||||
def display_url(self) -> str:
|
||||
"""URL with any userinfo removed — safe to show in the browser."""
|
||||
return re.sub(r"^(https?://)[^/@]+@", r"\1", self.url)
|
||||
return strip_userinfo(self.url)
|
||||
|
||||
|
||||
def parse_targets(raw: str) -> tuple[list[Target], list[str]]:
|
||||
|
||||
@@ -328,6 +328,10 @@ def parse_host_snapshot(name: str, snapshot: dict[str, Any], collector: HostColl
|
||||
"forks_per_s": forks_rate,
|
||||
"procs_running": cpu_raw.get("procs_running"),
|
||||
"procs_blocked": cpu_raw.get("procs_blocked"),
|
||||
# The card metric for "cpu temp" is this one; `temperatures.cpu_c` below is the
|
||||
# same measurement (the agent's own `cpu_temperature_celsius`) and `/api/summary`
|
||||
# flattens this field, so the card and the overview cannot disagree. Pinned by
|
||||
# tests/test_system_contract.py.
|
||||
"temp_c": cpu_temp,
|
||||
# Additive: the pages colour from this verdict instead of comparing the number
|
||||
# to a threshold of their own. Keys mirror the rated fields in this section.
|
||||
@@ -364,6 +368,10 @@ def parse_host_snapshot(name: str, snapshot: dict[str, Any], collector: HostColl
|
||||
"rx_bytes_per_s": rx_total if any_net else None,
|
||||
"tx_bytes_per_s": tx_total if any_net else None,
|
||||
"interfaces": net_rows,
|
||||
# Deliberately no `severity` section here, unlike cpu/memory/disks/gpus: a byte
|
||||
# rate has no good/bad (900 MB/s is idle on 10 GbE, 2 KB/s can be an incident),
|
||||
# so `severity.THRESHOLDS` has no `net.*` row and a colour here would be a
|
||||
# verdict nobody measured. tests/test_system_contract.py pins its absence.
|
||||
},
|
||||
"gpus": gpus,
|
||||
"temperatures": {"cpu_c": cpu_temp, "max_c": max_temp, "nvme_max_c": nvme_max,
|
||||
|
||||
@@ -37,83 +37,19 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="card flat" aria-label="Fleet totals">
|
||||
<div class="body">
|
||||
<div class="metrics">
|
||||
<div class="metric"><div class="label">Hosts answering</div>
|
||||
<div class="value"><span id="f-hosts">n/a</span><span class="unit" id="f-hosts-of"></span></div></div>
|
||||
<div class="metric"><div class="label">CPU average</div>
|
||||
<div class="value"><span id="f-cpu">n/a</span><span class="unit">%</span></div></div>
|
||||
<div class="metric"><div class="label">Load 1 (sum)</div>
|
||||
<div class="value" id="f-load">n/a</div></div>
|
||||
<div class="metric"><div class="label">Memory in use</div>
|
||||
<div class="value"><span id="f-mem">n/a</span><span class="unit" id="f-mem-of"></span></div></div>
|
||||
<div class="metric"><div class="label">Worst filesystem</div>
|
||||
<div class="value"><span id="f-disk">n/a</span><span class="unit">%</span></div></div>
|
||||
<div class="metric"><div class="label">Network in</div>
|
||||
<div class="value"><span id="f-rx">n/a</span><span class="unit" id="f-rx-unit"></span></div></div>
|
||||
<div class="metric"><div class="label">Network out</div>
|
||||
<div class="value"><span id="f-tx">n/a</span><span class="unit" id="f-tx-unit"></span></div></div>
|
||||
<div class="metric"><div class="label">GPU busy (avg)</div>
|
||||
<div class="value"><span id="f-gpu">n/a</span><span class="unit">%</span></div></div>
|
||||
<div class="metric"><div class="label">Power</div>
|
||||
<div class="value sm" title="No whole-host power source is wired up yet; no script writes this, so it stays n/a rather than guessing.">n/a</div></div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="grid cols-2" style="margin-top:.85rem" aria-label="Fleet charts">
|
||||
<article class="card">
|
||||
<header><h3>CPU utilisation</h3><span class="spacer"></span><span class="faint">% · per host</span></header>
|
||||
<div class="body"><div class="chart" id="chart-cpu"></div><div class="legend" id="legend-cpu"></div></div>
|
||||
</article>
|
||||
<article class="card">
|
||||
<header><h3>Memory used</h3><span class="spacer"></span><span class="faint">% of MemAvailable basis</span></header>
|
||||
<div class="body"><div class="chart" id="chart-mem"></div><div class="legend" id="legend-mem"></div></div>
|
||||
</article>
|
||||
<article class="card">
|
||||
<header><h3>Disk I/O throughput</h3><span class="spacer"></span>
|
||||
<span class="faint">selected host · all physical devices summed</span></header>
|
||||
<div class="body"><div class="chart" id="chart-disk"></div><div class="legend" id="legend-disk"></div></div>
|
||||
</article>
|
||||
<article class="card">
|
||||
<header><h3>Network throughput</h3><span class="spacer"></span><span class="faint">selected host · physical interfaces</span></header>
|
||||
<div class="body"><div class="chart" id="chart-net"></div><div class="legend" id="legend-net"></div></div>
|
||||
</article>
|
||||
<article class="card">
|
||||
<header><h3>GPU utilisation</h3><span class="spacer"></span><span class="faint">one series per host GPU</span></header>
|
||||
<div class="body"><div class="chart" id="chart-gpu"></div><div class="legend" id="legend-gpu"></div></div>
|
||||
</article>
|
||||
<article class="card">
|
||||
<header><h3>Temperature</h3><span class="spacer"></span><span class="faint">CPU package · °C</span></header>
|
||||
<div class="body"><div class="chart" id="chart-temp"></div><div class="legend" id="legend-temp"></div></div>
|
||||
</article>
|
||||
</section>
|
||||
|
||||
<section class="card" style="margin-top:.85rem" aria-label="Series detail">
|
||||
<header>
|
||||
<h3>Detail</h3>
|
||||
<span class="spacer"></span>
|
||||
<label class="field" for="detail-host">host</label>
|
||||
<select id="detail-host"></select>
|
||||
<label class="field" for="detail-metric">metric</label>
|
||||
<select id="detail-metric"></select>
|
||||
</header>
|
||||
<div class="body">
|
||||
<div class="chart" id="chart-detail" style="height:200px"></div>
|
||||
<div class="legend" id="legend-detail"></div>
|
||||
<div class="faint" id="detail-note" style="margin-top:.4rem;font-size:11.5px"></div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section style="margin-top:.85rem">
|
||||
<!-- No fleet-wide chart and no host/metric picker survives here (phase 03 task 03): every
|
||||
chart is drawn inside one host panel below, and the question "which machine stands
|
||||
out?" belongs to the overview's alerts and fleet table. -->
|
||||
<section>
|
||||
<div class="page-head"><h2 style="font-size:.95rem;margin:0">Per host</h2>
|
||||
<span class="sub">agent-reported values; rates measured over the window in each header</span></div>
|
||||
<span class="sub">agent-reported values; rates are measured over the Δ window in the
|
||||
footer, and each panel's charts cover only that host</span></div>
|
||||
<div class="grid auto" id="hosts"></div>
|
||||
<div class="empty-state" id="hosts-empty" hidden>
|
||||
<h3>No hosts reporting</h3>
|
||||
<p>Start the agent on a machine and add it to <code>HOST_TARGETS</code>:</p>
|
||||
<p class="mono">python3 agent/host_agent.py --host 0.0.0.0 --port 9120 --token …</p>
|
||||
<p class="mono">python3 agent/host_agent.py --host 0.0.0.0 --port 9120 \
|
||||
--token-file /etc/dashboards-agent.token</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
@@ -123,7 +59,7 @@
|
||||
</section>
|
||||
|
||||
<footer class="pagefoot">
|
||||
<span>numbers are measured, not inferred: rates are Δcounter/Δt over the poll window</span>
|
||||
<span>numbers are measured, not inferred: rates are Δcounter/Δt over the window below</span>
|
||||
<span class="spacer" style="margin-inline-start:auto"></span>
|
||||
<span class="mono" id="foot-note"></span>
|
||||
</footer>
|
||||
|
||||
@@ -110,9 +110,9 @@ const Dash = (() => {
|
||||
*
|
||||
* Thresholds live in `src/dashboards/severity.py` and travel in the payload as
|
||||
* `severity`; `severityFrom()` below is how a page reads them. This function survives
|
||||
* only because `page-system.js`, `page-llm.js` and `page-ups.js` still call it with
|
||||
* their own numbers (phases 03–05 convert them page by page, so each page's change is
|
||||
* reviewable on its own — `/` was already converted in phase 02). Do not add a new call
|
||||
* only because `page-llm.js` and `page-ups.js` still call it with their own numbers
|
||||
* (phases 04–05 convert them page by page, so each page's change is reviewable on its
|
||||
* own — `/` was converted in phase 02, `/system` in phase 03). Do not add a new call
|
||||
* site: a threshold written in JS is a threshold that can disagree with the other pages.
|
||||
*/
|
||||
function severityFor(value, warnAt, failAt) {
|
||||
|
||||
@@ -43,7 +43,7 @@ const DEFAULT_FORMAT = (v) => (Math.abs(v) >= 100 ? v.toFixed(0) : v.toFixed(1))
|
||||
class LineChart {
|
||||
/**
|
||||
* container: element to fill (its width is used; height comes from opts.height)
|
||||
* opts: {height, yFormat, xSpan, yMin, yMax, area, legend, showAxis, min, unit}
|
||||
* opts: {height, yFormat, xSpan, yMin, yMax, area, legend, showAxis, min, unit, ariaLabel}
|
||||
*/
|
||||
constructor(container, opts) {
|
||||
this.el = typeof container === 'string' ? document.querySelector(container) : container;
|
||||
@@ -69,6 +69,13 @@ class LineChart {
|
||||
this.svg = document.createElementNS('http://www.w3.org/2000/svg', 'svg');
|
||||
this.svg.setAttribute('width', '100%');
|
||||
this.svg.setAttribute('height', this.opts.height);
|
||||
// A chart is an image to assistive tech, which means silence unless the page names it —
|
||||
// the legend below is a sibling, not a label. `ariaLabel` is optional so pages adopt it
|
||||
// one at a time (`/system` names every chart it draws; see page-system.js).
|
||||
if (this.opts.ariaLabel) {
|
||||
this.svg.setAttribute('role', 'img');
|
||||
this.svg.setAttribute('aria-label', String(this.opts.ariaLabel));
|
||||
}
|
||||
this.el.appendChild(this.svg);
|
||||
this.empty = document.createElement('div');
|
||||
this.empty.className = 'empty';
|
||||
@@ -337,17 +344,6 @@ function renderLegend(container, series, format, showPeak) {
|
||||
}).join('');
|
||||
}
|
||||
|
||||
/** Column heat bars for per-core usage: 48 cores read as a pattern, not a table. */
|
||||
function renderCores(container, values, format) {
|
||||
const f = format || ((v) => (v == null ? 'n/a' : v.toFixed(0) + '%'));
|
||||
container.innerHTML = (values || []).map((v, i) => {
|
||||
const pct = v == null ? 0 : Math.max(0, Math.min(100, v));
|
||||
const hue = pct > 85 ? 'var(--fail)' : pct > 60 ? 'var(--warn)' : 'var(--accent)';
|
||||
return `<span class="core" title="core ${i}: ${escapeHtml(f(v))}">` +
|
||||
`<i style="height:${pct.toFixed(1)}%;background:${hue}"></i></span>`;
|
||||
}).join('');
|
||||
}
|
||||
|
||||
/** Small non-interactive sparkline for cards where a full chart would shout. */
|
||||
function sparkline(container, points, opts) {
|
||||
const o = Object.assign({ height: 34, color: cssVar('--series-1'), min: null, max: null }, opts || {});
|
||||
@@ -374,5 +370,5 @@ function sparkline(container, points, opts) {
|
||||
container.innerHTML = svg;
|
||||
}
|
||||
|
||||
window.DashCharts = { LineChart, niceTicks, renderLegend, renderCores, sparkline, cssVar,
|
||||
window.DashCharts = { LineChart, niceTicks, renderLegend, sparkline, cssVar,
|
||||
CHART_COLORS, DEFAULT_FORMAT, escapeHtml };
|
||||
|
||||
@@ -282,6 +282,27 @@ main { padding: 1.15rem 1.25rem 3rem; max-width: 1680px; margin: 0 auto; }
|
||||
/* with no verdict on either the hero or the bar the fill stays the neutral accent: the
|
||||
bar shows *how much*, never a judgement nobody measured. */
|
||||
|
||||
/* The per-host hero row: six glanceable numbers — CPU, MEM, TEMP, DISK, GPU, NET — in one
|
||||
line on a wide card, wrapping to 3×2 and then 2×3 as the card narrows. Tracks are a
|
||||
`minmax` of the widest plain hero value (`100 %`), so a row never squints into ellipsis. */
|
||||
.heroes {
|
||||
display: grid; gap: .9rem 1rem;
|
||||
grid-template-columns: repeat(auto-fit, minmax(min(118px, 100%), 1fr));
|
||||
}
|
||||
/* a slot that holds more than one hero (one per GPU card) without breaking the row */
|
||||
.hero-slot { display: grid; gap: .7rem; min-width: 0; align-content: start; }
|
||||
/* Network is one hero with two lines: in and out are one wire seen in two directions, and
|
||||
two big numbers side by side read as a competition. It is deliberately colourless —
|
||||
a byte rate has no good/bad, so no verdict ever paints here (severity.py has no net.*
|
||||
row); `n/a` comes from `.na` and is as faint as any other unmeasured value. */
|
||||
.hero .hero-lines {
|
||||
display: grid; gap: .15rem;
|
||||
font-family: var(--mono); font-size: 1.28rem; font-weight: 650; line-height: 1.25;
|
||||
font-variant-numeric: tabular-nums; /* same fixed digit width as .hero .value */
|
||||
color: var(--text);
|
||||
}
|
||||
.hero .hero-lines .dir { color: var(--text-muted); }
|
||||
|
||||
/* ---- bars, dots, chips -------------------------------------------------- */
|
||||
.bar {
|
||||
position: relative; height: 7px; border-radius: 999px; background: var(--bg-sunken);
|
||||
@@ -357,9 +378,15 @@ table.data td.fail { color: var(--fail); }
|
||||
table.data.scroll { display: block; overflow: auto; max-height: 320px; }
|
||||
|
||||
/* ---- charts ------------------------------------------------------------- */
|
||||
/* The chart vocabulary every page draws with. `LineChart` puts the y tick labels in the
|
||||
`grid` group and the x labels in `axis`, so both groups need the same text style — the
|
||||
grid text used to inherit SVG's black fill, which meant the numbers beside a utilisation
|
||||
chart were invisible on the dark theme. */
|
||||
.chart { position: relative; width: 100%; }
|
||||
.chart svg { display: block; width: 100%; height: 100%; overflow: visible; }
|
||||
.chart .axis text { fill: var(--text-faint); font-family: var(--mono); font-size: 10px; }
|
||||
.chart .axis text, .chart .grid text {
|
||||
fill: var(--text-faint); font-family: var(--mono); font-size: 10px;
|
||||
}
|
||||
.chart .axis line, .chart .grid line { stroke: var(--border); stroke-width: 1; }
|
||||
.chart .grid line { stroke-dasharray: 2 4; opacity: .75; }
|
||||
.chart .zero line { stroke: var(--border-strong); }
|
||||
@@ -375,6 +402,20 @@ table.data.scroll { display: block; overflow: auto; max-height: 320px; }
|
||||
.legend .swatch { width: 9px; height: 9px; border-radius: 2px; }
|
||||
.legend .lv { font-family: var(--mono); color: var(--text); }
|
||||
|
||||
/* The per-host chart stack inside a `/system` panel: supporting evidence under a number,
|
||||
not the main event. One utilisation chart, one network chart, and a GPU chart only for a
|
||||
host that reports cards (`page-system.js` adds and removes that slot with `host.gpus`),
|
||||
each with its legend glued underneath. They are stacked rather than side by side because
|
||||
a 200px-wide plot cannot show an hour of CPU without its axis labels colliding — and no
|
||||
chart in here ever contains a second host. */
|
||||
.host-charts { display: grid; gap: .7rem; }
|
||||
.host-chart { display: grid; gap: .15rem; min-width: 0; }
|
||||
/* `sm` is the slot's floor before `LineChart` writes its measured height inline, so an
|
||||
empty panel is the height it is about to be rather than a collapsing card. */
|
||||
.chart.sm { min-height: 120px; }
|
||||
.host-chart .legend { font-size: 10.5px; gap: .2rem .6rem; }
|
||||
.host-chart .legend .swatch { width: 7px; height: 7px; }
|
||||
|
||||
.tooltip {
|
||||
position: absolute; z-index: 30; pointer-events: none; opacity: 0;
|
||||
background: color-mix(in srgb, var(--bg-raised) 95%, transparent);
|
||||
@@ -386,11 +427,6 @@ table.data.scroll { display: block; overflow: auto; max-height: 320px; }
|
||||
.tooltip .row { display: flex; align-items: center; gap: .35rem; }
|
||||
.tooltip .row .val { margin-inline-start: auto; font-family: var(--mono); }
|
||||
|
||||
/* per-core usage heat bars */
|
||||
.cores { display: grid; grid-template-columns: repeat(auto-fill, minmax(9px, 1fr)); gap: 2px; }
|
||||
.cores .core { height: 22px; border-radius: 2px; background: var(--bg-sunken); position: relative; }
|
||||
.cores .core > i { position: absolute; bottom: 0; left: 0; right: 0; background: var(--accent); border-radius: 2px; }
|
||||
|
||||
/* ---- lists -------------------------------------------------------------- */
|
||||
ul.events { list-style: none; margin: 0; padding: 0; display: grid; gap: .1rem; }
|
||||
ul.events li {
|
||||
@@ -497,11 +533,13 @@ dialog footer { padding: .7rem .9rem; border-top: 1px solid var(--border); displ
|
||||
|
||||
.stale { opacity: .62; }
|
||||
|
||||
select, input[type="text"] {
|
||||
/* Only the UPS confirmation dialog takes text now — `/system`'s host/metric pickers were the
|
||||
last `select` on the app and left with the shared charts, along with the `label.field` caption
|
||||
that sat above them. */
|
||||
input[type="text"] {
|
||||
background: var(--bg-sunken); color: var(--text); border: 1px solid var(--border);
|
||||
border-radius: var(--radius-sm); padding: .3rem .45rem; font: 500 12.5px var(--sans);
|
||||
}
|
||||
label.field { font-size: 11px; text-transform: uppercase; letter-spacing: .05em; color: var(--text-faint); }
|
||||
|
||||
footer.pagefoot {
|
||||
margin-top: 1.6rem; padding-top: .8rem; border-top: 1px solid var(--border);
|
||||
|
||||
@@ -2,71 +2,43 @@
|
||||
*
|
||||
* Host cards are created once and patched in place (keyed by host name) — rebuilding the
|
||||
* DOM every 10s kills hover, transitions, and any scroll position inside the tables.
|
||||
* Charts get their history from /api/system/history: one request per chart, N hosts each.
|
||||
*
|
||||
* Every chart on this page belongs to exactly one host and is drawn inside that host's own
|
||||
* panel (`/api/system/history`, one request per host per refresh). Nothing here plots two
|
||||
* machines on one axis and there is no host/metric picker: "which machine stands out?" is
|
||||
* the overview's job (alerts + fleet table), and `/system` answers "what is this machine
|
||||
* doing?". See the per-host chart section below.
|
||||
*/
|
||||
'use strict';
|
||||
|
||||
const { $, $$, fetchJSON, setTxt, setValue, setBar, setDot, bytes, pct, num, duration, poll,
|
||||
staleness, markStale, severityFor } = Dash;
|
||||
const { $, fetchJSON, setTxt, setDot, bytes, num, duration, poll,
|
||||
markStale } = Dash;
|
||||
|
||||
const state = {
|
||||
range: 3600,
|
||||
detailHost: null,
|
||||
detailMetric: 'cpu.usage_pct',
|
||||
hosts: new Map(), // name -> card element
|
||||
charts: {},
|
||||
charts: new Map(), // name -> { util, net, gpu|null } LineChart instances
|
||||
poller: null,
|
||||
pollMs: 10000,
|
||||
metricList: [],
|
||||
};
|
||||
|
||||
const DETAIL_METRICS = [
|
||||
['cpu.usage_pct', 'CPU utilisation', 'pct'],
|
||||
['cpu.user_pct', 'CPU user', 'pct'],
|
||||
['cpu.system_pct', 'CPU system', 'pct'],
|
||||
['cpu.iowait_pct', 'CPU iowait', 'pct'],
|
||||
['cpu.steal_pct', 'CPU steal', 'pct'],
|
||||
['cpu.load1', 'Load average (1m)', 'num'],
|
||||
['cpu.load15', 'Load average (15m)', 'num'],
|
||||
['cpu.freq_mhz', 'CPU frequency', 'mhz'],
|
||||
['cpu.context_switches_per_s', 'Context switches', 'num'],
|
||||
['mem.used_pct', 'Memory used', 'pct'],
|
||||
['mem.used_bytes', 'Memory used', 'bytes'],
|
||||
['mem.available_bytes', 'Memory available', 'bytes'],
|
||||
['mem.swap_used_pct', 'Swap used', 'pct'],
|
||||
['disk.used_pct', 'Filesystem used (worst)', 'pct'],
|
||||
['disk.read_bytes_per_s', 'Disk read', 'bytes'],
|
||||
['disk.write_bytes_per_s', 'Disk write', 'bytes'],
|
||||
['disk.util_pct', 'Disk busy', 'pct'],
|
||||
['net.rx_bytes_per_s', 'Network in', 'bytes'],
|
||||
['net.tx_bytes_per_s', 'Network out', 'bytes'],
|
||||
['temp.cpu_c', 'CPU temperature', 'temp'],
|
||||
['temp.nvme_max_c', 'NVMe temperature', 'temp'],
|
||||
['host.uptime_seconds', 'Uptime', 'dur'],
|
||||
['host.processes', 'Process count', 'num'],
|
||||
];
|
||||
|
||||
const FORMATS = {
|
||||
pct: { fn: (v) => num(v, 1), unit: '%', axis: (v) => num(v, 0) },
|
||||
bytes: { fn: (v) => bytes(v, 1), unit: 'B/s', axis: (v) => (Dash.fmtBytes(v, 0) || { text: '0' }).text +
|
||||
((Dash.fmtBytes(v, 0) || { unit: '' }).unit) },
|
||||
mhz: { fn: (v) => num(v, 0) + ' MHz', unit: 'MHz', axis: (v) => num(v, 0) },
|
||||
temp: { fn: (v) => num(v, 1) + ' °C', unit: '°C', axis: (v) => num(v, 0) },
|
||||
dur: { fn: (v) => duration(v), unit: '', axis: (v) => duration(v) },
|
||||
num: { fn: (v) => num(v, 1), unit: '', axis: (v) => num(v, 0) },
|
||||
};
|
||||
|
||||
function formatFor(kind) { return FORMATS[kind] || FORMATS.num; }
|
||||
|
||||
function metricKind(name) {
|
||||
if (name.endsWith('_pct')) return 'pct';
|
||||
if (name.endsWith('_bytes') || name.includes('bytes_per_s')) return 'bytes';
|
||||
if (name.includes('_c') && name.startsWith('temp.')) return 'temp';
|
||||
if (name === 'host.uptime_seconds') return 'dur';
|
||||
return 'num';
|
||||
}
|
||||
|
||||
// ---- host cards -----------------------------------------------------------
|
||||
//
|
||||
// Six heroes per host, in one fixed order: CPU, MEM, TEMP, DISK, GPU, NET. Every colour
|
||||
// comes from the payload's `severity` dicts — `severity.py` owns the thresholds, this file
|
||||
// compares no number to any bound — and `Dash.hero()` prints `n/a` with **no** colour
|
||||
// class when the number or its verdict is missing, so gateway's absent CPU sensor cannot
|
||||
// borrow last poll's green.
|
||||
//
|
||||
// `data-sev` names the verdict each slot paints; tests/test_api.py pins the six slots and
|
||||
// their order on the served bytes. The NET slot carries none on purpose: `network` has no
|
||||
// `severity` (a byte rate has no good/bad), so nothing here may colour it.
|
||||
//
|
||||
// Per-core usage is not plotted here or anywhere on this page — the series is still
|
||||
// stored as `cpu.usage_pct@coreN`, the page just stopped drawing a pattern nobody reads.
|
||||
// Everything that is not glance-worthy (load averages, clock, disk read/write/busy,
|
||||
// uptime, GPU power and temperature, filesystems, top processes) lives in "More detail",
|
||||
// collapsed by default: the TODO asked for a simpler surface, not a less complete page.
|
||||
function hostCard(name) {
|
||||
const card = document.createElement('article');
|
||||
card.className = 'card';
|
||||
@@ -80,38 +52,110 @@ function hostCard(name) {
|
||||
<span class="faint mono updated"></span>
|
||||
</header>
|
||||
<div class="body stack">
|
||||
<div class="metrics">
|
||||
<div class="metric"><div class="label">CPU</div><div class="value"><span data-k="cpu">n/a</span><span class="unit">%</span></div>
|
||||
<div class="bar" data-bar="cpu"><i></i></div></div>
|
||||
<div class="metric"><div class="label">Load 1 / 15</div><div class="value sm" data-k="load">n/a</div></div>
|
||||
<div class="metric"><div class="label">CPU temp</div><div class="value sm"><span data-k="temp">n/a</span><span class="unit">°C</span></div></div>
|
||||
<div class="metric"><div class="label">Clock</div><div class="value sm"><span data-k="freq">n/a</span><span class="unit">GHz</span></div></div>
|
||||
<div class="metric"><div class="label">Memory</div><div class="value sm"><span data-k="mem">n/a</span><span class="unit"></span></div>
|
||||
<div class="bar" data-bar="mem"><i></i></div></div>
|
||||
<div class="metric"><div class="label">Worst fs</div><div class="value sm"><span data-k="disk">n/a</span><span class="unit"></span></div>
|
||||
<div class="bar" data-bar="disk"><i></i></div></div>
|
||||
<div class="heroes">
|
||||
<div class="hero hero-bar" data-k="cpu" data-sev="cpu.severity.usage_pct"></div>
|
||||
<div class="hero hero-bar" data-k="mem" data-sev="memory.severity.used_pct"></div>
|
||||
<div class="hero" data-k="temp" data-sev="cpu.severity.temp_c"></div>
|
||||
<div class="hero hero-bar" data-k="disk" data-sev="disks.severity.worst_used_pct"></div>
|
||||
<div class="hero-slot" data-k="gpus" data-sev="gpus[].severity.busy_pct"></div>
|
||||
<div class="hero" data-k="net">
|
||||
<div class="label">Net</div>
|
||||
<div class="hero-lines">
|
||||
<div class="line"><span class="dir">↓</span><span data-k="rx">n/a</span></div>
|
||||
<div class="line"><span class="dir">↑</span><span data-k="tx">n/a</span></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="label faint" style="font-size:10.5px;letter-spacing:.06em;text-transform:uppercase">Per-core</div>
|
||||
<div class="cores" data-k="cores" style="margin-top:.25rem"></div>
|
||||
<div class="host-charts">
|
||||
<div class="host-chart">
|
||||
<div class="chart sm" data-chart="util"></div>
|
||||
<div class="legend" data-legend="util"></div>
|
||||
</div>
|
||||
<div class="host-chart">
|
||||
<div class="chart sm" data-chart="net"></div>
|
||||
<div class="legend" data-legend="net"></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="metrics">
|
||||
<div class="metric"><div class="label">Disk read</div><div class="value sm"><span data-k="dr">n/a</span><span class="unit"></span></div></div>
|
||||
<div class="metric"><div class="label">Disk write</div><div class="value sm"><span data-k="dw">n/a</span><span class="unit"></span></div></div>
|
||||
<div class="metric"><div class="label">Disk busy</div><div class="value sm"><span data-k="dutil">n/a</span><span class="unit">%</span></div></div>
|
||||
<div class="metric"><div class="label">Net in</div><div class="value sm"><span data-k="rx">n/a</span><span class="unit"></span></div></div>
|
||||
<div class="metric"><div class="label">Net out</div><div class="value sm"><span data-k="tx">n/a</span><span class="unit"></span></div></div>
|
||||
<div class="metric"><div class="label">Up</div><div class="value sm" data-k="uptime">n/a</div></div>
|
||||
</div>
|
||||
<div data-k="gpus"></div>
|
||||
<details>
|
||||
<summary class="faint" style="cursor:pointer;font-size:12px">Filesystems & processes</summary>
|
||||
<summary class="faint" style="cursor:pointer;font-size:12px">More detail</summary>
|
||||
<div class="metrics" style="margin-top:.6rem">
|
||||
<div class="metric"><div class="label">Load 1 / 5 / 15</div><div class="value sm" data-k="load">n/a</div></div>
|
||||
<div class="metric"><div class="label">Clock</div><div class="value sm" data-k="freq">n/a</div></div>
|
||||
<div class="metric"><div class="label">Disk read</div><div class="value sm" data-k="dr">n/a</div></div>
|
||||
<div class="metric"><div class="label">Disk write</div><div class="value sm" data-k="dw">n/a</div></div>
|
||||
<div class="metric"><div class="label">Disk busy</div><div class="value sm" data-k="dutil">n/a</div></div>
|
||||
<div class="metric"><div class="label">Up</div><div class="value sm" data-k="uptime">n/a</div></div>
|
||||
</div>
|
||||
<div data-k="gpu-extra"></div>
|
||||
<div style="margin-top:.6rem" data-k="details"></div>
|
||||
</details>
|
||||
</div>`;
|
||||
return card;
|
||||
}
|
||||
|
||||
/** `200 / 253 GiB` — the percentage is the verdict, the bytes are the detail. */
|
||||
function memorySub(mem) {
|
||||
const used = Dash.fmtBytes(mem.used_bytes, 1);
|
||||
const total = Dash.fmtBytes(mem.total_bytes, 0);
|
||||
if (!used || !total) return null;
|
||||
if (used.unit === total.unit) return `${used.text} / ${total.text} ${total.unit}`;
|
||||
return `${bytes(mem.used_bytes, 1)} of ${bytes(mem.total_bytes, 0)}`;
|
||||
}
|
||||
|
||||
/** Model, VRAM used / total, and the honest APU pool view: a Strix Halo's model lives in
|
||||
* GTT, so "VRAM 3%" and "pool 78%" are both true and the card shows both, not whichever
|
||||
* flatters the machine. */
|
||||
function gpuSub(gpu) {
|
||||
const bits = [gpu.model || 'GPU'];
|
||||
bits.push(gpu.vram_used_bytes == null || gpu.vram_total_bytes == null
|
||||
? 'VRAM n/a'
|
||||
: `VRAM ${bytes(gpu.vram_used_bytes, 1)} / ${bytes(gpu.vram_total_bytes, 1)}`);
|
||||
if (gpu.mem_used_pct != null) bits.push(`pool ${num(gpu.mem_used_pct, 0)}%`);
|
||||
return bits.join(' · ');
|
||||
}
|
||||
|
||||
/** The GPU slot of the hero row: one hero per card, or the honest no-GPU state. A box
|
||||
* with no card is not a box at 0% busy, and gateway must not look broken for it. */
|
||||
function renderGpuSlot(cell, gpus) {
|
||||
if (cell.dataset.count !== String(gpus.length)) {
|
||||
cell.innerHTML = '';
|
||||
for (let i = 0; i < Math.max(gpus.length, 1); i += 1) {
|
||||
const row = document.createElement('div');
|
||||
row.className = 'hero hero-bar';
|
||||
cell.appendChild(row);
|
||||
}
|
||||
cell.dataset.count = String(gpus.length);
|
||||
}
|
||||
const rows = Array.from(cell.children);
|
||||
if (!gpus.length) {
|
||||
Dash.hero(rows[0], { label: 'GPU', value: null, sub: 'No GPU reported by this agent' });
|
||||
return;
|
||||
}
|
||||
gpus.forEach((gpu, i) => {
|
||||
Dash.hero(rows[i], {
|
||||
label: gpus.length > 1 ? `GPU ${i}` : 'GPU',
|
||||
value: gpu.busy_pct, unit: '%', digits: 0,
|
||||
severity: (gpu.severity || {}).busy_pct, bar: true,
|
||||
sub: gpuSub(gpu),
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/** What the GPU hero left out, in "More detail": power, temperature, both memory pools. */
|
||||
function renderGpuExtra(cell, gpus) {
|
||||
if (!gpus.length) { cell.innerHTML = ''; return; }
|
||||
cell.innerHTML = `
|
||||
<div class="label faint" style="font-size:10.5px;letter-spacing:.06em;text-transform:uppercase;margin-top:.6rem">GPU detail</div>
|
||||
<table class="data" style="margin-top:.25rem"><thead><tr>
|
||||
<th>Card</th><th>Power</th><th>Temp</th><th>VRAM</th><th>GTT</th></tr></thead>
|
||||
<tbody>${gpus.map((gpu) => `<tr>
|
||||
<td class="name mono" style="font-size:11.5px">${DashCharts.escapeHtml(gpu.model || 'GPU')}</td>
|
||||
<td class="num">${gpu.power_watts == null ? 'n/a' : `${num(gpu.power_watts, 0)} W`}</td>
|
||||
<td class="num">${gpu.temp_c == null ? 'n/a' : `${num(gpu.temp_c, 0)} °C`}</td>
|
||||
<td class="num mute">${bytes(gpu.vram_used_bytes, 1)} / ${bytes(gpu.vram_total_bytes, 1)}</td>
|
||||
<td class="num mute">${bytes(gpu.gtt_used_bytes, 1)}</td></tr>`).join('')}</tbody></table>`;
|
||||
}
|
||||
|
||||
function renderHostCard(card, host) {
|
||||
const q = (key) => card.querySelector(`[data-k="${key}"]`);
|
||||
const stateRow = host.state || {};
|
||||
@@ -125,65 +169,41 @@ function renderHostCard(card, host) {
|
||||
markStale(card, stateRow.ok === false || Dash.ageSeconds(stateRow.last_ok) > 5 * (host.window || 10));
|
||||
|
||||
const cpu = host.cpu || {};
|
||||
setValue(q('cpu'), cpu.usage_pct, 0);
|
||||
setBar(card.querySelector('[data-bar="cpu"]'), cpu.usage_pct, severityFor(cpu.usage_pct, 75, 90));
|
||||
setTxt(q('load'), cpu.load1 == null ? 'n/a' : `${num(cpu.load1, 2)} / ${num(cpu.load15, 2)}`);
|
||||
setValue(q('temp'), cpu.temp_c, 1);
|
||||
setValue(q('freq'), cpu.freq_mhz == null ? null : cpu.freq_mhz / 1000, 2);
|
||||
|
||||
const mem = host.memory || {};
|
||||
const memText = Dash.fmtBytes(mem.used_bytes, 1);
|
||||
const memTotal = Dash.fmtBytes(mem.total_bytes, 0);
|
||||
setTxt(q('mem'), memText ? memText.text : 'n/a');
|
||||
setTxt(q('mem').nextElementSibling, memText ? `${memText.unit}/${memTotal ? memTotal.text + memTotal.unit : ''}` : '');
|
||||
setBar(card.querySelector('[data-bar="mem"]'), mem.used_pct, severityFor(mem.used_pct, 80, 92));
|
||||
|
||||
const disks = host.disks || {};
|
||||
setTxt(q('disk'), disks.worst_used_pct == null ? 'n/a' : num(disks.worst_used_pct, 0) + '%');
|
||||
setTxt(q('disk').nextElementSibling, disks.worst_mount ? ' ' + disks.worst_mount : '');
|
||||
setBar(card.querySelector('[data-bar="disk"]'), disks.worst_used_pct, severityFor(disks.worst_used_pct, 80, 92));
|
||||
const net = host.network || {};
|
||||
|
||||
DashCharts.renderCores(q('cores'), cpu.core_usage_pct || []);
|
||||
// The six heroes. The verdict is whatever the payload carried (`severity` dicts written
|
||||
// by severity.py, pinned by tests/test_system_contract.py); a missing number or a
|
||||
// missing verdict prints `n/a` with no colour class at all.
|
||||
Dash.hero(q('cpu'), { label: 'CPU', value: cpu.usage_pct, unit: '%', digits: 0,
|
||||
severity: (cpu.severity || {}).usage_pct, bar: true });
|
||||
Dash.hero(q('mem'), { label: 'Memory', value: mem.used_pct, unit: '%', digits: 0,
|
||||
severity: (mem.severity || {}).used_pct, bar: true,
|
||||
sub: memorySub(mem) });
|
||||
Dash.hero(q('temp'), { label: 'CPU temp', value: cpu.temp_c, unit: '°C', digits: 1,
|
||||
severity: (cpu.severity || {}).temp_c });
|
||||
// the mount is the `.sub` because "80 %" without "of what" is not actionable
|
||||
Dash.hero(q('disk'), { label: 'Disk', value: disks.worst_used_pct, unit: '%', digits: 0,
|
||||
severity: (disks.severity || {}).worst_used_pct, bar: true,
|
||||
sub: disks.worst_mount });
|
||||
renderGpuSlot(q('gpus'), host.gpus || []);
|
||||
// One wire seen in two directions, two lines, never two competing numbers — and a null
|
||||
// (no physical interface reported) stays `n/a` rather than becoming a confident 0 B/s.
|
||||
// No severity by design: `network` carries none, so this slot never colours.
|
||||
setTxt(q('rx'), net.rx_bytes_per_s == null ? null : Dash.rate(net.rx_bytes_per_s));
|
||||
setTxt(q('tx'), net.tx_bytes_per_s == null ? null : Dash.rate(net.tx_bytes_per_s));
|
||||
|
||||
setValue(q('dr'), disks.read_bytes_per_s == null ? null : disks.read_bytes_per_s, null);
|
||||
if (disks.read_bytes_per_s != null) {
|
||||
const parts = Dash.fmtBytes(disks.read_bytes_per_s, 1);
|
||||
setTxt(q('dr'), parts.text); setTxt(q('dr').nextElementSibling, parts.unit + '/s');
|
||||
const w = Dash.fmtBytes(disks.write_bytes_per_s, 1);
|
||||
setTxt(q('dw'), w ? w.text : 'n/a'); setTxt(q('dw').nextElementSibling, w ? w.unit + '/s' : '');
|
||||
} else { setTxt(q('dr'), 'n/a'); setTxt(q('dw'), 'n/a'); }
|
||||
setValue(q('dutil'), disks.util_pct, 0);
|
||||
const rx = Dash.fmtBytes((host.network || {}).rx_bytes_per_s, 1);
|
||||
const tx = Dash.fmtBytes((host.network || {}).tx_bytes_per_s, 1);
|
||||
setTxt(q('rx'), rx ? rx.text : 'n/a'); setTxt(q('rx').nextElementSibling, rx ? rx.unit + '/s' : '');
|
||||
setTxt(q('tx'), tx ? tx.text : 'n/a'); setTxt(q('tx').nextElementSibling, tx ? tx.unit + '/s' : '');
|
||||
setTxt(q('uptime'), duration((host.info || {}).uptime_seconds));
|
||||
|
||||
// GPUs: one row each, with both memory pools named (an APU's model lives in GTT)
|
||||
const gpus = host.gpus || [];
|
||||
const gpuHost = q('gpus');
|
||||
if (!gpus.length) {
|
||||
gpuHost.innerHTML = '<div class="faint" style="font-size:12px">No GPU reported by this agent</div>';
|
||||
} else {
|
||||
gpuHost.innerHTML = gpus.map((gpu) => `
|
||||
<div class="stack" style="gap:.2rem;margin-top:.35rem">
|
||||
<div class="gap" style="justify-content:space-between">
|
||||
<span class="mono" style="font-size:12px">${DashCharts.escapeHtml(gpu.model || 'GPU')}</span>
|
||||
<span class="mono" style="font-size:12px">
|
||||
${gpu.busy_pct == null ? 'n/a' : num(gpu.busy_pct, 0) + '% busy'} ·
|
||||
${gpu.power_watts == null ? 'power n/a' : num(gpu.power_watts, 0) + ' W'} ·
|
||||
${gpu.temp_c == null ? 'temp n/a' : num(gpu.temp_c, 0) + '°C'}</span>
|
||||
</div>
|
||||
<div class="bar thin"><i style="width:${(gpu.busy_pct || 0).toFixed(1)}%"></i></div>
|
||||
<div class="gap" style="justify-content:space-between;font-size:11.5px" class="faint">
|
||||
<span class="faint">VRAM ${gpu.vram_used_bytes == null ? 'n/a' : bytes(gpu.vram_used_bytes, 0)}
|
||||
${gpu.vram_total_bytes ? 'of ' + bytes(gpu.vram_total_bytes, 0) : ''}</span>
|
||||
<span class="faint">shared (GTT) ${gpu.gtt_used_bytes == null ? 'n/a' : bytes(gpu.gtt_used_bytes, 0)}
|
||||
${gpu.mem_used_pct == null ? '' : ' · pool ' + num(gpu.mem_used_pct, 0) + '%'}</span>
|
||||
</div>
|
||||
<div class="bar thin"><i style="width:${(gpu.mem_used_pct || 0).toFixed(1)}%;background:var(--series-2)"></i></div>
|
||||
</div>`).join('');
|
||||
}
|
||||
// "More detail": everything the glance does not need but the page still owes.
|
||||
setTxt(q('load'), cpu.load1 == null && cpu.load5 == null && cpu.load15 == null
|
||||
? null : `${num(cpu.load1, 2)} / ${num(cpu.load5, 2)} / ${num(cpu.load15, 2)}`);
|
||||
setTxt(q('freq'), cpu.freq_mhz == null ? null : `${num(cpu.freq_mhz / 1000, 2)} GHz`);
|
||||
setTxt(q('dr'), disks.read_bytes_per_s == null ? null : Dash.rate(disks.read_bytes_per_s));
|
||||
setTxt(q('dw'), disks.write_bytes_per_s == null ? null : Dash.rate(disks.write_bytes_per_s));
|
||||
setTxt(q('dutil'), disks.util_pct == null ? null : `${num(disks.util_pct, 0)}%`);
|
||||
const up = (host.info || {}).uptime_seconds;
|
||||
setTxt(q('uptime'), up == null ? null : duration(up));
|
||||
renderGpuExtra(q('gpu-extra'), host.gpus || []);
|
||||
|
||||
const filesystems = (disks.filesystems || []).filter((fs) => fs.used_pct != null);
|
||||
const procs = host.processes || {};
|
||||
@@ -226,180 +246,175 @@ function stalenessText(row) {
|
||||
return seconds > 30 ? `stale ${duration(seconds)}` : `updated ${seconds < 2 ? 'now' : duration(seconds)}`;
|
||||
}
|
||||
|
||||
// ---- charts ---------------------------------------------------------------
|
||||
function makeCharts() {
|
||||
const common = { height: 165, xSpan: state.range };
|
||||
state.charts.cpu = new DashCharts.LineChart('#chart-cpu', Object.assign({}, common, {
|
||||
yMin: 0, yMax: 100, yFormat: (v) => num(v, 0) }));
|
||||
state.charts.mem = new DashCharts.LineChart('#chart-mem', Object.assign({}, common, {
|
||||
yMin: 0, yMax: 100, yFormat: (v) => num(v, 0) }));
|
||||
state.charts.disk = new DashCharts.LineChart('#chart-disk', Object.assign({}, common, {
|
||||
yMin: 0, yFormat: (v) => (Dash.fmtBytes(v, 0) || { text: '0' }).text }));
|
||||
state.charts.net = new DashCharts.LineChart('#chart-net', Object.assign({}, common, {
|
||||
yMin: 0, yFormat: (v) => (Dash.fmtBytes(v, 0) || { text: '0' }).text }));
|
||||
state.charts.gpu = new DashCharts.LineChart('#chart-gpu', Object.assign({}, common, {
|
||||
yMin: 0, yMax: 100, yFormat: (v) => num(v, 0) }));
|
||||
state.charts.temp = new DashCharts.LineChart('#chart-temp', Object.assign({}, common, {
|
||||
yFormat: (v) => num(v, 0) }));
|
||||
state.charts.detail = new DashCharts.LineChart('#chart-detail', Object.assign({}, common, {
|
||||
height: 200, yFormat: (v) => num(v, 1) }));
|
||||
}
|
||||
// ---- per-host charts ------------------------------------------------------
|
||||
//
|
||||
// One host, one panel, at most three charts — and no chart on this page contains a second
|
||||
// host. Six metrics x three hosts as six separate charts is the unreadable version of
|
||||
// "per-host charts", so the metrics that share a unit share an axis:
|
||||
//
|
||||
// util cpu.usage_pct + mem.used_pct + disk.used_pct one 0-100 % axis
|
||||
// net net.rx_bytes_per_s + net.tx_bytes_per_s one byte-rate axis
|
||||
// gpu gpu.busy_pct@<index> per card the agent reports (no chart when there is none)
|
||||
//
|
||||
// Each host costs **one** `/api/system/history` request per refresh: the endpoint takes a
|
||||
// `hosts=` and a `metrics=` list, so the XHR budget for a three-host fleet is one snapshot
|
||||
// plus three history calls plus the event feed, whatever the chart count. GPU stream names
|
||||
// come from the payload's own `gpus[].index` (`sources/host.py` writes `gpu.busy_pct@<i>`),
|
||||
// so nothing here has to discover them through /api/metrics the way the deleted
|
||||
// fleet-wide GPU chart did.
|
||||
//
|
||||
// The charts live in a Map keyed by host name and are torn down with the card: a panel for
|
||||
// a host that has left the payload would otherwise keep drawing its last series forever.
|
||||
const CHART_HEIGHT = 120;
|
||||
const UTIL_SERIES = [
|
||||
['cpu.usage_pct', 'CPU'],
|
||||
['mem.used_pct', 'Mem'],
|
||||
['disk.used_pct', 'Disk (worst)'],
|
||||
];
|
||||
const NET_SERIES = [
|
||||
['net.rx_bytes_per_s', '↓ in'],
|
||||
['net.tx_bytes_per_s', '↑ out'],
|
||||
];
|
||||
const PCT_FORMAT = (v) => `${num(v, 1)}%`;
|
||||
const RATE_FORMAT = (v) => `${bytes(v, 1)}/s`;
|
||||
const AXIS_PCT = (v) => num(v, 0);
|
||||
const AXIS_RATE = (v) => {
|
||||
const parts = Dash.fmtBytes(v, 0) || { text: '0', unit: '' };
|
||||
return `${parts.text}${parts.unit}`;
|
||||
};
|
||||
|
||||
function setChartSpan(span) {
|
||||
Object.values(state.charts).forEach((chart) => { chart.opts.xSpan = span; });
|
||||
}
|
||||
|
||||
function seriesFor(response, metric) {
|
||||
return (response.series || []).filter((s) => s.metric === metric).map((s) => ({
|
||||
name: s.host, points: s.points.map((p) => ({ ts: p.ts, value: p.value })),
|
||||
}));
|
||||
}
|
||||
|
||||
async function refreshCharts(snapshot) {
|
||||
const hostNames = (snapshot.hosts || []).map((h) => h.name);
|
||||
const detailHost = state.detailHost || hostNames[0] || null;
|
||||
const span = state.range;
|
||||
const requests = {
|
||||
cpu: fetchJSON(`/api/system/history?metrics=cpu.usage_pct&range=${span}`).catch(() => null),
|
||||
mem: fetchJSON(`/api/system/history?metrics=mem.used_pct&range=${span}`).catch(() => null),
|
||||
temp: fetchJSON(`/api/system/history?metrics=temp.cpu_c&range=${span}`).catch(() => null),
|
||||
disk: detailHost ? fetchJSON(`/api/system/history?hosts=${encodeURIComponent(detailHost)}&` +
|
||||
`metrics=disk.read_bytes_per_s,disk.write_bytes_per_s&range=${span}`).catch(() => null) : null,
|
||||
net: detailHost ? fetchJSON(`/api/system/history?hosts=${encodeURIComponent(detailHost)}&` +
|
||||
`metrics=net.rx_bytes_per_s,net.tx_bytes_per_s&range=${span}`).catch(() => null) : null,
|
||||
};
|
||||
const [cpu, mem, temp, disk, net] = await Promise.all([requests.cpu, requests.mem,
|
||||
requests.temp, requests.disk, requests.net]);
|
||||
|
||||
if (cpu) {
|
||||
const series = seriesFor(cpu, 'cpu.usage_pct');
|
||||
state.charts.cpu.setData(series);
|
||||
DashCharts.renderLegend($('#legend-cpu'), series, (v) => num(v, 1) + '%', true);
|
||||
}
|
||||
if (mem) {
|
||||
const series = seriesFor(mem, 'mem.used_pct');
|
||||
state.charts.mem.setData(series);
|
||||
DashCharts.renderLegend($('#legend-mem'), series, (v) => num(v, 1) + '%');
|
||||
}
|
||||
if (temp) {
|
||||
const series = seriesFor(temp, 'temp.cpu_c');
|
||||
state.charts.temp.setData(series);
|
||||
DashCharts.renderLegend($('#legend-temp'), series, (v) => num(v, 1) + '°C', true);
|
||||
}
|
||||
if (disk) {
|
||||
const read = (disk.series.find((s) => s.metric === 'disk.read_bytes_per_s') || { points: [] })
|
||||
.points.map((p) => ({ ts: p.ts, value: p.value }));
|
||||
const write = (disk.series.find((s) => s.metric === 'disk.write_bytes_per_s') || { points: [] })
|
||||
.points.map((p) => ({ ts: p.ts, value: p.value }));
|
||||
const series = [{ name: 'read', points: read }, { name: 'write', points: write }];
|
||||
state.charts.disk.setData(series);
|
||||
DashCharts.renderLegend($('#legend-disk'), series, (v) => bytes(v, 1) + '/s', true);
|
||||
}
|
||||
if (net) {
|
||||
const rx = (net.series.find((s) => s.metric === 'net.rx_bytes_per_s') || { points: [] })
|
||||
.points.map((p) => ({ ts: p.ts, value: p.value }));
|
||||
const tx = (net.series.find((s) => s.metric === 'net.tx_bytes_per_s') || { points: [] })
|
||||
.points.map((p) => ({ ts: p.ts, value: p.value }));
|
||||
const series = [{ name: 'in', points: rx }, { name: 'out', points: tx }];
|
||||
state.charts.net.setData(series);
|
||||
DashCharts.renderLegend($('#legend-net'), series, (v) => bytes(v, 1) + '/s', true);
|
||||
}
|
||||
await refreshGpuChart(hostNames);
|
||||
await refreshDetailChart();
|
||||
}
|
||||
|
||||
async function refreshGpuChart(hostNames) {
|
||||
if (!hostNames.length) {
|
||||
state.charts.gpu.setData([]);
|
||||
$('#legend-gpu').innerHTML = '<span class="faint">no hosts</span>';
|
||||
return;
|
||||
}
|
||||
const series = [];
|
||||
for (const host of hostNames) {
|
||||
// per-stream names are discovered, not guessed: gpu.busy_pct@0, @1, ...
|
||||
const found = await fetchJSON(`/api/metrics?source=host&host=${encodeURIComponent(host)}`)
|
||||
.catch(() => ({ metrics: [] }));
|
||||
const names = (found.metrics || []).filter((m) => m.startsWith('gpu.busy_pct@'));
|
||||
for (const name of names) {
|
||||
const data = await fetchJSON(`/api/history?source=host&host=${encodeURIComponent(host)}` +
|
||||
`&metric=${encodeURIComponent(name)}&range=${state.range}`).catch(() => null);
|
||||
if (data) series.push({ name: `${host} ${name.split('@')[1]}`,
|
||||
points: data.points.map((p) => ({ ts: p.ts, value: p.value })) });
|
||||
}
|
||||
}
|
||||
state.charts.gpu.setData(series);
|
||||
DashCharts.renderLegend($('#legend-gpu'), series, (v) => num(v, 0) + '%', true);
|
||||
}
|
||||
|
||||
async function refreshDetailChart() {
|
||||
const host = state.detailHost;
|
||||
if (!host) { state.charts.detail.setData([]); return; }
|
||||
const kind = metricKind(state.detailMetric);
|
||||
const format = formatFor(kind);
|
||||
const data = await fetchJSON(`/api/history?source=host&host=${encodeURIComponent(host)}` +
|
||||
`&metric=${encodeURIComponent(state.detailMetric)}&range=${state.range}`).catch(() => null);
|
||||
if (!data) return;
|
||||
const series = [{ name: state.detailMetric,
|
||||
points: data.points.map((p) => ({ ts: p.ts, value: p.value })) }];
|
||||
state.charts.detail.opts.yFormat = format.axis;
|
||||
state.charts.detail.opts.yMin = state.detailMetric.endsWith('_pct') ? 0 : null;
|
||||
state.charts.detail.opts.yMax = state.detailMetric.endsWith('_pct') ? 100 : null;
|
||||
state.charts.detail.setData(series);
|
||||
DashCharts.renderLegend($('#legend-detail'), series, format.fn);
|
||||
const last = series[0].points.slice(-1)[0];
|
||||
$('#detail-note').textContent = `${series[0].points.length} samples over ${duration(state.range)} · ` +
|
||||
`last ${last ? format.fn(last.value) : 'n/a'} · ${state.range <= 86400 ? 'raw resolution' : 'rolled up'}`;
|
||||
}
|
||||
|
||||
async function populateDetailMetrics(host) {
|
||||
const select = $('#detail-metric');
|
||||
const found = await fetchJSON(`/api/metrics?source=host&host=${encodeURIComponent(host)}`)
|
||||
.catch(() => ({ metrics: [] }));
|
||||
const curated = new Set(DETAIL_METRICS.map(([name]) => name));
|
||||
const extras = (found.metrics || []).filter((m) => !curated.has(m)).sort();
|
||||
select.innerHTML = '';
|
||||
const groupMain = document.createElement('optgroup');
|
||||
groupMain.label = 'Common';
|
||||
DETAIL_METRICS.forEach(([name, label]) => {
|
||||
const option = document.createElement('option');
|
||||
option.value = name; option.textContent = label;
|
||||
groupMain.appendChild(option);
|
||||
/** The GPU streams one host reports, named from the payload rather than guessed. */
|
||||
function gpuStreams(host) {
|
||||
return (host.gpus || []).map((gpu, i) => {
|
||||
const index = gpu.index == null ? i : gpu.index;
|
||||
return { metric: `gpu.busy_pct@${index}`, label: gpu.model || `GPU ${index}` };
|
||||
});
|
||||
select.appendChild(groupMain);
|
||||
if (extras.length) {
|
||||
const groupExtra = document.createElement('optgroup');
|
||||
groupExtra.label = `Per-device (${extras.length})`;
|
||||
extras.forEach((name) => {
|
||||
const option = document.createElement('option');
|
||||
option.value = name; option.textContent = name;
|
||||
groupExtra.appendChild(option);
|
||||
});
|
||||
select.appendChild(groupExtra);
|
||||
}
|
||||
|
||||
/**
|
||||
* The chart objects inside one host panel, built once and reused every poll.
|
||||
*
|
||||
* The GPU chart is added the first time the host reports a card and removed the moment it
|
||||
* stops reporting one — a machine whose GPU went missing must not keep a plot of it.
|
||||
*/
|
||||
function hostChartSet(card, host) {
|
||||
let set = state.charts.get(host.name);
|
||||
if (!set) {
|
||||
// no `area` here on purpose — `drawHostChart()` decides the fill per draw, from how many
|
||||
// series actually have points, which is the only question worth asking
|
||||
const common = { height: CHART_HEIGHT, xSpan: state.range };
|
||||
// Every chart names itself for assistive tech: the panel heading says *who*, the
|
||||
// aria-label says what is plotted. An unlabelled SVG is silence, and a screen reader
|
||||
// must not need the colour-coded legend to know which machine it is listening to.
|
||||
set = {
|
||||
util: new DashCharts.LineChart(card.querySelector('[data-chart="util"]'),
|
||||
Object.assign({}, common, { yMin: 0, yMax: 100, yFormat: AXIS_PCT,
|
||||
ariaLabel: `${host.name} — utilisation over the selected window: CPU, memory and worst disk in per cent` })),
|
||||
net: new DashCharts.LineChart(card.querySelector('[data-chart="net"]'),
|
||||
Object.assign({}, common, { yMin: 0, yFormat: AXIS_RATE,
|
||||
ariaLabel: `${host.name} — network traffic over the selected window: bytes in and bytes out` })),
|
||||
gpu: null,
|
||||
};
|
||||
state.charts.set(host.name, set);
|
||||
}
|
||||
select.value = DETAIL_METRICS.some(([name]) => name === state.detailMetric)
|
||||
? state.detailMetric : (extras[0] || DETAIL_METRICS[0][0]);
|
||||
state.detailMetric = select.value || DETAIL_METRICS[0][0];
|
||||
const wantsGpu = gpuStreams(host).length > 0;
|
||||
if (wantsGpu && !set.gpu) {
|
||||
const slot = document.createElement('div');
|
||||
slot.className = 'host-chart';
|
||||
slot.dataset.chartSlot = 'gpu';
|
||||
slot.innerHTML = '<div class="chart sm" data-chart="gpu"></div>' +
|
||||
'<div class="legend" data-legend="gpu"></div>';
|
||||
card.querySelector('.host-charts').appendChild(slot);
|
||||
set.gpu = new DashCharts.LineChart(slot.querySelector('[data-chart="gpu"]'), {
|
||||
height: CHART_HEIGHT, xSpan: state.range, yMin: 0, yMax: 100, yFormat: AXIS_PCT,
|
||||
ariaLabel: `${host.name} — GPU busy per cent over the selected window: ` +
|
||||
gpuStreams(host).map((s) => s.label).join(', ') });
|
||||
}
|
||||
if (!wantsGpu && set.gpu) {
|
||||
// the whole slot goes: an empty wrapper or a plot of a card that is no longer
|
||||
// reported is a machine's panel lying about hardware it does not have
|
||||
card.querySelectorAll('[data-chart-slot="gpu"]').forEach((node) => node.remove());
|
||||
set.gpu = null;
|
||||
}
|
||||
return set;
|
||||
}
|
||||
|
||||
/**
|
||||
* `[metric, label][]` -> LineChart series, from one host's history response.
|
||||
*
|
||||
* The legend names the *metric*, never the host: a chart here can only ever hold one
|
||||
* machine, so repeating the host name three times under a chart is ink with no information.
|
||||
*/
|
||||
function seriesFor(response, definitions) {
|
||||
return definitions.map(([metric, label]) => {
|
||||
const found = (response.series || []).find((s) => s.metric === metric);
|
||||
return { name: label,
|
||||
points: ((found || {}).points || []).map((p) => ({ ts: p.ts, value: p.value })) };
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Plot one series set into one panel chart and repaint its legend.
|
||||
*
|
||||
* The fill is decided per draw on purpose: a single series reads as a shape worth shading
|
||||
* (GPU busy), while two or three filled areas stack into one block and bury the smaller
|
||||
* line — which is how a quiet CPU hides behind a busy memory plot.
|
||||
*/
|
||||
function drawHostChart(chart, card, key, series, format) {
|
||||
chart.opts.xSpan = state.range;
|
||||
chart.opts.area = series.filter((s) => s.points.length).length < 2;
|
||||
chart.setData(series);
|
||||
DashCharts.renderLegend(card.querySelector(`[data-legend="${key}"]`), series, format, true);
|
||||
}
|
||||
|
||||
/** One host's charts, from one request. Called for every host on every refresh. */
|
||||
async function renderHostCharts(card, host) {
|
||||
const charts = hostChartSet(card, host);
|
||||
const gpu = gpuStreams(host);
|
||||
const wanted = UTIL_SERIES.concat(NET_SERIES, gpu.map((s) => [s.metric, s.label]));
|
||||
const query = `/api/system/history?hosts=${encodeURIComponent(host.name)}` +
|
||||
`&metrics=${encodeURIComponent(wanted.map(([metric]) => metric).join(','))}` +
|
||||
`&range=${state.range}`;
|
||||
const response = await fetchJSON(query).catch(() => null);
|
||||
if (!response || !state.charts.has(host.name)) return; // the panel went away mid-flight
|
||||
|
||||
drawHostChart(charts.util, card, 'util', seriesFor(response, UTIL_SERIES), PCT_FORMAT);
|
||||
drawHostChart(charts.net, card, 'net', seriesFor(response, NET_SERIES), RATE_FORMAT);
|
||||
if (charts.gpu) {
|
||||
drawHostChart(charts.gpu, card, 'gpu',
|
||||
seriesFor(response, gpu.map((s) => [s.metric, s.label])), PCT_FORMAT);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The Δ window the rates were actually measured over, for the page footnote.
|
||||
*
|
||||
* `window_seconds` beside the fleet is the *poll interval*; the window behind a rate is the
|
||||
* estimator's (`RATE_WINDOW_SECONDS`) and the two need not agree, so the footnote reads each
|
||||
* host's own measured `window_seconds` instead of repeating a config number as a measurement
|
||||
* (invariant 2: a rate says how it was obtained). One poll loop drives every host, so the
|
||||
* windows differ by milliseconds; a real spread prints as a range rather than picking a winner.
|
||||
*/
|
||||
function windowLabel(values, fallback) {
|
||||
const nums = values.filter((value) => value != null && Number.isFinite(value));
|
||||
if (!nums.length) return fallback;
|
||||
const lo = Math.min(...nums);
|
||||
const hi = Math.max(...nums);
|
||||
return lo.toFixed(1) === hi.toFixed(1) ? `Δ${hi.toFixed(1)}s`
|
||||
: `Δ${lo.toFixed(1)}–${hi.toFixed(1)}s`;
|
||||
}
|
||||
|
||||
/** Drop cards (and charts) for hosts that are no longer in the payload. */
|
||||
function pruneHosts(living) {
|
||||
Array.from(state.hosts.keys()).forEach((name) => {
|
||||
if (living.includes(name)) return;
|
||||
state.hosts.get(name).remove();
|
||||
state.hosts.delete(name);
|
||||
state.charts.delete(name);
|
||||
});
|
||||
}
|
||||
|
||||
// ---- snapshot -------------------------------------------------------------
|
||||
function renderFleet(fleet) {
|
||||
if (!fleet) return;
|
||||
setTxt($('#f-hosts'), fleet.hosts_live == null ? 'n/a' : String(fleet.hosts_live));
|
||||
setTxt($('#f-hosts-of'), fleet.hosts_total == null ? '' : `of ${fleet.hosts_total}`);
|
||||
setValue($('#f-cpu'), fleet.cpu_usage_pct_avg, 0);
|
||||
setValue($('#f-load'), fleet.load1_total, 2);
|
||||
const used = Dash.fmtBytes(fleet.memory_used_bytes, 0);
|
||||
const total = Dash.fmtBytes(fleet.memory_total_bytes, 0);
|
||||
setTxt($('#f-mem'), used ? used.text : 'n/a');
|
||||
setTxt($('#f-mem-of'), used && total ? `${used.unit} of ${total.text} ${total.unit}` : '');
|
||||
setValue($('#f-disk'), fleet.disk_worst_used_pct, 0);
|
||||
const rx = Dash.fmtBytes(fleet.net_rx_bytes_per_s, 1);
|
||||
const tx = Dash.fmtBytes(fleet.net_tx_bytes_per_s, 1);
|
||||
setTxt($('#f-rx'), rx ? rx.text : 'n/a'); setTxt($('#f-rx-unit'), rx ? rx.unit + '/s' : '');
|
||||
setTxt($('#f-tx'), tx ? tx.text : 'n/a'); setTxt($('#f-tx-unit'), tx ? tx.unit + '/s' : '');
|
||||
setValue($('#f-gpu'), fleet.gpu_busy_pct_avg, 0);
|
||||
}
|
||||
|
||||
async function refresh() {
|
||||
let snapshot;
|
||||
try {
|
||||
@@ -411,6 +426,7 @@ async function refresh() {
|
||||
const states = {};
|
||||
(snapshot.states || []).forEach((row) => { states[row.key.split(':')[1]] = row; });
|
||||
const hostNames = [];
|
||||
const chartJobs = [];
|
||||
(snapshot.hosts || []).forEach((host) => {
|
||||
hostNames.push(host.name);
|
||||
host.state = states[host.name];
|
||||
@@ -420,43 +436,33 @@ async function refresh() {
|
||||
card = hostCard(host.name);
|
||||
state.hosts.set(host.name, card);
|
||||
$('#hosts').appendChild(card);
|
||||
card.addEventListener('click', () => {
|
||||
if (state.detailHost !== host.name) {
|
||||
state.detailHost = host.name;
|
||||
$('#detail-host').value = host.name;
|
||||
populateDetailMetrics(host.name).then(refreshDetailChart);
|
||||
}
|
||||
});
|
||||
}
|
||||
renderHostCard(card, host);
|
||||
// one history request per host, all of its charts fed from the same response
|
||||
chartJobs.push(renderHostCharts(card, host));
|
||||
});
|
||||
// a host that left HOST_TARGETS must take its panel — and its live charts — with it
|
||||
pruneHosts(hostNames);
|
||||
$('#hosts-empty').hidden = hostNames.length > 0;
|
||||
|
||||
const hostSelect = $('#detail-host');
|
||||
if (hostSelect.dataset.names !== hostNames.join(',')) {
|
||||
hostSelect.dataset.names = hostNames.join(',');
|
||||
hostSelect.innerHTML = hostNames.map((name) =>
|
||||
`<option value="${DashCharts.escapeHtml(name)}">${DashCharts.escapeHtml(name)}</option>`).join('');
|
||||
if (!state.detailHost || !hostNames.includes(state.detailHost)) {
|
||||
state.detailHost = hostNames[0] || null;
|
||||
}
|
||||
if (state.detailHost) hostSelect.value = state.detailHost;
|
||||
if (state.detailHost) await populateDetailMetrics(state.detailHost);
|
||||
}
|
||||
|
||||
renderFleet(snapshot.fleet);
|
||||
const fleet = snapshot.fleet || {};
|
||||
$('#sub').textContent = `${fleet.hosts_live || 0}/${fleet.hosts_total || 0} hosts answering · ` +
|
||||
`${fleet.cpu_cores || 0} cores · ${fleet.gpu_count || 0} GPUs · ` +
|
||||
`window ${duration(state.range)}`;
|
||||
$('#foot-note').textContent = `rates over Δ${snapshot.window_seconds}s · ` +
|
||||
`poll ${Math.round(state.pollMs / 1000)}s`;
|
||||
await refreshCharts(snapshot);
|
||||
// the Δ comes from the payloads, never from the interval the fleet happens to answer on
|
||||
$('#foot-note').textContent = `rates over ${windowLabel(
|
||||
(snapshot.hosts || []).map((host) => host.window_seconds),
|
||||
`Δ${snapshot.window_seconds}s`, // nothing measured yet — the configured interval
|
||||
)} · poll ${Math.round(state.pollMs / 1000)}s`;
|
||||
await Promise.all(chartJobs);
|
||||
}
|
||||
|
||||
async function refreshEvents() {
|
||||
try {
|
||||
const data = await fetchJSON('/api/events?limit=25&source=host');
|
||||
// `system`, not `host`: reachability/reboot events are logged under the *dashboard* name
|
||||
// (`poller.py` writes `Event(ts, self.dashboard, …)`; samples use `host`, events do not).
|
||||
// Querying `source=host` matched nothing and left this feed forever empty.
|
||||
const data = await fetchJSON('/api/events?limit=25&source=system');
|
||||
const list = $('#events');
|
||||
if (!(data.events || []).length) {
|
||||
list.innerHTML = '<li class="muted" style="grid-template-columns:1fr">No host transitions ' +
|
||||
@@ -471,7 +477,6 @@ async function refreshEvents() {
|
||||
}
|
||||
|
||||
document.addEventListener('DOMContentLoaded', async () => {
|
||||
makeCharts();
|
||||
let pollMs = 10000;
|
||||
try {
|
||||
const config = await fetchJSON('/api/config');
|
||||
@@ -482,22 +487,12 @@ document.addEventListener('DOMContentLoaded', async () => {
|
||||
} catch (err) { /* config is nice-to-have */ }
|
||||
state.pollMs = pollMs;
|
||||
|
||||
// the window chips re-fetch every panel's history: the range is per page, not per chart
|
||||
Dash.rangeControl($('#range'), state.range, (seconds) => {
|
||||
state.range = seconds;
|
||||
setChartSpan(seconds);
|
||||
refresh();
|
||||
});
|
||||
|
||||
$('#detail-host').addEventListener('change', async (event) => {
|
||||
state.detailHost = event.target.value;
|
||||
await populateDetailMetrics(state.detailHost);
|
||||
await refreshDetailChart();
|
||||
});
|
||||
$('#detail-metric').addEventListener('change', async (event) => {
|
||||
state.detailMetric = event.target.value;
|
||||
await refreshDetailChart();
|
||||
});
|
||||
|
||||
state.poller = poll(async () => { await refresh(); await refreshEvents(); }, state.pollMs);
|
||||
Dash.bindPause(state.poller);
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user