services/dashboards
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Read the real UPSes: upsd lives on 3051, not 3493

Browse Source 
Nothing was listening on 3493, so the dashboard said "connection refused"
and fell back to the simulator while upsd was healthy the whole time. It
binds a nonstandard port: `ss -ltnH` + a NUT handshake found it, and
`upsc -L localhost:3051` lists both batteries. No host config had to
change — dev now forwards 3494 -> 3051 and `/ups` shows real numbers,
including an honest "battery is 5.3 years old" on the APC.

Docs now say what to check before blaming the service (`ss -ltnH`,
`printf 'LIST UPS\n' | nc 127.0.0.1 PORT`, `upsc -L host:port`), and the
remaining UPS gap is rewritten: reads work, instant commands still need a
scoped upsd.users entry — which needs a *read* of that file, because
`tee` writing it blind would drop the users upsmon authenticates as.

Real-hardware quirks recorded while verifying: battery.mfr.date is the
date that means something (battery.date is often an unset placeholder),
CyberPower reports `battery.mfr.date: CPS`, and neither driver reports a
battery temperature — all three already render as unknown rather than a
invented number.

LLM-Generated: true
This commit is contained in:
ducoterra
2026-10-16 19:00:44 -04:00
parent b389bb4a94
commit 1e454a8f81
4 changed files with 32 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.dev.example
+6
View File
@@ -12,6 +12,12 @@
export DEV_TUNNELS="9121:driveripper-ops:9120 9122:deskwork-ops:9120 9123:gateway-ops:9120"
export DEV_FORWARDS="3494:driveripper-ops:3493"
# Do not assume upsd is on 3493 — nonstandard ports are common and a refused connection then
# reads as "upsd is broken" while the service is fine. Ask the host what actually speaks NUT:
# ss -ltn # what is listening
# printf 'LIST UPS\n' | nc 127.0.0.1 PORT # the one that answers "BEGIN LIST UPS"
# upsc -L localhost:PORT # names + descriptions of every UPS
# Point the dashboard at those tunnels.
export HOST_TARGETS="driveripper=http://127.0.0.1:9121 deskwork=http://127.0.0.1:9122 gateway=http://127.0.0.1:9123"
export GUFO_BASE_URL=http://127.0.0.1:8731
.env.example
+7 -3
View File
@@ -37,9 +37,13 @@ RATE_WINDOW_SECONDS=10
# --------------------------------------------------------------------------- #
# UPS dashboard: the NUT data server (upsd)
# upsd must be reachable from this process. Keep it off the public interface: bind it to
# loopback (`LISTEN 127.0.0.1` in upsd.conf) and reach it with an ssh forward or a tailnet
# address, or bind a LAN address only if a firewall rule limits it to this host.
# upsd must be reachable from this process. Confirm the port before assuming 3493 — a wrong
# one looks exactly like a dead service:
# ss -ltnH # what the NUT host is listening on
# printf 'LIST UPS\n' | nc 127.0.0.1 PORT # NUT answers: BEGIN LIST UPS
# Keep it off the public interface: bind it to loopback (`LISTEN 127.0.0.1` in upsd.conf) and
# reach it with an ssh forward or a tailnet address, or bind a LAN address only if a firewall
# rule limits it to this host.
#
# Reads need no user in stock NUT. For the battery-test buttons add one, scoped as narrowly
# as the buttons need — `instcmds` beats `actions = -`, which grants load.off and shutdown.*
AGENTS.md
+18 -12
View File
@@ -123,12 +123,16 @@ That scheme is the history API's key — treat it as a public interface.
- Agents run from `~/lab-agent/host_agent.py` on each host, forwarded to `127.0.0.1:9121/9122/9123`
by `scripts/dev-tunnels.sh` (`DEV_TUNNELS`). Nothing listens on the hosts in dev.
- Engines are reached directly: gufo `deskwork.reeselink.com:8731`, vLLM `driveripper.reeselink.com:8000`.
- NUT 2.8.5 on driveripper: **configs live in `/etc/ups/`, not `/etc/nut/`**. `nut-server`
is active and `nut-driver@apc` + `nut-driver@cyberpower` are both running, so `ups.conf`
already exports two driver instances; a CyberPower PR1500LCDRT2U is attached on USB
(`0764:0601`). **`upsd` does not bind 3493**, so reads fail with *Connection refused* and
the dashboard falls back to the labelled simulator. A restart of `nut-server` does not fix
it. See *Known gaps* — this needs root logs and a config read.
- NUT 2.8.5 on driveripper: **configs live in `/etc/ups/`, not `/etc/nut/`**, and **upsd is
on port 3051, not 3493**. Nothing listens on 3493 at all, so a default client reports
*Connection refused* and it looks like a dead service — `ss -ltnH` plus `printf 'LIST UPS\n'
| nc 127.0.0.1 3051` (answers `BEGIN LIST UPS`) is how that was pinned down. Two UPSes are
exported over USB and read live through `DEV_FORWARDS="3494:driveripper-ops:3051"`:
`apc` = APC Back-UPS RS 1500MS, `cyberpower` = CyberPower PR1500LCDRT2U.
Real-hardware quirks the parsers already handle: `battery.mfr.date` is the useful date
(`battery.date` is often an unset placeholder like `2001-09-25`), CyberPower reports
`battery.mfr.date: CPS` (not a date → `unknown`, never a fake age), neither driver reports
a battery temperature, and `ups.test.result: No test initiated` is the factory state.
- Host access is via ssh keys as `ops`. Passwordless sudo exists but is command-scoped; the
set we rely on is documented in `deploy/nut-sudoers` (additions are listed there for the
work that is still blocked).
@@ -186,12 +190,14 @@ unconfigured. Escape anything upstream-provided through `DashCharts.escapeHtml`.
## Known gaps / next work
1. **Real UPS data (blocking issue, host-side).** `upsd` on driveripper runs but binds
nothing. Needed: root journal for `nut-server` (`journalctl -u nut-server`), a read of
`/etc/ups/{upsd.conf,upsd.users,ups.conf}`, then `LISTEN 127.0.0.1` (plus a `dashboards`
user with scoped `instcmds`). Then: `DEV_FORWARDS="3494:driveripper-ops:3493"`,
`NUT_HOST=127.0.0.1 NUT_PORT=3494`, unset `NUT_SIMULATE`, and re-verify `/ups` against a
real battery. Until then every UPS number is modelled, and the page says so.
1. **UPS instant commands are read-only in practice.** Reads work against real upsd (see
above), but every button is gated off with "NUT_USER / NUT_PASSWORD must name a user with
`actions` in upsd.users" because no such user exists yet. Creating one safely needs a
*read* of `/etc/ups/upsd.users` first — `sudo tee` can write the file but overwriting it
blind would delete the users upsmon already authenticates as, which can take the box down
at the wrong moment. `deploy/nut-sudoers` lists the read rules; once they are in, add
`[dashboards]` with `instcmds` limited to the battery tests, put the credentials in
`.env.dev`, and the buttons come alive. Never grant `actions = -`.
2. **GPU coverage is AMD only.** `host_agent.gpus()` reads amdgpu DRM sysfs
(`gpu_busy_percent`, `mem_info_vram_*`, hwmon temps, clocks). That covers the current
fleet (R9700 + Strix Halo iGPU; gateway has none). An NVIDIA host would report no GPU at
README.md
+1 -1
View File
@@ -87,7 +87,7 @@ documentation and is never loaded, and `scripts/dev-server.sh` exports `.env.dev
| `HOST_INTERVAL_SECONDS` | poll interval for hosts (default 10) |
| `GUFO_BASE_URL` / `VLLM_BASE_URL` | LLM engines; unset means *not deployed here*, and its half of the page disappears |
| `RATE_WINDOW_SECONDS` | the Δ window behind every measured token rate (default 10) |
| `NUT_HOST` / `NUT_PORT` | upsd address. Reads work unauthenticated in stock NUT |
| `NUT_HOST` / `NUT_PORT` | upsd address — **whatever port it really listens on**, not always 3493. Reads are unauthenticated in stock NUT |
| `NUT_USER` / `NUT_PASSWORD` | only needed for instant commands (battery tests) |
| `NUT_ALLOW_ACTIONS` | `1` enables the one write endpoint in the app |
| `NUT_SIMULATE` | `1` replaces upsd with a **labelled** simulator (development only) |