phase: 14_store_performance
**Phase 14 `store_performance` — final verification pass (all 4 tasks already in `complete/`)** - Re-verified every task's work in the tree: read caches (`series_names`/`stats`), capped compound `pick_res_many`, `history_many` + `api._history_batch`, windowed `maintain()` + per-resolution prune + `PRAGMA optimize`/`automatic_analyze=0`/WAL pragmas, `store_perf.BUDGETS` (14 ceilings), probe, live watch. - **Completion criteria:** baseline+after side by side in `data/logs/NOTES-store-perf.md` ✅ · budget tests green (`uv run pytest -m store_perf --store-perf` → **4 passed**, 14/14 inside) ✅ · 25-min dev-server watch re-run **independently** (74 passes, `scripts/watch-store-perf.py`): **0 `store-error`** (total still 190, newest 2026‑10‑18 06:33:30), WAL 7.77 MB ≤ 67.1 MB, medians 1.7–3.2 ms, p95 720/613 ms = the documented cold-cache window; a 3rd 27-pass watch on a freshly restarted server reproduced it (3.1/708, 1.8/572) ✅ · `maintain()` 5,368 ms → 2.7–6.8 ms and WAL bounded across two cycles (test + notes) ✅ · no behaviour change (all four history routes pinned against `query()`) ✅ - **Full gates:** `uv run pytest` → **1267 passed, 15 skipped**; `bash .agents/validate.sh` → **exit 0**, coverage **98.75%** (floor 94), `ruff check src tests agent` clean, `pyright` 0 errors, `node --check` ok, repo hygiene ok; `uv run pytest -m live --live` → **11 passed**; dev server up, 9/9 sources `ok`, pages 200 in 1.5–3.3 ms. - **Docs fixed:** README/AGENTS still claimed a flat "20 s per pass" while `host_agent.py` (in this same uncommitted change) scales the budget to 5 s × disk count — corrected in both plus the two illustrative "20 s of wedged disk" sentences; broken bold (`` `driveripper** ``) repaired; stale gap-4 title + "tuning is its own phase" wording updated to say phase 14 ran; README's absolute "nothing … is answered by a whole-table scan" qualified to *per call* with the cold-window cost named; "third opt-in marker" → second; conftest `--store-perf` help named the real marker; **`data/logs/watch-store-perf.py` moved to `scripts/watch-store-perf.py`** (both docs told a clone to run a gitignored path) with root-anchored paths, `DASH_WATCH_OUT`, and dead code removed. - **New finding recorded:** gap 9's "~10 s process-wide stall is still gap 4's / `14_store_performance`'s problem" is now closed — coincident >8 s `llm` + >12 s `ups` raw-stream gaps went 393 (00:00–12:00) and 198 (12:00–21:36) to **0** across 110+ min of the phase-14 store; written into AGENTS.md gaps 4 and 9 and the notes table. - **Deviations:** did *not* pull either deferred cost knob (`STATS_CACHE_TTL_SECONDS`, warming caches from the maintenance tick) — AGENTS.md records that decision and its reasoning; `uv pip list --outdated` still reports certifi/fastapi/starlette/websockets (pre-existing advisory, no dependency touched this phase, gate treats it as advisory). Overwrote nothing: task 04's watch preserved as `store-perf-watch.task04.*`, my two runs as `.verify25.*` / `.current-code.*`. - **Next pending phase:** `15_agent_mtls`, then `16_agent_smart_pass_budget` (owns the scaled-budget tests + its doc criterion, now partly pre-satisfied by this pass), `17_installer_device_filter`.
This commit is contained in:
@@ -0,0 +1,66 @@
|
||||
# Phase 14 — The store: fast reads, short writes, no "database is locked"
|
||||
|
||||
**Source:** `TODO.md` L7 — "Pages take forever to load and seem to lock up the database. Fix that."
|
||||
**Story:** n/a
|
||||
**Context:** `src/dashboards/store.py` (the `sample`/`event`/`meta` schema, `pick_res`, `history`, `series_names`, `stats`, `maintain`, the single `self._lock`, `busy_timeout=5000`), `src/dashboards/api.py` (the N-hosts × M-metrics history loops, `/api/metrics`, the polled `/api/health`), `src/dashboards/poller.py` (`add_many` on every successful poll — the write side that loses the `busy_timeout` race), `tests/test_store.py`, `tests/test_api.py`, `data/logs/` (the ~610 MB dev DB per AGENTS.md known gap 4 — the `store-error: OperationalError: database is locked` line)
|
||||
|
||||
## Objective
|
||||
|
||||
The page load time and the `database is locked` lines are one disease: the hot read
|
||||
paths of a 610 MB / 140 MB-WAL database are **full-table operations on every request**
|
||||
(`series_names()` scans for `DISTINCT metric`, the browser-polled `/api/health` runs
|
||||
`stats()` which counts the whole `sample` table, `pick_res()` pays three COUNT
|
||||
round-trips per series), while `maintain()`'s rollups are long whole-table write
|
||||
transactions that hold the WAL write lock long enough for a poller's `add_many` to
|
||||
outlive `busy_timeout=5000` and surface as the reported `store-error` (AGENTS.md
|
||||
known gap 4). This phase makes every hot path bounded (indexes, one grouped query,
|
||||
short-TTL caches) and every write short (windowed rollups, WAL checkpoint tuning),
|
||||
with timing budgets pinned as tests so a regression is a red test, not a slow page.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- the foundation (complete) — `store.py`/`api.py`/`poller.py` as shipped; the rollup
|
||||
and retention contract in AGENTS.md invariant 9.
|
||||
- `11_admin_login` (todo) — **shared files, run-order note**: 11 rewrites `api.py`
|
||||
(the session gate) and adds `get_meta`/`set_meta`/`delete_meta` to `store.py`. Apply
|
||||
this phase's `store.py`/`api.py` changes on top of the file state at the time it
|
||||
runs; never restore a line the other phase owns. The two phases touch disjoint
|
||||
*regions* (11: the meta helpers + the auth routes; 14: the query paths + the
|
||||
maintain rollups) — the coupling is the file, not the logic.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_baseline_store_costs.md` — a production-shaped seeded store + the timing/EXPLAIN baseline for every hot path, recorded in `data/logs/`.
|
||||
2. `02_no_full_scan_on_hot_paths.md` — `series_names`/`stats` off the per-request full scan (bounded caches), `pick_res` to one grouped query, the N×M history endpoints to one grouped round trip.
|
||||
3. `03_windowed_rollups_and_wal_tuning.md` — `maintain()` rolls up only the window since `last_maintain`; WAL checkpoint + journal-size tuning so the 140 MB WAL stops growing between checkpoints.
|
||||
4. `04_budget_gates_and_live_proof.md` — the timing budgets pinned as tests, the live dev-server proof (page load measured, zero `store-error` over a watch window), docs updated.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit/integration: `tests/test_store.py` — the budget tests (a seeded
|
||||
production-shaped store: the task-01 recipe), the cache-invalidation behaviour, the
|
||||
windowed-rollup correctness (a rollup run twice rolls the new window only, and the
|
||||
1m/1h values match a whole-table rollup on the same data — the windowing must not
|
||||
change a single averaged value).
|
||||
- Integration: `tests/test_api.py` — the N×M history endpoints' response shape is
|
||||
byte-identical to today (only the cost changes), under the new budget.
|
||||
- Live: the dev-server watch per task 04 (AGENTS.md invariant 10 — the dev server stays
|
||||
up while work happens; invariant 9 — the DB stays bounded, on a volume in
|
||||
deployment).
|
||||
- Coverage: **>90%** on new/modified code; the 94% gate must hold.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] the task-01 baseline and the task-04 "after" numbers are both in
|
||||
`data/logs/NOTES-store-perf.md`, side by side
|
||||
- [ ] the budget tests are green: the full `/system` history request, `/api/metrics`,
|
||||
`/api/health` and a 90d `history()` call all inside their pinned budgets on the
|
||||
seeded store
|
||||
- [ ] a 25-minute dev-server watch (the task-04 recipe) shows **zero** `store-error`
|
||||
events and page loads inside the budget — the known-gap-4 failure mode is gone,
|
||||
not hidden
|
||||
- [ ] `maintain()`'s write transaction is measurably shorter (the task-03 note) and
|
||||
the WAL stays bounded across two maintain cycles
|
||||
- [ ] `uv run pytest` green, `bash .agents/validate.sh` green
|
||||
- [ ] no behavior change in completed phases: every history payload, rollup value and
|
||||
event answer is identical to today's shape (the `tests/test_api.py` pins)
|
||||
@@ -0,0 +1,68 @@
|
||||
# Task 01 — What the hot paths actually cost
|
||||
|
||||
**Phase:** `14_store_performance` · **Source:** `TODO.md:7` — "Pages take forever to load and seem to lock up the database. Fix that."
|
||||
|
||||
## Objective
|
||||
|
||||
A before/after exists only if the "before" is measured, not remembered. Seed a
|
||||
production-shaped store, time every hot path of `store.py`/`api.py` against it,
|
||||
`EXPLAIN QUERY PLAN` the queries that will change, and record the baseline in
|
||||
`data/logs/NOTES-store-perf.md` — the numbers task 04 compares against.
|
||||
|
||||
## Work
|
||||
|
||||
1. `tests/test_store.py` — the seeded-store recipe (a fixture the budget tests of this
|
||||
phase all reuse): an on-disk SQLite file (WAL, like production — the in-memory DB
|
||||
hides the lock behaviour) seeded to the dev DB's *shape and retention semantics*
|
||||
(`raw_keep_hours=48`, `minute_keep_days=30`, `hour_keep_days=400` — the
|
||||
`SampleStore` defaults, i.e. what a 90 d chart actually reads: 1m/1h, never raw
|
||||
beyond 48 h):
|
||||
- ~3 hosts × ~30 series per host, 10 s cadence for the 48 h raw window;
|
||||
- their 1m rollups for the 30 d window and 1h rollups for the 400 d window
|
||||
(generated by calling `maintain()` — a few hundred MB, the dev DB's order of
|
||||
magnitude, a temp file, never the dev DB itself: tests stay hermetic per the
|
||||
AGENTS.md testing policy);
|
||||
- ~2 000 event rows and the `meta` rows the app writes (`last_maintain` included).
|
||||
The test asserts the row counts, not the wall time of seeding.
|
||||
- ASSUMPTION: "production-shaped" means the dev DB's *shape and magnitude* (~610 MB,
|
||||
per AGENTS.md known gap 4), reproduced in a temp file — the dev DB itself is
|
||||
never opened by a test (tests stay hermetic, the AGENTS.md testing policy), and
|
||||
a temp DB of the same shape is the honest substitute.
|
||||
2. Measure (a script or the test's `--store-perf` marker; record in
|
||||
`data/logs/NOTES-store-perf.md` with the machine's specs):
|
||||
- `store.series_names()` (what `/api/metrics` runs per call) — wall time +
|
||||
`EXPLAIN QUERY PLAN` (expected: a full scan of `sample`);
|
||||
- `store.stats()` (what the browser-polled `/api/health` runs) — wall time +
|
||||
EXPLAIN (expected: a full COUNT scan of `sample` + `event`);
|
||||
- `store.pick_res()` × 3 + `store.history()` for one 90 d window (what one chart
|
||||
series costs) — wall time + EXPLAIN;
|
||||
- the full `/api/system/history` request shape (3 hosts × 9 series — the
|
||||
`tests/test_system_contract.py` table, 27 `_history` calls) through the test app
|
||||
— total wall time;
|
||||
- a `maintain()` run on the seeded store — the write transaction's wall time
|
||||
(the lock-holding window; note the `to_hour` step already scans the *entire*
|
||||
30 d 1m window on every run — the suspected long write) and the WAL file's
|
||||
size before/after;
|
||||
3. Record the baseline table in `data/logs/NOTES-store-perf.md`: path, wall time,
|
||||
EXPLAIN shape, and the wall time of the poller's `add_many` (a 30-sample insert)
|
||||
issued **concurrently** with a `stats()`/`series_names()` call — the race known gap
|
||||
4 describes, made visible as a number (how often does the insert wait, and does it
|
||||
ever outlive `busy_timeout=5000`? run the race loop ~100× and count the waits).
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: the seeded-store fixture in `tests/test_store.py` (asserts row counts and the
|
||||
rollup values match a second `maintain()` pass — determinism check).
|
||||
- No behaviour changes in this task — measurement only.
|
||||
- Coverage: **>90%** on the fixture code; the 94% gate must hold.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `data/logs/NOTES-store-perf.md` exists: the full baseline table (wall times,
|
||||
EXPLAIN shapes, the concurrent-insert wait count), the machine's specs, and the
|
||||
seeded DB's row counts
|
||||
- [ ] the baseline confirms (or refutes — record the refutation) the three suspect
|
||||
mechanisms: `series_names` full scan, `stats` full scan, `maintain`'s long write
|
||||
window
|
||||
- [ ] the fixture is green and deterministic (two runs, same rollup values)
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
@@ -0,0 +1,70 @@
|
||||
# Task 02 — No full table scan on a per-request path
|
||||
|
||||
**Phase:** `14_store_performance` · **Source:** `TODO.md:7` — "Pages take forever to load and seem to lock up the database. Fix that."
|
||||
|
||||
## Objective
|
||||
|
||||
Three per-request paths pay a whole-table price on a 610 MB database:
|
||||
`series_names()` (a `SELECT DISTINCT metric` scan, on every `/api/metrics` call),
|
||||
`stats()` (a `COUNT(*) GROUP BY res` over all of `sample` plus `event`, on every
|
||||
browser-polled `/api/health`), and the N-hosts × M-metrics history endpoints
|
||||
(`pick_res()` is three COUNT round-trips per series, so one `/system` chart request
|
||||
is ~108 serialized round-trips through the single `self._lock`). Make every hot path
|
||||
bounded: short-TTL caches for the two scanned answers, one grouped query where three
|
||||
stand, and one grouped round trip where N×M stand. The response shapes are byte-
|
||||
identical — only the cost moves.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/store.py` — `series_names()`: an in-memory cache of the distinct
|
||||
metric set (per `source`/`host` argument tuple, a 60 s TTL, invalidated on
|
||||
`add`/`add_many`/`delete_series` via a dirty flag — a cache that outlives a write
|
||||
answers "why is this chart empty" with yesterday's truth). `stats()`: cache the
|
||||
`sample`/`event` counts the same way (30 s TTL) — the `/api/health` poll (the
|
||||
browser's `app.js` masthead loop) must never run a full COUNT scan; `last_maintain`
|
||||
stays read through `get_meta()` as today (the meta table is tiny; do not cache
|
||||
what a three-row table costs nothing to read).
|
||||
2. `src/dashboards/store.py` — `pick_res()`: replace the three per-`res` COUNT queries
|
||||
with **one** `SELECT res, COUNT(*) … WHERE … ts >= ? GROUP BY res` (the three
|
||||
resolutions in a single index-range pass), same decision logic, same return value.
|
||||
3. `src/dashboards/api.py` — the four history endpoints' loops (system/disks/llm/ups —
|
||||
the `[host, metric]` nesting in each) call `store.history()` per pair, which is
|
||||
`pick_res()` (now 1 query) + the thinning query. Add the batched door in
|
||||
`store.py`: `pick_res_many(hosts, metrics, span, max_points)` and
|
||||
`history_many(hosts, metrics, span, max_points)` — one grouped COUNT over the whole
|
||||
`host IN (…) AND metric IN (…) AND res IN (…) AND ts >= ?` range, and one
|
||||
per-resolution thinning query (the per-series `LIMIT ?` tail-thinning kept by a
|
||||
window function or a per-series subquery — the shape of each series' answer must
|
||||
stay identical: ascending `(ts, value)`, `tail=True` semantics preserved). The
|
||||
endpoints' response dicts are unchanged (the same series, the same order); the
|
||||
`tests/test_api.py` pins are the contract.
|
||||
- ASSUMPTION: one grouped round trip (plus one thinning query per resolution
|
||||
present) is the target for the N×M endpoints — the alternative (N×M separate
|
||||
queries, each now cheaper) leaves ~100 serialized round-trips on a 3-host request,
|
||||
which on a 610 MB file is still the page-load tail; the grouped shape is the fix
|
||||
the TODO asks for. The per-series thinning keeps `tail=True` (the "the chart must
|
||||
have 'now'" docstring) exactly as today.
|
||||
4. `tests/test_store.py` — the pins: `series_names()`/`stats()` answer from cache
|
||||
within the TTL (a write after the read invalidates `series_names`; `stats` expires
|
||||
on the clock); `pick_res_many`/`history_many` equal the per-series answers on the
|
||||
seeded store (the byte-identical-shape test); a cache hit makes a second
|
||||
`series_names()` call measurably not scan (assert via a query-counter hook or the
|
||||
wall-time budget of task 04).
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit/integration: per the work list, on the task-01 seeded store.
|
||||
- Regression: every pre-existing `tests/test_store.py` / `tests/test_api.py` test
|
||||
passes unmodified in intent — the response shapes are the contract.
|
||||
- Coverage: **>90%** on this task's new/modified code; the 94% gate must hold.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] no per-request path in `api.py` triggers a full-table scan of `sample` or
|
||||
`event` (the task-04 budgets are the proof; `EXPLAIN QUERY PLAN` of the grouped
|
||||
queries shows index-range access, recorded in the notes file)
|
||||
- [ ] the N×M endpoints' answers are byte-identical to the per-series path on the
|
||||
seeded store (the equality test)
|
||||
- [ ] a fresh `series_names()`/`stats()` call costs the full scan (documented — the
|
||||
first poll after a process start pays it once), a repeated call costs the cache
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
@@ -0,0 +1,78 @@
|
||||
# Task 03 — A rollup that holds the write lock for seconds, not minutes
|
||||
|
||||
**Phase:** `14_store_performance` · **Source:** `TODO.md:7` — "Pages take forever to load and seem to lock up the database. Fix that."
|
||||
|
||||
## Objective
|
||||
|
||||
`maintain()` is "safe to call every minute" and is idempotent — but on a real-sized
|
||||
store the `to_hour` step's `SELECT … GROUP BY` scans the **entire 30-day 1m window**
|
||||
on every run (and `to_minute` the entire 48 h raw window), and that GROUP BY is
|
||||
inside the one `with self._lock, self._db:` transaction: a multi-second write
|
||||
transaction that holds the WAL write lock long enough for a poller's `add_many` to
|
||||
outlive `busy_timeout=5000` and surface as the `store-error: OperationalError:
|
||||
database is locked` line (AGENTS.md known gap 4). Window each step to *what became
|
||||
due since the last run* — the idempotent `INSERT OR REPLACE` makes a slightly wider
|
||||
re-scan safe, so the window is a cost bound, not a correctness device — and tune the
|
||||
WAL so its 140 MB growth stops being a per-run surprise.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/dashboards/store.py` — `maintain()`, both rollup steps:
|
||||
- read `last_maintain` through `get_meta()` **before** the transaction (the
|
||||
existing `stats()` note applies: the meta read is outside the non-reentrant
|
||||
lock);
|
||||
- `to_minute`: the raw scan bound moves from `ts >= now - raw_keep_seconds` to
|
||||
`ts >= max(now - raw_keep_seconds, last_maintain - MINUTE - 60)` — steady state
|
||||
(a run per minute) scans ~one minute of raw rows; a missed interval scans the
|
||||
catch-up window; a first run on a fresh DB (no `last_maintain` row) scans the
|
||||
full retention window exactly as today;
|
||||
- `to_hour`: the same pattern against the 1m rows (`ts >= max(now -
|
||||
minute_keep_seconds, last_maintain - HOUR - 60)`). The `ts < now - {MINUTE|HOUR}
|
||||
- 60` incompleteness guard stays — a bucket is still only rolled once its whole
|
||||
window is in the past.
|
||||
- the `DELETE … pruned` step is unchanged (the retention window is the correct
|
||||
bound for a prune; it is `DELETE`s on indexed ranges, not a GROUP BY).
|
||||
2. `src/dashboards/store.py` — the WAL tuning in `__init__` (after the existing
|
||||
pragmas): `PRAGMA wal_autocheckpoint = 1000` (the default 1000 pages ≈ 4 MB
|
||||
auto-checkpoint is already the default — the 140 MB WAL means checkpoints lose to
|
||||
the write rate; the real lever is the journal-size cap) and
|
||||
`PRAGMA journal_size_limit = 67108864` (64 MB — the WAL stops growing past it;
|
||||
checkpoints then truncate it). Keep `journal_mode=WAL` and
|
||||
`synchronous=NORMAL` (the "a lost sample is fine, corruption is not" comment
|
||||
stands); keep `busy_timeout=5000` — the fix is a shorter write, not a longer wait.
|
||||
- ASSUMPTION: 64 MB is the journal-size cap — the dev DB's 140 MB WAL is the
|
||||
measured "too big" (AGENTS.md known gap 4), 64 MB is the nearest power of two
|
||||
below it, and the cap is a pragma, not a migration; if the live watch (task 04)
|
||||
shows checkpoint truncation starving writers, the cap moves (named constant in
|
||||
`__init__`, one-line change).
|
||||
3. `tests/test_store.py` — the pins:
|
||||
- **windowing correctness**: seed a store, run `maintain()` (full window, no
|
||||
`last_maintain` yet), advance time and add new 1m buckets, run `maintain()`
|
||||
again — the second run's `to_hour`/`to_minute` scan (assert via the EXPLAIN/row
|
||||
count hook, or a `sqlite3` trace) touches only the since-`last_maintain` window,
|
||||
**and** the rolled 1m/1h values are byte-identical to a reference whole-window
|
||||
rollup on the same data (the windowing must not change a single averaged value —
|
||||
`INSERT OR REPLACE` idempotency is what makes this safe, prove it);
|
||||
- **catch-up**: no `maintain` for 10 minutes, then one run — the 10 minutes of
|
||||
buckets are all rolled (the wider window, not a gap);
|
||||
- **the lock race**: the task-01 concurrent loop (a poller-style `add_many` during
|
||||
a `maintain()` run) now completes with **zero** `OperationalError`s over the
|
||||
100× loop (the baseline's wait count is the before, this is the after).
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit: per the work list, on the task-01 seeded store.
|
||||
- Regression: every pre-existing `tests/test_store.py` test passes unmodified in
|
||||
intent (the idempotency/retention tests are the load-bearing ones).
|
||||
- Coverage: **>90%** on this task's new/modified code; the 94% gate must hold.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] the second `maintain()` run's write transaction is measured shorter than the
|
||||
task-01 baseline (both numbers in `data/logs/NOTES-store-perf.md`), and the
|
||||
rollup values are unchanged (the equality test)
|
||||
- [ ] the 100× concurrent `add_many`-during-`maintain` loop: zero `OperationalError`
|
||||
(baseline wait count vs after, in the notes file)
|
||||
- [ ] two `maintain()` cycles on the seeded store: the WAL file size stays bounded by
|
||||
the 64 MB cap (before/after sizes recorded)
|
||||
- [ ] `bash .agents/validate.sh` green
|
||||
@@ -0,0 +1,78 @@
|
||||
# Task 04 — The budgets, and the dev server that proves them
|
||||
|
||||
**Phase:** `14_store_performance` · **Source:** `TODO.md:7` — "Pages take forever to load and seem to lock up the database. Fix that."
|
||||
|
||||
## Objective
|
||||
|
||||
The "after" numbers, pinned as tests so a regression is red instead of slow: the
|
||||
task-01 baseline vs the fixed store, side by side in `data/logs/NOTES-store-perf.md`,
|
||||
plus the live dev-server proof on the real 610 MB DB — page loads inside budget, zero
|
||||
`store-error` events over a watch window (the known-gap-4 failure mode gone, not
|
||||
hidden), and the docs updated.
|
||||
|
||||
## Work
|
||||
|
||||
1. `tests/test_store.py` / `tests/test_api.py` — the budget tests (the task-01 recipe
|
||||
on the seeded store; budgets are the task-01 baseline's *target* — record the
|
||||
chosen numbers in the notes file with the reasoning):
|
||||
- the full `/api/system/history` request (3 hosts × 9 series, 1 h and 90 d):
|
||||
wall time budget (the 90 d case reads the 1m/1h rollups — the one that used to
|
||||
be the page-load tail);
|
||||
- `store.series_names()` / `store.stats()` second call (cache hit): wall time
|
||||
budget, asserted to be a fraction of the first call's (the scan is gone);
|
||||
- one 90 d `store.history()` call: wall time budget;
|
||||
- the budgets are generous (2–3× the measured "after"), so a slower CI machine does
|
||||
not flake red — the notes file records both the machine's specs and the margin.
|
||||
2. The live proof (AGENTS.md invariant 10 — the dev server stays up while work
|
||||
happens, and a fix is screenshotted, not claimed; invariant 9 — the DB is the
|
||||
volume's business in deployment, the dev DB is the test bench):
|
||||
- `./scripts/dev-server.sh restart` (its `stop` waits for the old pid to die —
|
||||
known gap 4's "two servers on one file" cause stays fixed), `status` shows the
|
||||
sources;
|
||||
- page loads: `curl -o /dev/null -s -w '%{time_total}'` on `/` and on the
|
||||
history-heavy pages' API calls (signed in, per phase 11), five runs each,
|
||||
recorded; the browser's `/llm` or `/system` page with a 90 d window selected is
|
||||
the worst case — screenshot `data/logs/store-perf-live.png` (the page loaded,
|
||||
the charts drawn, the masthead dots `ok`);
|
||||
- the watch: ≥ 25 minutes of `./scripts/dev-server.sh logs` (longer than the
|
||||
gap-4 baseline's "a few times an hour" rate) with **zero**
|
||||
`store-error: OperationalError: database is locked` lines; the `store.stats()`
|
||||
`event` count for `kind='store-error'` over the window is 0 (the `sqlite3`
|
||||
check on the dev DB — read-only);
|
||||
- the WAL file's size after the watch (the cap from task 03, in the real DB).
|
||||
- ASSUMPTION: the watch window is 25 minutes against a "few times an hour"
|
||||
baseline rate — enough to cover > 0.4 of the baseline's expected occurrences at
|
||||
the mean rate; if the baseline was at the mean, a zero-count 25-minute window
|
||||
already moves the needle, and the `sqlite3` event-table check over *the whole
|
||||
dev DB's* `store-error` rows (not just the window) shows when the last
|
||||
occurrence predates this phase.
|
||||
3. Docs: `README.md` — the *Configuration*/*Development* notes gain the two new
|
||||
`PRAGMA`s' one-line "why" (the journal cap, and that it is a cap not a target);
|
||||
`AGENTS.md` — known gap 4's entry is updated to record the fix (the windowed
|
||||
rollups, the journal cap, the budget tests) and the gap's "what remains" line
|
||||
(checkpoint starvation was never shown to be a cause — the controlled 25 s read
|
||||
transaction produced none — so the entry's honest wording stays, with the
|
||||
`database is locked` line now attributed to the rollup's write window). (The
|
||||
AGENTS.md/README update is the phase's *normal work* per the repo's definition of
|
||||
done — this instruction is for the phase's executor, not for the conversion.)
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Unit/integration: the budget tests per step 1.
|
||||
- Live: the step-2 watch, screenshots and log evidence in `data/logs/`.
|
||||
- Coverage: **>90%** on new/modified code; the 94% gate must hold.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `data/logs/NOTES-store-perf.md` has the baseline and the after table side by
|
||||
side (every task-01 path, wall time before/after, EXPLAIN shapes)
|
||||
- [ ] the budget tests are green with the recorded margins; `bash .agents/validate.sh`
|
||||
green
|
||||
- [ ] the 25-minute watch: zero `store-error` lines in the dev-server logs and zero
|
||||
new `store-error` rows in the dev DB's `event` table; `data/logs/` holds the
|
||||
page-load numbers and `store-perf-live.png`
|
||||
- [ ] the WAL's post-watch size is inside the 64 MB cap on the real DB
|
||||
- [ ] `README.md`/`AGENTS.md` carry the updated, accurate wording (gap 4: fixed, with
|
||||
its "what remains" honest)
|
||||
- [ ] no behavior change in completed phases: the `tests/test_api.py` payload pins all
|
||||
pass — the answers are the same, only faster
|
||||
Reference in New Issue
Block a user