phase: 187_sync_peak_memory_line
**Phase 187 — final verification pass: all criteria met, repo 100% green.**
Verified (all four tasks already in `complete/`)
- `app/core/memory.py`: `rss_mb` / `rss_peak_mb` / `cgroup_peak_mb` / `fmt_mb`, pure stdlib, never-raise, **100%** coverage, 24 branch tests.
- Lifecycle lines append-only: `sync: started … rss_mb=`, all three terminals `… rss_mb= rss_peak_mb=` (failed = sanitized one-liner beside the surviving `logger.exception`), `import: summary … rss_mb=`; prefixes byte-identical.
- `GET /api/sync/status`: additive `rss_mb`/`rss_peak_mb` (`int|null`), keyed off the ONE `_sample_rss` (run start, phase transitions, forced terminals, 5 s-throttled hook), `peak >= current` invariant, anonymous 403.
- `sources.js` `fmtSyncPeakClause`: ` · peak <N> MB` trailing the counts line / "Sync cancelled", nothing when null, no new control/live region.
Gates (exact commands)
- `uv run pytest --cov=app --cov-report=term-missing` → **4570 passed**, TOTAL **99%**, `app/api/sync.py` **100%**, `app/core/memory.py` **100%**
- `uv run ruff check .` → clean · `uv run pyright` → **0 errors**
- `uv run pytest tests/e2e/test_sync_peak_memory_visible.py -v --no-cov` → **5 passed in isolation** (ran twice)
- All 27 touched E2E families re-run **in isolation**, all green (progress_box 8, cancel 3, status_freshness 4, summary_visibility 3, sync_button 3, session_reset 4, git_freeze 2, git_log_stream_dates 5, PDF/image suites, …)
Completion criteria: every checkbox ✅ except the commit + phase-dir move, which belong to the harness.
Documentation
- README reviewed in full (pages, commands, config tables cross-checked against `app/config.py` — 85 env rows, all defaults match; phase-187 ops section, greps, process-vs-cgroup caveat): consistent, no changes needed.
- `AGENTS.md`: **fixed one pre-existing inconsistency** — rule 6 claimed the no-CDN test covers "the index page"; it actually parametrizes 10 pages (all but `/doc-edit.html`).
Notable
- Two failures early in the pass were self-inflicted by running the E2E module concurrently with the full suite on the shared Postgres (TRUNCATE deadlock + one order-dependent `test_overview` flake); both vanish under the documented serial isolation — no code defect, no change made.
- **PLAN §9 still needs the owner's reconciliation** (phase 187 adds the sync memory fields + two status keys; PLAN is never agent-edited — AGENTS rule 10 carries the spec). `TODO.md` holds an unrelated owner note, left untouched.
Next pending phase: `188_chunks_hnsw_index`.
This commit is contained in:
@@ -0,0 +1,54 @@
|
||||
# Phase 187 — Make the sync's memory a logged, visible number
|
||||
|
||||
**Source:** owner incident 2026-10-15 — reconstructing "how much RAM did the sync actually use?" meant reading `systemd` unit-teardown lines from three boots (`brain-of-code-app.service: … 3.3G memory peak`), and boot −1 (the run that killed the box) had NO teardown record at all because the VM never shut down cleanly.
|
||||
**Story:** n/a (incident-derived, owner directive 2026-10-15)
|
||||
**Depends on:** none functionally; lands after 183–186 so the numbers it reports describe the fixed sync.
|
||||
|
||||
**Context:** `.agents/PLAN.md` §9 Observability (the log-format authority — PLAN is never edited by an agent, so the new fields are documented in `README.md` + `AGENTS.md` rule 10 and the owner reconciles PLAN); `AGENTS.md` rule 10 ("Amply log, visibly feed back"); `app/core/response_budget.py` (the house precedent this phase mirrors: a pure-stdlib `app/core/` module, one greppable logger name — `grep ' :: api '`, `BOR_LOG_LARGE_RESPONSE_BYTES`, and `scripts/api_payload_audit.py` as the offline audit); `app/api/sync.py` — the `SyncStatus` dataclass (**L276**, phase-64 "keep the final counts in the terminal" convention), `sync_status()` (**L385**, its additive-keys contract "Additive only — every pre-existing key keeps its name, type, and meaning"), the `sync: started repos=%d origin=%s git=%d local=%d` line (**L591–594**), the phase transitions (`_status.phase = "import"` **L718**, `"overview"` **L743**, `"summaries"`), `sync: done detail=%s` (**L903**), the cancel terminal's `sync: cancelled by user state=cancelled files_done=%d files_total=%d` (**L933–936**), the failed terminal (`logger.exception("sync: failed")` **L938**); `app/rag/importer.py` — `ImportSummary.format_counts` (**L513**) + `ImportSummary.log()` (**L520**, the cron-safe greppable summary line PLAN §9 names); `frontend/assets/sources.js` — the sync progress box + result line (`setSyncProgress(…)` around **L1325**, the label/counts helper at **L1080–1100** and **L1378**, the aria-live announcements), `frontend/index.html` (the progress box markup), `tests/unit/test_sync_api*.py` / `tests/integration/test_sync_api.py` (the status-shape pins), `tests/e2e/test_sync_summary_visibility.py`, `tests/e2e/test_sync_progress_box.py`, `tests/e2e/test_sync_status_freshness.py`, `tests/e2e/test_sync_cancel.py`.
|
||||
|
||||
## Objective
|
||||
Make "how much memory did that sync use?" answerable from one log line and visible on the page that started it: a small `app/core/memory.py` reading current + peak RSS, appended to every sync lifecycle line and the importer's summary line, plus an additive `rss_peak_mb` key on `GET /api/sync/status` and a `peak <N> MB` clause on the Sources page's sync result line.
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
- **D1 — `app/core/memory.py` never raises and never lies.** Three pure helpers: `rss_mb() -> int | None` (Linux: parse `VmRSS` from `/proc/self/status`; non-Linux/failure: `resource.getrusage(RUSAGE_SELF).ru_maxrss // 1024` as the best available proxy), `rss_peak_mb() -> int | None` (`ru_maxrss // 1024` — the process high-water mark), and `cgroup_peak_mb() -> int | None` (read `/sys/fs/cgroup/memory.peak` when present, else `None`). Docstring states the honest nuance the incident turned on: **inside a container the process high-water and the host's cgroup `memory.peak` are different numbers, and the cgroup number includes page cache** (the incident box's app cgroup showed 2.16 GB of `file` against 211 MB of `anon` while idle) — so the app logs the PROCESS number and the operator's dashboard may show a larger one; neither is wrong, and the log says which it is.
|
||||
- **D2 — the format is append-only, in the §9 style (task 02).** `sync: started … rss_mb=N`, and every terminal (`sync: done detail=… rss_mb=N rss_peak_mb=M`, `sync: cancelled by user … rss_mb=N rss_peak_mb=M`, and the failed funnel) gains the same two trailing fields; `ImportSummary.log()` gains a trailing ` rss_mb=N`. A `None` renders as `rss_mb=?` — a helper `fmt_mb(v: int | None) -> str` keeps that in one place. No existing field is renamed, reordered or re-typed (the phase-167/182 append-only convention), so every existing `grep` keeps working.
|
||||
- **D3 — sampled, not polled per file (task 03).** `_status` gains `rss_mb: int | None` and `rss_peak_mb: int | None`, reset at run start; sampled at the run start, at each `_status.phase` transition (`import` L718, `overview` L743, `summaries`), and in EVERY terminal (which keep the final value next to the counts — the phase-64 convention). The per-file progress hook samples at most once per 5 s via a `time.monotonic()` guard, so a 27k-file walk costs ~600 `/proc` reads, not 27k, and the operator watching the page sees the number move (PLAN §7.4's never-stale contract).
|
||||
- **D4 — the status key and the UI clause are additive and admin-only (task 03).** `sync_status()` returns two new keys (`rss_mb`, `rss_peak_mb`, `int | null`); `sources.js` appends ` · peak 512 MB` to the existing result/counts line — a clause on text that already exists, never a new control, never replacing a state string, and the aria-live announcement keeps its current wording (a screen-reader user gets the same announcement plus the clause; the peak is supplementary, so it must not become the announcement's only content). The Sources view is admin-only (PLAN B5), which is the exposure guard; no memory number appears on any anonymous surface (chat, shared chat, health).
|
||||
- **D5 — an offline reader for the log (task 02, small).** Document `grep ' :: ' | grep ' rss_peak_mb='` in `README.md`'s operations section. NO new audit script: `scripts/api_payload_audit.py` exists for bytes; a memory equivalent would be a second parser for one number. Explicitly out of scope (A3).
|
||||
- **NOT touched:** `PLAN.md` (never), the sync pipeline's sequence/errors/cancel semantics, the response-budget logger, the chat's per-turn line (this phase is about the sync), the container/quadlet memory limits (owner's own task), the retrieval path.
|
||||
|
||||
## Dependencies
|
||||
— (none; after 183–186 in the roadmap so the reported numbers describe the bounded sync)
|
||||
|
||||
## Tasks
|
||||
1. `01_memory_helper.md` — `app/core/memory.py` (the three helpers + `fmt_mb`), unit-tested with `/proc` and `resource` stubbed (Linux path, missing-file path, garbage-file path, non-Linux fallback).
|
||||
2. `02_sync_log_lines.md` — the four sync lifecycle lines + `ImportSummary.log()` gain `rss_mb=` / `rss_peak_mb=`, the `fmt_mb` single-sourcing, the log-format assertions in the integration tests, the README operations note.
|
||||
3. `03_status_key_and_ui.md` — the `SyncStatus` fields + sampling (start, phase transitions, terminals, the 5 s-throttled progress sample), the two additive `sync_status()` keys, the `sources.js` clause + aria-live behaviour, the shape pins.
|
||||
4. `04_e2e_peak_memory.md` — `tests/e2e/test_sync_peak_memory_visible.py` (the phase's dedicated Playwright suite), the full gates, the docs pass, the atomic commit.
|
||||
|
||||
## Testing & Quality
|
||||
- **Unit:** `tests/unit/test_core_memory.py` — every helper branch (real `/proc` on Linux + stubbed failure modes, the `None` rendering, the kB→MB arithmetic), `tests/unit/test_settings*` untouched.
|
||||
- **Integration:** `tests/integration/test_sync_api.py` — the lifecycle lines carry the fields (caplog assertions on the `sync: started` / `sync: done` / `sync: cancelled` / `sync: failed` shapes with `rss_mb=?` accepted on a platform without `/proc`), the status payload's two new keys with their types and the terminal-keeps-final-value convention, every pre-existing key byte-identical.
|
||||
- **E2E (AGENTS rule 4, task 04):** `tests/e2e/test_sync_peak_memory_visible.py`, run **in isolation**.
|
||||
- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
|
||||
- **Lint/types:** `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Documentation & Consistency
|
||||
- Final pass reviews `README.md` and `AGENTS.md` in their entirety against the post-phase app and fixes every inconsistency found, pre-existing ones included.
|
||||
- `AGENTS.md` rule 10: the required sync lines now carry `rss_mb=` / `rss_peak_mb=` (extend rule 10's parenthetical, do not rewrite it). `README.md`: the operations grep, what each number means, and the container-vs-cgroup caveat (page cache included in the host's number) written in one short paragraph. PLAN §9 is the owner's to reconcile — flag it in the phase report, do not edit it.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `app/core/memory.py` exists with the four documented functions, every branch unit-tested, and never raises.
|
||||
- [ ] All four sync lifecycle lines + `import: summary …` carry `rss_mb=` (and the terminals `rss_peak_mb=`), appended after the existing fields; no existing field changed.
|
||||
- [ ] `GET /api/sync/status` exposes `rss_mb` / `rss_peak_mb` (`int | null`), keeps the final value in every terminal, and every pre-existing key is unchanged.
|
||||
- [ ] The Sources page's sync result line shows the ` · peak <N> MB` clause with no new control and no regression in the existing state strings.
|
||||
- [ ] `uv run pytest` green, `app/` coverage >90%, ruff + pyright clean.
|
||||
- [ ] `uv run pytest tests/e2e/test_sync_peak_memory_visible.py -v --no-cov` green **in isolation**.
|
||||
- [ ] `README.md` and `AGENTS.md` accurate and consistent with the whole project (the PLAN §9 reconciliation flagged, not performed).
|
||||
- [ ] One atomic `--no-gpg-sign` Conventional Commit (e.g. `feat(observability): log and show the sync's RSS so memory is a number, not a forensic exercise`); the phase dir moves to `.agents/phases/complete/`.
|
||||
|
||||
## Locked decisions (owner-confirmed 2026-10-15)
|
||||
- **A1 — the app logs PROCESS RSS, not the cgroup number**, with the caveat documented (`anon` vs `file` differ by an order of magnitude and the cgroup figure includes page cache — demonstrated during the incident investigation).
|
||||
- **A2 — append-only to the existing lines and keys.** No new log line type, no renamed field, no separate metrics endpoint (A12: no aux services), no Prometheus.
|
||||
- **A3 — no memory audit script** this phase; the grep is documented instead.
|
||||
- **A4 — sampling is throttled (5 s) and never per-file**, and the UI clause is text on a line that already exists on an admin-only view.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Task 01 — `app/core/memory.py`
|
||||
|
||||
**Phase:** `187_sync_peak_memory_line` · **Source:** owner incident 2026-10-15
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
One pure-stdlib module that answers "how much RAM is this process using, and what was its high-water mark" without ever raising, mirroring `app/core/response_budget.py`'s shape (no FastAPI, no imports from `app.api`, logger only if needed).
|
||||
|
||||
## Work
|
||||
1. `app/core/memory.py`:
|
||||
- `rss_mb() -> int | None` — parse `VmRSS` from `/proc/self/status` (kB → MB, integer, floor); any `OSError`/`KeyError`/parse failure returns `None`. Non-Linux fallback: `resource.getrusage(resource.RUSAGE_SELF).ru_maxrss // 1024` (documented as a high-water proxy, not a current value).
|
||||
- `rss_peak_mb() -> int | None` — `resource.getrusage(RUSAGE_SELF).ru_maxrss // 1024`, `None` only if `resource` is unavailable (wrap in a guarded import for platforms without it).
|
||||
- `cgroup_peak_mb() -> int | None` — read `/sys/fs/cgroup/memory.peak` (bytes → MB) when it exists and parses, else `None`. Docstring: inside a container this file is usually absent (the host reads it), and the host's number counts PAGE CACHE too — the incident box's app cgroup showed ~2.16 GB `file` against ~211 MB `anon` while idle, and `systemd` reported `3.3G memory peak` for the unit while the uvicorn process was a fraction of that. This module therefore reports PROCESS numbers and this distinction is written here once, so no future reader conflates them.
|
||||
- `fmt_mb(v: int | None) -> str` — `"?"` for `None`, else `str(v)` (so log lines never print `rss_mb=None`).
|
||||
- Module docstring: the incident (one paragraph), the three helpers, the never-raise contract, and the container/cgroup caveat above.
|
||||
2. `tests/unit/test_core_memory.py` — every branch, with the filesystem and `resource` stubbed: a real read (Linux runner: `/proc/self/status` has `VmRSS`, so `rss_mb()` is a positive int), a missing file, an unparseable file, a file without a `VmRSS` line, the non-Linux `resource` fallback path (patch the `/proc` open to raise + patch `getrusage`), `rss_peak_mb()`'s arithmetic, `cgroup_peak_mb()` present/absent/garbage, `fmt_mb(None) == "?"` / `fmt_mb(0) == "0"` (0 is a value, not missing).
|
||||
3. Verify: `uv run pytest tests/unit/test_core_memory.py -v`; `uv run ruff check . && uv run pyright` clean; `uv run pytest` green.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: 100% branch coverage of the new module (well above the >90% gate).
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `app/core/memory.py` with the four functions, the never-raise contract, and the container/cgroup caveat documented.
|
||||
- [ ] `uv run pytest tests/unit/test_core_memory.py -v` green, every branch covered.
|
||||
- [ ] No import cycle (`app.core.memory` imports stdlib only — prove it in the module docstring's "pure stdlib" line and by the absence of `app.` imports).
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 02 — The lifecycle lines carry the numbers
|
||||
|
||||
**Phase:** `187_sync_peak_memory_line` · **Source:** owner incident 2026-10-15
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
Append `rss_mb=` to the sync's start line, `rss_mb=`/`rss_peak_mb=` to all three terminals, and `rss_mb=` to the importer's cron-safe summary line — append-only, so every existing grep and every existing field keeps working.
|
||||
|
||||
## Work
|
||||
1. `app/api/sync.py` — the four lifecycle log sites, using `app.core.memory.rss_mb/rss_peak_mb/fmt_mb` (imported at module level, matching the file's import style):
|
||||
- **L591–594** `sync: started repos=%d origin=%s git=%d local=%d` → append ` rss_mb=%s` (`fmt_mb(rss_mb())`).
|
||||
- **L903** `sync: done detail=%s` → append ` rss_mb=%s rss_peak_mb=%s`.
|
||||
- **L933–936** `sync: cancelled by user state=cancelled files_done=%d files_total=%d` → append the same two fields.
|
||||
- **L938** the failed funnel: `logger.exception("sync: failed")` stays as-is (the traceback), and the FAILED line gains a dedicated `logger.error("sync: failed state=failed files_done=%d files_total=%d rss_mb=%s rss_peak_mb=%s error=%s", …)` immediately after the `_status.error = _sanitize_error(str(e))` assignment (sanitized exactly like the status field — AGENTS/security: the raw exception string never reaches the log un-sanitized). Keep `logger.exception` so the traceback survives.
|
||||
2. `app/rag/importer.py` — `ImportSummary.log()` (**L520**): append ` rss_mb=%s` to the existing `import: summary …` line (the `format_counts()` helper at L513 stays untouched — the memory field belongs to the LOG line, not the counts formatter, so the CLI's other consumers of `format_counts()` are byte-identical).
|
||||
3. `tests/integration/test_sync_api.py` — caplog assertions (the house caplog pattern): after a run to `success`, the captured records include a `sync: started … rss_mb=` line and a `sync: done … rss_mb=… rss_peak_mb=…` line whose values are `?` or a positive integer; the same for the cancelled terminal (reuse the existing cancel test's fixture) and the failed terminal (reuse the existing failure test — `sync: model check failed` or a stubbed raise). Assert the PRE-existing prefixes are unchanged (`line.startswith("sync: started repos=")` etc.) — the append-only pin.
|
||||
4. `tests/integration/test_importer_e2e.py` (or the importer family's log-pin home): the `import: summary …` line ends with ` rss_mb=<int|?>` and its earlier fields are unchanged.
|
||||
5. `README.md` — the operations grep (`grep ' :: ' … | grep ' rss_peak_mb='`), the field meanings, and the process-vs-cgroup caveat in one short paragraph (D1/A1). `AGENTS.md` rule 10: extend the parenthetical to mention the sync's new fields (do not rewrite the rule).
|
||||
6. Verify: `uv run pytest tests/integration/test_sync_api.py tests/integration/test_importer_e2e.py -v`; `uv run pytest`; ruff + pyright.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: the four lifecycle terminals + the importer summary line pin the fields and the unchanged prefixes.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] All four sync lifecycle lines and `import: summary …` carry the new fields, appended after the existing ones; the failed terminal gains the sanitized one-line summary while `logger.exception` still writes the traceback.
|
||||
- [ ] `uv run pytest tests/integration/test_sync_api.py -v` green (fields present, pre-existing prefixes byte-identical).
|
||||
- [ ] `README.md` + `AGENTS.md` rule 10 updated; `uv run pytest` green, coverage >90%, ruff + pyright clean.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Task 03 — The status key and the Sources-page clause
|
||||
|
||||
**Phase:** `187_sync_peak_memory_line` · **Source:** owner incident 2026-10-15
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
Put the number where the operator is: two additive keys on `GET /api/sync/status` (sampled at the run start, each phase transition, every terminal, and at most every 5 s during the walk) and a ` · peak <N> MB` clause on the Sources page's existing sync result line.
|
||||
|
||||
## Work
|
||||
1. `app/api/sync.py`:
|
||||
- `SyncStatus` (**L276**): two new fields `rss_mb: int | None = None`, `rss_peak_mb: int | None = None`, documented as phase-187 additive state with the phase-64 "terminals keep the final value" convention and the run-start reset.
|
||||
- A single private sampler on the module: `_sample_rss(force: bool = False) -> None` — reads `app.core.memory.rss_mb()` / `rss_peak_mb()`, stores both, keeps a `time.monotonic()` guard so a non-forced call is a no-op within 5 s (D3/A4); forced at the run start, each `_status.phase = "import"|"overview"|"summaries"` transition (**L718** and friends), and every terminal (success L903 path, cancelled L933 path, failed L938 path) so the final value is exact.
|
||||
- `sync_status()` (**L385**): append `"rss_mb"` and `"rss_peak_mb"` to the dict (the file's own words: "Additive only — every pre-existing key keeps its name, type, and meaning") and document the two keys + the terminal convention in its docstring.
|
||||
- The per-file `_hook` (L677) calls `_sample_rss()` (unforced → throttled).
|
||||
2. `frontend/assets/sources.js`: on the terminal render path, append the clause to the EXISTING result/counts text — ` · peak 512 MB` — sourced from `status.rss_peak_mb ?? status.rss_mb`, and render NOTHING when both are null (never `peak ? MB`). Touch the result-line helper used at **L1080–1100** / **L1378** and the terminal/announce path around **L1314–1330**; the existing state strings ("Sync cancelled", the counts line, the error copy) keep their exact wording with the clause appended, and the aria-live announcement keeps announcing what it announces today plus the clause (no new announcement). No new DOM control, no layout shift on the 375px viewport (the clause wraps inside the existing line's `.cell-clip`-free text; verify visually in the E2E at mobile width).
|
||||
3. Pins:
|
||||
- `tests/integration/test_sync_api.py`: the payload carries both keys with `int | None`; after a run the terminal payload's `rss_peak_mb` is non-null on Linux and `>= rss_mb` when both are set; idle (never-run) reports `null`/`null`; every pre-existing key/value shape asserted unchanged (the additive pin); anonymous `GET /api/sync/status` still 403 (no new anonymous surface — A4).
|
||||
- `tests/unit/test_sync_status_shape.py` (or this file's existing home for the status-shape matrix): the key set is exactly the old set ∪ {`rss_mb`, `rss_peak_mb`}.
|
||||
- `frontend`: the clause appears in the terminal line and NOT in `running`; a null peak renders no clause.
|
||||
4. Verify: `uv run pytest tests/integration/test_sync_api.py -v`; the three sync E2E neighbours in isolation (`test_sync_summary_visibility.py`, `test_sync_progress_box.py`, `test_sync_status_freshness.py`) still green (they assert the existing strings — the clause must not break them; if one asserts an exact full-line equality, update it to the clause-appended form and say so in the task note); `uv run pytest`; ruff + pyright.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: the shape + additive + admin-gating pins above.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `SyncStatus` + `sync_status()` carry the two additive keys; sampling happens at start / phase transitions / every terminal and is throttled to 5 s in the per-file hook.
|
||||
- [ ] The Sources page shows ` · peak <N> MB` on the terminal line, renders nothing when the value is null, and adds no new control.
|
||||
- [ ] The three existing sync E2E families are green (any exact-string update documented).
|
||||
- [ ] `uv run pytest` green, coverage >90%, ruff + pyright clean.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Task 04 — E2E: the operator sees the peak (the phase's dedicated Playwright suite)
|
||||
|
||||
**Phase:** `187_sync_peak_memory_line` · **Source:** owner incident 2026-10-15
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
Prove through the browser that the number is there and honest: a real sync from the Sources page ends with a `peak <N> MB` clause on the result line, the status payload's `rss_peak_mb` is a positive integer that survives the terminal, the running state shows no clause, and nothing anonymous can read it. Then the full gates, the docs pass, the atomic commit.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_sync_peak_memory_visible.py` — a NEW module, mock-only, the `tests/e2e/test_sync_progress_box.py` / `test_sync_summary_visibility.py` house shapes (dedicated module app + port, mock LLM, dedicated `BOR_SOURCES_DIR`, shared E2E Postgres truncated before AND after each test, admin login via `tests/e2e/auth_helpers.py`).
|
||||
- **Test 1 `test_result_line_shows_the_peak`:** sync a small fixture → after the `success` terminal, the progress box's result line matches `/peak \d+ MB/`, the existing wording ("Sync complete"/the counts line) is still present, and the aria-live region contains the same clause.
|
||||
- **Test 2 `test_status_payload_reports_rss`:** during `running` and after the terminal, `GET /api/sync/status` via `page.request` → `rss_peak_mb` is a positive int in the terminal, `>= rss_mb` when both are set, and pre-existing keys are unchanged (the additive pin end to end). During `running` the result line shows NO `peak` clause (the clause is terminal-only in the UI).
|
||||
- **Test 3 `test_anonymous_cannot_read_memory`:** logged-out `GET /api/sync/status` → 403 (the A4 exposure guard, unchanged).
|
||||
- **Test 4 `test_cancelled_terminal_keeps_the_peak`:** cancel a run (the `test_sync_cancel.py` shape) → the neutral "Sync cancelled" state still renders, with the peak clause present (terminals keep the value).
|
||||
- **Mobile width:** assert the clause at 375 px does not overflow the progress box (the PLAN §7 container rule).
|
||||
- Module docstring: the incident reference, the fields + the process-vs-cgroup caveat pointer to `README.md`, the test→observable mapping, the run-in-isolation command, prerequisites (`podman compose up -d db`).
|
||||
2. **Docs pass:** `README.md` + `AGENTS.md` in full against the post-phase app, every inconsistency fixed (pre-existing included); confirm rule 10's extension and the operations grep landed in task 02; flag the PLAN §9 reconciliation in the phase report (never edit PLAN.md).
|
||||
3. **The full gates:**
|
||||
- `uv run pytest tests/e2e/test_sync_peak_memory_visible.py -v --no-cov` — green **in isolation**.
|
||||
- `uv run pytest tests/e2e/test_sync_progress_box.py -v --no-cov`, `tests/e2e/test_sync_status_freshness.py -v --no-cov`, `tests/e2e/test_sync_cancel.py -v --no-cov` — green in isolation (the surfaces the clause touches).
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` — full suite green, `app/` coverage **>90%**.
|
||||
- `uv run ruff check . && uv run pyright` — clean.
|
||||
4. **Commit + move:** one atomic `--no-gpg-sign` Conventional Commit — e.g. `feat(observability): log and show the sync's RSS so memory is a number, not a forensic exercise` — covering `app/core/memory.py`, `app/api/sync.py`, `app/rag/importer.py`, `frontend/assets/sources.js`, the tests, the docs and `.agents/`. Move the phase dir to `.agents/phases/complete/187_sync_peak_memory_line/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: the dedicated suite + the three sync UI families in isolation.
|
||||
- Coverage: **>90%** on `app/`; ruff + pyright clean.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_sync_peak_memory_visible.py -v --no-cov` green **in isolation** (clause on both terminals, absent while running, payload types, anonymous 403, mobile width clean).
|
||||
- [ ] The three regression E2E families green in isolation.
|
||||
- [ ] `uv run pytest --cov=app --cov-report=term-missing` green with `app/` coverage >90%; ruff + pyright clean.
|
||||
- [ ] `README.md` and `AGENTS.md` accurate and consistent with the whole project; the PLAN §9 reconciliation flagged for the owner.
|
||||
- [ ] One atomic `--no-gpg-sign` Conventional Commit; the phase dir moved to `.agents/phases/complete/`.
|
||||
Reference in New Issue
Block a user