phase: 153_document_viewer_context
Build and Push Containers / build-and-push-app (push) Successful in 2m45s
Build and Push Containers / build-and-push-db (push) Successful in 19s

That last `--co` (collect-only) run was my probe mistake — it shows 34% because no tests execute under collect-only. The valid coverage run was the real gate command executed earlier: `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (exit 0, all 3544 tests executed). All gates are verified.

## Phase 153 — final verification pass report

**Verified (all tasks 01–04 complete in `complete/`):**
- Modal `fullPageUrl(source, path, back = null)` + `openDocumentModal(…, back = null)` with module-level `currentBack`; 5 chat call sites pass `"/"`, sources.js call site unchanged (byte-identical Sources path); `documentUrl` comment now true
- Copy-path control (shared `renderDocument` core, icon+text, text = accessible name) on both surfaces; per-page clipboard helpers with the fallback-field + status-line contract; `#doc-copy-announcer` live region; styles + 44px touch floor
- Unit pins: `&back=%2F` / no-`back` byte-identity, `currentBack` single-writer guard, copy strings byte-pinned, phase-13 regression pins untouched

**Test / lint / coverage results:**
- `uv run pytest` → **3544 passed**, 0 failed (full unit + integration)
- `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (>90% gate)
- `uv run pytest tests/e2e/test_document_viewer_context.py -v --no-cov` → **8 passed** in 47.33s, in isolation (DB up, mock LLM)
- `uv run ruff check . && uv run pyright` → clean (0 errors/warnings)

**Completion criteria:** all met — (1) chat→modal→Full page: `href="/"` + "Chat"; Sources→modal→Full page: `href="/sources.html"` + "Sources" with `back=` asserted absent; (2) no-`back` URLs byte-identical to pre-phase; (3) Copy path: "Path copied." on success (clipboard readback), prefilled select-on-focus fallback field + "Path ready — copy it from the field." on rejected clipboard, both surfaces parameterized; (4) copied string exactly `docs/homelab/kubernetes.md`; (5) full suite green, E2E isolated green, lint clean, coverage 99%; (6) commit by harness.

**Documentation:** README.md reviewed in full — phase-153 feature already named in the Chat section; routes, env reference, workflows, deployment all consistent — no changes needed. AGENTS.md reviewed in full — rules, gates, and quick-reference commands verified against the repo (incl. `test_smoke.py`, `.env.example`, gitignore entries) — consistent, no changes needed.

**Notable:** pytest 9.1.1 quirk — a doubled `-q` (pyproject `addopts` + explicit `-q`) swallows the final "N passed" summary line while still exiting 0; the plain `uv run pytest` gate command prints it. No defects found in prior work.

**Next pending phase:** `154_sync_status_freshness`.
This commit is contained in:
2026-10-04 09:23:54 -04:00
parent 12fa69b7ea
commit 966c4b0540
59 changed files with 4304 additions and 34 deletions
@@ -0,0 +1,42 @@
# Phase 155 — folder summaries in varied, natural language
**Source:** owner UX audit 2026-10-03 (live interactive-browser pass): "All `lite` folder descriptions open with 'This folder contains…' — formulaic; a prompt nudge for variety would help."
**Story:** n/a (audit-derived phase; the folder-summary machinery is phase 94's — `94_ls_tree_drilldown` (complete))
**Context:** `app/rag/folder_summaries.py::FOLDER_SUMMARY_INSTRUCTION` (L102–108) — the current neutral instruction: "From the document list below, write a 1-3 sentence plain-text summary of what this folder's documents cover, in natural language. Do not use markdown. Do not invent anything that is not in the list." The `FOLDER_SUMMARY_MODE` marker (L91) and the `FOLDER_HEADER_PREFIX` (`"Folder: "`) are the E2E mock's keys — `tests/e2e/mock_llm.py` keys on the marker + the header line (a canned reply that is a pure function of the request), NOT on the instruction text — an instruction change is mock-safe as long as the marker + header prefix stay byte-identical. `generate_folder_summaries(db, llm, *, skip, only_missing, changed_docs, on_progress)` (L440) — `changed_docs=None` is the FULL regeneration (every candidate folder; the `scripts/import_docs.py` CLI's default); the sync path is change-gated (phase 127). PLAN's "persona text changes through the plan, not in code" does NOT bind this constant — it is a one-shot prompt constant (the phase-94/127 idiom), not the chat persona; the marker contract is the load-bearing piece and it is untouched. LLM calls are free (A21) — a full regeneration's cost is not a constraint.
## Objective
New and regenerated folder summaries open in varied, natural language — no more catalogue of "This folder contains…" — via an instruction nudge, plus a one-shot `scripts/regen_folder_summaries.py` that refreshes every stored summary against the new instruction (the live RAG catalogue's 15+ source-root rows currently all carry the formulaic opener).
## Dependencies
- `94_ls_tree_drilldown` (complete) — the folder-summary machinery (grouping, prompt, upsert, prune).
- `127_sync_incremental_folder_summaries` (complete) — the `changed_docs` mode contract the new script's FULL mode (`changed_docs=None`) must not disturb.
## Design (shared by all tasks — the executor reads this, not the chat)
- **The nudge (task 01):** `FOLDER_SUMMARY_INSTRUCTION` gains the variety direction: keep the 1–3 sentence / plain-text / no-markdown / no-invention constraints UNCHANGED; add: "Open in a varied, natural way — state what the folder is for, the way a person would describe it. Do NOT begin with 'This folder contains' or 'The <folder name> folder contains'." (ASSUMPTION A1, owner-locked — the exact wording above; the executor may polish grammar but not the two forbidden openers, which are the pin.) `FOLDER_SUMMARY_MODE` and `FOLDER_HEADER_PREFIX` stay byte-identical (the mock contract — a unit pin asserts it).
- **The regen script (task 02):** `scripts/regen_folder_summaries.py` — the `scripts/import_docs.py` entrypoint idiom (argparse; the app settings/`get_settings()` fail-loud boot; the DB session construction; the `LLMClient` construction — read `scripts/import_docs.py`'s main path and mirror its exact setup, the `--limit N` debug passthrough included if import_docs has one — check and match). Body: `await generate_folder_summaries(session, llm)` — FULL mode (`changed_docs=None` the default), `only_missing=False` (a REGEN — every stored row is refreshed, not just gaps), `skip=False`. End: the importer's cron-safe single summary log line idiom (one line: counts out of the returned stats dict) + the per-`uv run` exit code 0 only on success. NO new API surface, NO schema change, NO change to the sync path's change-gated behavior (phase 127's contract is untouched — the script is a standalone maintenance entrypoint).
- **The mock (task 04's suite prerequisite):** `tests/e2e/mock_llm.py`'s CANNED folder-summary reply may evolve (it is a pure function of the request — the marker/header keys are unchanged); its canned text for a folder is updated to the new voice (e.g. "Documentation for <folder>, covering <first-doc-theme> …" — a natural opener, never the forbidden ones) so the E2E can assert the stored text's shape. The mock's OTHER markers/contracts are untouched.
## Tasks
1. `01_prompt_nudge.md` — the instruction update + the marker/header byte-identity pins.
2. `02_regen_script.md` — `scripts/regen_folder_summaries.py` (full mode, the import_docs idiom).
3. `03_unit_pins.md` — the instruction pin, the mock-contract guard, the script wiring.
4. `04_e2e_folder_voice.md` — the dedicated Playwright suite (the mock's new canned voice + the RAG table rendering it).
## Testing & Quality
- Unit: `FOLDER_SUMMARY_INSTRUCTION` pinned (the two forbidden openers present as the constraint; the 1–3 sentence/plain-text/no-markdown/no-invention words preserved); `FOLDER_SUMMARY_MODE` + `FOLDER_HEADER_PREFIX` byte-identical (the mock contract); the script's wiring (a fake LLM + a fixture catalogue: `generate_folder_summaries` called with `changed_docs=None`, `only_missing=False`, `skip=False` — the stats dict surfaces in the summary line; a bad settings boot fails loud — the app's validator idiom).
- Integration: a full-mode regeneration over a fixture catalogue upserts every candidate row (the phase-94 idiom) and prunes nothing (no catalogue change — the prune rule is count-driven; a pure regen touches only `updated_at` + text).
- Coverage: **>90%** on new/modified code (`app/rag/folder_summaries.py`, the script).
- The phase is "Complete" only when the **entire** project test suite passes 100% (every test in the project, not just this phase's).
- E2E: `uv run pytest tests/e2e/test_folder_summary_voice.py -v --no-cov` in isolation (DB up: `podman compose up -d db`; the deterministic `tests/e2e/mock_llm.py`).
## Documentation & Consistency
- The final pass reviews `README.md` and `AGENTS.md` in full against the project as it stands after this phase — fixing every inconsistency found, **including pre-existing ones not caused by the phase**.
- Tasks update `README.md`/`AGENTS.md` immediately when their own changes are user-visible (the new script is a user-visible command — it joins the README's import/update workflow list beside `import_docs`; the summary voice is a visible content change).
## Completion Criteria
- [ ] `FOLDER_SUMMARY_MODE` + `FOLDER_HEADER_PREFIX` are byte-identical to pre-phase (the mock contract pin)
- [ ] `uv run python -m scripts.regen_folder_summaries` runs a FULL regeneration (the stats summary line logs; exit 0; every candidate row's `updated_at` advances, unchanged folders' text is the mock's new voice)
- [ ] the sync path's change-gated behavior is byte-identical (phase 127's pins stay green — no `app/api/sync.py` change)
- [ ] the E2E: a mock-LLM folder summary is stored + rendered in the RAG table WITHOUT the forbidden openers
- [ ] `uv run pytest` green; the dedicated E2E green in isolation; `uv run ruff check . && uv run pyright` clean; `app/` coverage >90%
- [ ] one atomic commit per AGENTS.md rule 8 (`--no-gpg-sign`)