ducoterra
7b5ea7c2ee
All gates green. Final state verified.
**Implemented / verified**
- Task 03's work was **never actually done** (its report was a stub, `tests/e2e/test_prune_cascade_cleanup.py` did not exist) → wrote it: 3 tests — 9-file local source synced from the Sources page (`9 added`), 6 files deleted upstream, re-sync (`0 added · 3 unchanged · 6 pruned`), documents vanished from the tree + `GET /api/documents/content` 404 + total 3, **0 orphan chunk rows** via the test's own SQL, survivor chunk total unchanged; partial-prune chat turn (grounded, non-deflected, `query_log` names the survivor and no pruned doc); total source removal (`DELETE /api/git-sources/{id}`) wipes every doc+chunk of the source with 0 orphans, sibling source untouched. Mock-only (`E2E_REAL_LLM=1` skipped), own port 8199, truncate before/after.
- Tasks 01/02 re-verified: `passive_deletes=True` + mapper pin, probe flipped to `EXPECTED_CHUNK_SELECTS_PER_PRUNED_DOC = 0`, `_prune`/`delete_git_source` docstrings.
- **Drift guard proven non-tautological by my own one-off run**: scratch DB (`alembic upgrade head`) → FK `confdeltype='c'`, guard PASSES, CASCADE delete leaves 0 chunks; FK recreated `ON DELETE NO ACTION` → guard FAILS verbatim and a `passive_deletes` delete raises `ForeignKeyViolation` (never silent orphans). Scratch DB dropped; shared test DB untouched (`'c'`).
**Gates**
- `uv run pytest --cov=app --cov-report=term-missing` → **4488 passed**, `app/` TOTAL **99%** (>90%)
- `uv run pytest tests/integration/test_prune_cascade_statements.py -v` → 4 passed
- `uv run pytest tests/e2e/test_prune_cascade_cleanup.py -v --no-cov` → **3 passed in isolation**; `test_source_removal_cleanup.py` 6 passed, `test_force_push_resync.py` 2 passed (both isolated)
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors
**Completion criteria**: passive_deletes+pin ✅ · probe post-fix ✅ · drift guard green + proven-failing variant ✅ · full suite/coverage/lint ✅ · E2E in isolation ✅ · docs consistent ✅ · commit = harness-only (per pipeline rule I ran no git write commands).
**Docs**: fixed 4 pre-existing README inconsistencies — per-module E2E ports (claim of "one pair of fixed ports" predates phase 79), debugpy `launch.json` snippet vs the shipped file (`name`/`remoteRoot`), and the page-naming wobble (navbar labels `/sources.html` **RAG**, `/git-sources.html` **Sources** — the Sync button's home is now named correctly in 4 places); added the phase-183 E2E pointer. AGENTS.md verified accurate (rule 12 matches the delete-site audit).
**Deviation**: none from the locked decisions; the E2E is a behaviour pin (byte-identical owner-visible behaviour), the statement-count probe remains the memory teeth, as phase 183 D1/A4 specify.
**Next pending phase**: `184_sync_bounded_session`.
151 lines
9.4 KiB
Markdown
151 lines
9.4 KiB
Markdown
# AGENTS.md — Operating Rules for All Agents on This Repo
|
||
|
||
1. **Always read `.agents/PLAN.md` first.** It contains the architecture, the
|
||
LOCKED DECISIONS, the UI/UX standards, the data model, and the roadmap.
|
||
Do not design anything that contradicts it.
|
||
2. **Follow the phased execution protocol in `.agents/phases/`.** Work through
|
||
`.agents/phases/todo/` in numeric order. When a phase is complete, move its
|
||
directory to `.agents/phases/complete/`.
|
||
3. **Strictly adhere to the LOCKED DECISIONS** (PLAN §2). If you believe a
|
||
LOCKED decision is wrong, stop and flag it — do not silently deviate.
|
||
4. **"One Story, One Phase":** each user story in `.agents/user_stories/`
|
||
corresponds to a distinct execution phase, and **every** phase — story-driven
|
||
or owner-directed — includes its own dedicated Playwright E2E test suite
|
||
(one test file for the phase, run in isolation).
|
||
5. **"UI Structure Check":** before finalizing any UI component, verify that
|
||
it follows the layout principles in `.agents/PLAN.md` §7 (proper container
|
||
usage, the 72rem centered reading column — every page rides the 72rem
|
||
container at every viewport; phase 100 supersedes §7.1's 46rem chat
|
||
column, full-width tables on Sources — no skinny wasted-space lists;
|
||
the catalogue tables (RAG, Sources, History) are bounded scroll
|
||
panels — `max-height: min(70vh, 64rem)` — with a sticky header row;
|
||
a clipped action label shows an ellipsis hint (`.cell-clip`) with
|
||
the full text kept reachable via the control's aria-label
|
||
(phase 156); an in-flight turn never paints an empty answer
|
||
bubble — the box is created empty as the reasoning anchor and
|
||
`.msg .bubble:empty { display: none }` hides it until the first
|
||
answer token (phase 166); an answer's inline `[text](url)`,
|
||
`` and `####`+ heading render through the ONE shared
|
||
renderer into an allowlist-gated `a.md-link` (underlined, never a
|
||
`javascript:`/`data:` element), a container-capped
|
||
`img.md-img` (`max-width: 100%` — no horizontal overflow at 375px)
|
||
and an `h5`/`h6` sized into the existing type scale — never printed
|
||
reference text (phase 180); an SVG document paints as a
|
||
graphic — the viewer's image block adds
|
||
`.doc-image-img--vector` (`width: 100%`, `object-fit: contain`)
|
||
when `format` is `svg` so a `viewBox`-only drawing spans the
|
||
`.doc-image` card instead of the 300×150 default box, while the
|
||
chat figure and Sources thumbnail keep their fixed caps and the
|
||
bytes stay an `<img>` src (never inlined) (phase 181); the
|
||
document viewer's page strip is
|
||
CLASS-scoped —
|
||
the primary class shows the strip in place of the text, the
|
||
additive class never hides the text: Text/Pages toggle group
|
||
(`aria-pressed`, Text default) with the strip behind it
|
||
(phase 163); a lazily-drilled KB-tree level keeps the level the
|
||
user is ON rendered while its node fetch runs, and its ONE
|
||
in-flight row (`role="status"`) / alert + Retry row
|
||
(`role="alert"`, a real `<button>`) sits inside that same bounded
|
||
scroll panel — and the stat cards read the server's `totals`,
|
||
never a walk of a depth-limited tree (phase 167); an answer's
|
||
code block is EITHER verbatim-and-badged OR plainly the model's —
|
||
never a silently-substituted hybrid: the ` ```bor ` locator fence
|
||
renders through the ONE shared renderer as
|
||
`.verbatim[data-bor-locator]` with the model's body DROPPED
|
||
(`renderMarkdown(md, { verbatim: true })`, off by default so every
|
||
other surface is byte-identical), styled from palette variables only
|
||
with a `min(40rem, 70vh)` scroll body, a sticky `source/path`
|
||
caption and a `user-select: none` line-number gutter, while the
|
||
plain `.code-block` wrapper carries a Copy button and no badge
|
||
(phase 182); those placeholders hydrate against the frame's
|
||
`answer_text` through the ONE panel client
|
||
(`frontend/assets/verbatim.js` — a classic script loaded AFTER
|
||
`markdown.js` on the chat AND shared pages, no second locator grammar,
|
||
a plain string compare against the server's canonical `locator`, the
|
||
caption's numbers always the SERVER's extent, an unresolvable locator
|
||
painting the pinned sentence `I could not verify this against the
|
||
knowledge base` with no badge, no body and none of the model's bytes,
|
||
no panel left at "Retrieving the exact text…" after a terminal frame
|
||
(the `error` frame, the 300 s guard, Stop and the failure funnel all
|
||
settle it), the Copy button's clipboard content equal to the server
|
||
string exactly, and the shared page's panel Copy button + plain
|
||
`/document.html` link the ONE owner-directed exception to that page's
|
||
zero-controls rule) — phase 182 task 07) and
|
||
meets WCAG 2.1 AA basics (semantic
|
||
landmarks, labels, contrast ≥4.5:1, focus-visible, aria-live for the
|
||
stream).
|
||
6. **"No CDN Rule":** all CSS, JS, fonts, and images must be served statically
|
||
from within the FastAPI application. No `<script src="https://…">` or
|
||
`<link href="https://…">` tags in HTML templates unless bundled locally.
|
||
An integration test enforces this on the index page.
|
||
7. **"Debugpy Check":** any new entrypoint must keep the conditional import —
|
||
`debugpy` is imported **only** when `DEBUGPY=1` (see
|
||
`app/core/debugging.py`); default off, never imported otherwise.
|
||
8. **Git protocol:** Git is mandatory. One **atomic, professional commit per
|
||
completed phase** (Conventional Commits, e.g. `feat(rag): stream RAG
|
||
answers over SSE with source citations`). Always append `--no-gpg-sign`
|
||
(the repo also sets `commit.gpgsign=false`). `.agents/` is **tracked and
|
||
committed** with the phase — only runtime artifacts (`.agents/phase-sessions/`,
|
||
`.agents/pipeline.log`) are gitignored.
|
||
9. **Test gates are non-negotiable:** a phase is complete only when unit +
|
||
integration tests pass, `app/` coverage is **>90%**
|
||
(`uv run pytest --cov=app --cov-report=term-missing`), and the phase's
|
||
Playwright E2E file passes **in isolation**
|
||
(`uv run pytest tests/e2e/test_<story>.py -v --no-cov`).
|
||
10. **Amply log, visibly feed back:** server-side, log the required
|
||
per-turn line (PLAN §9 — phase 182 appends `verbatim_blocks=N` after
|
||
`scaffold_stripped=N`: the verbatim display blocks actually SENT on
|
||
the `done` frame, `0` on every turn without them, deflected included)
|
||
**and** the per-response byte-budget line
|
||
(`app/core/response_budget.py` — one
|
||
`api method=… path=… status=… bytes=… enc=… duration_ms=…` line per
|
||
`/api/*` response, INFO at/over `BOR_LOG_LARGE_RESPONSE_BYTES`, read it
|
||
with `grep ' :: api '`, audit it with
|
||
`uv run python -m scripts.api_payload_audit`, phase 167); UI-side,
|
||
honor the "never stale" feedback contract (PLAN §7.4).
|
||
11. **Never block the event loop:** anything blocking (a `subprocess`,
|
||
synchronous file/network I/O, a heavy walk) reached from an `async`
|
||
handler or a spawned background coroutine goes through
|
||
`asyncio.to_thread` (or `create_subprocess_exec`). The sync's git
|
||
calls do (phase 179: a long `git clone` used to park the single
|
||
uvicorn loop and freeze the whole HTTP surface, the 2 s
|
||
`GET /api/sync/status` poll included — pinned by
|
||
`tests/integration/test_sync_git_freeze.py` with a 3 s stub and by
|
||
`tests/e2e/test_sync_git_freeze.py` with a real ~9 s `file://`
|
||
clone). A cooperative cancel unblocks the *run* at the `to_thread`
|
||
await; the worker thread itself runs to completion (threads are not
|
||
interruptible) while the loop keeps serving — pinned by the same E2E
|
||
module's cancel-during-clone test.
|
||
12. **Never hydrate what you delete:** `Document.chunks` is
|
||
`passive_deletes=True` (phase 183), so deleting a document removes its
|
||
chunk rows **in the database** (`chunks_document_id_fkey` is
|
||
`ON DELETE CASCADE`, alembic 0001) and a mass prune costs
|
||
O(documents). The importer's prune and the source-removal endpoint both
|
||
rely on it. Do not remove the keyword, and do not add a delete path
|
||
that SELECTs `chunks` (or a `Chunk.embedding`) on its way to deleting
|
||
them — one hydrated 768-dim embedding is ~24.8 KiB of Python, which is
|
||
how the 2026-10-15 sync's 3,008-document prune reached 4 GB and locked
|
||
up the VM. Pinned by
|
||
`tests/integration/test_prune_cascade_statements.py` (statement count,
|
||
zero orphan chunk rows, and a live-FK drift guard that fails the suite
|
||
if the cascade clause ever disappears — because then
|
||
`passive_deletes=True` would silently orphan rows). The
|
||
`doc.chunks = [...]` **re-embed** path is the deliberate, documented
|
||
exception (collection replacement legitimately needs the loaded
|
||
collection; changed files only).
|
||
|
||
## Quick reference
|
||
```bash
|
||
podman compose up -d db # start Postgres 17 + pgvector
|
||
cp .env.example .env # once; then edit
|
||
uv run alembic upgrade head # apply migrations
|
||
uv run uvicorn app.main:app --reload # dev server (add DEBUGPY=1 to debug)
|
||
uv run pytest # unit + integration
|
||
uv run pytest --cov=app --cov-report=term-missing
|
||
uv run playwright install chromium # once
|
||
uv run pytest tests/e2e/test_smoke.py -v --no-cov
|
||
uv run ruff check . && uv run pyright # lint + types
|
||
uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt # retrieval regression battery (phases 119/162)
|
||
uv run python -m scripts.api_payload_audit # endpoint byte audit vs the 256 KB budget (phase 167; app must be running)
|
||
```
|