phase: 167_api_payload_diet

All gates green. Final report:

**Verified (phase 167 final pass — all 6 tasks already in `complete/`)**
- Live re-measure of the headline criterion on the **1,000-doc live-replica corpus** (restored `data/bor_live_kb_replica.dump.sql` into a scratch DB `bor_audit`, alembic 0011→0027, dropped after; the shared dev DB was never touched): `GET /api/docs/tree` default depth=1 = **8,474 identity → 1,128 B with `Content-Encoding: gzip`** (< 40 KB bar; ~2 MB defect fixed), repeat load **304 / 0 bytes**; depth=8 265,147→23,515; node drill 20,844→1,065; `/api/docs` 246,366→45,104; `/api/documents/content` 892,124→60,776 (`LOOK`, accepted-with-why). Reproduces `AUDIT.md` to within 1 byte.
- Transport contracts proven live: `POST /api/chat` **no** `Content-Encoding` + frames arriving live + budget line `… enc=identity duration_ms=7380 stream=1`; HTML still `no-cache` with the `?v=` rewrite (and gzipped); A20 headers on 200 **and** 304; ETag identical across identity/gzip; `depth=0|9|abc`→422; anonymous tree/node→403; unknown folder/source→404 `unknown node`.

**Gates (all 100% green)**
- `uv run pytest` → **3980 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (`compression.py`, `http_cache.py`, `response_budget.py` 100%; `api/docs.py` 99%, one defensive dedupe line).
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- Phase + regression E2E **in isolation** (`--junitxml`, exit codes): `test_kb_tree_lazy_load` 11, `test_kb_tree` 8, `test_kb_tree_nav` 4, `test_navbar_refresh` 7, `test_document_dates` 6, `test_git_source_dates` 4, `test_sync_summary_visibility` 3, `test_table_ux_polish` 6, `test_sources_midstream_bug` 6, `test_ls_tree_drilldown` 3, `test_smoke` 3 — **0 failures/errors** (the earlier `-q -q` run had simply suppressed pytest's summary line).

**Documentation** — README.md + AGENTS.md reviewed in full (1,436 / 80 lines):
- Fixed (pre-existing, phase-167-exposed): README *Caching & deploys* claimed "One Starlette middleware (`app/core/caching.py`) applies the rule" while the section now describes three transport layers → rewritten to name caching/compression/http_cache, the real outermost→innermost order, and why `bytes=` is uncompressed.
- AGENTS.md rule 5 (UI Structure Check) gained the phase-167 clause: a lazily-drilled level keeps the current level rendered, its in-flight `role="status"` / `role="alert"`+Retry row lives in the same bounded scroll panel, and stat cards read server `totals` (never a walk of a depth-limited tree).
- Cross-checked and consistent: every `BOR_*` in `.env.example` is in the README table, every `Settings` field is documented, all 15 README-cited API routes exist in the app, no stale "one fetch / drill client-side" claim remains anywhere.

**Notable decisions / deviations**
- `compression.py` adds a documented `_RebufferSmallResponses` seam beyond the D1 sketch (the caching layer's `BaseHTTPMiddleware` re-framing otherwise defeats Starlette's `minimum_size` exemption and inflates tiny responses) — integration-pinned, math still Starlette's.
- A node-route **404 = vanished folder → honest reset to top level** (per `00_phase.md` D4/D6), not task-06's "alert + Retry"; 500 shows alert+Retry with the level preserved — both pinned, nothing weakened.
- Retrieval battery not measurable (shared E2E DB holds no corpus), but `git diff --name-only -- app/rag` is empty, so retrieval is unchanged by construction. `AUDIT.md` left byte-identical (it lives under `todo/`, off-limits to this pass); no git index/history commands run.

**Next pending phase:** none — `167_api_payload_diet` is the only entry in `.agents/phases/todo/`; the harness moves it to `complete/` and commits.

tests/e2e/test_kb_tree.py

@@ -9,6 +9,24 @@ the level's files — and the owner edits (or clears) any directory's
 description with the phase-57 inline affordance. Everything is pinned
 against the real app + the deterministic mock.
 
+PHASE 167 (task 06, LOCKED A3) — THE CATALOG LOAD, MIGRATED NOT DELETED:
+the view no longer downloads the whole tree. The boot is ONE
+``GET /api/docs/tree?depth=1`` (the sources plus one level plus the
+server's ``totals``), and the FIRST time the owner opens a folder the
+payload only NAMES, the client fetches that one level through
+``GET /api/docs/tree/node?source=…&path=…`` (the phase-97 "ONE fetch of
+/api/docs/tree, the drill is client-side" contract, superseded by the
+TODO L5 2 MB fix — see
+``tests/e2e/test_kb_tree_lazy_load.py`` for the new load-shape suite).
+EVERY claim this suite owns stays and is re-pointed at the level the
+owner is ON: the breadcrumb text, the back navigation, the table
+columns, the dates, the summary-editor round trip, the pending markers,
+the empty state and the anonymous gate. A level the client already
+holds (the top level, a source root, a folder drilled into once — and
+the breadcrumb always walks back through a level it drilled) still
+renders with ZERO requests, so the pre-phase navigation claims below
+remain literally true at those levels.
+
 Run in isolation (DB must be up: ``podman compose up -d db``):
 
     uv run pytest tests/e2e/test_kb_tree.py -v --no-cov
@@ -498,18 +516,21 @@ def _run_sync(app_url: str) -> dict[str, Any]:
 
 def _wait_top_level(page: Page) -> None:
     """The admin boot has rendered the top level: the two source rows
-    are in the folders table (the tree's single fetch settled) and the
-    top-level file table is empty (files are per-source — always hidden
-    at the top)."""
+    are in the folders table (the ONE ``?depth=1`` catalog fetch has
+    settled — phase 167) and the top-level file table is empty (files
+    are per-source — always hidden at the top)."""
     expect(page.locator("#folders-tbody tr")).to_have_count(2, timeout=30_000)
     expect(page.locator("#docs-tbody tr")).to_have_count(0)
 
 
 def _drill(page: Page, *names: str) -> None:
-    """Drill one level at a time (client-side — no fetch, no URL
-    change): each name is the EXACT text of the source/folder link at
-    the current level (the row builders' link text: the source name,
-    or the folder's last path segment)."""
+    """Drill one level at a time (no URL change — the navigation is a
+    re-render; phase 167: a level the depth-1 payload already holds
+    costs ZERO requests, the first open of one it only NAMES costs
+    exactly one ``/api/docs/tree/node`` fetch): each name is the EXACT
+    text of the source/folder link at the current level (the row
+    builders' link text: the source name, or the folder's last path
+    segment)."""
     for name in names:
         page.click(f'#folders-tbody a.folder-link:text-is("{name}")')
 
@@ -668,7 +689,10 @@ def test_drill_into_folder(
     expect(page.locator("#folders-wrap")).to_be_hidden()
 
     # The breadcrumb link on the source goes back UP to the source
-    # level (client-side — no fetch, no URL change).
+    # level — a re-render with no fetch and no URL change, because
+    # reaching ``two`` means the client holds ``alpha`` (phase 167:
+    # the breadcrumb only ever walks back through a level it drilled,
+    # so this stays the zero-request path it was pre-phase).
     page.click(f'#kb-crumb a.kb-crumb-link:text-is("{ALPHA}")')
     expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text(ALPHA)
     expect(page.locator("#kb-crumb a.kb-crumb-link")).to_have_count(1)