phase: 167_api_payload_diet

All gates green. Final report:

**Verified (phase 167 final pass — all 6 tasks already in `complete/`)**
- Live re-measure of the headline criterion on the **1,000-doc live-replica corpus** (restored `data/bor_live_kb_replica.dump.sql` into a scratch DB `bor_audit`, alembic 0011→0027, dropped after; the shared dev DB was never touched): `GET /api/docs/tree` default depth=1 = **8,474 identity → 1,128 B with `Content-Encoding: gzip`** (< 40 KB bar; ~2 MB defect fixed), repeat load **304 / 0 bytes**; depth=8 265,147→23,515; node drill 20,844→1,065; `/api/docs` 246,366→45,104; `/api/documents/content` 892,124→60,776 (`LOOK`, accepted-with-why). Reproduces `AUDIT.md` to within 1 byte.
- Transport contracts proven live: `POST /api/chat` **no** `Content-Encoding` + frames arriving live + budget line `… enc=identity duration_ms=7380 stream=1`; HTML still `no-cache` with the `?v=` rewrite (and gzipped); A20 headers on 200 **and** 304; ETag identical across identity/gzip; `depth=0|9|abc`→422; anonymous tree/node→403; unknown folder/source→404 `unknown node`.

**Gates (all 100% green)**
- `uv run pytest` → **3980 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (`compression.py`, `http_cache.py`, `response_budget.py` 100%; `api/docs.py` 99%, one defensive dedupe line).
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- Phase + regression E2E **in isolation** (`--junitxml`, exit codes): `test_kb_tree_lazy_load` 11, `test_kb_tree` 8, `test_kb_tree_nav` 4, `test_navbar_refresh` 7, `test_document_dates` 6, `test_git_source_dates` 4, `test_sync_summary_visibility` 3, `test_table_ux_polish` 6, `test_sources_midstream_bug` 6, `test_ls_tree_drilldown` 3, `test_smoke` 3 — **0 failures/errors** (the earlier `-q -q` run had simply suppressed pytest's summary line).

**Documentation** — README.md + AGENTS.md reviewed in full (1,436 / 80 lines):
- Fixed (pre-existing, phase-167-exposed): README *Caching & deploys* claimed "One Starlette middleware (`app/core/caching.py`) applies the rule" while the section now describes three transport layers → rewritten to name caching/compression/http_cache, the real outermost→innermost order, and why `bytes=` is uncompressed.
- AGENTS.md rule 5 (UI Structure Check) gained the phase-167 clause: a lazily-drilled level keeps the current level rendered, its in-flight `role="status"` / `role="alert"`+Retry row lives in the same bounded scroll panel, and stat cards read server `totals` (never a walk of a depth-limited tree).
- Cross-checked and consistent: every `BOR_*` in `.env.example` is in the README table, every `Settings` field is documented, all 15 README-cited API routes exist in the app, no stale "one fetch / drill client-side" claim remains anywhere.

**Notable decisions / deviations**
- `compression.py` adds a documented `_RebufferSmallResponses` seam beyond the D1 sketch (the caching layer's `BaseHTTPMiddleware` re-framing otherwise defeats Starlette's `minimum_size` exemption and inflates tiny responses) — integration-pinned, math still Starlette's.
- A node-route **404 = vanished folder → honest reset to top level** (per `00_phase.md` D4/D6), not task-06's "alert + Retry"; 500 shows alert+Retry with the level preserved — both pinned, nothing weakened.
- Retrieval battery not measurable (shared E2E DB holds no corpus), but `git diff --name-only -- app/rag` is empty, so retrieval is unchanged by construction. `AUDIT.md` left byte-identical (it lives under `todo/`, off-limits to this pass); no git index/history commands run.

**Next pending phase:** none — `167_api_payload_diet` is the only entry in `.agents/phases/todo/`; the harness moves it to `complete/` and commits.

AGENTS.md

@@ -28,7 +28,12 @@
    the primary class shows the strip in place of the text, the
    additive class never hides the text: Text/Pages toggle group
    (`aria-pressed`, Text default) with the strip behind it
-   (phase 163)) and
+   (phase 163); a lazily-drilled KB-tree level keeps the level the
+   user is ON rendered while its node fetch runs, and its ONE
+   in-flight row (`role="status"`) / alert + Retry row
+   (`role="alert"`, a real `<button>`) sits inside that same bounded
+   scroll panel — and the stat cards read the server's `totals`,
+   never a walk of a depth-limited tree (phase 167)) and
    meets WCAG 2.1 AA basics (semantic
    landmarks, labels, contrast ≥4.5:1, focus-visible, aria-live for the
    stream).
@@ -51,8 +56,13 @@
    Playwright E2E file passes **in isolation**
    (`uv run pytest tests/e2e/test_<story>.py -v --no-cov`).
 10. **Amply log, visibly feed back:** server-side, log the required
-    per-turn line (PLAN §9); UI-side, honor the "never stale" feedback
-    contract (PLAN §7.4).
+    per-turn line (PLAN §9) **and** the per-response byte-budget line
+    (`app/core/response_budget.py` — one
+    `api method=… path=… status=… bytes=… enc=… duration_ms=…` line per
+    `/api/*` response, INFO at/over `BOR_LOG_LARGE_RESPONSE_BYTES`, read it
+    with `grep ' :: api '`, audit it with
+    `uv run python -m scripts.api_payload_audit`, phase 167); UI-side,
+    honor the "never stale" feedback contract (PLAN §7.4).
 
 ## Quick reference
 ```bash
@@ -66,4 +76,5 @@ uv run playwright install chromium           # once
 uv run pytest tests/e2e/test_smoke.py -v --no-cov
 uv run ruff check . && uv run pyright        # lint + types
 uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt   # retrieval regression battery (phases 119/162)
+uv run python -m scripts.api_payload_audit                      # endpoint byte audit vs the 256 KB budget (phase 167; app must be running)
 ```