Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Move phase 167 audit to complete
Build and Push Containers / build-and-push-app (push) Successful in 14s
Details
Build and Push Containers / build-and-push-db (push) Successful in 11s
Details

Browse Source
This commit is contained in:
ducoterra
2026-10-09 10:27:51 -04:00
parent a2b2f283fd
commit a12fdcdf0a
1 changed files with 0 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/todo/167_api_payload_diet/AUDIT.md
-236
View File
@@ -1,236 +0,0 @@
# Phase 167 · Task 05 — Measured API payload audit (the byte budget)
Owner TODO L5: "GET `/api/docs/tree` loads 2 MB in production … Fix that **any any
other endpoints** that could load a ton of data." This file is the measured answer —
every number below is pasted from `scripts/api_payload_audit.py` (or the `curl` /
server-log captures), never hand-written. LOCKED A5: the budget is a log line plus a
report, never a 413 — nothing was rejected, nothing was capped.
## How it was run
| | |
|---|---|
| Date | 2026-10-08 |
| Corpus | the **1,000-document live-replica snapshot** (`data/bor_live_kb_replica.dump.sql`, the `restore-test-db` skill) — fingerprint verified `docs=1000`, `chunks=8866`, `sources=deploy,homelab,homepage,ServMon,skills,vibe-bot`; the snapshot's `alembic=0011` was upgraded to head `0027` afterwards (the phase-119/162 precedent) |
| App | `uv run uvicorn app.main:app --port 8000` (local dev, this working tree — tasks 01-04 of this phase included) |
| BEFORE run | the same app with `BOR_COMPRESS_MIN_BYTES=0` — the gzip layer is **not added at all**, i.e. byte-for-byte the transport the pre-phase app had |
| AFTER run | the same app with the shipped defaults (`BOR_COMPRESS_MIN_BYTES=860`, depth default 1, ETag on the three catalogue reads) |
| Command | `uv run python -m scripts.api_payload_audit --base-url http://localhost:8000` (budget default `262144`) |
| Budget log capture | `BOR_LOG_LARGE_RESPONSE_BYTES=0` (the loud mode — every `/api/*` response at INFO), used only for the log excerpts below |
The script is **read-only**: it signs in through `POST /api/login` (the session
cookie — the single-admin login is password-only, PLAN A10, so there is no
`--username`) and GETs. It never mutates, so it is safe against a production target
(`--base-url https://…`).
### Corpus caveat (read this before comparing with the 2 MB report)
The 2026-09-05 snapshot carries **0 `folder_summaries` rows and 0 image documents**
(verified: `select count(*) from folder_summaries` → `0`,
`select count(*) from documents where is_image` → `0`). The production 2 MB payload
was 1,000 documents **plus** the hundreds of inlined AI folder descriptions
(phase 127/155) and the image documents' full vision descriptions (phase 122/124) —
none of which exist in the snapshot. **So the identity bytes below are a lower bound
on the production payload**, and what transfers from this measurement is the
*structure*, i.e. the ratios: the depth limit cut the tree by **31×** (it ships the
ONE level the RAG view renders instead of the whole recursion below it — and every
level of a summary-carrying folder is where the production payload's fat lives),
gzip bought a further **5.5× to 20×** across these payloads (**7.5×** on the depth-1
tree, **11×** on the full tree), and a repeat load costs **0 bytes**. The verdict
table below names the live re-run as the follow-up.
## BEFORE — no gzip middleware (`BOR_COMPRESS_MIN_BYTES=0`), full tree available
> Read the BEFORE table's `gzip` column as "no compressor existed": ratio
> `1.000` on every row, the route's bytes ARE the transfer. Its `304` verdicts
> are NOT a pre-phase property — the conditional re-requests hit the validators
> tasks 01-04 added, which this run left in place because only the transport was
> being switched off (the honest way to measure "the same bytes, uncompressed").
> Pre-phase there were no validators at all, so every repeat load cost the
> identity column again.
```
api_payload_audit: base=http://localhost:8000 budget=262,144 bytes (LOOK threshold)
api_payload_audit: corpus documents=1000 chunks_total=8866 sources=6 tree totals documents=1000 chunks=8866 saved_chats=0 biggest_doc=tests/fixtures/advisory/bodhi_exiv2.json
endpoint identity gzip ratio ms verdict
----------------------------------------------------------------------------------------------------------------
tree depth=1 (default) 8,474 8,474 1.000 12 304
tree depth=8 (full) 265,147 265,147 1.000 37 304
tree node homelab/active 20,844 20,844 1.000 11 304
docs catalogue 246,366 246,366 1.000 14 304
document content (biggest: tests/fixtures/advisory/bodhi_exiv2.json) 892,124 892,124 1.000 8 LOOK (> 262144)
note: content field 772573 chars
saved chats list 12 12 1.000 1 ok
saved chat (one) — — — — n/a
note: no saved chats
git sources 188 188 1.000 1 ok
suggestions 74 74 1.000 1 ok
sync status (polled) 272 272 1.000 1 ok
tokens 13 13 1.000 2 ok
config 188 188 1.000 1 ok
whoami 37 37 1.000 1 ok
api_payload_audit: ms is the gzip (real-client) variant; identity is the route's own bytes — the pair the ' :: api ' log line reports.
```
What the pre-phase browser actually paid for the **Sources page first paint** is the
sum of the rows it fetches on load, and the pre-phase tree route answered the FULL
recursive tree by default (`tree depth=8 (full)` is that shape):
`265,147 + 246,366 + 188 + 188 + 37 = 511,926 bytes identity`, and the pre-phase app
published **no validators at all**, so every repeat load re-paid the same 512 KB.
## AFTER — shipped defaults (depth-1 default + gzip + ETag)
api_payload_audit: base=http://localhost:8000 budget=262,144 bytes (LOOK threshold)
api_payload_audit: corpus documents=1000 chunks_total=8866 sources=6 tree totals documents=1000 chunks=8866 saved_chats=0 biggest_doc=tests/fixtures/advisory/bodhi_exiv2.json
endpoint identity gzip ratio ms verdict
----------------------------------------------------------------------------------------------------------------
tree depth=1 (default) 8,474 1,129 0.133 13 304
tree depth=8 (full) 265,147 23,520 0.089 41 304
tree node homelab/active 20,844 1,064 0.051 11 304
docs catalogue 246,366 45,107 0.183 23 304
document content (biggest: tests/fixtures/advisory/bodhi_exiv2.json) 892,124 60,776 0.068 14 LOOK (> 262144)
note: content field 772573 chars
saved chats list 12 12 1.000 2 ok
saved chat (one) — — — — n/a
note: no saved chats
git sources 188 188 1.000 1 ok
suggestions 74 74 1.000 2 ok
sync status (polled) 272 272 1.000 1 ok
tokens 13 13 1.000 2 ok
config 188 188 1.000 1 ok
whoami 37 37 1.000 1 ok
api_payload_audit: ms is the gzip (real-client) variant; identity is the route's own bytes — the pair the ' :: api ' log line reports.
```
Sources-page first paint, same sum against the AFTER gzip column:
`1,129 + 45,107 + 188 + 188 + 37 = 46,649 bytes on the wire` (from 511,926 identity,
and every one of those five rows revalidates to **304 / 0 bytes** on the next load).
The tree itself — the endpoint TODO L5 named — goes **265,147 → 1,129 = 235×**
(31× from the depth limit × 7.5× from gzip), and the second visit is free.
## `curl` proof on the audited corpus
```
$ curl -s -H 'Accept-Encoding: identity' -D- -o b1.json http://localhost:8000/api/docs/tree
etag: W/"53c86d5d554efd8487ad52ca6cd6f913"
cache-control: private, must-revalidate
content-length: 8474
vary: Cookie, Accept-Encoding
8474 bytes # wc -c b1.json
$ curl -s -H 'Accept-Encoding: gzip' -D- -o b2.gz http://localhost:8000/api/docs/tree
etag: W/"53c86d5d554efd8487ad52ca6cd6f913" # the validator survives the encoding
content-encoding: gzip
1129 bytes # wc -c b2.gz
$ curl -s -H 'If-None-Match: W/"53c86d5d554efd8487ad52ca6cd6f913"' -o /dev/null \
-w 'status=%{http_code} size=%{size_download}\n' http://localhost:8000/api/docs/tree
status=304 size=0
```
## The byte-budget log line (the answer to "which endpoint is heavy, right now")
Captured with `BOR_LOG_LARGE_RESPONSE_BYTES=0` (every `/api/*` response at INFO);
the shipped default is `65536`, so at rest only the big ones reach the log.
`grep ' :: api ' ` isolates them. `bytes=` is what the ROUTE produced, `enc=` what
went on the wire — the two together are the "how fat, and how fat did the client see
it" pair.
```
INFO app.response_budget :: api method=GET path=/api/docs/tree status=200 bytes=8474 enc=identity duration_ms=14
INFO app.response_budget :: api method=GET path=/api/docs/tree status=200 bytes=8474 enc=gzip duration_ms=12
INFO app.response_budget :: api method=GET path=/api/docs/tree status=304 bytes=0 enc=identity duration_ms=11
INFO app.response_budget :: api method=GET path=/api/docs/tree status=200 bytes=265147 enc=identity duration_ms=34
INFO app.response_budget :: api method=GET path=/api/docs/tree status=200 bytes=265147 enc=gzip duration_ms=40
INFO app.response_budget :: api method=GET path=/api/docs/tree/node status=200 bytes=20844 enc=gzip duration_ms=10
INFO app.response_budget :: api method=GET path=/api/docs status=200 bytes=246366 enc=gzip duration_ms=22
INFO app.response_budget :: api method=GET path=/api/documents/content status=200 bytes=892124 enc=gzip duration_ms=12
INFO app.response_budget :: api method=GET path=/api/sync/status status=200 bytes=272 enc=identity duration_ms=0
INFO app.response_budget :: api method=GET path=/api/whoami status=200 bytes=37 enc=identity duration_ms=0
```
`POST /api/chat` (SSE, A15) is not in the audit table — a stream has no bounded
payload to budget — and it is skipped by the gzip layer by exact path (which is why
its line reads `enc=identity`). One real turn against this corpus, measured:
```
INFO app.response_budget :: api method=POST path=/api/chat status=200 bytes=39527 enc=identity duration_ms=45002 stream=1
```
39,527 bytes of SSE frames streamed, `duration_ms` the stream's life (this capture
ended because the measuring `curl --max-time 45` hung up — which is itself the
other half of the contract: a stream the client walks away from still writes
exactly ONE line, with the bytes that actually got out, never one line per frame
and never a missing line). Pinned by
`tests/unit/test_response_budget.py::test_a_cancelled_stream_still_logs_one_line_with_what_got_out`
and by the integration pin
`tests/integration/test_chat_api.py::test_chat_turn_logs_one_budget_line_with_the_stream_flag`.
## Verdict per candidate endpoint
| Endpoint | Measured (identity / wire) | Verdict |
|---|---|---|
| `GET /api/docs/tree` (default `depth=1`) | 265,147 → **8,474** identity, **1,129** gzip, repeat **304 / 0** | **FIXED** — the TODO L5 defect. Depth-limited default (task 02) + selective gzip (task 01) + ETag (task 03). Phase criterion "< 40 KB gzipped" met at 1,129 B with a 1,000-document corpus. |
| `GET /api/docs/tree?depth=8` | 265,147 / 23,520 gzip, repeat 304 / 0 | **FIXED (bounded by the route's own contract)** — the explicit full tree stays available for tooling and the migrated date suites; it compresses 11× and revalidates to 0. Nothing asks for it in the UI anymore (`grep` of `frontend/` shows only `depth=1` + the node route). |
| `GET /api/docs/tree/node` (the drill) | 20,844 / 1,064 gzip, repeat 304 / 0 | **FIXED** — one level per click (task 03/04), so the level the RAG view renders is ~1 KB on the wire instead of a re-ship of the corpus. |
| `GET /api/docs` (the catalogue) | 246,366 / 45,107 gzip, repeat 304 / 0 | **FIXED within scope**, and a **watch item**: 246 KB identity is 6 KB under the 256 KB LOOK bar. It already carries the ETag pair (task 03) and gzips 5.5×. Follow-up if the corpus grows: `depth`-style paging or a per-source filter param — named, not applied (A5: raise nothing, reject nothing). |
| `GET /api/documents/content` (biggest doc, 772 KB JSON) | 892,124 / 60,776 gzip | **ACCEPTED with why** — one document, on demand, when the owner opens it from a citation; the wire cost is 60 KB gzipped, and LOCKED A5 says the app's own documents are legitimately large. **Follow-up (named, not applied):** task 03 recorded this route as deliberately un-validated; if the `:: api ` log shows repeat 200s for the same document, give it the ETag pair keyed on `(source, path, content_hash)` (the row is already loaded, so the hash is free). |
| `GET /api/chats` | 12 (empty corpus) | **ok** — one row per saved chat, no payloads in the list (phase 50 shape). Re-measure on the live corpus with the saved chats. |
| `GET /api/chats/{id}` | n/a — `no saved chats` in the snapshot | **ACCEPTED with why** — the owner opening ONE saved conversation; the payload is that chat's raw `bor.chat.v1` JSONB, which is exactly what the page must render. Not measurable on this snapshot (the replica carries 0 saved chats) — **follow-up**: re-run this script against the live app, where the chats exist. |
| `GET /api/git-sources` | 188 / 188 (under the 860 floor → identity, correctly) | **ok** — the registry is a handful of rows; no remedy needed, and task 05 item 6's conditional ETag is therefore NOT applied (it grows past the budget only with hundreds of sources). |
| `GET /api/suggestions` | 74 | **ok** — three opening questions (phase 103). |
| `GET /api/sync/status` | 272 | **ACCEPTED with why (polled)** — tiny, but the UI polls it every 2 s, which is precisely why the byte-budget line exists: the log shows a chatty poll, and 272 B says the poll is not a payload problem. |
| `GET /api/tokens` | 13 | **ok** — secret-free list, one row per token (phase 79 A4). |
| `GET /api/config` | 188 | **ok** — display strings only. |
| `GET /api/whoami` | 37 | **ok** — two fields. |
| `POST /api/chat` (SSE) | not auditable as a payload (skipped by gzip, A15) | **UNCHANGED by design** — the log line covers it with ` stream=1`; frame-by-frame delivery is pinned by `tests/unit/test_compression.py` and `tests/integration/test_chat_api.py`. |
## What the audit changed, and what it proved
* **No new remedy was needed beyond tasks 01-04.** The only over-budget row
(`/api/documents/content`) is the documented accepted case, and
`/api/git-sources` measures 188 bytes, so task 05 item 6's conditional ETag
was **not** applied — a validator on a sub-860-byte response costs a hash
and saves nothing (it would not even compress).
* **The browser confirms the wire.** `tests/e2e/test_kb_tree_lazy_load.py` (task 06)
drives the real Sources view in Chromium and pins what this file measured: the
boot is ONE `?depth=1` request whose body carries `totals` and no deep children,
the response arrives `Content-Encoding: gzip` with `Accept-Encoding` in `Vary`
(Chromium's own negotiation — no stub), a reload re-sends `If-None-Match` and the
wire answers `304` for a fraction of the first transfer while the table still
renders from the cache, and a level the payload only names costs exactly one
`/api/docs/tree/node` fetch (zero on every re-entry). Because Chromium hands
`fetch` the cached 200 on a revalidation, that last pair is asserted on the CDP
Network events — the DevTools-panel truth, not Playwright's post-cache view.
* **The depth limit is where the win is, not gzip.** depth-1 vs depth-8 on
the same corpus is 8,474 vs 265,147 identity bytes (**31×**) — gzip is the
second ~11× on top of it (265,147 → 23,520), and the ETag makes the repeat
load free. Had the phase shipped gzip alone, the Sources page would still
have paid 23 KB/request forever, and on the summary-inflated production
corpus proportionally more.
* **`/api/docs` is the next one to watch** (246 KB identity, 6 KB under the
LOOK bar, already validated + compressed). The byte-budget line at its
default `65536` threshold logs it at INFO on every uncached load, so the
corpus growth that pushes it over shows up in the log before it shows up
for the owner.
* **Two limits of this measurement, stated instead of hidden:** (1) the
snapshot has no folder summaries and no image docs, so the identity
numbers are a lower bound on the production payload (see the caveat
above); (2) it has no saved chats, so `/api/chats/{id}` is `n/a` rather
than measured — the script is read-only by design (it must stay safe
against the deployed app), so it does not invent a chat to size. Both are
re-measurable with one command against the live corpus.
## Re-running this audit
```bash
# 1. the corpus (destructive to the local DB — see the skill's own backup step)
# restore-test-db skill: data/bor_live_kb_replica.dump.sql → brain_of_reese
uv run alembic upgrade head # the snapshot carries an old alembic version
# 2. the app
uv run uvicorn app.main:app --port 8000
# 3. the measurement (BOR_ADMIN_PASSWORD from .env; --password to override)
uv run python -m scripts.api_payload_audit --base-url http://localhost:8000
# the BEFORE transport, for a before/after pair like the tables above:
BOR_COMPRESS_MIN_BYTES=0 uv run python -m scripts.api_payload_audit
```