phase: 164_chat_error_honesty
Build and Push Containers / build-and-push-app (push) Successful in 2m11s
Build and Push Containers / build-and-push-db (push) Successful in 29s

All gates green. Report:

- **Verified (phase 164, all 4 tasks already in `complete/`)**: `HistoryTurn`/`ChatMessage`/`ChatThinkingSegment` caps → 600 000, `history`/saved-messages lists → 500, new `SavedChatCreate/Update` total-8 000 000-char validator; exported `errorDetailText` at the two previously unguarded chat paths (pre-stream `!res.ok` + SSE `error` frame); `FAILED_REQUEST_TEXT` + `bubbleText` threading + phase-114 banner hint; dedicated E2E `tests/e2e/test_chat_error_honesty.py`.
- **Final-pass additions**: `tests/integration/test_chat_api.py::test_history_rejects_one_char_over_the_text_cap` (endpoint-level >600k → 422 with the pydantic list-of-objects `msg` — the criterion's upper half was only unit-pinned); README env rows for `BOR_APP_VERSION` + `BOR_SESSION_COOKIE`; restored `test_document_viewer.py`'s tracked screenshot artifact dirtied by the E2E run (via read-only `git show`, no index/history touched).
- **Tests/lint (exact commands)**: `uv run pytest --cov=app --cov-report=term-missing` → **3837 passed**, `app/` coverage **99%** (>90%); `uv run pytest tests/e2e/test_chat_error_honesty.py -v --no-cov` → **4 passed in isolation**; per-file loop over **all 148** E2E files (`uv run pytest tests/e2e/<f> -q --no-cov`) → **148 × exit 0**, incl. `test_failed_turn_retry`, `test_chat_save_payload_limits`, `test_edit_and_resend`, `test_llm_history`, `test_smoke`; `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- **Completion criteria**: pre-stream 422 shows the distinct line + verbatim `msg`, `[object Object]` unrepresentable + no `input` in DOM — **met** (unit totality/source pins + E2E asserts 1–2); 525k history accepted (200 settles) / >600k → 422 / saved chat 201 + 8M bound — **met**; transport + SSE-frame settles keep `FAILED_TURN_TEXT`, stop/abort untouched, §7.4 re-enables — **met**; budget trim, `message` 4000, the 7 guarded detail-readers and all SSE shapes byte-identical (diff touches only caps + the two chat paths) — **met**; README/AGENTS full review + proxy deploy note — **met**; commit/`complete/` move = pipeline gate.
- **Docs**: README reviewed in full (env tables cross-checked programmatically: 74 `BOR_*` rows vs `Settings` defaults — no mismatches; every `.env.example` var documented; referenced scripts/pages/endpoints/`query_log` columns exist) — found & fixed the 2 undocumented real knobs above; the phase's payload-limits section, failed-turn copy and `client_max_body_size ≥ 16 MB` deploy note verified accurate. `AGENTS.md` verified consistent (rules, gitignore list, quick-reference commands, referenced paths all correct) — no changes needed.
- **Deviations**: `ChatThinkingSegment.text` raised 32 000 → 600 000 too (not itemized in A1, but its own docstring mirrors `HistoryTurn.thinking`; without it a long-thinking turn 422s its own auto-save — same mirror invariant). No live-infra changes made (the proxy limit lives in the separate deployment repo; README note only).
- **Next pending phase**: none — `164_chat_error_honesty` was the only entry in `todo/`.
This commit is contained in:
2026-10-07 18:00:12 -04:00
parent 6ba46e735a
commit 34d60efbc1
42 changed files with 2601 additions and 84 deletions
+86 -3
View File
@@ -2204,18 +2204,101 @@ def test_history_rejects_unknown_who(client, db) -> None:
assert r.status_code == 422
def test_history_rejects_more_than_100_entries(client, db) -> None:
"""Schema pin: the DoS sanity ceiling is 100 turns — 101 is a 422
def test_history_rejects_more_than_500_entries(client, db) -> None:
"""Schema pin: the DoS sanity ceiling is 500 turns (phase 164, locked
A1 — raised from 100 because the client sends the WHOLE
``bor.chat.v1`` record minus the current question) — 501 is a 422
(the config budgets do the real trimming; this only keeps a
pathological body from wasting the mapper's work)."""
r = client.post(
"/api/chat",
json={
"message": "hi",
"history": [{"who": "user", "text": f"q{i}"} for i in range(101)],
"history": [{"who": "user", "text": f"q{i}"} for i in range(501)],
},
)
assert r.status_code == 422
assert r.json()["detail"][0]["loc"] == ["body", "history"]
def test_history_rejects_one_char_over_the_text_cap(client, db) -> None:
"""The upper half of the phase-164 boundary at the ENDPOINT (task 01,
locked A1): one character past the 600 000 per-message cap is a 422 —
and the ``detail`` FastAPI hands the browser is the pydantic LIST OF
OBJECTS (``loc``/``msg``), the exact shape the client's serializer
must read ``msg`` from (the live "[object Object]"). The 4000-char
``message`` cap is unchanged (the composer mirrors it client-side)."""
r = client.post(
"/api/chat",
json={"message": "hi", "history": [{"who": "brain", "text": "a" * 600_001}]},
)
assert r.status_code == 422
(entry,) = r.json()["detail"]
assert entry["loc"] == ["body", "history", 0, "text"]
assert entry["type"] == "string_too_long"
assert entry["msg"] == "String should have at most 600000 characters"
def test_history_accepts_a_full_length_prior_answer(
client,
db,
seeded_kb: FakeRagLLM,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Phase 164 task 01 — the live 422. The turn AFTER a full-length
answer (525k chars ≈ a 131 072-token generation) sends that answer
back as ``history``; the 32 000 ceiling rejected it, so the
follow-up died with "[object Object]". It is now ACCEPTED: the
stream opens (200) and settles with ``done``, no 422 anywhere.
What the model SEES is the locked budget trim's business (A4, not
touched here): a turn over ``history_max_chars`` is dropped WHOLE,
so this giant turn trims itself out of the prompt and
``history_msgs=0`` — the follow-up works with a trimmed window.
The acceptance is what this pin owns."""
long_answer = "a" * 525_000
history = [
{"who": "user", "text": "Write me the whole k3s runbook."},
{"who": "brain", "text": long_answer},
]
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
try:
caplog.set_level(logging.INFO, logger="app.chat")
frames = _stream_chat_with_history(client, "And the rollback part?", history)
finally:
fastapi_app.dependency_overrides.clear()
assert frames[-1]["type"] == "done" # accepted: the turn settled normally
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
assert lines and "history_msgs=0" in lines[-1]
(messages,) = seeded_kb.seen_messages
assert messages[1] == {"role": "user", "content": "And the rollback part?"}
def test_history_accepts_a_full_length_answer_bounded_by_the_char_budget(
client,
db,
seeded_kb: FakeRagLLM,
) -> None:
"""The same request with the giant answer NOT the newest turn: the
newer small turns fit ``history_max_chars`` and reach the model,
while the 525k turn is dropped WHOLE (locked A3/A4 — never cut,
never a 422). Proves the raise changed what ARRIVES, not what the
trim keeps."""
history = [
{"who": "brain", "text": "a" * 525_000},
{"who": "user", "text": "Short follow-up question."},
]
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
try:
_stream_chat_with_history(client, "And the rollback part?", history)
finally:
fastapi_app.dependency_overrides.clear()
(messages,) = seeded_kb.seen_messages
# The giant turn is gone whole; the fitting newer turn is kept.
assert {"role": "user", "content": "Short follow-up question."} in messages
assert all(len(m["content"]) < 525_000 for m in messages)
def test_done_event_serializes_column_maximum_source_refs() -> None: