phase: 164_chat_error_honesty
All gates green. Report: - **Verified (phase 164, all 4 tasks already in `complete/`)**: `HistoryTurn`/`ChatMessage`/`ChatThinkingSegment` caps → 600 000, `history`/saved-messages lists → 500, new `SavedChatCreate/Update` total-8 000 000-char validator; exported `errorDetailText` at the two previously unguarded chat paths (pre-stream `!res.ok` + SSE `error` frame); `FAILED_REQUEST_TEXT` + `bubbleText` threading + phase-114 banner hint; dedicated E2E `tests/e2e/test_chat_error_honesty.py`. - **Final-pass additions**: `tests/integration/test_chat_api.py::test_history_rejects_one_char_over_the_text_cap` (endpoint-level >600k → 422 with the pydantic list-of-objects `msg` — the criterion's upper half was only unit-pinned); README env rows for `BOR_APP_VERSION` + `BOR_SESSION_COOKIE`; restored `test_document_viewer.py`'s tracked screenshot artifact dirtied by the E2E run (via read-only `git show`, no index/history touched). - **Tests/lint (exact commands)**: `uv run pytest --cov=app --cov-report=term-missing` → **3837 passed**, `app/` coverage **99%** (>90%); `uv run pytest tests/e2e/test_chat_error_honesty.py -v --no-cov` → **4 passed in isolation**; per-file loop over **all 148** E2E files (`uv run pytest tests/e2e/<f> -q --no-cov`) → **148 × exit 0**, incl. `test_failed_turn_retry`, `test_chat_save_payload_limits`, `test_edit_and_resend`, `test_llm_history`, `test_smoke`; `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - **Completion criteria**: pre-stream 422 shows the distinct line + verbatim `msg`, `[object Object]` unrepresentable + no `input` in DOM — **met** (unit totality/source pins + E2E asserts 1–2); 525k history accepted (200 settles) / >600k → 422 / saved chat 201 + 8M bound — **met**; transport + SSE-frame settles keep `FAILED_TURN_TEXT`, stop/abort untouched, §7.4 re-enables — **met**; budget trim, `message` 4000, the 7 guarded detail-readers and all SSE shapes byte-identical (diff touches only caps + the two chat paths) — **met**; README/AGENTS full review + proxy deploy note — **met**; commit/`complete/` move = pipeline gate. - **Docs**: README reviewed in full (env tables cross-checked programmatically: 74 `BOR_*` rows vs `Settings` defaults — no mismatches; every `.env.example` var documented; referenced scripts/pages/endpoints/`query_log` columns exist) — found & fixed the 2 undocumented real knobs above; the phase's payload-limits section, failed-turn copy and `client_max_body_size ≥ 16 MB` deploy note verified accurate. `AGENTS.md` verified consistent (rules, gitignore list, quick-reference commands, referenced paths all correct) — no changes needed. - **Deviations**: `ChatThinkingSegment.text` raised 32 000 → 600 000 too (not itemized in A1, but its own docstring mirrors `HistoryTurn.thinking`; without it a long-thinking turn 422s its own auto-save — same mirror invariant). No live-infra changes made (the proxy limit lives in the separate deployment repo; README note only). - **Next pending phase**: none — `164_chat_error_honesty` was the only entry in `todo/`.
This commit is contained in:
@@ -2204,18 +2204,101 @@ def test_history_rejects_unknown_who(client, db) -> None:
|
||||
assert r.status_code == 422
|
||||
|
||||
|
||||
def test_history_rejects_more_than_100_entries(client, db) -> None:
|
||||
"""Schema pin: the DoS sanity ceiling is 100 turns — 101 is a 422
|
||||
def test_history_rejects_more_than_500_entries(client, db) -> None:
|
||||
"""Schema pin: the DoS sanity ceiling is 500 turns (phase 164, locked
|
||||
A1 — raised from 100 because the client sends the WHOLE
|
||||
``bor.chat.v1`` record minus the current question) — 501 is a 422
|
||||
(the config budgets do the real trimming; this only keeps a
|
||||
pathological body from wasting the mapper's work)."""
|
||||
r = client.post(
|
||||
"/api/chat",
|
||||
json={
|
||||
"message": "hi",
|
||||
"history": [{"who": "user", "text": f"q{i}"} for i in range(101)],
|
||||
"history": [{"who": "user", "text": f"q{i}"} for i in range(501)],
|
||||
},
|
||||
)
|
||||
assert r.status_code == 422
|
||||
assert r.json()["detail"][0]["loc"] == ["body", "history"]
|
||||
|
||||
|
||||
def test_history_rejects_one_char_over_the_text_cap(client, db) -> None:
|
||||
"""The upper half of the phase-164 boundary at the ENDPOINT (task 01,
|
||||
locked A1): one character past the 600 000 per-message cap is a 422 —
|
||||
and the ``detail`` FastAPI hands the browser is the pydantic LIST OF
|
||||
OBJECTS (``loc``/``msg``), the exact shape the client's serializer
|
||||
must read ``msg`` from (the live "[object Object]"). The 4000-char
|
||||
``message`` cap is unchanged (the composer mirrors it client-side)."""
|
||||
r = client.post(
|
||||
"/api/chat",
|
||||
json={"message": "hi", "history": [{"who": "brain", "text": "a" * 600_001}]},
|
||||
)
|
||||
assert r.status_code == 422
|
||||
(entry,) = r.json()["detail"]
|
||||
assert entry["loc"] == ["body", "history", 0, "text"]
|
||||
assert entry["type"] == "string_too_long"
|
||||
assert entry["msg"] == "String should have at most 600000 characters"
|
||||
|
||||
|
||||
def test_history_accepts_a_full_length_prior_answer(
|
||||
client,
|
||||
db,
|
||||
seeded_kb: FakeRagLLM,
|
||||
caplog: pytest.LogCaptureFixture,
|
||||
) -> None:
|
||||
"""Phase 164 task 01 — the live 422. The turn AFTER a full-length
|
||||
answer (525k chars ≈ a 131 072-token generation) sends that answer
|
||||
back as ``history``; the 32 000 ceiling rejected it, so the
|
||||
follow-up died with "[object Object]". It is now ACCEPTED: the
|
||||
stream opens (200) and settles with ``done``, no 422 anywhere.
|
||||
|
||||
What the model SEES is the locked budget trim's business (A4, not
|
||||
touched here): a turn over ``history_max_chars`` is dropped WHOLE,
|
||||
so this giant turn trims itself out of the prompt and
|
||||
``history_msgs=0`` — the follow-up works with a trimmed window.
|
||||
The acceptance is what this pin owns."""
|
||||
long_answer = "a" * 525_000
|
||||
history = [
|
||||
{"who": "user", "text": "Write me the whole k3s runbook."},
|
||||
{"who": "brain", "text": long_answer},
|
||||
]
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
|
||||
try:
|
||||
caplog.set_level(logging.INFO, logger="app.chat")
|
||||
frames = _stream_chat_with_history(client, "And the rollback part?", history)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
assert frames[-1]["type"] == "done" # accepted: the turn settled normally
|
||||
lines = [r.getMessage() for r in caplog.records if "question=" in r.getMessage()]
|
||||
assert lines and "history_msgs=0" in lines[-1]
|
||||
(messages,) = seeded_kb.seen_messages
|
||||
assert messages[1] == {"role": "user", "content": "And the rollback part?"}
|
||||
|
||||
|
||||
def test_history_accepts_a_full_length_answer_bounded_by_the_char_budget(
|
||||
client,
|
||||
db,
|
||||
seeded_kb: FakeRagLLM,
|
||||
) -> None:
|
||||
"""The same request with the giant answer NOT the newest turn: the
|
||||
newer small turns fit ``history_max_chars`` and reach the model,
|
||||
while the 525k turn is dropped WHOLE (locked A3/A4 — never cut,
|
||||
never a 422). Proves the raise changed what ARRIVES, not what the
|
||||
trim keeps."""
|
||||
history = [
|
||||
{"who": "brain", "text": "a" * 525_000},
|
||||
{"who": "user", "text": "Short follow-up question."},
|
||||
]
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: seeded_kb
|
||||
try:
|
||||
_stream_chat_with_history(client, "And the rollback part?", history)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
(messages,) = seeded_kb.seen_messages
|
||||
# The giant turn is gone whole; the fitting newer turn is kept.
|
||||
assert {"role": "user", "content": "Short follow-up question."} in messages
|
||||
assert all(len(m["content"]) < 525_000 for m in messages)
|
||||
|
||||
|
||||
def test_done_event_serializes_column_maximum_source_refs() -> None:
|
||||
|
||||
Reference in New Issue
Block a user