phase: 118_summary_seed_context
**Phase 118 final verification pass — complete.** All criteria verified; 4 pre-existing defects found and fixed.
- **Verified:** summary-seed wiring (`select_suggested` top-5 no-floor → summary blocks, no full text in HIGH prompt), all-doc markdown summaries + NULL backfill (`summary_backfilled`, no `sources_meta` bump), `read` adds full text with `read_docs`-only dedupe, `done.sources` = suggested+read / durable record = suggested+related+read + `suggested=N` log line (seen live in E2E), byte-locked PERSONA/LOW/TOOLS_SECTION, battery gate PASS recorded in `TOOL_CALLING_TESTING.md` §10 (turbo 2026-09-16: 1/2/4 GREEN, cond-3 reported 9/10 per A7, contract 21/21, caps 0).
- **Defects fixed (all pre-existing, none phase-118):** ① `ChatMessage` schema missing the phase-113 `related` key → `extra="forbid"` 422'd every done-time auto-save of grounded turns with a related tier, leaving `message_count=1` (root cause of `test_share_chat` 3F; browser-level instrumentation proved the PUT 422) — added the field + unit/integration pins; ② `test_theme_semantic_completion` pins stale vs phase-117 debox (border/chip removed) — re-targeted to assert border/chip *absence*; ③ `test_header_consistency` `<26`px pin red on 26.125px native date-input line — bound relaxed to `<34` (wrap-detection intent kept); ④ `test_navbar_refresh` bor.chat.v1 key set updated for `related`.
- **Test/lint/coverage:** `uv run pytest --cov=app --cov-report=term-missing` → **2506 passed, app/ 99%** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors.
- **E2E:** new story suite in isolation → **2 passed**; full 103-suite matrix sweep (each isolated) → **all 103 green** after the fixes; `test_share_chat` 4 passed, `test_theme_semantic_completion` 8 passed, `test_header_consistency` 3 passed, `test_navbar_refresh` 7 passed.
- **Deviations:** none from LOCKED decisions. Note: orphaned diagnostic uvicorn processes briefly made E2E sessions exercise stale code — killed and re-verified; a sweep-regenerated tracked screenshot was restored. No commits made (harness commits).
- **Completion criteria:** all 7 ✅ (commit/phase-move is the harness's step).
- **Next pending phase:** none — `todo/` holds only this phase's overview pending the harness move.
This commit is contained in:
@@ -46,9 +46,14 @@ Test → contract mapping (six tests, one per contract bullet):
|
||||
the ``Indexed`` badge (D8 — the date at the top of the clicked
|
||||
document), with the source/format/indexed/chunks badges intact.
|
||||
4. ``test_old_correct_beats_new_similar`` — THE OWNER SCENARIO end to
|
||||
end: the real retriever + the DEFAULT recency boost (0.0007 / 365 d)
|
||||
over the mock's token-overlap embeddings ranks the OLDER correct
|
||||
document as the first cited source over the newer similar one.
|
||||
end (phase 118, A4): the real retriever + the DEFAULT recency boost
|
||||
(0.0007 / 365 d) over the mock's token-overlap embeddings ranks the
|
||||
OLDER correct document first in the suggested tier — the chip row IS
|
||||
the suggested tier (top-5, NO floor; this four-doc KB therefore
|
||||
chips ALL four docs in rank order: retention, draft, forward,
|
||||
old-doc) and the related row is absent (no rank-6+ doc exists).
|
||||
The newer similar doc (the boost's intended beneficiary) stays
|
||||
second — the boost never lets it outrank the one that answers.
|
||||
5. ``test_date_edit_and_sync_preserves`` — the admin-only editor in
|
||||
the real UI: set → Save → the badge re-renders from the RESPONSE
|
||||
(never the optimistic input) → the API round-trips; a re-import
|
||||
@@ -69,9 +74,12 @@ deterministic for fixed text — measured with
|
||||
``app.rag.retriever._vector_candidates`` / ``_lexical_candidates`` /
|
||||
``retrieve()`` against a real Postgres + the mock:
|
||||
|
||||
* ``retention.md`` (correct, 2020) — rank 1 in BOTH lists (cosine
|
||||
0.6222; the lexical tsquery ``how|did|i|configure|backup|retention|
|
||||
policy`` after stopword removal matches it most densely).
|
||||
* ``retention.md`` (correct, 2020) — rank 1 in BOTH lists (content
|
||||
chunk cosine 0.6222; the lexical tsquery ``how|did|i|configure|
|
||||
backup|retention|policy`` after stopword removal matches it most
|
||||
densely). Phase 118 (A2): its EMBEDDED summary is a retrieval
|
||||
candidate too (best-chunk cosine 0.7133 — the seeded summary ranks,
|
||||
which is the point of seeding it).
|
||||
* ``retention-draft.md`` (similar, now) — rank 4 in the vector list
|
||||
(cosine 0.1443) and rank 2 in the lexical list. Its wording was
|
||||
tuned for exactly this: it shares ONLY the three "backup retention
|
||||
@@ -85,14 +93,18 @@ deterministic for fixed text — measured with
|
||||
(0.1875) rank 2–3 in the vector list (unrelated content) and match
|
||||
the tsquery not at all.
|
||||
|
||||
Fused (RRF k=60) + the default boost (0.0007 · exp(−age/365d)):
|
||||
retention.md 1/61+1/61 = 0.0327878 (+ ≈ 0, 6.7 half-lives old) vs
|
||||
retention-draft.md 1/64+1/62 = 0.0317540 (+ the FULL zero-age 0.0007
|
||||
= 0.0324540) → the older correct doc wins by 0.000334 WITH the boost
|
||||
on (it would win by 0.001034 with the boost off — the scenario holds
|
||||
both ways; the boost never lets the newer similar doc outrank the one
|
||||
that answers the question). ``select_documents`` (top-2) cites
|
||||
retention.md first, the draft second. The boost defaults are owned by
|
||||
Fused (RRF k=60) + the default boost (0.0007 · exp(−age/365d)),
|
||||
probe-verified against the current candidate set (the embedded summary
|
||||
chunks join the walk — phase 118 A2): retention.md 0.032523 vs
|
||||
retention-draft.md 0.031498 (the full zero-age boost included) → the
|
||||
older correct doc wins by ≈ 0.001025 WITH the boost on (the margin
|
||||
only grew once the summary chunks ranked — the scenario holds both
|
||||
ways; the boost never lets the newer similar doc outrank the one that
|
||||
answers the question). The phase-118 suggested tier (top-5, NO floor)
|
||||
carries all four docs in that rank order — retention.md first, the
|
||||
draft second, then forward.md (0.016325) and old-doc.md (0.015874)
|
||||
(no floor filters the unrelated docs); the related tier is EMPTY
|
||||
(no rank-6+ doc in a four-doc KB). The boost defaults are owned by
|
||||
task 07 — untouched here.
|
||||
|
||||
DB isolation: every test TRUNCATEs the KB tables (the
|
||||
@@ -529,11 +541,14 @@ def test_old_correct_beats_new_similar(
|
||||
) -> None:
|
||||
"""``How did I configure the backup retention policy?`` → grounded,
|
||||
and the FIRST cited source is the OLDER correct doc (2020) — the
|
||||
NEWER similar one (now, "under review") is cited second. The real
|
||||
hybrid retriever + the DEFAULT recency boost (0.0007 / 365 d) over
|
||||
the mock's token-overlap embeddings (the module docstring records
|
||||
the measured fused scores: 0.0327878 vs 0.0324540 — margin
|
||||
0.000334 WITH the full zero-age boost on the newer doc)."""
|
||||
NEWER similar one (now, "under review") is cited second. Phase 118
|
||||
(A4): the chip row IS the suggested tier (top-5, NO floor) — this
|
||||
four-doc KB chips ALL four docs in rank order — and the related row
|
||||
is absent (no rank-6+ doc). The real hybrid retriever + the DEFAULT
|
||||
recency boost (0.0007 / 365 d) over the mock's token-overlap
|
||||
embeddings (the module docstring records the measured fused scores:
|
||||
0.032523 vs 0.031498 — margin ≈ 0.001025 WITH the full zero-age
|
||||
boost on the newer doc)."""
|
||||
_reset_db(mock_llm, dates_tree)
|
||||
page.set_default_timeout(30_000)
|
||||
login(page, app_url, next="/") # phase 79: chat is require_user-gated
|
||||
@@ -548,22 +563,34 @@ def test_old_correct_beats_new_similar(
|
||||
# top_score 0.6222 ≥ the e2e threshold 0.30).
|
||||
expect(page.locator(".msg.brain.is-deflected")).to_have_count(0)
|
||||
|
||||
# The OLDER correct doc is the FIRST source chip; the NEWER similar
|
||||
# one (the boost's intended beneficiary) is cited — but second.
|
||||
# Phase 118 (A4): the chip row IS the suggested tier — top-5, NO
|
||||
# floor, and this KB has exactly four docs, so ALL four are
|
||||
# suggested (chipped) in fused rank order: the OLDER correct doc
|
||||
# first, the NEWER similar one (the boost's intended beneficiary)
|
||||
# second, then the two unrelated docs (no floor filters them — the
|
||||
# LLM decides what the summaries earn).
|
||||
chips = page.locator(".msg.brain .source-chip")
|
||||
expect(chips).to_have_count(2, timeout=30_000)
|
||||
expect(chips).to_have_count(4, timeout=30_000)
|
||||
assert chips.nth(0).inner_text() == f"{dates_tree.name}/{RETENTION_MD}"
|
||||
assert chips.nth(1).inner_text() == f"{dates_tree.name}/{DRAFT_MD}"
|
||||
assert chips.nth(2).inner_text() == f"{dates_tree.name}/{FORWARD_MD}"
|
||||
assert chips.nth(3).inner_text() == f"{dates_tree.name}/{OLDDOC_MD}"
|
||||
# No rank-6+ doc exists in this four-doc KB → the related row is
|
||||
# absent (the de-emphasized row renders only when it has entries).
|
||||
expect(page.locator(".msg.brain .related-docs")).to_have_count(0)
|
||||
|
||||
# Durable record: one row, grounded, both docs cited in rank order.
|
||||
# Durable record: one row, grounded, the FULL retrieval (suggested
|
||||
# tier + related + read, deduped — here: all four docs) in rank
|
||||
# order (LOCKED A3).
|
||||
with SessionLocal() as db:
|
||||
row = db.scalars(select(QueryLog)).one()
|
||||
assert row.question == QUESTION
|
||||
assert row.deflected is False
|
||||
assert row.top_score >= 0.30 # the e2e mock-calibrated threshold
|
||||
assert (row.fts_hits or 0) >= 1
|
||||
assert f"{dates_tree.name}/{RETENTION_MD}" in row.sources
|
||||
assert row.sources.index(RETENTION_MD) < row.sources.index(DRAFT_MD)
|
||||
assert row.sources == ", ".join(
|
||||
f"{dates_tree.name}/{p}" for p in (RETENTION_MD, DRAFT_MD, FORWARD_MD, OLDDOC_MD)
|
||||
), row.sources
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -728,12 +755,20 @@ def test_anonymous_gate_and_editor_a11y(
|
||||
" === document.querySelector('#doc-modal .doc-date-input')"
|
||||
)
|
||||
# Keyboard traversal runs through the editor's controls (the modal
|
||||
# focus trap): Tab from Save → Cancel, and Tab from Revert wraps to
|
||||
# the panel's first control (the trap's edge behavior). (The CDP
|
||||
# key dispatch of headless Chromium does NOT perform the native
|
||||
# focus move off <input type="date"> itself — a harness artifact,
|
||||
# not a product defect: the same Tab works from every text input
|
||||
# and button in the editor, pinned here through the buttons.)
|
||||
# focus trap — the visible focusable order is "Full page" → Close →
|
||||
# the content's controls, in DOM order): Tab from Save → Cancel,
|
||||
# Tab from Revert → the summary editor's button (phase 118, A2: the
|
||||
# markdown doc is summarized TOO, so the modal's summary-edit
|
||||
# button is visible and joins the trap after the date editor — it
|
||||
# did not exist in the focusable set when this pin was written,
|
||||
# which is why the wrap below is now reached from IT), and Tab from
|
||||
# the summary editor wraps to the panel's FIRST control, the
|
||||
# "Full page" link (the trap's edge behavior the pin originally
|
||||
# carried). (The CDP key dispatch of headless Chromium does NOT
|
||||
# perform the native focus move off <input type="date"> itself — a
|
||||
# harness artifact, not a product defect: the same Tab works from
|
||||
# every text input and button in the editor, pinned here through
|
||||
# the buttons.)
|
||||
page.evaluate("() => document.querySelector('#doc-modal .doc-date-save').focus()")
|
||||
page.keyboard.press("Tab")
|
||||
assert page.evaluate(
|
||||
@@ -742,6 +777,12 @@ def test_anonymous_gate_and_editor_a11y(
|
||||
)
|
||||
page.evaluate("() => document.querySelector('#doc-modal .doc-date-revert').focus()")
|
||||
page.keyboard.press("Tab")
|
||||
assert page.evaluate(
|
||||
"() => !!document.activeElement"
|
||||
" && document.activeElement.classList.contains('doc-summary-edit')"
|
||||
)
|
||||
page.evaluate("() => document.querySelector('#doc-modal .doc-summary-edit').focus()")
|
||||
page.keyboard.press("Tab")
|
||||
assert page.evaluate(
|
||||
"() => !!document.activeElement"
|
||||
" && document.activeElement.classList.contains('doc-modal-open')"
|
||||
|
||||
Reference in New Issue
Block a user