phase: 118_summary_seed_context
**Phase 118 final verification pass — complete.** All criteria verified; 4 pre-existing defects found and fixed.
- **Verified:** summary-seed wiring (`select_suggested` top-5 no-floor → summary blocks, no full text in HIGH prompt), all-doc markdown summaries + NULL backfill (`summary_backfilled`, no `sources_meta` bump), `read` adds full text with `read_docs`-only dedupe, `done.sources` = suggested+read / durable record = suggested+related+read + `suggested=N` log line (seen live in E2E), byte-locked PERSONA/LOW/TOOLS_SECTION, battery gate PASS recorded in `TOOL_CALLING_TESTING.md` §10 (turbo 2026-09-16: 1/2/4 GREEN, cond-3 reported 9/10 per A7, contract 21/21, caps 0).
- **Defects fixed (all pre-existing, none phase-118):** ① `ChatMessage` schema missing the phase-113 `related` key → `extra="forbid"` 422'd every done-time auto-save of grounded turns with a related tier, leaving `message_count=1` (root cause of `test_share_chat` 3F; browser-level instrumentation proved the PUT 422) — added the field + unit/integration pins; ② `test_theme_semantic_completion` pins stale vs phase-117 debox (border/chip removed) — re-targeted to assert border/chip *absence*; ③ `test_header_consistency` `<26`px pin red on 26.125px native date-input line — bound relaxed to `<34` (wrap-detection intent kept); ④ `test_navbar_refresh` bor.chat.v1 key set updated for `related`.
- **Test/lint/coverage:** `uv run pytest --cov=app --cov-report=term-missing` → **2506 passed, app/ 99%** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors.
- **E2E:** new story suite in isolation → **2 passed**; full 103-suite matrix sweep (each isolated) → **all 103 green** after the fixes; `test_share_chat` 4 passed, `test_theme_semantic_completion` 8 passed, `test_header_consistency` 3 passed, `test_navbar_refresh` 7 passed.
- **Deviations:** none from LOCKED decisions. Note: orphaned diagnostic uvicorn processes briefly made E2E sessions exercise stale code — killed and re-verified; a sweep-regenerated tracked screenshot was restored. No commits made (harness commits).
- **Completion criteria:** all 7 ✅ (commit/phase-move is the harness's step).
- **Next pending phase:** none — `todo/` holds only this phase's overview pending the harness move.
This commit is contained in:
+113
-13
@@ -35,12 +35,16 @@
|
||||
ranked documents (up to ``BOR_RELATED_MAX_DOCS``) become the related
|
||||
tier.
|
||||
|
||||
The product requirement (LOCKED A7, revised 2026-08-24): the LLM receives
|
||||
the **entire relevant document**, not just the chunk — chunk hits map back
|
||||
to their parents, dedupe, rank by best fused score, and the full text of
|
||||
the top-N documents is always fed through, never truncated. If a future KB
|
||||
ever makes the prompt too large for the model, the ``LLMError`` → SSE
|
||||
``error`` path surfaces it loudly — no silent partial context.
|
||||
The product requirement (A7, re-revised by the phase-118 owner directive,
|
||||
LOCKED A6, 2026-09-15): the retrieval path seeds **summaries** — the
|
||||
suggestion tier (:func:`select_suggested`, top-N distinct documents, no
|
||||
cosine floor, LOCKED A3) whose summary blocks are the grounded prompt's
|
||||
``<documents>`` starting points. The full text of a document enters the
|
||||
context ONLY through the agent's capped ``read`` tool
|
||||
(:mod:`app.rag.agent`; ``BOR_READ_MAX_CHARS`` + :data:`TRUNCATION_MARKER`),
|
||||
never through the retrieval seeding. A document's content itself is still
|
||||
carried on its rows byte-identical — the ``read`` tool serves it whole,
|
||||
un-truncated up to its cap.
|
||||
|
||||
Deterministic tie-break for equal fused scores:
|
||||
``(−fused, −cosine, document.path, chunk.position)``.
|
||||
@@ -61,9 +65,12 @@ from sqlalchemy.orm import Session
|
||||
from app.config import get_settings
|
||||
from app.models import Chunk, Document
|
||||
|
||||
#: Shared overflow marker — now used by the steering (<tuning>) section
|
||||
#: only (phase 15; imported by ``app.rag.prompts``). The document context
|
||||
#: path never truncates (A7 revised, owner permission 2026-08-24).
|
||||
#: Shared overflow marker (phase 15; imported by ``app.rag.prompts``)
|
||||
#: — used by the steering (<tuning>) section, the phase-118 NULL-summary
|
||||
#: suggestion preview fallback (A5), and the capped agent ``read``
|
||||
#: result. The seeded summary blocks and the ``read``-served document
|
||||
#: content never truncate silently (A6 re-revised): full text enters the
|
||||
#: context only through the capped ``read`` tool.
|
||||
TRUNCATION_MARKER = "[…truncated…]"
|
||||
|
||||
#: Alphanumeric tokens of a question (``to_tsquery`` input, OR-joined),
|
||||
@@ -574,6 +581,11 @@ def select_documents_tiered(
|
||||
"""Tier chunk hits into the cited and the related parent documents
|
||||
(phase 113, LOCKED A2/A4 — the usefulness bar).
|
||||
|
||||
Phase 118 retired the full-text seeding role (A6); the suggested
|
||||
tier (:func:`select_suggested`) seeds the prompt now — this helper
|
||||
stays as a dormant public helper (env back-compat for the settings
|
||||
it was calibrated by).
|
||||
|
||||
Distinct parent documents are ranked exactly like :func:`select_documents`
|
||||
(best fused score first — the same stable score-descending walk, so a
|
||||
document's rank position is fixed by its FIRST seen chunk) and each
|
||||
@@ -603,7 +615,11 @@ def select_documents_tiered(
|
||||
|
||||
The returned rows carry the full document content, byte-identical —
|
||||
a matched parent document is **never truncated** (A7 revised, owner
|
||||
permission 2026-08-24).
|
||||
permission 2026-08-24; A6 re-revised 2026-09-15: the retrieval path
|
||||
seeds SUMMARIES — the cited tier's full texts no longer ride the
|
||||
grounded prompt, full text enters the context only through the
|
||||
capped ``read`` tool; the rows themselves still carry the whole
|
||||
content).
|
||||
"""
|
||||
top_n = n if n is not None else get_settings().top_n_docs
|
||||
no_bar = floor <= 0.0
|
||||
@@ -642,12 +658,20 @@ def select_documents(
|
||||
) -> list[Document]:
|
||||
"""Map chunk hits to distinct parent documents, ranked by best fused score.
|
||||
|
||||
Phase 118 retired the full-text seeding role (A6); the suggested
|
||||
tier (:func:`select_suggested`) seeds the prompt now — this helper
|
||||
stays as a dormant public helper (env back-compat for the settings
|
||||
it was calibrated by).
|
||||
|
||||
At most *n* documents are returned (default ``BOR_TOP_N_DOCS``). The
|
||||
returned rows carry the full document content, byte-identical — a
|
||||
matched parent document is **never truncated** (A7 revised, owner
|
||||
permission 2026-08-24). There is deliberately no context budget: an
|
||||
oversized prompt must fail loudly through the ``LLMError`` → SSE
|
||||
``error`` path, never arrive as silent partial context.
|
||||
permission 2026-08-24; A6 re-revised 2026-09-15: the seeded prompt
|
||||
now carries SUMMARIES — the full text reaches the context only
|
||||
through the capped ``read`` tool, not through this selection). There
|
||||
is deliberately no context budget: an oversized prompt must fail
|
||||
loudly through the ``LLMError`` → SSE ``error`` path, never arrive
|
||||
as silent partial context.
|
||||
|
||||
Phase 113: a thin wrapper on :func:`select_documents_tiered` — the
|
||||
legacy "any score, top-N" behavior is the cited tier with a zero
|
||||
@@ -656,3 +680,79 @@ def select_documents(
|
||||
"""
|
||||
cited, _ = select_documents_tiered(chunks, n, 0.0, 0)
|
||||
return cited
|
||||
|
||||
|
||||
def select_suggested(
|
||||
chunks: Sequence[RetrievedChunk],
|
||||
n: int | None = None,
|
||||
) -> list[Document]:
|
||||
"""Top-N distinct parent documents in fused rank order — the phase-118
|
||||
"start here" suggestion tier (LOCKED A3), with NO cosine floor.
|
||||
|
||||
Distinct parent documents are walked in the SAME stable score-
|
||||
descending order as :func:`select_documents_tiered` (a document's rank
|
||||
position is fixed by its FIRST seen chunk; dedupe by ``document.id``),
|
||||
and at most *n* of them are returned (default the
|
||||
``BOR_SUGGESTED_DOCS`` setting, 5). Unlike the phase-113 cited tier,
|
||||
the usefulness bar NEVER filters here: a lexical-only hit with
|
||||
cosine 0.0 is suggested when it ranks. Suggestions are opt-in
|
||||
starting points, not citations — the seeded prompt carries the
|
||||
document's summary, and the LLM decides whether to extend its context
|
||||
by reading the document's full text.
|
||||
|
||||
The returned rows carry the full document content, byte-identical —
|
||||
the content is what the agent's ``read`` tool serves later (never
|
||||
truncated; A6 re-revises A7: full text enters the context only through
|
||||
the capped ``read`` tool).
|
||||
"""
|
||||
top_n = n if n is not None else get_settings().suggested_docs
|
||||
|
||||
order: list[Document] = []
|
||||
seen: set[uuid.UUID] = set()
|
||||
for rc in sorted(chunks, key=lambda c: c.score, reverse=True):
|
||||
if len(order) >= top_n:
|
||||
break
|
||||
doc = rc.document
|
||||
if doc.id in seen:
|
||||
continue
|
||||
seen.add(doc.id)
|
||||
order.append(doc)
|
||||
return order
|
||||
|
||||
|
||||
def select_related(
|
||||
chunks: Sequence[RetrievedChunk],
|
||||
excluded_ids: set[uuid.UUID],
|
||||
cap: int,
|
||||
) -> list[Document]:
|
||||
"""The documents ranked AFTER *excluded_ids* — the phase-118 related
|
||||
tier (rank 6+ for the contiguous top-5 suggestion set), up to *cap*
|
||||
(``BOR_RELATED_MAX_DOCS``).
|
||||
|
||||
The SAME stable score-descending walk as
|
||||
:func:`select_documents_tiered` / :func:`select_suggested` (a
|
||||
document's rank position is fixed by its FIRST seen chunk; dedupe by
|
||||
``document.id``), skipping every document whose id is in
|
||||
*excluded_ids* and admitting at most *cap* documents. There is NO
|
||||
cosine floor: the related tier is the ranked remainder (a lexical-
|
||||
only cosine 0.0 hit is included) — its job on the ``done`` frame is
|
||||
visibility (the UI's de-emphasized "nearby docs" row), not
|
||||
citation. With the turn wiring's exclusion — exactly the suggested
|
||||
tier's document ids (LOCKED A3: a contiguous top-N, no floor) —
|
||||
"excluding the suggested" is exactly "rank 6+".
|
||||
|
||||
The returned rows carry the full document content, byte-identical
|
||||
(the tier is metadata for the ``done`` frame and the durable
|
||||
record; the prompt and ``read`` contract are untouched).
|
||||
"""
|
||||
out: list[Document] = []
|
||||
seen: set[uuid.UUID] = set()
|
||||
for rc in sorted(chunks, key=lambda c: c.score, reverse=True):
|
||||
if len(out) >= cap:
|
||||
break
|
||||
doc = rc.document
|
||||
if doc.id in seen or doc.id in excluded_ids:
|
||||
continue
|
||||
seen.add(doc.id)
|
||||
out.append(doc)
|
||||
return out
|
||||
|
||||
Reference in New Issue
Block a user