phase: 118_summary_seed_context
Build and Push Containers / build-and-push-app (push) Successful in 2m2s
Build and Push Containers / build-and-push-db (push) Successful in 14s

**Phase 118 final verification pass — complete.** All criteria verified; 4 pre-existing defects found and fixed.

- **Verified:** summary-seed wiring (`select_suggested` top-5 no-floor → summary blocks, no full text in HIGH prompt), all-doc markdown summaries + NULL backfill (`summary_backfilled`, no `sources_meta` bump), `read` adds full text with `read_docs`-only dedupe, `done.sources` = suggested+read / durable record = suggested+related+read + `suggested=N` log line (seen live in E2E), byte-locked PERSONA/LOW/TOOLS_SECTION, battery gate PASS recorded in `TOOL_CALLING_TESTING.md` §10 (turbo 2026-09-16: 1/2/4 GREEN, cond-3 reported 9/10 per A7, contract 21/21, caps 0).
- **Defects fixed (all pre-existing, none phase-118):** ① `ChatMessage` schema missing the phase-113 `related` key → `extra="forbid"` 422'd every done-time auto-save of grounded turns with a related tier, leaving `message_count=1` (root cause of `test_share_chat` 3F; browser-level instrumentation proved the PUT 422) — added the field + unit/integration pins; ② `test_theme_semantic_completion` pins stale vs phase-117 debox (border/chip removed) — re-targeted to assert border/chip *absence*; ③ `test_header_consistency` `<26`px pin red on 26.125px native date-input line — bound relaxed to `<34` (wrap-detection intent kept); ④ `test_navbar_refresh` bor.chat.v1 key set updated for `related`.
- **Test/lint/coverage:** `uv run pytest --cov=app --cov-report=term-missing` → **2506 passed, app/ 99%** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors.
- **E2E:** new story suite in isolation → **2 passed**; full 103-suite matrix sweep (each isolated) → **all 103 green** after the fixes; `test_share_chat` 4 passed, `test_theme_semantic_completion` 8 passed, `test_header_consistency` 3 passed, `test_navbar_refresh` 7 passed.
- **Deviations:** none from LOCKED decisions. Note: orphaned diagnostic uvicorn processes briefly made E2E sessions exercise stale code — killed and re-verified; a sweep-regenerated tracked screenshot was restored. No commits made (harness commits).
- **Completion criteria:** all 7 ✅ (commit/phase-move is the harness's step).
- **Next pending phase:** none — `todo/` holds only this phase's overview pending the harness move.
This commit is contained in:
2026-09-16 06:57:49 -04:00
parent 21aad84a6d
commit 9820c361b0
80 changed files with 4690 additions and 1302 deletions
+44 -32
View File
@@ -97,12 +97,13 @@ task 04):
context; ``grep`` — which searches the whole document — is the
follow-up), with the truncation recorded on the holder so the loop
yields a :class:`app.rag.llm.ToolResultPiece` (task 02 → the SSE
``tool_result`` frame + UI marker). **A7 scope clarification:** the
never-truncated contract is for the retrieval ``<documents>`` path
(the top-2 seed documents stay whole — "this should never happen");
the ``read`` TOOL path is the only capped read, per the owner's
explicit request — the two paths are distinct (retrieval seeds vs.
agent-requested additions). And ``grep`` greps the
``tool_result`` frame + UI marker). **A6 re-revised contract
(phase 118, owner directive 2026-09-15):** the retrieval
``<documents>`` path seeds SUMMARIES only — a suggested document's
full text never enters the prompt on the retrieval path; full text
enters the context ONLY through this ``read`` TOOL path, which is
the only capped read (the phase-95 cap unchanged). And ``grep``
greps the
indexed documents (or the one document a combined ``source/path``
names) for a case-insensitive fixed substring and returns up to 20
``source/path:line: text`` match lines (owner-locked A5, phase 68),
@@ -138,7 +139,9 @@ task 04):
:data:`NOT_A_FOLDER`, the drill-down teaching with the argument
echoed and the deepest existing ancestor's direct subfolders
listed, so the model self-corrects in the next round; a document
already in context (seed or previously read) →
already READ into full-text context (phase 118: the suggested
seeds are summary blocks in the prompt, not full text — only an
already-read document is refused) →
:data:`ALREADY_IN_CONTEXT` (phase 72, task 05 gate iteration:
the line names the correct action — answer from the text already
in the prompt, do not call read again — so a fired refusal ends
@@ -301,19 +304,21 @@ AGENT_TOOLS: list[dict[str, Any]] = [
"function": {
"name": "read",
"description": (
"Do not call this tool for a document already shown in "
"the <documents> section, even when the user asks you to "
"open or read it — its full text is already in your "
"prompt; answer directly from it. Use it only to add a "
"document NOT already in <documents> to your context, "
"by its combined `source/path` string. Very large "
"documents are truncated: you receive the first part "
"plus a TRUNCATED notice naming how many more characters "
"exist — the notice is authoritative, the document did "
"NOT end where it stopped. Follow it and use `grep` "
"(pattern) to locate the rest — it searches the whole "
"document. Call one tool at a time — wait for this "
"result before your next call."
"The <documents> section shows the SUMMARIES of the "
"top-ranked documents — their full texts are NOT in "
"your prompt yet. Use this tool to add one of them (or "
"any other document) to your context, by its combined "
"`source/path` string, exactly as shown in the `ls` "
"output or the <documents> blocks. Do not re-read a "
"document you have already read — its full text is "
"already in your prompt. Very large documents are "
"truncated: you receive the first part plus a TRUNCATED "
"notice naming how many more characters exist — the "
"notice is authoritative, the document did NOT end "
"where it stopped. Follow it and use `grep` (pattern) "
"to locate the rest — it searches the whole document. "
"Call one tool at a time — wait for this result before "
"your next call."
),
"parameters": {
"type": "object",
@@ -326,10 +331,9 @@ AGENT_TOOLS: list[dict[str, Any]] = [
"shown in the `ls` output (e.g. "
"'homelab/active/container_caddy/caddy.md'). "
"A bare document path (without the source "
"name) will not resolve. Only pass a document "
"NOT already shown in the <documents> "
"section — it is already in your context; do "
"not re-read it."
"name) will not resolve. Do not re-read a "
"document you have already read — its full "
"text is already in your prompt."
),
}
},
@@ -1152,10 +1156,15 @@ def _execute_tool(
arg = raw_path.strip() if isinstance(raw_path, str) else ""
if not arg:
return MISSING_READ_ARGS
known = {(doc.source, doc.path) for doc in (*seed_docs, *holder.read_docs)}
# Phase 118 (A6): the dedupe set is ``holder.read_docs`` ONLY —
# the ``seed_docs`` are SUMMARY blocks in the prompt, not full
# text, so a FIRST read of a suggested document adds its full
# text through the path below; only a document ALREADY READ is
# refused.
known = {(doc.source, doc.path) for doc in holder.read_docs}
# The dedupe check needs no DB: the split pair of a combined
# identity that is in context is in `known` as-is (the resolve
# below would find the same document).
# identity that is already in full-text context is in `known`
# as-is (the resolve below would find the same document).
if "/" in arg:
src, _, p = arg.partition("/")
if (src, p) in known:
@@ -1326,11 +1335,14 @@ async def run_agent(
the capture mechanism for new registry entries; *holder* accumulates
the turn's ``scaffold_stripped`` total for the API layer's log line.
``seed_docs`` are the documents the retrieval already put in context
(they shape the *system_prompt* the caller built); re-reading one of
them is rejected with :data:`ALREADY_IN_CONTEXT` (the phase-72
teaching line — answer from the text already in the prompt) — the
rejection counts in nothing, but it still consumes a round.
``seed_docs`` are the suggested documents whose SUMMARY blocks the
caller put in the *system_prompt* (phase 118: the retrieval seeds
summaries, never full texts); reading one of them ADDS its full
text to the context through the ordinary ``read`` path — only a
document ALREADY READ is rejected with :data:`ALREADY_IN_CONTEXT`
(the phase-72 teaching line — answer from the text already in the
prompt); the rejection counts in nothing, but it still consumes a
round.
DB sessions (SEC-14-04): *db_factory* is a callable that returns a
new :class:`sqlalchemy.orm.Session` (e.g. ``lambda: SessionLocal()``).
+62 -12
View File
@@ -10,7 +10,8 @@ the two-phase upsert:
2. embed the new chunks in batches and attach the vectors
3. commit — one transaction per file, so a failed embedding leaves the
database untouched and the file is simply retried on the next run
4. non-markdown files only (phase 30): generate a ``lite``-model summary
4. every file (phase 30; phase 118, A2: markdown included — the
non-markdown-only scope is retired): generate a ``lite``-model summary
and, best-effort, store it on ``documents.summary`` plus one extra
embedded chunk (``is_summary``, position −1). The document row and its
content chunks are already committed at this point, so a summary
@@ -42,6 +43,17 @@ guard) and counted in ``summary.dates_updated`` — unless the row
carries the owner's manual correction (``created_at_manual``, D1), which
the sync never touches.
NULL-summary backfill (phase 118, A2): an UNCHANGED file (same
``content_hash``) whose ``documents.summary`` is still NULL — a
pre-phase-30 row, or an earlier fail-soft miss — gets the same
best-effort summary pass on every sync until it sticks. A success counts
``summary_backfilled`` (never ``summaries``) and touches nothing else: no
content re-embed, no added/updated/pruned count — so no
``sources_meta`` bump, no KB-overview/folder-summary regeneration. The
backfill runs BEFORE the ``created_at_manual`` early-return (the manual
flag protects the DATE only, D1) and the strict ``is None`` check leaves
owner-set summaries (even empty strings, phase 57) alone.
``import_sources`` accepts an optional per-file ``progress`` callback
(phase 64, task 01) reporting the file being processed right now.
"""
@@ -99,12 +111,18 @@ class ImportSummary:
errors: int = 0
chunks: int = 0
embed_batches: int = 0
#: Non-markdown files whose lite summary was generated + indexed
#: (phase 30). One ``is_summary`` chunk per success.
#: Files whose lite summary was generated + indexed (phase 30; phase
#: 118, A2: every A9 format, markdown included). One ``is_summary``
#: chunk per success.
summaries: int = 0
#: Non-markdown files whose summary generation failed (best-effort —
#: the document is still indexed, without a summary).
#: Files whose summary generation failed (best-effort — the document
#: is still indexed, without a summary).
summary_errors: int = 0
#: Unchanged docs whose NULL summary was backfilled (phase 118, A2) —
#: one ``is_summary`` chunk per success; the content is untouched, so
#: a backfill NEVER counts added/updated/pruned (no
#: ``sources_meta`` bump, no overview/folder-summary regeneration).
summary_backfilled: int = 0
#: Files whose ``created_at`` was refreshed on the UNCHANGED path —
#: content untouched, date re-sourced (phase 106, D4: the date may
#: go OLDER; a date-only refresh NEVER counts added/updated/pruned,
@@ -125,7 +143,7 @@ class ImportSummary:
logger.info(
"import: summary files=%d added=%d updated=%d unchanged=%d pruned=%d "
"errors=%d chunks=%d embed_batches=%d summaries=%d summary_errors=%d "
"dates_updated=%d formats=%s",
"summary_backfilled=%d dates_updated=%d formats=%s",
self.files,
self.added,
self.updated,
@@ -136,6 +154,7 @@ class ImportSummary:
self.embed_batches,
self.summaries,
self.summary_errors,
self.summary_backfilled,
self.dates_updated,
self.format_counts(),
)
@@ -442,6 +461,20 @@ async def _index_file(
if doc is not None and doc.content_hash == digest:
summary.unchanged += 1
logger.info("import: unchanged source=%s path=%s", source, rel)
# Phase 118 (A2): an unchanged doc whose summary is still NULL
# (a pre-phase-30 row, or an earlier fail-soft miss) gets a
# summary-only backfill — one ``is_summary`` chunk, no content
# re-embed, and NEVER an added/updated/pruned count (so no
# ``sources_meta`` bump, no overview/folder-summary
# regeneration). Strict ``is None``: an empty-string summary is
# owner-set (phase 57) and is never overwritten. BEFORE the
# manual-date early-return: ``created_at_manual`` protects the
# DATE only (phase 106, D1), not the summary.
if doc.summary is None:
await _store_summary(
session, doc=doc, source=source, rel=rel, content=content,
llm=llm, summary=summary, backfill=True,
)
if doc.created_at_manual:
# D1/D4: the owner's correction survives the sync — no
# write at all (the phase-97 ``manually_edited`` precedent).
@@ -536,10 +569,11 @@ async def _index_file(
summary.chunks += len(chunks_text)
logger.info("import: %s source=%s path=%s chunks=%d", verb, source, rel, len(chunks_text))
# Phase 30: markdown is already natural language, so only the other A9
# formats (txt, yaml, yml, json, py) get a ``lite``-model summary.
if full_path.suffix.lower() in (".md", ".markdown"):
return
# Phase 30; phase 118 (A2, 2026-09-15): EVERY new/changed document
# gets a ``lite``-model summary — markdown included. Phase 30's
# "markdown is already natural language" exclusion is retired: the
# summary is the retrieval seed context (the phase-118 suggestion
# blocks), not a formatting convenience.
await _store_summary(
session, doc=doc, source=source, rel=rel, content=content, llm=llm, summary=summary
)
@@ -554,6 +588,7 @@ async def _store_summary(
content: str,
llm: Embedder,
summary: ImportSummary,
backfill: bool = False,
) -> None:
"""Best-effort ``lite`` summary for one already-committed document.
@@ -569,6 +604,11 @@ async def _store_summary(
:class:`EmbeddingError` only rolls back the summary rows — the file
stays indexed, without a summary, and the failure is counted in
``summary_errors`` (PLAN phase 30).
``backfill`` (phase 118, A2): the unchanged-doc NULL-summary path —
a success counts ``summary_backfilled`` instead of ``summaries``
(the doc content is untouched, so the import's KB-change signal must
not move); the rest of the mechanics are identical.
"""
try:
text = await generate_summary(llm, source=source, path=rel, content=content)
@@ -588,8 +628,18 @@ async def _store_summary(
# ``expire_on_commit=False`` — reflects the committed state.
doc.chunks.append(chunk)
session.commit()
summary.summaries += 1
logger.info("import: summary source=%s path=%s chars=%d", source, rel, len(text))
if backfill:
# Phase 118 (A2): the backfill counts itself apart from fresh
# imports — the doc content is unchanged, so ``summaries``
# (a KB-change signal) must not move.
summary.summary_backfilled += 1
logger.info(
"import: summary-backfill source=%s path=%s chars=%d",
source, rel, len(text),
)
else:
summary.summaries += 1
logger.info("import: summary source=%s path=%s chars=%d", source, rel, len(text))
except (LLMError, EmbeddingError) as e:
session.rollback()
summary.summary_errors += 1
+3 -2
View File
@@ -1,8 +1,9 @@
"""Async OpenAI-compatible client for the self-hosted aipi endpoint (PLAN A5).
Provides the embeddings surface (importer, retrieval), one-shot chat
completions (phase 30: the ``lite`` model summarizes non-markdown
documents at import time), and chat streaming (PLAN A15) for the RAG
completions (phase 30: the ``lite`` model summarizes documents at import
time — every A9 format, markdown included since phase 118), and chat
streaming (PLAN A15) for the RAG
pipeline. Chat streaming yields typed :class:`StreamPiece` values
(phase 17) and — when the caller passes a ``tools`` list —
:class:`ToolCallPiece` values (phase 37): aipi's ``turbo`` model streams
+3 -2
View File
@@ -100,8 +100,9 @@ def build_overview_prompt(
* ``user`` — one line per document,
``source — path — title — {first line of summary}``, joined with
newlines. The summary field is omitted when the document has no
summary (markdown docs and the fail-soft path — no dangling
dash). The list is capped at *max_chars* (default
summary (pre-phase-30 rows, the fail-soft path, and pre-backfill
NULL rows — no dangling dash). The list is capped at *max_chars*
(default
``BOR_OVERVIEW_INPUT_MAX_CHARS``): overflow is cut exactly at the
cap and the shared ``[…truncated…]`` marker is appended on its own
line, so the model never sees more than the cap and the cut is
+99 -41
View File
@@ -7,7 +7,14 @@ no mandated deflection opening; the honesty gate itself is unchanged.)
Two modes:
* ``HIGH`` — grounded turn: full top-document texts under ``<documents>``.
* ``HIGH`` — grounded turn: the top-ranked documents' SUMMARIES under
``<documents>`` (phase 118, LOCKED A6 re-revising A7 — the "start here"
suggestion seeding: each block is the document's stored summary, never
the full content; the LLM extends its context by ``read``-ing a
document's full text through the capped ``read`` tool, the ONLY full-
text path. A NULL/blank summary — a fail-soft import miss — falls back
to a ``suggestion_preview_chars`` content preview + the shared
``[…truncated…]`` marker; no LLM call at chat time, LOCKED A5).
* ``LOW`` — deflection turn: weak-hit *titles only* plus the
``DEFLECT_MODE`` marker (the E2E mock LLM keys on that marker).
@@ -124,6 +131,24 @@ _KB_INTRO = (
"(generated at import time):\n"
)
#: The ``<documents>`` section's start-here intro (phase 118, task 03 —
#: the owner directive, TODO L3): the seeded blocks are the SUMMARIES of
#: the top-ranked documents for the question, opt-in starting points —
#: start here if one seems right; ``read`` the document's combined
#: ``source/path`` to add its FULL content before answering specifics
#: (the full text is NOT in the prompt until you read it); cite the
#: document(s) you used, by path. Rendered BEFORE the first block, only
#: when at least one block is present. The ``<documents>`` tag and the
#: per-block ``<document>`` markup stay byte-stable around it (the E2E
#: mock's block parser and the ``read``-tool copy key off both).
SUGGEST_INTRO = (
"The blocks below are the summaries of the top-ranked documents for "
"your question — start here if one seems right to you: call `read` "
"with that document's combined `source/path` to add its full content "
"before answering specifics (its full text is not in the prompt until "
"you read it). Cite the document(s) you used, by path."
)
#: The ``<tools>`` instructions section — **HIGH prompt only** (phase 37,
#: task 03; phase 70: the copy is rewritten for the harness-aligned
#: ``ls`` / ``read`` / ``grep`` shapes, names/args exactly as the
@@ -158,12 +183,15 @@ _KB_INTRO = (
#: the user's "open it / read it" and ``read`` seed-context documents,
#: then repeated the refused call; run 2: 8/18, 44% — the repeat is
#: gone, but a grep turn batched two calls per reply and the harness
#: runs only the first of a batch): the do-not-read rule names the
#: user-command scenario (a document already in the ``<documents>``
#: section: do not call ``read``, answer from the text already in the
#: prompt — an anchor on the concrete ``<document path="...">`` markup
#: was tried and REVERTED: it primed the model to latch the seed
#: documents' paths as ``ls`` scopes, regressing the incident turn);
#: runs only the first of a batch): the ``read`` clause carries the
#: phase-118 summary-seed contract (the ``<documents>`` section holds
#: SUMMARIES — a suggested document's full text is not in the prompt
#: until ``read`` adds it; do not re-read a document you have already
#: read — answer from the text already in the prompt; if the user
#: asks to open or read a suggested document, ``read`` it — that is
#: the point of the section — the phase-72 do-not-read rule and the
#: reverted ``<document path="...">`` anchor are retired with the
#: full-text seeds);
#: the one-call rule names the consequence (a batched second call is
#: discarded — runs only the first); the never-repeat rule says why
#: (the refusal already told you the correct form); the ``grep`` clause
@@ -171,10 +199,11 @@ _KB_INTRO = (
#: live runs showed the model scoping ``grep`` with an ``ls``-style
#: source name — the incident shape, but on grep). The behavioral
#: contract lives in the ``AGENT_TOOLS`` descriptions as well (the most
#: local text at call time): ``read`` must not be called for a
#: ``<documents>`` document at all; ``grep`` with only ``pattern``
#: searches the whole knowledge base, and a source name is not a
#: document.
#: local text at call time): ``read`` adds a document's FULL text by
#: its combined ``source/path`` identity — the ``<documents>``
#: summaries are the starting points, not the content; ``grep`` with
#: only ``pattern`` searches the whole knowledge base, and a source
#: name is not a document.
TOOLS_SECTION: str = (
"<tools>\n"
"You may extend your context with three tools. `ls` lists the "
@@ -189,12 +218,16 @@ TOOLS_SECTION: str = (
"`source: X | path: Y | title: Z`; to find one specific document "
"without listing, use `grep`. `read` pulls in one document by its "
"combined `source/path` string, exactly as shown in the `ls` "
"output — including the source name — adding its full content to "
"your context. Do not call `read` for a document already shown in "
"the <documents> section, even when the user asks you to open or "
"read it — its full text is already in your prompt; answer "
"directly from it. For `read`, a bare document path (without the "
"source name) will not resolve. Very large documents are capped: a "
"output — including the source name — or in the <documents> "
"summary blocks — adding its full content to your context. The "
"<documents> section holds SUMMARIES — the full text of a "
"suggested document is not in your prompt until you `read` it. Do "
"not re-read a document you have already read — its full text is "
"already in your prompt; answer directly from it. If the user "
"asks you to open or read a suggested document, `read` it — that "
"is the point of the section. For `read`, a bare document path "
"(without the source name) will not resolve. Very large documents "
"are capped: a "
"cut read returns the first part plus a TRUNCATED notice — the "
"document did not end where it stopped; use `grep` (pattern) to "
"find the rest, it searches the whole document. `grep` locates an exact string "
@@ -355,17 +388,42 @@ def build_kb_section(overview: str, max_chars: int | None = None) -> str:
return ""
def _document_body(doc: Document) -> str:
"""The body of one ``<document>`` suggestion block (phase 118, LOCKED
A6): the document's stored summary (stripped) — NEVER the full
content.
Defensive fallback ONLY when the summary is missing (``None`` or
whitespace — a fail-soft import miss, LOCKED A5): the first
``suggestion_preview_chars`` characters of the content plus the
shared :data:`TRUNCATION_MARKER` on its own line — the settings read
happens on this fallback path ONLY (a prompt built from
summary-bearing docs makes no settings call for the cap). Content at
or under the cap rides whole, unmarked (nothing was cut). No LLM
call at chat time — the preview is deterministic.
"""
summary = (doc.summary or "").strip()
if summary:
return summary
limit = get_settings().suggestion_preview_chars
content = doc.content
if len(content) > limit:
return content[:limit] + "\n" + TRUNCATION_MARKER
return content
def build_high_prompt(
documents: Sequence[Document],
notes: Sequence[str] | None = None,
kb_overview: str | None = None,
) -> str:
"""Grounded turn: locked persona (+ steering, + KB overview) + full
texts of the top documents + the ``<tools>`` instructions (phase 37;
phase 70: the harness-aligned ``ls`` / ``read`` / ``grep`` shapes;
phase 72: the copy states the document-identity contract — the
source-name ``ls`` scope, the combined ``source/path`` identity for
``read``/``grep`` — up front).
"""Grounded turn: locked persona (+ steering, + KB overview) + the
top-ranked documents' SUMMARY blocks + the ``<tools>`` instructions
(phase 37; phase 70: the harness-aligned ``ls`` / ``read`` /
``grep`` shapes; phase 72: the document-identity contract up front;
phase 118, LOCKED A6 re-revising A7: the ``<documents>`` section
seeds SUMMARIES, never full texts — full text enters the context
only through the capped ``read`` tool).
Section order: ``<relevance>`` → ``<knowledge_base>`` → ``<tuning>``
→ ``<documents>`` → ``<tools>``; empty steering/overview omit their
@@ -373,42 +431,42 @@ source-name ``ls`` scope, the combined ``source/path`` identity for
cap — not the prompt — decides whether the tools are actually
offered to the model, see :mod:`app.rag.agent`).
When at least one block is present, the section leads with the
start-here :data:`SUGGEST_INTRO` line (before the first block — the
phase-15 ``_STEERING_INTRO`` / phase-31 ``_KB_INTRO`` precedent): the
blocks are the summaries of the top-ranked documents, ``read`` adds
the full text, and the answer cites the document(s) used by path.
This is NOT the reverted phase-72 in-context reminder (that copy
taught the seed documents as already-read context; this one teaches
the summary-as-starting-point contract the A6 re-revision requires).
Each ``<document>`` block carries the identity attributes
``source`` / ``path`` / ``title`` — and, since phase 106 (D5),
``date`` (the row's ``created_at`` UTC date part, ``YYYY-MM-DD``,
APPENDED after ``title`` — the only position; always present,
``created_at`` is NOT NULL) — plus the document's full text.
Gate-iteration note (task 05, 2026-09-03/04): an in-context reminder
LEADING this section (the document texts are already context — do
not ``read`` one the user asked to open) was tried and REVERTED:
it never flipped the seed-doc reads (15/15 across gate runs 1-5)
and correlated with the incident-turn regression (the model latched
the seed documents' paths as ``ls`` scopes — cap reached on the
"list the files in this directory" turn) whenever the copy named
the ``<document>`` blocks explicitly.
``created_at`` is NOT NULL) — plus the document's summary
(:func:`_document_body`; the NULL/whitespace-summary preview
fallback is LOCKED A5).
"""
# 2026-09-04 (controlled tool-calling fast loop): the do-not-read
# rule for seed documents lives in TOOLS_SECTION and the ``read``
# tool descriptions (the copy levers that stuck — see the gate's
# telemetry in TOOL_CALLING_TESTING.md). A per-block instruction
# attribute at the ``source``/``path`` copy site was TRIED and
# REVERTED the same day (no improvement across runs; the block stays
# exactly the document identity + full text).
# Phase 106 (D5): every document the model sees carries its
# creation date — the block's ``date`` attribute (the row's
# ``created_at`` UTC date part, appended after ``title`` — the
# only position; always present, ``created_at`` is NOT NULL).
# Phase 118 (A6): the block body is the summary — the full content
# stays out of the prompt (the capped ``read`` tool is the only
# full-text path).
blocks = [
f'<document source="{doc.source}" path="{doc.path}" title="{doc.title}" '
f'date="{doc.created_at:%Y-%m-%d}">\n'
f"{doc.content}\n"
f"{_document_body(doc)}\n"
"</document>"
for doc in documents
]
body = "\n\n".join(blocks) if blocks else (
"(no documents matched — do not invent specifics)"
)
if blocks:
body = SUGGEST_INTRO + "\n\n" + body
prompt = _base("HIGH")
for part in (build_kb_section(kb_overview or ""), build_steering_section(notes or [])):
if part:
+113 -13
View File
@@ -35,12 +35,16 @@
ranked documents (up to ``BOR_RELATED_MAX_DOCS``) become the related
tier.
The product requirement (LOCKED A7, revised 2026-08-24): the LLM receives
the **entire relevant document**, not just the chunk — chunk hits map back
to their parents, dedupe, rank by best fused score, and the full text of
the top-N documents is always fed through, never truncated. If a future KB
ever makes the prompt too large for the model, the ``LLMError`` → SSE
``error`` path surfaces it loudly — no silent partial context.
The product requirement (A7, re-revised by the phase-118 owner directive,
LOCKED A6, 2026-09-15): the retrieval path seeds **summaries** — the
suggestion tier (:func:`select_suggested`, top-N distinct documents, no
cosine floor, LOCKED A3) whose summary blocks are the grounded prompt's
``<documents>`` starting points. The full text of a document enters the
context ONLY through the agent's capped ``read`` tool
(:mod:`app.rag.agent`; ``BOR_READ_MAX_CHARS`` + :data:`TRUNCATION_MARKER`),
never through the retrieval seeding. A document's content itself is still
carried on its rows byte-identical — the ``read`` tool serves it whole,
un-truncated up to its cap.
Deterministic tie-break for equal fused scores:
``(−fused, −cosine, document.path, chunk.position)``.
@@ -61,9 +65,12 @@ from sqlalchemy.orm import Session
from app.config import get_settings
from app.models import Chunk, Document
#: Shared overflow marker — now used by the steering (<tuning>) section
#: only (phase 15; imported by ``app.rag.prompts``). The document context
#: path never truncates (A7 revised, owner permission 2026-08-24).
#: Shared overflow marker (phase 15; imported by ``app.rag.prompts``)
#: — used by the steering (<tuning>) section, the phase-118 NULL-summary
#: suggestion preview fallback (A5), and the capped agent ``read``
#: result. The seeded summary blocks and the ``read``-served document
#: content never truncate silently (A6 re-revised): full text enters the
#: context only through the capped ``read`` tool.
TRUNCATION_MARKER = "[…truncated…]"
#: Alphanumeric tokens of a question (``to_tsquery`` input, OR-joined),
@@ -574,6 +581,11 @@ def select_documents_tiered(
"""Tier chunk hits into the cited and the related parent documents
(phase 113, LOCKED A2/A4 — the usefulness bar).
Phase 118 retired the full-text seeding role (A6); the suggested
tier (:func:`select_suggested`) seeds the prompt now — this helper
stays as a dormant public helper (env back-compat for the settings
it was calibrated by).
Distinct parent documents are ranked exactly like :func:`select_documents`
(best fused score first — the same stable score-descending walk, so a
document's rank position is fixed by its FIRST seen chunk) and each
@@ -603,7 +615,11 @@ def select_documents_tiered(
The returned rows carry the full document content, byte-identical —
a matched parent document is **never truncated** (A7 revised, owner
permission 2026-08-24).
permission 2026-08-24; A6 re-revised 2026-09-15: the retrieval path
seeds SUMMARIES — the cited tier's full texts no longer ride the
grounded prompt, full text enters the context only through the
capped ``read`` tool; the rows themselves still carry the whole
content).
"""
top_n = n if n is not None else get_settings().top_n_docs
no_bar = floor <= 0.0
@@ -642,12 +658,20 @@ def select_documents(
) -> list[Document]:
"""Map chunk hits to distinct parent documents, ranked by best fused score.
Phase 118 retired the full-text seeding role (A6); the suggested
tier (:func:`select_suggested`) seeds the prompt now — this helper
stays as a dormant public helper (env back-compat for the settings
it was calibrated by).
At most *n* documents are returned (default ``BOR_TOP_N_DOCS``). The
returned rows carry the full document content, byte-identical — a
matched parent document is **never truncated** (A7 revised, owner
permission 2026-08-24). There is deliberately no context budget: an
oversized prompt must fail loudly through the ``LLMError`` → SSE
``error`` path, never arrive as silent partial context.
permission 2026-08-24; A6 re-revised 2026-09-15: the seeded prompt
now carries SUMMARIES — the full text reaches the context only
through the capped ``read`` tool, not through this selection). There
is deliberately no context budget: an oversized prompt must fail
loudly through the ``LLMError`` → SSE ``error`` path, never arrive
as silent partial context.
Phase 113: a thin wrapper on :func:`select_documents_tiered` — the
legacy "any score, top-N" behavior is the cited tier with a zero
@@ -656,3 +680,79 @@ def select_documents(
"""
cited, _ = select_documents_tiered(chunks, n, 0.0, 0)
return cited
def select_suggested(
chunks: Sequence[RetrievedChunk],
n: int | None = None,
) -> list[Document]:
"""Top-N distinct parent documents in fused rank order — the phase-118
"start here" suggestion tier (LOCKED A3), with NO cosine floor.
Distinct parent documents are walked in the SAME stable score-
descending order as :func:`select_documents_tiered` (a document's rank
position is fixed by its FIRST seen chunk; dedupe by ``document.id``),
and at most *n* of them are returned (default the
``BOR_SUGGESTED_DOCS`` setting, 5). Unlike the phase-113 cited tier,
the usefulness bar NEVER filters here: a lexical-only hit with
cosine 0.0 is suggested when it ranks. Suggestions are opt-in
starting points, not citations — the seeded prompt carries the
document's summary, and the LLM decides whether to extend its context
by reading the document's full text.
The returned rows carry the full document content, byte-identical —
the content is what the agent's ``read`` tool serves later (never
truncated; A6 re-revises A7: full text enters the context only through
the capped ``read`` tool).
"""
top_n = n if n is not None else get_settings().suggested_docs
order: list[Document] = []
seen: set[uuid.UUID] = set()
for rc in sorted(chunks, key=lambda c: c.score, reverse=True):
if len(order) >= top_n:
break
doc = rc.document
if doc.id in seen:
continue
seen.add(doc.id)
order.append(doc)
return order
def select_related(
chunks: Sequence[RetrievedChunk],
excluded_ids: set[uuid.UUID],
cap: int,
) -> list[Document]:
"""The documents ranked AFTER *excluded_ids* — the phase-118 related
tier (rank 6+ for the contiguous top-5 suggestion set), up to *cap*
(``BOR_RELATED_MAX_DOCS``).
The SAME stable score-descending walk as
:func:`select_documents_tiered` / :func:`select_suggested` (a
document's rank position is fixed by its FIRST seen chunk; dedupe by
``document.id``), skipping every document whose id is in
*excluded_ids* and admitting at most *cap* documents. There is NO
cosine floor: the related tier is the ranked remainder (a lexical-
only cosine 0.0 hit is included) — its job on the ``done`` frame is
visibility (the UI's de-emphasized "nearby docs" row), not
citation. With the turn wiring's exclusion — exactly the suggested
tier's document ids (LOCKED A3: a contiguous top-N, no floor) —
"excluding the suggested" is exactly "rank 6+".
The returned rows carry the full document content, byte-identical
(the tier is metadata for the ``done`` frame and the durable
record; the prompt and ``read`` contract are untouched).
"""
out: list[Document] = []
seen: set[uuid.UUID] = set()
for rc in sorted(chunks, key=lambda c: c.score, reverse=True):
if len(out) >= cap:
break
doc = rc.document
if doc.id in seen or doc.id in excluded_ids:
continue
seen.add(doc.id)
out.append(doc)
return out