phase: 118_summary_seed_context
**Phase 118 final verification pass — complete.** All criteria verified; 4 pre-existing defects found and fixed.
- **Verified:** summary-seed wiring (`select_suggested` top-5 no-floor → summary blocks, no full text in HIGH prompt), all-doc markdown summaries + NULL backfill (`summary_backfilled`, no `sources_meta` bump), `read` adds full text with `read_docs`-only dedupe, `done.sources` = suggested+read / durable record = suggested+related+read + `suggested=N` log line (seen live in E2E), byte-locked PERSONA/LOW/TOOLS_SECTION, battery gate PASS recorded in `TOOL_CALLING_TESTING.md` §10 (turbo 2026-09-16: 1/2/4 GREEN, cond-3 reported 9/10 per A7, contract 21/21, caps 0).
- **Defects fixed (all pre-existing, none phase-118):** ① `ChatMessage` schema missing the phase-113 `related` key → `extra="forbid"` 422'd every done-time auto-save of grounded turns with a related tier, leaving `message_count=1` (root cause of `test_share_chat` 3F; browser-level instrumentation proved the PUT 422) — added the field + unit/integration pins; ② `test_theme_semantic_completion` pins stale vs phase-117 debox (border/chip removed) — re-targeted to assert border/chip *absence*; ③ `test_header_consistency` `<26`px pin red on 26.125px native date-input line — bound relaxed to `<34` (wrap-detection intent kept); ④ `test_navbar_refresh` bor.chat.v1 key set updated for `related`.
- **Test/lint/coverage:** `uv run pytest --cov=app --cov-report=term-missing` → **2506 passed, app/ 99%** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors.
- **E2E:** new story suite in isolation → **2 passed**; full 103-suite matrix sweep (each isolated) → **all 103 green** after the fixes; `test_share_chat` 4 passed, `test_theme_semantic_completion` 8 passed, `test_header_consistency` 3 passed, `test_navbar_refresh` 7 passed.
- **Deviations:** none from LOCKED decisions. Note: orphaned diagnostic uvicorn processes briefly made E2E sessions exercise stale code — killed and re-verified; a sweep-regenerated tracked screenshot was restored. No commits made (harness commits).
- **Completion criteria:** all 7 ✅ (commit/phase-move is the harness's step).
- **Next pending phase:** none — `todo/` holds only this phase's overview pending the harness move.
This commit is contained in:
+44
-32
@@ -97,12 +97,13 @@ task 04):
|
||||
context; ``grep`` — which searches the whole document — is the
|
||||
follow-up), with the truncation recorded on the holder so the loop
|
||||
yields a :class:`app.rag.llm.ToolResultPiece` (task 02 → the SSE
|
||||
``tool_result`` frame + UI marker). **A7 scope clarification:** the
|
||||
never-truncated contract is for the retrieval ``<documents>`` path
|
||||
(the top-2 seed documents stay whole — "this should never happen");
|
||||
the ``read`` TOOL path is the only capped read, per the owner's
|
||||
explicit request — the two paths are distinct (retrieval seeds vs.
|
||||
agent-requested additions). And ``grep`` greps the
|
||||
``tool_result`` frame + UI marker). **A6 re-revised contract
|
||||
(phase 118, owner directive 2026-09-15):** the retrieval
|
||||
``<documents>`` path seeds SUMMARIES only — a suggested document's
|
||||
full text never enters the prompt on the retrieval path; full text
|
||||
enters the context ONLY through this ``read`` TOOL path, which is
|
||||
the only capped read (the phase-95 cap unchanged). And ``grep``
|
||||
greps the
|
||||
indexed documents (or the one document a combined ``source/path``
|
||||
names) for a case-insensitive fixed substring and returns up to 20
|
||||
``source/path:line: text`` match lines (owner-locked A5, phase 68),
|
||||
@@ -138,7 +139,9 @@ task 04):
|
||||
:data:`NOT_A_FOLDER`, the drill-down teaching with the argument
|
||||
echoed and the deepest existing ancestor's direct subfolders
|
||||
listed, so the model self-corrects in the next round; a document
|
||||
already in context (seed or previously read) →
|
||||
already READ into full-text context (phase 118: the suggested
|
||||
seeds are summary blocks in the prompt, not full text — only an
|
||||
already-read document is refused) →
|
||||
:data:`ALREADY_IN_CONTEXT` (phase 72, task 05 gate iteration:
|
||||
the line names the correct action — answer from the text already
|
||||
in the prompt, do not call read again — so a fired refusal ends
|
||||
@@ -301,19 +304,21 @@ AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
"function": {
|
||||
"name": "read",
|
||||
"description": (
|
||||
"Do not call this tool for a document already shown in "
|
||||
"the <documents> section, even when the user asks you to "
|
||||
"open or read it — its full text is already in your "
|
||||
"prompt; answer directly from it. Use it only to add a "
|
||||
"document NOT already in <documents> to your context, "
|
||||
"by its combined `source/path` string. Very large "
|
||||
"documents are truncated: you receive the first part "
|
||||
"plus a TRUNCATED notice naming how many more characters "
|
||||
"exist — the notice is authoritative, the document did "
|
||||
"NOT end where it stopped. Follow it and use `grep` "
|
||||
"(pattern) to locate the rest — it searches the whole "
|
||||
"document. Call one tool at a time — wait for this "
|
||||
"result before your next call."
|
||||
"The <documents> section shows the SUMMARIES of the "
|
||||
"top-ranked documents — their full texts are NOT in "
|
||||
"your prompt yet. Use this tool to add one of them (or "
|
||||
"any other document) to your context, by its combined "
|
||||
"`source/path` string, exactly as shown in the `ls` "
|
||||
"output or the <documents> blocks. Do not re-read a "
|
||||
"document you have already read — its full text is "
|
||||
"already in your prompt. Very large documents are "
|
||||
"truncated: you receive the first part plus a TRUNCATED "
|
||||
"notice naming how many more characters exist — the "
|
||||
"notice is authoritative, the document did NOT end "
|
||||
"where it stopped. Follow it and use `grep` (pattern) "
|
||||
"to locate the rest — it searches the whole document. "
|
||||
"Call one tool at a time — wait for this result before "
|
||||
"your next call."
|
||||
),
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
@@ -326,10 +331,9 @@ AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
"shown in the `ls` output (e.g. "
|
||||
"'homelab/active/container_caddy/caddy.md'). "
|
||||
"A bare document path (without the source "
|
||||
"name) will not resolve. Only pass a document "
|
||||
"NOT already shown in the <documents> "
|
||||
"section — it is already in your context; do "
|
||||
"not re-read it."
|
||||
"name) will not resolve. Do not re-read a "
|
||||
"document you have already read — its full "
|
||||
"text is already in your prompt."
|
||||
),
|
||||
}
|
||||
},
|
||||
@@ -1152,10 +1156,15 @@ def _execute_tool(
|
||||
arg = raw_path.strip() if isinstance(raw_path, str) else ""
|
||||
if not arg:
|
||||
return MISSING_READ_ARGS
|
||||
known = {(doc.source, doc.path) for doc in (*seed_docs, *holder.read_docs)}
|
||||
# Phase 118 (A6): the dedupe set is ``holder.read_docs`` ONLY —
|
||||
# the ``seed_docs`` are SUMMARY blocks in the prompt, not full
|
||||
# text, so a FIRST read of a suggested document adds its full
|
||||
# text through the path below; only a document ALREADY READ is
|
||||
# refused.
|
||||
known = {(doc.source, doc.path) for doc in holder.read_docs}
|
||||
# The dedupe check needs no DB: the split pair of a combined
|
||||
# identity that is in context is in `known` as-is (the resolve
|
||||
# below would find the same document).
|
||||
# identity that is already in full-text context is in `known`
|
||||
# as-is (the resolve below would find the same document).
|
||||
if "/" in arg:
|
||||
src, _, p = arg.partition("/")
|
||||
if (src, p) in known:
|
||||
@@ -1326,11 +1335,14 @@ async def run_agent(
|
||||
the capture mechanism for new registry entries; *holder* accumulates
|
||||
the turn's ``scaffold_stripped`` total for the API layer's log line.
|
||||
|
||||
``seed_docs`` are the documents the retrieval already put in context
|
||||
(they shape the *system_prompt* the caller built); re-reading one of
|
||||
them is rejected with :data:`ALREADY_IN_CONTEXT` (the phase-72
|
||||
teaching line — answer from the text already in the prompt) — the
|
||||
rejection counts in nothing, but it still consumes a round.
|
||||
``seed_docs`` are the suggested documents whose SUMMARY blocks the
|
||||
caller put in the *system_prompt* (phase 118: the retrieval seeds
|
||||
summaries, never full texts); reading one of them ADDS its full
|
||||
text to the context through the ordinary ``read`` path — only a
|
||||
document ALREADY READ is rejected with :data:`ALREADY_IN_CONTEXT`
|
||||
(the phase-72 teaching line — answer from the text already in the
|
||||
prompt); the rejection counts in nothing, but it still consumes a
|
||||
round.
|
||||
|
||||
DB sessions (SEC-14-04): *db_factory* is a callable that returns a
|
||||
new :class:`sqlalchemy.orm.Session` (e.g. ``lambda: SessionLocal()``).
|
||||
|
||||
+62
-12
@@ -10,7 +10,8 @@ the two-phase upsert:
|
||||
2. embed the new chunks in batches and attach the vectors
|
||||
3. commit — one transaction per file, so a failed embedding leaves the
|
||||
database untouched and the file is simply retried on the next run
|
||||
4. non-markdown files only (phase 30): generate a ``lite``-model summary
|
||||
4. every file (phase 30; phase 118, A2: markdown included — the
|
||||
non-markdown-only scope is retired): generate a ``lite``-model summary
|
||||
and, best-effort, store it on ``documents.summary`` plus one extra
|
||||
embedded chunk (``is_summary``, position −1). The document row and its
|
||||
content chunks are already committed at this point, so a summary
|
||||
@@ -42,6 +43,17 @@ guard) and counted in ``summary.dates_updated`` — unless the row
|
||||
carries the owner's manual correction (``created_at_manual``, D1), which
|
||||
the sync never touches.
|
||||
|
||||
NULL-summary backfill (phase 118, A2): an UNCHANGED file (same
|
||||
``content_hash``) whose ``documents.summary`` is still NULL — a
|
||||
pre-phase-30 row, or an earlier fail-soft miss — gets the same
|
||||
best-effort summary pass on every sync until it sticks. A success counts
|
||||
``summary_backfilled`` (never ``summaries``) and touches nothing else: no
|
||||
content re-embed, no added/updated/pruned count — so no
|
||||
``sources_meta`` bump, no KB-overview/folder-summary regeneration. The
|
||||
backfill runs BEFORE the ``created_at_manual`` early-return (the manual
|
||||
flag protects the DATE only, D1) and the strict ``is None`` check leaves
|
||||
owner-set summaries (even empty strings, phase 57) alone.
|
||||
|
||||
``import_sources`` accepts an optional per-file ``progress`` callback
|
||||
(phase 64, task 01) reporting the file being processed right now.
|
||||
"""
|
||||
@@ -99,12 +111,18 @@ class ImportSummary:
|
||||
errors: int = 0
|
||||
chunks: int = 0
|
||||
embed_batches: int = 0
|
||||
#: Non-markdown files whose lite summary was generated + indexed
|
||||
#: (phase 30). One ``is_summary`` chunk per success.
|
||||
#: Files whose lite summary was generated + indexed (phase 30; phase
|
||||
#: 118, A2: every A9 format, markdown included). One ``is_summary``
|
||||
#: chunk per success.
|
||||
summaries: int = 0
|
||||
#: Non-markdown files whose summary generation failed (best-effort —
|
||||
#: the document is still indexed, without a summary).
|
||||
#: Files whose summary generation failed (best-effort — the document
|
||||
#: is still indexed, without a summary).
|
||||
summary_errors: int = 0
|
||||
#: Unchanged docs whose NULL summary was backfilled (phase 118, A2) —
|
||||
#: one ``is_summary`` chunk per success; the content is untouched, so
|
||||
#: a backfill NEVER counts added/updated/pruned (no
|
||||
#: ``sources_meta`` bump, no overview/folder-summary regeneration).
|
||||
summary_backfilled: int = 0
|
||||
#: Files whose ``created_at`` was refreshed on the UNCHANGED path —
|
||||
#: content untouched, date re-sourced (phase 106, D4: the date may
|
||||
#: go OLDER; a date-only refresh NEVER counts added/updated/pruned,
|
||||
@@ -125,7 +143,7 @@ class ImportSummary:
|
||||
logger.info(
|
||||
"import: summary files=%d added=%d updated=%d unchanged=%d pruned=%d "
|
||||
"errors=%d chunks=%d embed_batches=%d summaries=%d summary_errors=%d "
|
||||
"dates_updated=%d formats=%s",
|
||||
"summary_backfilled=%d dates_updated=%d formats=%s",
|
||||
self.files,
|
||||
self.added,
|
||||
self.updated,
|
||||
@@ -136,6 +154,7 @@ class ImportSummary:
|
||||
self.embed_batches,
|
||||
self.summaries,
|
||||
self.summary_errors,
|
||||
self.summary_backfilled,
|
||||
self.dates_updated,
|
||||
self.format_counts(),
|
||||
)
|
||||
@@ -442,6 +461,20 @@ async def _index_file(
|
||||
if doc is not None and doc.content_hash == digest:
|
||||
summary.unchanged += 1
|
||||
logger.info("import: unchanged source=%s path=%s", source, rel)
|
||||
# Phase 118 (A2): an unchanged doc whose summary is still NULL
|
||||
# (a pre-phase-30 row, or an earlier fail-soft miss) gets a
|
||||
# summary-only backfill — one ``is_summary`` chunk, no content
|
||||
# re-embed, and NEVER an added/updated/pruned count (so no
|
||||
# ``sources_meta`` bump, no overview/folder-summary
|
||||
# regeneration). Strict ``is None``: an empty-string summary is
|
||||
# owner-set (phase 57) and is never overwritten. BEFORE the
|
||||
# manual-date early-return: ``created_at_manual`` protects the
|
||||
# DATE only (phase 106, D1), not the summary.
|
||||
if doc.summary is None:
|
||||
await _store_summary(
|
||||
session, doc=doc, source=source, rel=rel, content=content,
|
||||
llm=llm, summary=summary, backfill=True,
|
||||
)
|
||||
if doc.created_at_manual:
|
||||
# D1/D4: the owner's correction survives the sync — no
|
||||
# write at all (the phase-97 ``manually_edited`` precedent).
|
||||
@@ -536,10 +569,11 @@ async def _index_file(
|
||||
summary.chunks += len(chunks_text)
|
||||
logger.info("import: %s source=%s path=%s chunks=%d", verb, source, rel, len(chunks_text))
|
||||
|
||||
# Phase 30: markdown is already natural language, so only the other A9
|
||||
# formats (txt, yaml, yml, json, py) get a ``lite``-model summary.
|
||||
if full_path.suffix.lower() in (".md", ".markdown"):
|
||||
return
|
||||
# Phase 30; phase 118 (A2, 2026-09-15): EVERY new/changed document
|
||||
# gets a ``lite``-model summary — markdown included. Phase 30's
|
||||
# "markdown is already natural language" exclusion is retired: the
|
||||
# summary is the retrieval seed context (the phase-118 suggestion
|
||||
# blocks), not a formatting convenience.
|
||||
await _store_summary(
|
||||
session, doc=doc, source=source, rel=rel, content=content, llm=llm, summary=summary
|
||||
)
|
||||
@@ -554,6 +588,7 @@ async def _store_summary(
|
||||
content: str,
|
||||
llm: Embedder,
|
||||
summary: ImportSummary,
|
||||
backfill: bool = False,
|
||||
) -> None:
|
||||
"""Best-effort ``lite`` summary for one already-committed document.
|
||||
|
||||
@@ -569,6 +604,11 @@ async def _store_summary(
|
||||
:class:`EmbeddingError` only rolls back the summary rows — the file
|
||||
stays indexed, without a summary, and the failure is counted in
|
||||
``summary_errors`` (PLAN phase 30).
|
||||
|
||||
``backfill`` (phase 118, A2): the unchanged-doc NULL-summary path —
|
||||
a success counts ``summary_backfilled`` instead of ``summaries``
|
||||
(the doc content is untouched, so the import's KB-change signal must
|
||||
not move); the rest of the mechanics are identical.
|
||||
"""
|
||||
try:
|
||||
text = await generate_summary(llm, source=source, path=rel, content=content)
|
||||
@@ -588,8 +628,18 @@ async def _store_summary(
|
||||
# ``expire_on_commit=False`` — reflects the committed state.
|
||||
doc.chunks.append(chunk)
|
||||
session.commit()
|
||||
summary.summaries += 1
|
||||
logger.info("import: summary source=%s path=%s chars=%d", source, rel, len(text))
|
||||
if backfill:
|
||||
# Phase 118 (A2): the backfill counts itself apart from fresh
|
||||
# imports — the doc content is unchanged, so ``summaries``
|
||||
# (a KB-change signal) must not move.
|
||||
summary.summary_backfilled += 1
|
||||
logger.info(
|
||||
"import: summary-backfill source=%s path=%s chars=%d",
|
||||
source, rel, len(text),
|
||||
)
|
||||
else:
|
||||
summary.summaries += 1
|
||||
logger.info("import: summary source=%s path=%s chars=%d", source, rel, len(text))
|
||||
except (LLMError, EmbeddingError) as e:
|
||||
session.rollback()
|
||||
summary.summary_errors += 1
|
||||
|
||||
+3
-2
@@ -1,8 +1,9 @@
|
||||
"""Async OpenAI-compatible client for the self-hosted aipi endpoint (PLAN A5).
|
||||
|
||||
Provides the embeddings surface (importer, retrieval), one-shot chat
|
||||
completions (phase 30: the ``lite`` model summarizes non-markdown
|
||||
documents at import time), and chat streaming (PLAN A15) for the RAG
|
||||
completions (phase 30: the ``lite`` model summarizes documents at import
|
||||
time — every A9 format, markdown included since phase 118), and chat
|
||||
streaming (PLAN A15) for the RAG
|
||||
pipeline. Chat streaming yields typed :class:`StreamPiece` values
|
||||
(phase 17) and — when the caller passes a ``tools`` list —
|
||||
:class:`ToolCallPiece` values (phase 37): aipi's ``turbo`` model streams
|
||||
|
||||
+3
-2
@@ -100,8 +100,9 @@ def build_overview_prompt(
|
||||
* ``user`` — one line per document,
|
||||
``source — path — title — {first line of summary}``, joined with
|
||||
newlines. The summary field is omitted when the document has no
|
||||
summary (markdown docs and the fail-soft path — no dangling
|
||||
dash). The list is capped at *max_chars* (default
|
||||
summary (pre-phase-30 rows, the fail-soft path, and pre-backfill
|
||||
NULL rows — no dangling dash). The list is capped at *max_chars*
|
||||
(default
|
||||
``BOR_OVERVIEW_INPUT_MAX_CHARS``): overflow is cut exactly at the
|
||||
cap and the shared ``[…truncated…]`` marker is appended on its own
|
||||
line, so the model never sees more than the cap and the cut is
|
||||
|
||||
+99
-41
@@ -7,7 +7,14 @@ no mandated deflection opening; the honesty gate itself is unchanged.)
|
||||
|
||||
Two modes:
|
||||
|
||||
* ``HIGH`` — grounded turn: full top-document texts under ``<documents>``.
|
||||
* ``HIGH`` — grounded turn: the top-ranked documents' SUMMARIES under
|
||||
``<documents>`` (phase 118, LOCKED A6 re-revising A7 — the "start here"
|
||||
suggestion seeding: each block is the document's stored summary, never
|
||||
the full content; the LLM extends its context by ``read``-ing a
|
||||
document's full text through the capped ``read`` tool, the ONLY full-
|
||||
text path. A NULL/blank summary — a fail-soft import miss — falls back
|
||||
to a ``suggestion_preview_chars`` content preview + the shared
|
||||
``[…truncated…]`` marker; no LLM call at chat time, LOCKED A5).
|
||||
* ``LOW`` — deflection turn: weak-hit *titles only* plus the
|
||||
``DEFLECT_MODE`` marker (the E2E mock LLM keys on that marker).
|
||||
|
||||
@@ -124,6 +131,24 @@ _KB_INTRO = (
|
||||
"(generated at import time):\n"
|
||||
)
|
||||
|
||||
#: The ``<documents>`` section's start-here intro (phase 118, task 03 —
|
||||
#: the owner directive, TODO L3): the seeded blocks are the SUMMARIES of
|
||||
#: the top-ranked documents for the question, opt-in starting points —
|
||||
#: start here if one seems right; ``read`` the document's combined
|
||||
#: ``source/path`` to add its FULL content before answering specifics
|
||||
#: (the full text is NOT in the prompt until you read it); cite the
|
||||
#: document(s) you used, by path. Rendered BEFORE the first block, only
|
||||
#: when at least one block is present. The ``<documents>`` tag and the
|
||||
#: per-block ``<document>`` markup stay byte-stable around it (the E2E
|
||||
#: mock's block parser and the ``read``-tool copy key off both).
|
||||
SUGGEST_INTRO = (
|
||||
"The blocks below are the summaries of the top-ranked documents for "
|
||||
"your question — start here if one seems right to you: call `read` "
|
||||
"with that document's combined `source/path` to add its full content "
|
||||
"before answering specifics (its full text is not in the prompt until "
|
||||
"you read it). Cite the document(s) you used, by path."
|
||||
)
|
||||
|
||||
#: The ``<tools>`` instructions section — **HIGH prompt only** (phase 37,
|
||||
#: task 03; phase 70: the copy is rewritten for the harness-aligned
|
||||
#: ``ls`` / ``read`` / ``grep`` shapes, names/args exactly as the
|
||||
@@ -158,12 +183,15 @@ _KB_INTRO = (
|
||||
#: the user's "open it / read it" and ``read`` seed-context documents,
|
||||
#: then repeated the refused call; run 2: 8/18, 44% — the repeat is
|
||||
#: gone, but a grep turn batched two calls per reply and the harness
|
||||
#: runs only the first of a batch): the do-not-read rule names the
|
||||
#: user-command scenario (a document already in the ``<documents>``
|
||||
#: section: do not call ``read``, answer from the text already in the
|
||||
#: prompt — an anchor on the concrete ``<document path="...">`` markup
|
||||
#: was tried and REVERTED: it primed the model to latch the seed
|
||||
#: documents' paths as ``ls`` scopes, regressing the incident turn);
|
||||
#: runs only the first of a batch): the ``read`` clause carries the
|
||||
#: phase-118 summary-seed contract (the ``<documents>`` section holds
|
||||
#: SUMMARIES — a suggested document's full text is not in the prompt
|
||||
#: until ``read`` adds it; do not re-read a document you have already
|
||||
#: read — answer from the text already in the prompt; if the user
|
||||
#: asks to open or read a suggested document, ``read`` it — that is
|
||||
#: the point of the section — the phase-72 do-not-read rule and the
|
||||
#: reverted ``<document path="...">`` anchor are retired with the
|
||||
#: full-text seeds);
|
||||
#: the one-call rule names the consequence (a batched second call is
|
||||
#: discarded — runs only the first); the never-repeat rule says why
|
||||
#: (the refusal already told you the correct form); the ``grep`` clause
|
||||
@@ -171,10 +199,11 @@ _KB_INTRO = (
|
||||
#: live runs showed the model scoping ``grep`` with an ``ls``-style
|
||||
#: source name — the incident shape, but on grep). The behavioral
|
||||
#: contract lives in the ``AGENT_TOOLS`` descriptions as well (the most
|
||||
#: local text at call time): ``read`` must not be called for a
|
||||
#: ``<documents>`` document at all; ``grep`` with only ``pattern``
|
||||
#: searches the whole knowledge base, and a source name is not a
|
||||
#: document.
|
||||
#: local text at call time): ``read`` adds a document's FULL text by
|
||||
#: its combined ``source/path`` identity — the ``<documents>``
|
||||
#: summaries are the starting points, not the content; ``grep`` with
|
||||
#: only ``pattern`` searches the whole knowledge base, and a source
|
||||
#: name is not a document.
|
||||
TOOLS_SECTION: str = (
|
||||
"<tools>\n"
|
||||
"You may extend your context with three tools. `ls` lists the "
|
||||
@@ -189,12 +218,16 @@ TOOLS_SECTION: str = (
|
||||
"`source: X | path: Y | title: Z`; to find one specific document "
|
||||
"without listing, use `grep`. `read` pulls in one document by its "
|
||||
"combined `source/path` string, exactly as shown in the `ls` "
|
||||
"output — including the source name — adding its full content to "
|
||||
"your context. Do not call `read` for a document already shown in "
|
||||
"the <documents> section, even when the user asks you to open or "
|
||||
"read it — its full text is already in your prompt; answer "
|
||||
"directly from it. For `read`, a bare document path (without the "
|
||||
"source name) will not resolve. Very large documents are capped: a "
|
||||
"output — including the source name — or in the <documents> "
|
||||
"summary blocks — adding its full content to your context. The "
|
||||
"<documents> section holds SUMMARIES — the full text of a "
|
||||
"suggested document is not in your prompt until you `read` it. Do "
|
||||
"not re-read a document you have already read — its full text is "
|
||||
"already in your prompt; answer directly from it. If the user "
|
||||
"asks you to open or read a suggested document, `read` it — that "
|
||||
"is the point of the section. For `read`, a bare document path "
|
||||
"(without the source name) will not resolve. Very large documents "
|
||||
"are capped: a "
|
||||
"cut read returns the first part plus a TRUNCATED notice — the "
|
||||
"document did not end where it stopped; use `grep` (pattern) to "
|
||||
"find the rest, it searches the whole document. `grep` locates an exact string "
|
||||
@@ -355,17 +388,42 @@ def build_kb_section(overview: str, max_chars: int | None = None) -> str:
|
||||
return ""
|
||||
|
||||
|
||||
def _document_body(doc: Document) -> str:
|
||||
"""The body of one ``<document>`` suggestion block (phase 118, LOCKED
|
||||
A6): the document's stored summary (stripped) — NEVER the full
|
||||
content.
|
||||
|
||||
Defensive fallback ONLY when the summary is missing (``None`` or
|
||||
whitespace — a fail-soft import miss, LOCKED A5): the first
|
||||
``suggestion_preview_chars`` characters of the content plus the
|
||||
shared :data:`TRUNCATION_MARKER` on its own line — the settings read
|
||||
happens on this fallback path ONLY (a prompt built from
|
||||
summary-bearing docs makes no settings call for the cap). Content at
|
||||
or under the cap rides whole, unmarked (nothing was cut). No LLM
|
||||
call at chat time — the preview is deterministic.
|
||||
"""
|
||||
summary = (doc.summary or "").strip()
|
||||
if summary:
|
||||
return summary
|
||||
limit = get_settings().suggestion_preview_chars
|
||||
content = doc.content
|
||||
if len(content) > limit:
|
||||
return content[:limit] + "\n" + TRUNCATION_MARKER
|
||||
return content
|
||||
|
||||
|
||||
def build_high_prompt(
|
||||
documents: Sequence[Document],
|
||||
notes: Sequence[str] | None = None,
|
||||
kb_overview: str | None = None,
|
||||
) -> str:
|
||||
"""Grounded turn: locked persona (+ steering, + KB overview) + full
|
||||
texts of the top documents + the ``<tools>`` instructions (phase 37;
|
||||
phase 70: the harness-aligned ``ls`` / ``read`` / ``grep`` shapes;
|
||||
phase 72: the copy states the document-identity contract — the
|
||||
source-name ``ls`` scope, the combined ``source/path`` identity for
|
||||
``read``/``grep`` — up front).
|
||||
"""Grounded turn: locked persona (+ steering, + KB overview) + the
|
||||
top-ranked documents' SUMMARY blocks + the ``<tools>`` instructions
|
||||
(phase 37; phase 70: the harness-aligned ``ls`` / ``read`` /
|
||||
``grep`` shapes; phase 72: the document-identity contract up front;
|
||||
phase 118, LOCKED A6 re-revising A7: the ``<documents>`` section
|
||||
seeds SUMMARIES, never full texts — full text enters the context
|
||||
only through the capped ``read`` tool).
|
||||
|
||||
Section order: ``<relevance>`` → ``<knowledge_base>`` → ``<tuning>``
|
||||
→ ``<documents>`` → ``<tools>``; empty steering/overview omit their
|
||||
@@ -373,42 +431,42 @@ source-name ``ls`` scope, the combined ``source/path`` identity for
|
||||
cap — not the prompt — decides whether the tools are actually
|
||||
offered to the model, see :mod:`app.rag.agent`).
|
||||
|
||||
When at least one block is present, the section leads with the
|
||||
start-here :data:`SUGGEST_INTRO` line (before the first block — the
|
||||
phase-15 ``_STEERING_INTRO`` / phase-31 ``_KB_INTRO`` precedent): the
|
||||
blocks are the summaries of the top-ranked documents, ``read`` adds
|
||||
the full text, and the answer cites the document(s) used by path.
|
||||
This is NOT the reverted phase-72 in-context reminder (that copy
|
||||
taught the seed documents as already-read context; this one teaches
|
||||
the summary-as-starting-point contract the A6 re-revision requires).
|
||||
|
||||
Each ``<document>`` block carries the identity attributes
|
||||
``source`` / ``path`` / ``title`` — and, since phase 106 (D5),
|
||||
``date`` (the row's ``created_at`` UTC date part, ``YYYY-MM-DD``,
|
||||
APPENDED after ``title`` — the only position; always present,
|
||||
``created_at`` is NOT NULL) — plus the document's full text.
|
||||
|
||||
Gate-iteration note (task 05, 2026-09-03/04): an in-context reminder
|
||||
LEADING this section (the document texts are already context — do
|
||||
not ``read`` one the user asked to open) was tried and REVERTED:
|
||||
it never flipped the seed-doc reads (15/15 across gate runs 1-5)
|
||||
and correlated with the incident-turn regression (the model latched
|
||||
the seed documents' paths as ``ls`` scopes — cap reached on the
|
||||
"list the files in this directory" turn) whenever the copy named
|
||||
the ``<document>`` blocks explicitly.
|
||||
``created_at`` is NOT NULL) — plus the document's summary
|
||||
(:func:`_document_body`; the NULL/whitespace-summary preview
|
||||
fallback is LOCKED A5).
|
||||
"""
|
||||
# 2026-09-04 (controlled tool-calling fast loop): the do-not-read
|
||||
# rule for seed documents lives in TOOLS_SECTION and the ``read``
|
||||
# tool descriptions (the copy levers that stuck — see the gate's
|
||||
# telemetry in TOOL_CALLING_TESTING.md). A per-block instruction
|
||||
# attribute at the ``source``/``path`` copy site was TRIED and
|
||||
# REVERTED the same day (no improvement across runs; the block stays
|
||||
# exactly the document identity + full text).
|
||||
# Phase 106 (D5): every document the model sees carries its
|
||||
# creation date — the block's ``date`` attribute (the row's
|
||||
# ``created_at`` UTC date part, appended after ``title`` — the
|
||||
# only position; always present, ``created_at`` is NOT NULL).
|
||||
# Phase 118 (A6): the block body is the summary — the full content
|
||||
# stays out of the prompt (the capped ``read`` tool is the only
|
||||
# full-text path).
|
||||
blocks = [
|
||||
f'<document source="{doc.source}" path="{doc.path}" title="{doc.title}" '
|
||||
f'date="{doc.created_at:%Y-%m-%d}">\n'
|
||||
f"{doc.content}\n"
|
||||
f"{_document_body(doc)}\n"
|
||||
"</document>"
|
||||
for doc in documents
|
||||
]
|
||||
body = "\n\n".join(blocks) if blocks else (
|
||||
"(no documents matched — do not invent specifics)"
|
||||
)
|
||||
if blocks:
|
||||
body = SUGGEST_INTRO + "\n\n" + body
|
||||
prompt = _base("HIGH")
|
||||
for part in (build_kb_section(kb_overview or ""), build_steering_section(notes or [])):
|
||||
if part:
|
||||
|
||||
+113
-13
@@ -35,12 +35,16 @@
|
||||
ranked documents (up to ``BOR_RELATED_MAX_DOCS``) become the related
|
||||
tier.
|
||||
|
||||
The product requirement (LOCKED A7, revised 2026-08-24): the LLM receives
|
||||
the **entire relevant document**, not just the chunk — chunk hits map back
|
||||
to their parents, dedupe, rank by best fused score, and the full text of
|
||||
the top-N documents is always fed through, never truncated. If a future KB
|
||||
ever makes the prompt too large for the model, the ``LLMError`` → SSE
|
||||
``error`` path surfaces it loudly — no silent partial context.
|
||||
The product requirement (A7, re-revised by the phase-118 owner directive,
|
||||
LOCKED A6, 2026-09-15): the retrieval path seeds **summaries** — the
|
||||
suggestion tier (:func:`select_suggested`, top-N distinct documents, no
|
||||
cosine floor, LOCKED A3) whose summary blocks are the grounded prompt's
|
||||
``<documents>`` starting points. The full text of a document enters the
|
||||
context ONLY through the agent's capped ``read`` tool
|
||||
(:mod:`app.rag.agent`; ``BOR_READ_MAX_CHARS`` + :data:`TRUNCATION_MARKER`),
|
||||
never through the retrieval seeding. A document's content itself is still
|
||||
carried on its rows byte-identical — the ``read`` tool serves it whole,
|
||||
un-truncated up to its cap.
|
||||
|
||||
Deterministic tie-break for equal fused scores:
|
||||
``(−fused, −cosine, document.path, chunk.position)``.
|
||||
@@ -61,9 +65,12 @@ from sqlalchemy.orm import Session
|
||||
from app.config import get_settings
|
||||
from app.models import Chunk, Document
|
||||
|
||||
#: Shared overflow marker — now used by the steering (<tuning>) section
|
||||
#: only (phase 15; imported by ``app.rag.prompts``). The document context
|
||||
#: path never truncates (A7 revised, owner permission 2026-08-24).
|
||||
#: Shared overflow marker (phase 15; imported by ``app.rag.prompts``)
|
||||
#: — used by the steering (<tuning>) section, the phase-118 NULL-summary
|
||||
#: suggestion preview fallback (A5), and the capped agent ``read``
|
||||
#: result. The seeded summary blocks and the ``read``-served document
|
||||
#: content never truncate silently (A6 re-revised): full text enters the
|
||||
#: context only through the capped ``read`` tool.
|
||||
TRUNCATION_MARKER = "[…truncated…]"
|
||||
|
||||
#: Alphanumeric tokens of a question (``to_tsquery`` input, OR-joined),
|
||||
@@ -574,6 +581,11 @@ def select_documents_tiered(
|
||||
"""Tier chunk hits into the cited and the related parent documents
|
||||
(phase 113, LOCKED A2/A4 — the usefulness bar).
|
||||
|
||||
Phase 118 retired the full-text seeding role (A6); the suggested
|
||||
tier (:func:`select_suggested`) seeds the prompt now — this helper
|
||||
stays as a dormant public helper (env back-compat for the settings
|
||||
it was calibrated by).
|
||||
|
||||
Distinct parent documents are ranked exactly like :func:`select_documents`
|
||||
(best fused score first — the same stable score-descending walk, so a
|
||||
document's rank position is fixed by its FIRST seen chunk) and each
|
||||
@@ -603,7 +615,11 @@ def select_documents_tiered(
|
||||
|
||||
The returned rows carry the full document content, byte-identical —
|
||||
a matched parent document is **never truncated** (A7 revised, owner
|
||||
permission 2026-08-24).
|
||||
permission 2026-08-24; A6 re-revised 2026-09-15: the retrieval path
|
||||
seeds SUMMARIES — the cited tier's full texts no longer ride the
|
||||
grounded prompt, full text enters the context only through the
|
||||
capped ``read`` tool; the rows themselves still carry the whole
|
||||
content).
|
||||
"""
|
||||
top_n = n if n is not None else get_settings().top_n_docs
|
||||
no_bar = floor <= 0.0
|
||||
@@ -642,12 +658,20 @@ def select_documents(
|
||||
) -> list[Document]:
|
||||
"""Map chunk hits to distinct parent documents, ranked by best fused score.
|
||||
|
||||
Phase 118 retired the full-text seeding role (A6); the suggested
|
||||
tier (:func:`select_suggested`) seeds the prompt now — this helper
|
||||
stays as a dormant public helper (env back-compat for the settings
|
||||
it was calibrated by).
|
||||
|
||||
At most *n* documents are returned (default ``BOR_TOP_N_DOCS``). The
|
||||
returned rows carry the full document content, byte-identical — a
|
||||
matched parent document is **never truncated** (A7 revised, owner
|
||||
permission 2026-08-24). There is deliberately no context budget: an
|
||||
oversized prompt must fail loudly through the ``LLMError`` → SSE
|
||||
``error`` path, never arrive as silent partial context.
|
||||
permission 2026-08-24; A6 re-revised 2026-09-15: the seeded prompt
|
||||
now carries SUMMARIES — the full text reaches the context only
|
||||
through the capped ``read`` tool, not through this selection). There
|
||||
is deliberately no context budget: an oversized prompt must fail
|
||||
loudly through the ``LLMError`` → SSE ``error`` path, never arrive
|
||||
as silent partial context.
|
||||
|
||||
Phase 113: a thin wrapper on :func:`select_documents_tiered` — the
|
||||
legacy "any score, top-N" behavior is the cited tier with a zero
|
||||
@@ -656,3 +680,79 @@ def select_documents(
|
||||
"""
|
||||
cited, _ = select_documents_tiered(chunks, n, 0.0, 0)
|
||||
return cited
|
||||
|
||||
|
||||
def select_suggested(
|
||||
chunks: Sequence[RetrievedChunk],
|
||||
n: int | None = None,
|
||||
) -> list[Document]:
|
||||
"""Top-N distinct parent documents in fused rank order — the phase-118
|
||||
"start here" suggestion tier (LOCKED A3), with NO cosine floor.
|
||||
|
||||
Distinct parent documents are walked in the SAME stable score-
|
||||
descending order as :func:`select_documents_tiered` (a document's rank
|
||||
position is fixed by its FIRST seen chunk; dedupe by ``document.id``),
|
||||
and at most *n* of them are returned (default the
|
||||
``BOR_SUGGESTED_DOCS`` setting, 5). Unlike the phase-113 cited tier,
|
||||
the usefulness bar NEVER filters here: a lexical-only hit with
|
||||
cosine 0.0 is suggested when it ranks. Suggestions are opt-in
|
||||
starting points, not citations — the seeded prompt carries the
|
||||
document's summary, and the LLM decides whether to extend its context
|
||||
by reading the document's full text.
|
||||
|
||||
The returned rows carry the full document content, byte-identical —
|
||||
the content is what the agent's ``read`` tool serves later (never
|
||||
truncated; A6 re-revises A7: full text enters the context only through
|
||||
the capped ``read`` tool).
|
||||
"""
|
||||
top_n = n if n is not None else get_settings().suggested_docs
|
||||
|
||||
order: list[Document] = []
|
||||
seen: set[uuid.UUID] = set()
|
||||
for rc in sorted(chunks, key=lambda c: c.score, reverse=True):
|
||||
if len(order) >= top_n:
|
||||
break
|
||||
doc = rc.document
|
||||
if doc.id in seen:
|
||||
continue
|
||||
seen.add(doc.id)
|
||||
order.append(doc)
|
||||
return order
|
||||
|
||||
|
||||
def select_related(
|
||||
chunks: Sequence[RetrievedChunk],
|
||||
excluded_ids: set[uuid.UUID],
|
||||
cap: int,
|
||||
) -> list[Document]:
|
||||
"""The documents ranked AFTER *excluded_ids* — the phase-118 related
|
||||
tier (rank 6+ for the contiguous top-5 suggestion set), up to *cap*
|
||||
(``BOR_RELATED_MAX_DOCS``).
|
||||
|
||||
The SAME stable score-descending walk as
|
||||
:func:`select_documents_tiered` / :func:`select_suggested` (a
|
||||
document's rank position is fixed by its FIRST seen chunk; dedupe by
|
||||
``document.id``), skipping every document whose id is in
|
||||
*excluded_ids* and admitting at most *cap* documents. There is NO
|
||||
cosine floor: the related tier is the ranked remainder (a lexical-
|
||||
only cosine 0.0 hit is included) — its job on the ``done`` frame is
|
||||
visibility (the UI's de-emphasized "nearby docs" row), not
|
||||
citation. With the turn wiring's exclusion — exactly the suggested
|
||||
tier's document ids (LOCKED A3: a contiguous top-N, no floor) —
|
||||
"excluding the suggested" is exactly "rank 6+".
|
||||
|
||||
The returned rows carry the full document content, byte-identical
|
||||
(the tier is metadata for the ``done`` frame and the durable
|
||||
record; the prompt and ``read`` contract are untouched).
|
||||
"""
|
||||
out: list[Document] = []
|
||||
seen: set[uuid.UUID] = set()
|
||||
for rc in sorted(chunks, key=lambda c: c.score, reverse=True):
|
||||
if len(out) >= cap:
|
||||
break
|
||||
doc = rc.document
|
||||
if doc.id in seen or doc.id in excluded_ids:
|
||||
continue
|
||||
seen.add(doc.id)
|
||||
out.append(doc)
|
||||
return out
|
||||
|
||||
Reference in New Issue
Block a user