phase: 124_image_chips_and_answers
All green. Final report: **Phase 124 final verification pass — all criteria verified, no defects found (nothing to fix)** - Verified the implemented phase (all 7 tasks done, changes in working tree): `BOR_ANSWER_IMAGES` toggle, `show_image` tool + refusals + `holder.shown_images`, toggle-gated prompt clause, `answer_image` frame field (None-omitting serializer) + `ChatMessage` persistence, `merge_cited_docs` auto-cite bug fix, answer-figure UI (live/restore/shared/CSS), mock triggers + E2E suite. Also confirmed the bytes route → public change matches the phase-123 shared-chat precedent (task 06 design). - `uv run pytest --cov=app --cov-report=term-missing` → **2848 passed**, TOTAL **99%** (>90% gate) - `bash .agents/validate.sh` → **validation OK** (tests + coverage + ruff + pyright) - `uv run ruff check . && uv run pyright` → clean, 0 errors - `uv run pytest tests/e2e/test_answer_images.py -v --no-cov` (isolated) → **5/5 passed** - Mock-inert spot checks: `tests/e2e/test_image_documents.py` 4/4, `tests/e2e/test_agent_document_tools.py` 4/4 **Completion criteria:** ① zero-read suggested-tier image doc chips with figure — verified (unit matrix, integration `test_zero_read_grounded_turn_auto_cites_suggested_image_doc`, E2E bug-fix flows); ② show_image → answer figure below text/above chips, second call refused, persists + restore + shared — verified (unit handler matrix, integration round-trip, E2E feature flow); ③ default-off byte-identical (3 tools, no clause, no key) — verified (unit + integration + E2E toggle-off); ④ full suite green / >90% / lint clean — verified above; ⑤ commit + phase move — harness responsibility (changes left uncommitted in working tree, no live-infrastructure changes made). **Next pending phase:** none — `todo/` holds only this phase; the pipeline is clear once the harness moves it.
This commit is contained in:
+82
-8
@@ -462,7 +462,14 @@ def plan_turn(
|
||||
fts_hits,
|
||||
False,
|
||||
build_high_prompt(
|
||||
suggested, notes=steering, kb_overview=kb_text, folder_lines=folder_lines
|
||||
suggested,
|
||||
notes=steering,
|
||||
kb_overview=kb_text,
|
||||
folder_lines=folder_lines,
|
||||
# Phase 124 (task 03, LOCKED A5): the grounded HIGH prompt
|
||||
# carries the show_image clause exactly when the
|
||||
# BOR_ANSWER_IMAGES toggle is on (off = byte-identical).
|
||||
show_image=settings.answer_images,
|
||||
),
|
||||
suggested,
|
||||
related_docs,
|
||||
@@ -486,6 +493,51 @@ def plan_turn(
|
||||
)
|
||||
|
||||
|
||||
def merge_cited_docs(
|
||||
read_docs: list[Document],
|
||||
suggested_docs: list[Document],
|
||||
deflected: bool,
|
||||
) -> list[Document]:
|
||||
"""Compose the citation tier (the done frame's ``sources``, PLAN §9).
|
||||
|
||||
Phase 119 (LOCKED A1): on grounded turns the agent's READ DOCS —
|
||||
deduped by ``(source, path)``, read order; on deflected turns
|
||||
nothing (a weak-hit answer is not a citation). Phase 124 (TODO L3)
|
||||
adds the rule's single documented exception: a suggested-tier
|
||||
IMAGE doc's seed block IS its entire content (content == summary
|
||||
== the vision description, phase 122 task 03) — a ``read`` would
|
||||
add only the marker line, so a real model never performs one and
|
||||
the doc's chip never appeared. "Suggested" is therefore equivalent
|
||||
to "read" for it, and it is auto-cited — in SUGGESTION order AFTER
|
||||
the read docs, deduped against them (an image doc the agent also
|
||||
explicitly read appears once, at its read position). Non-image
|
||||
suggested docs are never auto-cited: the phase-119 read-docs-only
|
||||
rule stays intact for text docs (a zero-read text turn still chips
|
||||
nothing).
|
||||
"""
|
||||
if deflected:
|
||||
return []
|
||||
cited: list[Document] = []
|
||||
seen: set[tuple[str, str]] = set()
|
||||
for doc in read_docs:
|
||||
key = (doc.source, doc.path)
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
cited.append(doc)
|
||||
# Phase 124 (TODO L3): the suggested-tier IMAGE docs join the
|
||||
# citation tier — "suggested" is equivalent to "read" for them
|
||||
# (their seed block is their entire content), so a zero-read
|
||||
# grounded turn chips them too. Suggestion order, deduped.
|
||||
for doc in suggested_docs:
|
||||
if not doc.is_image:
|
||||
continue
|
||||
key = (doc.source, doc.path)
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
cited.append(doc)
|
||||
return cited
|
||||
|
||||
|
||||
@router.post("/chat")
|
||||
async def chat(
|
||||
request: ChatRequest,
|
||||
@@ -1035,15 +1087,21 @@ async def chat(
|
||||
# phase-119 cite discipline, A5) — and [] on
|
||||
# deflected ones (a deflected answer cites nothing;
|
||||
# the weak hits stay in the durable record).
|
||||
# Phase 124 (TODO L3) gives that rule its single
|
||||
# documented exception — the suggested-tier IMAGE
|
||||
# docs (``merge_cited_docs``, below): a seed block
|
||||
# IS an image doc's entire content, so "suggested"
|
||||
# is equivalent to "read" for it and it is
|
||||
# auto-cited in suggestion order AFTER the read
|
||||
# docs, deduped (a read image doc stays at its read
|
||||
# position, once). Text docs keep the read-docs-only
|
||||
# rule — a zero-read text turn still chips nothing.
|
||||
# A cancelled turn (the generator closed by the
|
||||
# consumer) never reaches this step — no query_log row.
|
||||
cited_docs: list[Document] = []
|
||||
cited_seen: set[tuple[str, str]] = set()
|
||||
for doc in holder.read_docs:
|
||||
key = (doc.source, doc.path)
|
||||
if key not in cited_seen:
|
||||
cited_seen.add(key)
|
||||
cited_docs.append(doc)
|
||||
cited_docs = merge_cited_docs(
|
||||
holder.read_docs, plan.suggested_docs, plan.deflected
|
||||
)
|
||||
cited_seen = {(d.source, d.path) for d in cited_docs}
|
||||
record_docs: list[Document] = []
|
||||
seen: set[tuple[str, str]] = set()
|
||||
for doc in [*plan.suggested_docs, *plan.related_docs, *holder.read_docs]:
|
||||
@@ -1124,12 +1182,28 @@ async def chat(
|
||||
for d in plan.related_docs
|
||||
if (d.source, d.path) not in cited_seen
|
||||
]
|
||||
# Phase 124 (task 04): the answer's first-class image —
|
||||
# the shown document's ref (the LLM's ``show_image``
|
||||
# pick, task 02's ``holder.shown_images``), built by the
|
||||
# SAME shared ref builder (source/path/title +
|
||||
# ``image_url``). ``None`` on every other turn — a
|
||||
# deflected turn runs no agent (``shown_images`` is
|
||||
# always empty) and an image-less grounded turn never
|
||||
# called the tool — and ``None`` is DROPPED on
|
||||
# serialization, so those frames stay byte-identical to
|
||||
# pre-phase (no key, no null).
|
||||
answer_ref = (
|
||||
source_ref_with_image(holder.shown_images[0])
|
||||
if holder.shown_images
|
||||
else None
|
||||
)
|
||||
yield sse_event(
|
||||
ChatDoneEvent(
|
||||
deflected=plan.deflected,
|
||||
sources=cited_refs,
|
||||
related=related_refs,
|
||||
suggestions=plan.suggestions,
|
||||
answer_image=answer_ref,
|
||||
).model_dump()
|
||||
)
|
||||
finally:
|
||||
|
||||
+22
-10
@@ -28,12 +28,17 @@ BYTES route: the persistent copy behind an ``is_image`` document's
|
||||
``image_path``, served with the extension's ``Content-Type`` (the
|
||||
``app.rag.summarizer.IMAGE_MIMES`` map — one map, one truth) and a
|
||||
``Cache-Control: private, max-age=3600`` header (the bytes are
|
||||
content-hashed — long enough, bustable by re-upload). User-gated like
|
||||
the content endpoint (phase 79 — the ONLY anonymous surface is the
|
||||
shared chats): a missing doc, a non-image doc, a doc whose
|
||||
content-hashed — long enough, bustable by re-upload). PUBLIC (phase
|
||||
124 — the phase-123 chat-image precedent: the image is part of a
|
||||
shared chat's rendered content, and the shared chat — the app's ONLY
|
||||
anonymous surface — must render the answer figure faithfully for
|
||||
guests; the unguessable ``uuid4`` document id IS the credential, no
|
||||
enumeration value): a missing doc, a non-image doc, a doc whose
|
||||
``image_path`` is NULL, or a row whose copy was lost all map to 404
|
||||
``document not found`` (the router's unknown-document shape —
|
||||
traversal/UUID-guessing has no filesystem surface to hit).
|
||||
traversal/UUID-guessing has no filesystem surface to hit). The
|
||||
document's TEXT (the content endpoint) stays user-gated — only the
|
||||
bytes are public.
|
||||
|
||||
PATCH /api/folders/summary — the admin folder-description editor
|
||||
(phase 97, task 03): update / create / clear a stored
|
||||
@@ -212,16 +217,23 @@ def get_document_content(
|
||||
def get_document_image(
|
||||
doc_id: str,
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
_user: None = Depends(require_user), # noqa: B008 # phase 79 posture (see docstring)
|
||||
) -> FileResponse:
|
||||
"""The phase-122 (task 04) image BYTES route — the persistent copy
|
||||
behind an ``is_image`` document's ``image_path``.
|
||||
|
||||
Same auth posture as the document content endpoint (phase 79 —
|
||||
``require_user``: admin OR live token holder; the ONLY anonymous
|
||||
surface is the shared chats — the image is part of a document's
|
||||
content, so it travels under the same gate): anonymous callers get
|
||||
401 ``authentication required`` before any row is read.
|
||||
PUBLIC (phase 124, LOCKED A4 — "visible … on the shared page"): a
|
||||
guest opening a shared chat sees the brain answer's image figure,
|
||||
so the bytes must serve ANONYMOUSLY. The trust model is the
|
||||
phase-123 chat-image precedent (the question attachment travels in
|
||||
the SAME shared content): the document id is an unguessable
|
||||
server-minted ``uuid4`` — IT IS the credential, with no
|
||||
enumeration value — exactly like the saved-chat id (phase 55) and
|
||||
the chat-image filename (phase 123). (The phase-122/79
|
||||
``require_user`` posture was set before the image became part of a
|
||||
shared chat's rendered content; the shared page's figure — built
|
||||
on this route per the phase-124 design — 401'd and collapsed for
|
||||
every guest, contradicting the owner-locked A4. The document's
|
||||
TEXT stays user-gated: only the bytes are public.)
|
||||
|
||||
404 ``document not found`` (the router's unknown-document shape) in
|
||||
every non-servable case — a missing id (an unparseable string maps
|
||||
|
||||
@@ -412,6 +412,23 @@ class Settings(BaseSettings):
|
||||
#: (the ``upload_max_mb`` pattern).
|
||||
chat_image_max_mb: int = 10
|
||||
|
||||
# --- Answer images (phase 124: the LLM shows a supporting image in the answer) ---
|
||||
#: Feature toggle for the answer-image capability (phase 124,
|
||||
#: ``BOR_ANSWER_IMAGES``; ``0``/``false`` = off — the DEFAULT,
|
||||
#: LOCKED A2). The single knob that turns the whole feature on or
|
||||
#: off. Gates the ANSWER side ONLY: while on, the grounded agent is
|
||||
#: offered the ``show_image`` tool, the HIGH prompt carries its
|
||||
#: teaching clause, and a shown doc rides the done frame's
|
||||
#: ``answer_image`` key (and the saved record's). While off,
|
||||
#: behavior is byte-identical everywhere — no ``show_image`` tool
|
||||
#: offered, no prompt clause, no ``answer_image`` key in any frame
|
||||
#: or record. INDEPENDENT of ``images``/``BOR_IMAGES`` (LOCKED A2):
|
||||
#: that toggle indexes image documents, this one decides whether
|
||||
#: the model may show one in an answer — ``on`` with ``images`` off
|
||||
#: is a coherent (if image-less) state: the tool is offered, finds
|
||||
#: no image docs, and always refuses.
|
||||
answer_images: bool = False
|
||||
|
||||
# --- Docs push (phase 59: save a chat answer as documentation) ---
|
||||
#: The git repo a saved chat answer is committed to (phase 59, D3):
|
||||
#: **any** remote — a URL (``https://``, ``ssh://``, ``git@``) or a
|
||||
|
||||
+143
-2
@@ -390,6 +390,71 @@ AGENT_TOOLS: list[dict[str, Any]] = [
|
||||
},
|
||||
]
|
||||
|
||||
#: The 4th agent tool (phase 124, task 02; LOCKED A3/A5) — offered
|
||||
#: ONLY when ``settings.answer_images`` (``BOR_ANSWER_IMAGES``) is true:
|
||||
#: :func:`agent_tools` composes the offer list (the three base tools
|
||||
#: stay :data:`AGENT_TOOLS`, byte-identical). The description IS the
|
||||
#: relevance gate (A5 — "we don't want to just show every image that
|
||||
#: seems relevant"): call it only when the image itself contains the
|
||||
#: answer, at most ONE image per answer, the text answer is always
|
||||
#: given in full (the image supplements, never replaces). Same OpenAI
|
||||
#: function shape as the other three (one required string arg ``path``,
|
||||
#: the combined ``source/path`` identity — the bare-path-will-not-
|
||||
#: resolve teaching mirrored from ``read``).
|
||||
SHOW_IMAGE_TOOL: dict[str, Any] = {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "show_image",
|
||||
"description": (
|
||||
"Show a standalone image document to the user as part of "
|
||||
"your answer. Call it only when the image itself contains "
|
||||
"the answer — a diagram, screenshot, or photo of the thing "
|
||||
"the user asked about — never for a merely related image. "
|
||||
"Show at most one image per answer. The image supplements "
|
||||
"your text answer — answer fully in text either way. Its "
|
||||
"`path` is the combined `source/path` string, exactly as "
|
||||
"shown in the `ls` output or the `<documents>` blocks — a "
|
||||
"bare document path will not resolve. Call one tool at a "
|
||||
"time — wait for this result before your next call."
|
||||
),
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"path": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"The image document to show, as the combined "
|
||||
"`source/path` string exactly as shown in the "
|
||||
"`ls` output (e.g. "
|
||||
"'homelab/docs/architecture.png'). A bare "
|
||||
"document path (without the source name) will "
|
||||
"not resolve. Show at most one image per "
|
||||
"answer — a second call is refused."
|
||||
),
|
||||
}
|
||||
},
|
||||
"required": ["path"],
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def agent_tools(settings: Settings) -> list[dict[str, Any]]:
|
||||
"""The grounded turn's tool offer list (phase 124, task 02).
|
||||
|
||||
The three base tools (:data:`AGENT_TOOLS`, byte-identical) plus
|
||||
:data:`SHOW_IMAGE_TOOL` when ``settings.answer_images``
|
||||
(``BOR_ANSWER_IMAGES``) is true — otherwise exactly the three base
|
||||
tools (the pre-phase-124 offer, byte-identical: no 4th tool in the
|
||||
wire ``tools`` array). A FRESH list on every call — callers never
|
||||
mutate the result or the constants (``run_agent`` passes it straight
|
||||
to the LLM request).
|
||||
"""
|
||||
tools: list[dict[str, Any]] = list(AGENT_TOOLS)
|
||||
if settings.answer_images:
|
||||
tools.append(SHOW_IMAGE_TOOL)
|
||||
return tools
|
||||
|
||||
#: Tool refusal texts (phase 37): rejected calls count in nothing
|
||||
#: (``holder.tool_calls`` tracks executed calls); the round cap bounds
|
||||
#: their pathological repetition (phase 45). The in-context line is a
|
||||
@@ -407,6 +472,24 @@ ALREADY_IN_CONTEXT = (
|
||||
UNKNOWN_TOOL = "Unknown tool."
|
||||
MISSING_READ_ARGS = "read requires a string argument 'path'."
|
||||
MISSING_SEARCH_ARGS = "grep requires a string argument 'pattern'."
|
||||
MISSING_SHOW_ARGS = "show_image requires a string argument 'path'."
|
||||
|
||||
#: The ``show_image`` refusals (phase 124, task 02): the same contract
|
||||
#: as every other refusal — counts in nothing, consumes a round,
|
||||
#: changes no context. ``NOT_AN_IMAGE_DOC``: one ``{arg}`` field, the
|
||||
#: argument echoed as passed — a text doc is refused, never silently
|
||||
#: converted to a read (LOCKED A5). ``ALREADY_SHOWING``: the LOCKED A3
|
||||
#: one-image-per-answer gate, enforced SERVER-side (the model is taught
|
||||
#: it in the tool description + the prompt clause; the handler is the
|
||||
#: actual gate).
|
||||
NOT_AN_IMAGE_DOC = (
|
||||
"'{arg}' is not an image document — only standalone image "
|
||||
"documents can be shown as answers."
|
||||
)
|
||||
ALREADY_SHOWING = (
|
||||
"Already showing an image this answer — do not show another one. "
|
||||
"Answer from what you have."
|
||||
)
|
||||
|
||||
#: The read-truncation notice (phase 95, task 01) — appended, after the
|
||||
#: shared :data:`TRUNCATION_MARKER`, to a ``read`` result whose document
|
||||
@@ -1166,12 +1249,19 @@ class AgentHolder:
|
||||
as today — this list only carries the truncation signal the agent
|
||||
loop turns into :class:`app.rag.llm.ToolResultPiece` values (task 02
|
||||
surfaces them to the UI).
|
||||
``shown_images`` (phase 124, task 02): the documents ``show_image``
|
||||
showed, in call order — at most one in practice (LOCKED A3: a
|
||||
second call is refused with :data:`ALREADY_SHOWING`, which appends
|
||||
nothing). Read by ``app.api.chat`` after the stream for the done
|
||||
frame's ``answer_image`` field (task 04) — the ``read_docs``
|
||||
precedent (an API-layer read-after-stream signal).
|
||||
"""
|
||||
|
||||
read_docs: list[Document] = field(default_factory=list)
|
||||
tool_calls: int = 0
|
||||
scaffold_stripped: int = 0
|
||||
read_truncations: list[tuple[str, int, int]] = field(default_factory=list)
|
||||
shown_images: list[Document] = field(default_factory=list)
|
||||
|
||||
|
||||
def _execute_tool(
|
||||
@@ -1327,6 +1417,51 @@ def _execute_tool(
|
||||
f"{marker}"
|
||||
f"{doc.content}"
|
||||
)
|
||||
if call.name == "show_image":
|
||||
# Phase 124 (task 02; LOCKED A3/A5): the 4th agent tool — the
|
||||
# model shows a standalone image document as part of its
|
||||
# answer. Offered only when ``settings.answer_images``
|
||||
# (``agent_tools``); the handler enforces the contract
|
||||
# server-side — image docs only, ONE image per answer. Every
|
||||
# refusal below counts in nothing and consumes a round, like
|
||||
# every other refusal.
|
||||
raw_path = call.arguments.get("path")
|
||||
arg = raw_path.strip() if isinstance(raw_path, str) else ""
|
||||
if not arg:
|
||||
return MISSING_SHOW_ARGS
|
||||
doc, _source, _path = _resolve_path(db, arg)
|
||||
if doc is None:
|
||||
# The same phase-72 "did you mean …?" teaching as the read
|
||||
# branch (a refusal — not counted, no context).
|
||||
return _no_document_refusal(db, arg)
|
||||
if not doc.is_image:
|
||||
# LOCKED A5: a text doc is refused, never silently
|
||||
# converted to a read — the tool shows IMAGE documents
|
||||
# only.
|
||||
return NOT_AN_IMAGE_DOC.format(arg=arg)
|
||||
if holder.shown_images:
|
||||
# LOCKED A3: one image per answer — server-enforced, not
|
||||
# just taught.
|
||||
return ALREADY_SHOWING
|
||||
holder.shown_images.append(doc)
|
||||
holder.tool_calls += 1
|
||||
# The confirming result: a NEW first line (absent from ``read``
|
||||
# — the model must know the show registered) followed by the
|
||||
# exact document block ``read`` returns for an image doc at-or-
|
||||
# under-cap (the ``Document …`` header + phase-106 date line +
|
||||
# :data:`IMAGE_DOC_MARKER` — the header contract the E2E mock
|
||||
# pins on ``read`` results is deliberately reused here). An
|
||||
# image doc's content is the summary-capped vision description
|
||||
# — short by construction (phase 122, task 03), so NO
|
||||
# truncation machinery; the plain block stays correct even if
|
||||
# it ever exceeded the cap.
|
||||
return (
|
||||
"Image shown to the user — it is now part of your answer.\n"
|
||||
f"Document {doc.source}/{doc.path}:\n"
|
||||
f"date: {doc.created_at:%Y-%m-%d}\n"
|
||||
f"{IMAGE_DOC_MARKER}\n"
|
||||
f"{doc.content}"
|
||||
)
|
||||
if call.name == "grep":
|
||||
raw_pattern = call.arguments.get("pattern")
|
||||
pattern = raw_pattern.strip() if isinstance(raw_pattern, str) else ""
|
||||
@@ -1485,9 +1620,15 @@ async def run_agent(
|
||||
# Phase 45: no per-tool budgets — the tools stay offered for the
|
||||
# whole turn, bounded by the round cap. ``0`` is the no-tools kill
|
||||
# switch: exactly one request with ``tools=None`` (the pre-phase-37
|
||||
# path).
|
||||
# path). Phase 124 (task 02): the offer list is :func:`agent_tools`
|
||||
# — the three base tools (byte-identical) plus ``show_image`` only
|
||||
# when ``settings.answer_images`` (``BOR_ANSWER_IMAGES``) is on;
|
||||
# off (the default) the offer is element-for-element the
|
||||
# pre-phase-124 list.
|
||||
max_rounds = settings.agent_max_rounds
|
||||
tools: list[dict[str, Any]] | None = AGENT_TOOLS if max_rounds > 0 else None
|
||||
tools: list[dict[str, Any]] | None = (
|
||||
agent_tools(settings) if max_rounds > 0 else None
|
||||
)
|
||||
rounds = 0
|
||||
while True:
|
||||
calls: list[ToolCallPiece] = []
|
||||
|
||||
+67
-2
@@ -258,6 +258,43 @@ TOOLS_SECTION: str = (
|
||||
"</tools>"
|
||||
)
|
||||
|
||||
#: The ``show_image`` contract clause (phase 124, task 03, LOCKED A5) —
|
||||
#: appended INSIDE the ``<tools>`` section of the HIGH prompt, after the
|
||||
#: existing three-tool copy and before the closing tag, ONLY when
|
||||
#: :func:`build_high_prompt` is called with ``show_image=True`` (the
|
||||
#: ``BOR_ANSWER_IMAGES`` toggle, via ``app.api.chat``). It teaches the
|
||||
#: same relevance gate as the ``show_image`` tool description in
|
||||
#: :mod:`app.rag.agent` (LOCKED A5: the gate is the LLM's decision — no
|
||||
#: server-side image scoring), plus the image-doc nature the tool
|
||||
#: description cannot fully carry: the text of an image document (in a
|
||||
#: ``<documents>`` block or a ``read`` result) is a description
|
||||
#: GENERATED from the image, not the image's own words. The four gate
|
||||
#: elements, all stated here: (1) the image-doc nature; (2) the
|
||||
#: "contains the answer" bar — call it only when THAT image is what the
|
||||
#: user actually wants to see, never a merely related image, never to
|
||||
#: pad an answer; (3) at most ONE image per answer (also enforced
|
||||
#: server-side, task 02's ``ALREADY_SHOWING`` refusal); (4) the text
|
||||
#: answer is still provided in full — the image supplements, never
|
||||
#: replaces. ``TOOLS_SECTION`` itself stays BYTE-IDENTICAL (the
|
||||
#: pre-phase pins in :mod:`tests.unit.test_prompt_lock` hold); a
|
||||
#: ``show_image=False`` build (the default — every pre-phase call site)
|
||||
#: is byte-identical to the pre-task output. The ``<tools>`` marker the
|
||||
#: E2E mock keys on is untouched either way — the clause rides on its
|
||||
#: presence, not the wording. The LOW/deflection prompt never carries
|
||||
#: the clause (a deflected turn has no tools and never shows an image —
|
||||
#: the phase's locked design).
|
||||
SHOW_IMAGE_CLAUSE: str = (
|
||||
"`show_image` displays a standalone image document as part of your "
|
||||
"answer. The text of an image document — in a <documents> block or "
|
||||
"a `read` result — is a description GENERATED from the image, not "
|
||||
"the image's own words. Call `show_image` only when that image is "
|
||||
"what the user actually wants to see: a diagram, screenshot, or "
|
||||
"photo of the very thing you are answering about — never for a "
|
||||
"merely related image, and never to pad an answer. Show at most one "
|
||||
"image per answer. The image supplements your text — answer fully "
|
||||
"in text either way."
|
||||
)
|
||||
|
||||
|
||||
def history_to_messages(
|
||||
history: Sequence[HistoryTurn],
|
||||
@@ -426,6 +463,7 @@ def build_high_prompt(
|
||||
notes: Sequence[str] | None = None,
|
||||
kb_overview: str | None = None,
|
||||
folder_lines: Sequence[str] = (),
|
||||
show_image: bool = False,
|
||||
) -> str:
|
||||
"""Grounded turn: locked persona (+ steering, + KB overview) + the
|
||||
top-ranked documents' SUMMARY blocks + the ``<tools>`` instructions
|
||||
@@ -434,7 +472,8 @@ def build_high_prompt(
|
||||
phase 118, LOCKED A6 re-revising A7: the ``<documents>`` section
|
||||
seeds SUMMARIES, never full texts — full text enters the context
|
||||
only through the capped ``read`` tool; phase 119, LOCKED A4: the
|
||||
suggested-folder context lines, below).
|
||||
suggested-folder context lines, below; phase 124, task 03, LOCKED
|
||||
A5: the optional :data:`SHOW_IMAGE_CLAUSE`, below).
|
||||
|
||||
Section order: ``<relevance>`` → ``<knowledge_base>`` → ``<tuning>``
|
||||
→ ``<documents>`` → ``<tools>``; empty steering/overview omit their
|
||||
@@ -442,6 +481,23 @@ def build_high_prompt(
|
||||
cap — not the prompt — decides whether the tools are actually
|
||||
offered to the model, see :mod:`app.rag.agent`).
|
||||
|
||||
*show_image* (phase 124, task 03, LOCKED A5): when true (the
|
||||
caller passes the ``BOR_ANSWER_IMAGES`` toggle —
|
||||
``app.api.chat``'s grounded plan is the only call site that does),
|
||||
the :data:`SHOW_IMAGE_CLAUSE` rides INSIDE the ``<tools>`` section
|
||||
— appended after the existing three-tool copy and before the
|
||||
closing ``</tools>`` tag, so the ``<tools>`` marker the E2E mock
|
||||
keys on is untouched and the clause reaches the model together with
|
||||
the tool teaching. The section order is otherwise UNCHANGED and
|
||||
:data:`TOOLS_SECTION` itself is never edited: the built section is
|
||||
``TOOLS_SECTION``'s text minus its final ``</tools>`` + the clause
|
||||
+ the closing tag. The default ``False`` keeps EVERY pre-existing
|
||||
call byte-identical (the pre-task pins in
|
||||
:mod:`tests.unit.test_prompts` / :mod:`tests.unit.test_prompt_lock`
|
||||
hold untouched). The deflection prompt (:func:`build_deflect_prompt`)
|
||||
never carries the clause — a deflected turn has no tools and never
|
||||
shows an image.
|
||||
|
||||
When at least one block is present, the section leads with the
|
||||
start-here :data:`SUGGEST_INTRO` line (before the first block — the
|
||||
phase-15 ``_STEERING_INTRO`` / phase-31 ``_KB_INTRO`` precedent): the
|
||||
@@ -506,7 +562,16 @@ def build_high_prompt(
|
||||
for part in (build_kb_section(kb_overview or ""), build_steering_section(notes or [])):
|
||||
if part:
|
||||
prompt += "\n" + part
|
||||
return prompt + "\n<documents>\n" + body + "\n</documents>\n" + TOOLS_SECTION
|
||||
if show_image:
|
||||
# Phase 124 (task 03, LOCKED A5): the clause rides inside
|
||||
# <tools> — TOOLS_SECTION's text minus its final </tools>, the
|
||||
# clause (on its own line, after the existing three-tool copy),
|
||||
# then the closing tag. TOOLS_SECTION itself is unedited and the
|
||||
# show_image=False path keeps the exact pre-task concatenation.
|
||||
tools_section = TOOLS_SECTION[: -len("</tools>")] + SHOW_IMAGE_CLAUSE + "\n</tools>"
|
||||
else:
|
||||
tools_section = TOOLS_SECTION
|
||||
return prompt + "\n<documents>\n" + body + "\n</documents>\n" + tools_section
|
||||
|
||||
|
||||
def build_deflect_prompt(
|
||||
|
||||
+69
-10
@@ -242,6 +242,21 @@ class ChatDoneEvent(BaseModel):
|
||||
citation chip — while ``sources`` stays the citation surface. The
|
||||
field is ADDITIVE: old clients ignore unknown fields (PLAN §4 house
|
||||
contract) and old frames without it parse with the default ``[]``.
|
||||
|
||||
Phase 124 (task 04): ``answer_image`` — the brain answer's
|
||||
FIRST-CLASS image: the standalone image document the LLM chose to
|
||||
show as part of its answer (its ``show_image`` pick, behind
|
||||
``BOR_ANSWER_IMAGES``; LOCKED A5 — the relevance gate is the model's
|
||||
decision). It is the shown document's :class:`SourceRef` built by
|
||||
the shared :func:`app.rag.retriever.source_ref_with_image` (source /
|
||||
path / title + the ``image_url`` bytes route), distinct from a
|
||||
citation in ``sources`` — the document may be cited as well, but
|
||||
this key IS the answer's image (the UI's answer figure, task 06).
|
||||
``None`` (every deflected turn — they run no agent — every
|
||||
image-less turn, and the whole pre-phase) is DROPPED on
|
||||
serialization — ABSENT, never ``null`` — so those frames stay
|
||||
byte-identical to pre-phase (the byte-identical-off contract, task
|
||||
01; the :class:`SourceRef.image_url` omission precedent).
|
||||
"""
|
||||
|
||||
type: str = "done"
|
||||
@@ -249,6 +264,25 @@ class ChatDoneEvent(BaseModel):
|
||||
sources: list[SourceRef]
|
||||
related: list[SourceRef] = []
|
||||
suggestions: list[str] = []
|
||||
#: Phase 124 (task 04) — see the class docstring. A single
|
||||
#: :class:`SourceRef` (no list — LOCKED A3: one image per answer);
|
||||
#: the ref's own phase-83 field caps bind.
|
||||
answer_image: SourceRef | None = None
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
"""The phase-124 answer-image omission rule: ``answer_image``
|
||||
``None`` (every deflected / image-less / pre-phase turn)
|
||||
serializes WITHOUT the key — ABSENT, never ``null`` — so those
|
||||
done frames stay byte-identical to pre-phase (the
|
||||
:class:`SourceRef.image_url` / phase-123 ``image`` omission
|
||||
precedents). A shown ref keeps all its fields (its own
|
||||
serializer drops a ``None`` ``image_url`` — a shown doc is an
|
||||
image doc, so in practice it always carries it)."""
|
||||
data = handler(self)
|
||||
if self.answer_image is None:
|
||||
data.pop("answer_image", None)
|
||||
return data
|
||||
|
||||
|
||||
class ChatErrorEvent(BaseModel):
|
||||
@@ -967,22 +1001,47 @@ class ChatMessage(BaseModel):
|
||||
# omission precedent). The path is ≤ 500 chars — no phase-83
|
||||
# cap pressure (it is never a data URL, LOCKED A5).
|
||||
image: str | None = Field(default=None, max_length=500)
|
||||
# Phase 124 (task 04): the brain answer's FIRST-CLASS image — the
|
||||
# shown document's :class:`SourceRef` (the LLM's ``show_image``
|
||||
# pick, ``BOR_ANSWER_IMAGES``), on the BRAIN record only. It is a
|
||||
# DIFFERENT thing from the user record's phase-123 ``image`` (the
|
||||
# stored PATH STRING of the question's attachment): that field is
|
||||
# untouched and the two are semantically distinct — the attachment
|
||||
# belongs to the question, the answer image to the brain's reply
|
||||
# (a user record never carries ``answer_image``, a brain record
|
||||
# never carries ``image``). A single ref — no length cap needed
|
||||
# (the ref's own phase-83 field caps bind). The done-time
|
||||
# auto-save (``rememberBrainTurn``, task 06) persists the done
|
||||
# frame's ``answer_image`` into this key, and the restore path
|
||||
# re-renders the answer figure from it — the field must exist so
|
||||
# the ``extra="forbid"`` boundary does NOT 422 those saves (the
|
||||
# phase-113 lesson, applied up front: a missing key silently
|
||||
# unsaved every grounded brain record). ``None`` (every pre-phase
|
||||
# record, every image-less turn) serializes WITHOUT the key —
|
||||
# ABSENT, never ``null`` (the :meth:`_drop_image_when_absent`
|
||||
# serializer below), so pre-phase and image-less chats round-trip
|
||||
# byte-identically (the phase-50 contract).
|
||||
answer_image: SourceRef | None = None
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _drop_image_when_absent(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
"""The phase-123 image omission rule: ``image: None`` (every
|
||||
text-only record, and every pre-phase-123 record) serializes
|
||||
WITHOUT the key — ABSENT, never ``null`` — so the stored
|
||||
``bor.chat.v1`` JSONB and the saved/shared wire shape stay
|
||||
byte-identical to pre-phase for text-only chats (the phase-50
|
||||
round-trip contract). A record WITH an image keeps the path —
|
||||
the user bubble, the refreshed page, and the shared chat all
|
||||
render it from the served route (the image is part of the
|
||||
chat's content, so it rides the same public/credential-
|
||||
is-the-id trust model)."""
|
||||
"""The image omission rules: ``image: None`` (every text-only
|
||||
record, and every pre-phase-123 record) and ``answer_image:``
|
||||
``None`` (every record without a shown answer image, and every
|
||||
pre-phase-124 record) serialize WITHOUT the keys — ABSENT,
|
||||
never ``null`` — so the stored ``bor.chat.v1`` JSONB and the
|
||||
saved/shared wire shape stay byte-identical to pre-phase for
|
||||
those records (the phase-50 round-trip contract; the
|
||||
phase-122 ``SourceRef.image_url`` omission precedent). A
|
||||
record WITH one keeps it — the user bubble, the refreshed page,
|
||||
and the shared chat all render the image from the served route
|
||||
(the image is part of the chat's content, so it rides the same
|
||||
public/credential-is-the-id trust model)."""
|
||||
data = handler(self)
|
||||
if self.image is None:
|
||||
data.pop("image", None)
|
||||
if self.answer_image is None:
|
||||
data.pop("answer_image", None)
|
||||
return data
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user