phase: 125_image_anchor_context
All completion criteria verified. The repository is left functional with all changes in the working tree (no commits made, per the harness protocol). ## Final report — phase 125 `image_anchor_context` (verification pass) All 6 tasks were already complete; this pass verified every criterion and ran the full gates: - **Migration 0023** `documents.image_context` (TEXT nullable, reversible) — applied (`alembic current` = 0023 head); round-trip test (4 passed) exercises real upgrade → downgrade 0022 → upgrade. - **Criterion 1** (referenced image enriched + context line in retrieval): integration `test_markdown_referenced_image_gets_context_and_enriched_chunks` + E2E `test_sync_enriches_the_image_doc_with_markdown_context`, `test_chat_answer_quotes_the_context_line` (wire prompt pinned: `<document>` block leads with the one-line `Referenced in …: "alt"` line) — verified present and passing. - **Criterion 2** (standalone image byte-identical): E2E `test_standalone_*` (NULL `image_context`, block body exactly the bare description, no `Referenced in` anywhere in the prompt) + unit `test_describe_image_without_context_is_byte_identical` — passing. - **Criterion 3** (markdown-first: record stored on markdown doc, image picks up context later): integration `test_markdown_indexed_before_image_stores_record_and_image_picks_it_up` + post-index enrichment pass test (markdown file gone from disk) — passing. - **Gates:** `uv run pytest` → 2938 passed, exit 0 · `uv run pytest --cov=app --cov-report=term-missing` → exit 0, TOTAL **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings. - **Phase E2E in isolation:** `uv run pytest tests/e2e/test_image_anchor_context.py -v --no-cov` → 4 passed. - **Mock-LLM change regression check:** neighboring image E2E suites green in isolation — `test_smoke` (3), `test_image_documents` (16), `test_chat_image_questions` (17), `test_answer_images` (17). - No defects found; no code changes needed. No live-infrastructure changes (local dev DB only). No commit made (harness commits). - Next pending phase: check `.agents/phases/todo/` after the harness moves this phase.
This commit is contained in:
@@ -48,6 +48,8 @@ from app.rag.prompts import (
|
||||
SUGGEST_INTRO,
|
||||
TOOLS_SECTION,
|
||||
_base,
|
||||
_document_body,
|
||||
_extract_context_line,
|
||||
build_deflect_prompt,
|
||||
build_high_prompt,
|
||||
build_kb_section,
|
||||
@@ -74,6 +76,9 @@ def _doc(
|
||||
content: str,
|
||||
title: str,
|
||||
summary: str | None = None,
|
||||
*,
|
||||
is_image: bool = False,
|
||||
image_context: str | None = None,
|
||||
) -> Document:
|
||||
return Document(
|
||||
id=uuid.uuid4(),
|
||||
@@ -85,6 +90,9 @@ def _doc(
|
||||
content_hash="0" * 64,
|
||||
created_at=_FIXTURE_CREATED_AT,
|
||||
summary=summary,
|
||||
is_image=is_image,
|
||||
image_path="/tmp/pic.png" if is_image else None,
|
||||
image_context=image_context,
|
||||
)
|
||||
|
||||
|
||||
@@ -1141,3 +1149,301 @@ def test_show_image_clause_is_last_for_every_high_shape() -> None:
|
||||
assert on.index(section) < on.index("<tools>")
|
||||
assert on.index("K8s deploy notes.") < on.index("<tools>") # the summary still seeds
|
||||
assert SHOW_IMAGE_CLAUSE not in off # the toggle-off twin carries none of it
|
||||
|
||||
|
||||
# ---------- phase 125 (task 05, LOCKED A4): the image context line ----------
|
||||
# An enriched image doc (tasks 03/04: non-NULL ``image_context`` in the
|
||||
# "Referenced in …" block shape) leads its ``<document>`` suggestion
|
||||
# block with ONE short context line — the referring markdown's identity
|
||||
# + the reference's alt text — before the description, so the LLM knows
|
||||
# WHY the image exists, not only what it shows. Every other doc (every
|
||||
# text doc, every non-enriched image doc) is byte-identical to the
|
||||
# pre-phase body.
|
||||
|
||||
#: The exact stored block shape (tasks 03/04): one "Referenced in"
|
||||
#: block per referring markdown file, blank-line joined; the
|
||||
#: ``Context:`` body may span many lines (the surrounding paragraphs).
|
||||
CTX_SINGLE = (
|
||||
"Referenced in Homelab/homelab/kubernetes.md:\n"
|
||||
'- Alt text: "Cluster layout"\n'
|
||||
" Context: The diagram shows the three Talos nodes and Cilium.\n"
|
||||
"It is referenced again from the runbook."
|
||||
)
|
||||
CTX_TWO_BLOCKS = CTX_SINGLE + "\n\nReferenced in Homelab/ops/runbook.md:\n- Alt text: \"\""
|
||||
CTX_ALT_EMPTY = "Referenced in Homelab/ops/runbook.md:\n- Alt text: \"\""
|
||||
|
||||
#: The task-04 pending record — stored on MARKDOWN (text) docs, with no
|
||||
#: ``Referenced in`` header at all.
|
||||
MD_PENDING_RECORD = (
|
||||
'IMAGE_CONTEXT:\n- [image.png] (line 42): "Alt text here" — 2 paragraphs of context'
|
||||
)
|
||||
|
||||
|
||||
def _image_doc(
|
||||
image_context: str | None = None, summary: str | None = "A diagram of the cluster."
|
||||
) -> Document:
|
||||
return _doc(
|
||||
"cluster-diagram.png",
|
||||
"A diagram of the cluster.",
|
||||
"cluster-diagram",
|
||||
summary=summary,
|
||||
is_image=True,
|
||||
image_context=image_context,
|
||||
)
|
||||
|
||||
|
||||
# ---------- _extract_context_line: the parser contract ----------
|
||||
|
||||
|
||||
def test_extract_context_line_single_block_with_alt() -> None:
|
||||
"""One block with an alt → the short one-line locator: the stored
|
||||
identity (``<source>/<path>`` exactly as the header carries it) +
|
||||
the reference's alt between quotes."""
|
||||
assert _extract_context_line(CTX_SINGLE) == (
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
|
||||
)
|
||||
|
||||
|
||||
def test_extract_context_line_context_body_never_renders() -> None:
|
||||
"""The ``Context:`` paragraphs (LOCKED A3: they already reached the
|
||||
vision model at enrichment time) stay out of the line — the line is
|
||||
the locator + purpose, short enough not to bloat the prompt."""
|
||||
line = _extract_context_line(CTX_SINGLE)
|
||||
assert line is not None
|
||||
assert "The diagram shows" not in line
|
||||
assert "referenced again" not in line
|
||||
assert "Context:" not in line
|
||||
assert line.count("\n") == 0 # genuinely ONE line
|
||||
|
||||
|
||||
def test_extract_context_line_empty_alt_renders_the_bare_identity() -> None:
|
||||
"""A reference without an alt (````) → the clause is the
|
||||
bare identity — nothing to quote after the colon."""
|
||||
assert _extract_context_line(CTX_ALT_EMPTY) == "Referenced in Homelab/ops/runbook.md"
|
||||
|
||||
|
||||
def test_extract_context_line_two_blocks_join_with_semicolon() -> None:
|
||||
"""One clause per referring file after a single prefix — the second
|
||||
block's empty alt renders bare."""
|
||||
assert _extract_context_line(CTX_TWO_BLOCKS) == (
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"; '
|
||||
"Homelab/ops/runbook.md"
|
||||
)
|
||||
|
||||
|
||||
def test_extract_context_line_uses_the_first_reference_only() -> None:
|
||||
"""A block with two references (two ``- Alt text:`` bullets) → the
|
||||
FIRST reference's alt is the purpose; the second stays out."""
|
||||
text = (
|
||||
"Referenced in Homelab/a.md:\n"
|
||||
'- Alt text: "first purpose"\n'
|
||||
" Context: some paragraph.\n"
|
||||
'- Alt text: "second purpose"'
|
||||
)
|
||||
assert _extract_context_line(text) == 'Referenced in Homelab/a.md: "first purpose"'
|
||||
assert "second purpose" not in (_extract_context_line(text) or "")
|
||||
|
||||
|
||||
def test_extract_context_line_multiline_context_body_is_ignored() -> None:
|
||||
"""The ``Context:`` value is paragraph-joined (blank lines inside)
|
||||
— the parser walks it without breaking the block."""
|
||||
text = (
|
||||
"Referenced in Homelab/a.md:\n"
|
||||
'- Alt text: "purpose"\n'
|
||||
" Context: para one.\n"
|
||||
"\n"
|
||||
"para two, and it contains a line that\n"
|
||||
"looks like context continuation."
|
||||
)
|
||||
assert _extract_context_line(text) == 'Referenced in Homelab/a.md: "purpose"'
|
||||
|
||||
|
||||
def test_extract_context_line_alt_with_inner_quotes_survives() -> None:
|
||||
"""The block writer does not escape quotes: the greedy alt capture
|
||||
(between the opening quote and the line's final quote) keeps an
|
||||
alt with inner ``"`` intact."""
|
||||
text = 'Referenced in Homelab/x.md:\n- Alt text: "He said "hi""'
|
||||
assert _extract_context_line(text) == 'Referenced in Homelab/x.md: "He said "hi""'
|
||||
|
||||
|
||||
def test_extract_context_line_spurious_header_is_dropped() -> None:
|
||||
"""A look-alike ``Referenced in …:`` line WITHOUT its ``- Alt text:``
|
||||
bullet (context text that merely looks like a header) parses to
|
||||
``None`` — never a clause from the stray line."""
|
||||
assert _extract_context_line(
|
||||
"Referenced in Homelab/fake.md:\nsome context line that is not a bullet"
|
||||
) is None
|
||||
# A header with NO following line at all (end of value) is spurious too.
|
||||
assert _extract_context_line("Referenced in Homelab/fake.md:") is None
|
||||
# Spurious then REAL: only the real block's clause survives.
|
||||
mixed = (
|
||||
"Referenced in Homelab/fake.md:\nsome context line\n\n"
|
||||
+ CTX_SINGLE
|
||||
)
|
||||
assert _extract_context_line(mixed) == (
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
|
||||
)
|
||||
|
||||
|
||||
def test_extract_context_line_header_after_header_drops_the_bulletless_one() -> None:
|
||||
"""Two headers in a row: the first (never followed by its bullet) is
|
||||
dropped and the second block parses normally."""
|
||||
text = "Referenced in Homelab/fake.md:\nReferenced in Homelab/real.md:\n- Alt text: \"alt\""
|
||||
assert _extract_context_line(text) == 'Referenced in Homelab/real.md: "alt"'
|
||||
|
||||
|
||||
def test_extract_context_line_markdown_pending_record_is_none() -> None:
|
||||
"""The task-04 ``IMAGE_CONTEXT:`` record (the MARKDOWN doc's stored
|
||||
``image_context``) has no ``Referenced in`` header — it parses to
|
||||
``None`` (and it rides a text doc, which the ``is_image`` guard
|
||||
excludes from the line anyway)."""
|
||||
assert _extract_context_line(MD_PENDING_RECORD) is None
|
||||
|
||||
|
||||
def test_extract_context_line_blank_and_empty_are_none() -> None:
|
||||
assert _extract_context_line("") is None
|
||||
assert _extract_context_line(" \n \t ") is None
|
||||
# Garbage that contains the keyword somewhere is not a block.
|
||||
assert _extract_context_line("I was referenced in Homelab/a.md: yesterday") is None
|
||||
|
||||
|
||||
# ---------- _document_body: the block-body contract ----------
|
||||
|
||||
|
||||
def test_document_body_enriched_image_doc_leads_with_the_context_line() -> None:
|
||||
"""The completion criterion at the body level: the context line
|
||||
leads, a blank line, then the description — the exact two-part
|
||||
shape."""
|
||||
body = _document_body(_image_doc(image_context=CTX_SINGLE))
|
||||
assert body == (
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n\n'
|
||||
"A diagram of the cluster."
|
||||
)
|
||||
|
||||
|
||||
def test_document_body_image_doc_null_context_is_byte_identical_to_pre_phase() -> None:
|
||||
"""LOCKED A4 (the byte-identity criterion): a standalone / pre-phase
|
||||
image doc (NULL ``image_context``) → the body is EXACTLY the
|
||||
stripped summary — no line, no blank, no drift."""
|
||||
doc = _image_doc(image_context=None)
|
||||
assert _document_body(doc) == "A diagram of the cluster."
|
||||
# And the built prompt matches a text twin (same fields, not an
|
||||
# image) to the byte.
|
||||
twin = _doc("cluster-diagram.png", "A diagram of the cluster.", "cluster-diagram")
|
||||
assert build_high_prompt([doc]) == build_high_prompt([twin])
|
||||
|
||||
|
||||
def test_document_body_text_doc_with_image_context_is_unchanged() -> None:
|
||||
"""The guard is ``is_image``: a TEXT doc whose ``image_context``
|
||||
carries the task-04 pending record (the markdown case) keeps the
|
||||
pre-phase body — the record never renders."""
|
||||
doc = _doc("guide.md", "CONTENT", "Guide", summary="Guide summary.",
|
||||
image_context=MD_PENDING_RECORD)
|
||||
assert _document_body(doc) == "Guide summary."
|
||||
assert "IMAGE_CONTEXT" not in build_high_prompt([doc])
|
||||
assert "Referenced in" not in build_high_prompt([doc])
|
||||
|
||||
|
||||
def test_document_body_image_doc_unparsable_context_is_unchanged() -> None:
|
||||
"""Graceful degradation: an image doc whose ``image_context`` parses
|
||||
to nothing (owner-edited / unknown shape) → the description only,
|
||||
exactly pre-phase — the prompt build never raises."""
|
||||
for junk in ("", " ", "referenced in lowercase.md: no header shape", MD_PENDING_RECORD):
|
||||
assert _document_body(_image_doc(image_context=junk)) == "A diagram of the cluster."
|
||||
|
||||
|
||||
def test_document_body_enriched_image_doc_without_summary_precedes_the_preview() -> None:
|
||||
"""The line leads the fallback body too (the preview + marker of
|
||||
LOCKED A5) — the "context before the description" contract holds on
|
||||
both body shapes."""
|
||||
doc = _image_doc(image_context=CTX_SINGLE, summary=None)
|
||||
doc.content = "A" * 400 + "B" * 600 # past the 400-char preview cap
|
||||
body = _document_body(doc)
|
||||
line = 'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
|
||||
assert body.startswith(line + "\n\n")
|
||||
assert body[len(line) + 2 :] == "A" * 400 + "\n" + TRUNCATION_MARKER
|
||||
assert "B" * 40 not in body # beyond the cut, as pre-phase
|
||||
|
||||
|
||||
def test_enriched_image_doc_with_summary_makes_no_settings_call(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""House pattern (the LOCKED A5 preview-cap contract): the
|
||||
context-line path for a summary-bearing doc is pure — a dead
|
||||
``get_settings`` must not break such a build (extraction is pure;
|
||||
the cap is read on the fallback path only)."""
|
||||
from app.rag import prompts as prompts_mod
|
||||
|
||||
doc = _image_doc(image_context=CTX_SINGLE)
|
||||
|
||||
def _dead() -> None: # pragma: no cover - must never be called
|
||||
raise AssertionError("get_settings() called for a summarized doc")
|
||||
|
||||
monkeypatch.setattr(prompts_mod, "get_settings", _dead)
|
||||
assert _document_body(doc) == (
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n\n'
|
||||
"A diagram of the cluster."
|
||||
)
|
||||
|
||||
|
||||
# ---------- build_high_prompt: the line in the built block ----------
|
||||
|
||||
|
||||
def test_high_prompt_enriched_image_doc_block_exact_shape() -> None:
|
||||
"""The full block in the built prompt: identity attributes
|
||||
unchanged (phase 106 date appended after title), then the context
|
||||
line, a blank line, the description — the pre-phase block shape
|
||||
with the line spliced in."""
|
||||
doc = _image_doc(image_context=CTX_SINGLE)
|
||||
prompt = build_high_prompt([doc])
|
||||
assert (
|
||||
'<document source="Homelab" path="cluster-diagram.png" '
|
||||
'title="cluster-diagram" date="2024-06-15">\n'
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n'
|
||||
"\n"
|
||||
"A diagram of the cluster.\n"
|
||||
"</document>"
|
||||
) in prompt
|
||||
# The line is the FIRST thing in the block body (before the
|
||||
# description), inside <documents>.
|
||||
line = 'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
|
||||
block_start = prompt.index('<document source="Homelab"')
|
||||
assert prompt.index("<documents>") < prompt.index(line)
|
||||
assert prompt.index(line) < prompt.index("A diagram of the cluster.")
|
||||
assert block_start < prompt.index(line) < prompt.index("</document>")
|
||||
# The Context: body stays out of the whole prompt.
|
||||
assert "The diagram shows the three Talos nodes" not in prompt
|
||||
|
||||
|
||||
def test_high_prompt_enriched_image_doc_mixed_with_text_docs() -> None:
|
||||
"""A suggested tier with both text docs and one enriched image doc:
|
||||
the line rides ONLY the image doc's block; the text blocks are
|
||||
byte-identical to their summaries (no line, no drift)."""
|
||||
text_doc = _doc("kubernetes.md", "CONTENT", "K8s", summary="K8s summary.")
|
||||
img = _image_doc(image_context=CTX_SINGLE)
|
||||
prompt = build_high_prompt([text_doc, img])
|
||||
assert prompt.count('Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"') == 1
|
||||
# The text block is the bare summary, exactly as pre-phase.
|
||||
assert (
|
||||
'<document source="Homelab" path="kubernetes.md" '
|
||||
'title="K8s" date="2024-06-15">\nK8s summary.\n</document>'
|
||||
) in prompt
|
||||
# The image block carries the line + description.
|
||||
assert (
|
||||
'title="cluster-diagram" date="2024-06-15">\n'
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n\n'
|
||||
"A diagram of the cluster.\n</document>"
|
||||
) in prompt
|
||||
# Ordering unchanged: the text block (first doc) still precedes the
|
||||
# image block (second doc) — the line never re-orders the tier.
|
||||
assert prompt.index('path="kubernetes.md"') < prompt.index('path="cluster-diagram.png"')
|
||||
|
||||
|
||||
def test_high_prompt_image_doc_without_context_is_byte_identical_to_pre_phase() -> None:
|
||||
"""LOCKED A4 at the prompt level: an image doc with NULL
|
||||
``image_context`` builds the EXACT pre-phase prompt (the text-twin
|
||||
equality — no context line anywhere, not even a stray blank)."""
|
||||
img = _image_doc(image_context=None)
|
||||
twin = _doc("cluster-diagram.png", "A diagram of the cluster.", "cluster-diagram")
|
||||
assert build_high_prompt([img]) == build_high_prompt([twin])
|
||||
assert "Referenced in" not in build_high_prompt([img])
|
||||
|
||||
Reference in New Issue
Block a user