phase: 125_image_anchor_context
Build and Push Containers / build-and-push-app (push) Successful in 2m8s
Build and Push Containers / build-and-push-db (push) Failing after 1m38s

All completion criteria verified. The repository is left functional with all changes in the working tree (no commits made, per the harness protocol).

## Final report — phase 125 `image_anchor_context` (verification pass)

All 6 tasks were already complete; this pass verified every criterion and ran the full gates:

- **Migration 0023** `documents.image_context` (TEXT nullable, reversible) — applied (`alembic current` = 0023 head); round-trip test (4 passed) exercises real upgrade → downgrade 0022 → upgrade.
- **Criterion 1** (referenced image enriched + context line in retrieval): integration `test_markdown_referenced_image_gets_context_and_enriched_chunks` + E2E `test_sync_enriches_the_image_doc_with_markdown_context`, `test_chat_answer_quotes_the_context_line` (wire prompt pinned: `<document>` block leads with the one-line `Referenced in …: "alt"` line) — verified present and passing.
- **Criterion 2** (standalone image byte-identical): E2E `test_standalone_*` (NULL `image_context`, block body exactly the bare description, no `Referenced in` anywhere in the prompt) + unit `test_describe_image_without_context_is_byte_identical` — passing.
- **Criterion 3** (markdown-first: record stored on markdown doc, image picks up context later): integration `test_markdown_indexed_before_image_stores_record_and_image_picks_it_up` + post-index enrichment pass test (markdown file gone from disk) — passing.
- **Gates:** `uv run pytest` → 2938 passed, exit 0 · `uv run pytest --cov=app --cov-report=term-missing` → exit 0, TOTAL **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings.
- **Phase E2E in isolation:** `uv run pytest tests/e2e/test_image_anchor_context.py -v --no-cov` → 4 passed.
- **Mock-LLM change regression check:** neighboring image E2E suites green in isolation — `test_smoke` (3), `test_image_documents` (16), `test_chat_image_questions` (17), `test_answer_images` (17).
- No defects found; no code changes needed. No live-infrastructure changes (local dev DB only). No commit made (harness commits).
- Next pending phase: check `.agents/phases/todo/` after the harness moves this phase.
This commit is contained in:
2026-09-27 05:53:12 -04:00
parent 62e4f6b572
commit e0307fe3ff
60 changed files with 6583 additions and 82 deletions
+88
View File
@@ -517,6 +517,28 @@ Implements just enough of the aipi surface:
marker question always gets the echo whatever the honesty gate
says. Verified 2026-09-26: no existing E2E question or fixture
file contains the phrase, so every other suite is unaffected.
- user message containing ``quote the image reference``
(``CONTEXT_LINE_TRIGGER``, phase 125 task 06 — the image-anchor
context suite ``tests/e2e/test_image_anchor_context.py``) -> the
deterministic ZERO-READ answer that QUOTES THE CONTEXT LINE of the
seeded image doc's ``<document>`` block (the same image-doc pick
as ``DIAGRAM_TRIGGER``): an ENRICHED image doc's block (phase 125
task 05, LOCKED A4) leads with the one-line
``Referenced in <source>/<path>: "<alt>"`` context line BEFORE
the description, so the mock's quote — the block's exact first
line — proves the line WAS in the prompt (and that its text is
the task-05 one-line rendering, not the stored multi-line block).
A non-enriched (standalone) image doc's block is the bare
description (byte-identical pre-phase): its first line never
starts with ``Referenced in `` (the mock's fixed description
cannot begin that way), so the mock answers that the image
carries no reference line — the byte-identity criterion's
deterministic witness. Checked BEFORE the ``DEFLECT_MODE``
branch (the marker lives in the user message — the
``TABLE_TRIGGER``/``HISTORY_TRIGGER`` convention), so a marker
question always gets the echo whatever the honesty gate says.
Verified 2026-09-27: no existing E2E question or fixture file
contains the phrase, so every other suite is unaffected.
Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic
dead-endpoint behavior for the retry E2E suite (``tests/e2e/
@@ -1843,6 +1865,26 @@ SHOW_IMAGE_TRIGGER = "show me the picture of"
#: says.
DIAGRAM_TRIGGER = "which diagram is in your notes"
#: The phase-125 (task 06, image-anchor context, LOCKED A4) trigger
#: (case-insensitive): the mock QUOTES THE CONTEXT LINE of the seeded
#: image doc's ``<document>`` block (the same image-doc pick as
#: :data:`DIAGRAM_TRIGGER`). For an ENRICHED image doc the block
#: leads with the one-line ``Referenced in <source>/<path>: "<alt>"``
#: line (the task-05 rendering of the stored ``image_context``,
#: BEFORE the description) — quoting the block's first line proves
#: the line reached the prompt; for a non-enriched (standalone) doc
#: the block is the bare description (byte-identical pre-phase) and
#: its first line never starts with ``Referenced in `` (the mock's
#: fixed description cannot begin that way) — the mock then answers
#: that the image carries no reference line. Checked BEFORE the
#: ``DEFLECT_MODE`` branch in :func:`compose_answer` (the marker
#: lives in the user message — the TABLE/HISTORY convention), so a
#: marker question always gets the echo whatever the honesty gate
#: says. Disjoint from every other trigger phrase; verified
#: 2026-09-27: no existing E2E question or fixture file contains the
#: phrase, so every other suite is unaffected.
CONTEXT_LINE_TRIGGER = "quote the image reference"
#: The agent's ``show_image`` SUCCESS result's confirming first line
#: (``app.rag.agent``'s ``show_image`` branch — the line ``read``'s
#: result never carries): the SHOW_IMAGE flow's follow-up request
@@ -1878,6 +1920,26 @@ def _seeded_image_doc(system: str) -> tuple[str, str, str] | None:
return None
def _seed_context_line(block_content: str) -> str | None:
"""The phase-125 context line of a seeded image doc's block
(task 06, LOCKED A4) — or ``None``.
An ENRICHED image doc's ``<document>`` block leads with the
one-line ``Referenced in …`` rendering (the task-05
:func:`app.rag.prompts._extract_context_line` output, BEFORE the
description): its first line, verbatim, is the line the mock
quotes (the prompt-injection echo convention). A non-enriched
block is the bare description — its first line never starts with
``Referenced in `` (the mock's fixed
:data:`IMAGE_DESCRIPTION_ANSWER` cannot begin that way), so
``None`` pins the byte-identical pre-phase shape.
"""
first = block_content.split("\n", 1)[0].strip()
if first.startswith("Referenced in "):
return first
return None
def _show_image_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-124 SHOW_IMAGE request (see the module
docstring). Stateless over the messages, like the other marker
@@ -2467,6 +2529,32 @@ def compose_answer(body: dict[str, Any]) -> str:
else "Your notes don't carry that diagram — there is nothing to show. "
"(Deterministic mock answer for E2E.)"
)
elif CONTEXT_LINE_TRIGGER in user.lower():
# Phase 125 (task 06, image-anchor context, LOCKED A4): the
# ZERO-READ context-line echo — the mock QUOTES THE CONTEXT
# LINE of the seeded image doc's <document> block (the block's
# FIRST line — the task-05 one-line rendering for an enriched
# doc; a standalone doc's block is the bare description, whose
# first line never matches — the "no reference line" answer
# is the byte-identity witness). Checked BEFORE the
# DEFLECT_MODE branch (the marker lives in the user message —
# the TABLE/HISTORY convention), so a marker question always
# gets the echo whatever the honesty gate says.
image_doc = _seeded_image_doc(system)
if image_doc is None:
answer = (
"Your notes don't carry that image — there is nothing to reference. "
"(Deterministic mock answer for E2E.)"
)
else:
context_line = _seed_context_line(image_doc[2])
answer = (
f"The notes reference the image thus: {context_line} "
"(Deterministic mock answer for E2E.)"
if context_line is not None
else "The image stands alone — the notes carry no reference line for it. "
"(Deterministic mock answer for E2E.)"
)
elif "DEFLECT_MODE" in system:
answer = (
"Ah — I haven't done anything like that, so I don't want to make stuff up! "
+623
View File
@@ -0,0 +1,623 @@
"""Phase 125 E2E (Playwright): image-anchor context — the user path.
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_image_anchor_context.py -v --no-cov
The suite's app instance runs with ``BOR_IMAGES=true`` and
``BOR_ANSWER_IMAGES=false`` (module env override — the conftest
per-suite-app pattern, leak guards included): the fixture archive
carries ``docs/guide.md`` — a markdown file whose ``![alt](path)``
reference NAMES ``docs/diagram.png`` in the SAME source — so the real
upload → sync flow drives the whole phase-125 pipeline (LOCKED
A1/A2): the task-03 disk scan finds the reference at image-index time,
the vision model receives the context after the locked describe prompt,
and the image doc's ``image_context`` (and so its ``content``,
``summary`` and embedded chunks) becomes
``<context>\\n\n<description>``.
* the sync leaves the image doc ENRICHED (the ``Referenced in`` block
on ``image_context``, the enriched content + chunks, the markdown
doc's record NULL — the image was indexed first, the task-03 path
owns that case);
* a grounded chat turn whose suggested tier carries the enriched
image doc gets the task-05 context line: the mock (the
``CONTEXT_LINE_TRIGGER`` branch — the prompt parser extended for
this phase) QUOTES THE EXACT LINE of the image doc's ``<document>``
block, so the rendered answer AND the captured wire prompt prove
the block LEADS with the one-line ``Referenced in
<source>/<path>: "<alt>"`` rendering before the summary (the stored
block's shape carries the alt on the next line as a bullet — the
quote pins the task-05 rendering, not the stored block);
* a STANDALONE image (a second phase of the same app — the KB is
re-seeded with a single PNG no markdown references) stays
byte-identical to pre-phase: ``image_context`` NULL, the bare
description everywhere, the ``<document>`` block the description
alone, and the mock's "no reference line" answer (the deterministic
witness that no context line was in the prompt).
"""
from __future__ import annotations
import base64
import json
import os
import re
import subprocess
import sys
import zipfile
from collections.abc import Iterator
from pathlib import Path
import httpx
import pytest
from playwright.sync_api import Browser, Page, expect
from sqlalchemy import select, text
from app.config import Settings
from app.db import SessionLocal
from app.models import Chunk, Document
from e2e.auth_helpers import login
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
from e2e.mock_llm import IMAGE_DESCRIPTION_ANSWER
REPO = Path(__file__).resolve().parents[2]
GIT_SOURCES_URL = "/git-sources.html"
#: Phase A — the anchored source: the markdown references the image
#: from the SAME folder (``diagram.png`` next to ``guide.md`` → the
#: resolved in-source path ``docs/diagram.png``).
SOURCE_NAME = "e2e-anchor" # the archive stem (archive_source_name)
MD_PATH = "docs/guide.md"
DOC_PATH = "docs/diagram.png"
#: Phase B — the standalone source: a single PNG no markdown names.
STANDALONE_SOURCE_NAME = "e2e-anchor-standalone"
LOGO_PATH = "logo.png"
#: The fixture markdown — the reference on line 5, two context
#: paragraphs (the task-02 default window).
GUIDE_MD = (
"# Backup runbook\n"
"\n"
"The restore order diagram explains the backup procedure.\n"
"\n"
"![Restore order](diagram.png)\n"
"\n"
"Only run the restore script after the checks pass.\n"
)
#: The task-03-shape combined context the same-source disk scan builds
#: for ``docs/diagram.png`` (one referring markdown, one reference,
#: the task-02 window as the ``Context:`` body — the ``# Backup
#: runbook`` heading is the first of the two paragraphs before the
#: reference; the N=2 window keeps it).
BLOCK = (
"Referenced in e2e-anchor/docs/guide.md:\n"
'- Alt text: "Restore order"\n'
" Context: # Backup runbook\n"
"\n"
"The restore order diagram explains the backup procedure.\n"
"\n"
"Only run the restore script after the checks pass."
)
#: The enriched image doc's stored ``content`` (AND ``summary`` — the
#: phase-122 ``summary = content`` convention): the context FIRST
#: (for retrieval), then the vision description.
ENRICHED = f"{BLOCK}\n\n{IMAGE_DESCRIPTION_ANSWER}"
#: The content chunk's stored text — the plain-text packer joins the
#: enriched paragraphs with single newlines (the chunker's
#: normalization, PLAN §5); the ``is_summary`` chunk keeps the raw
#: enriched text.
PACKED = "\n".join(ENRICHED.split("\n\n"))
#: The task-05 ONE-LINE rendering of :data:`BLOCK` (the context line
#: that leads the enriched image doc's ``<document>`` block body):
#: the referring markdown's identity + the reference's alt, the
#: ``Context:`` body never rendered.
CONTEXT_LINE = 'Referenced in e2e-anchor/docs/guide.md: "Restore order"'
#: The grounded question — the mock's ``CONTEXT_LINE_TRIGGER`` marker
#: (the zero-read context-line echo) + the phase-122 grounding phrase
#: (token-overlap cosine ≈0.47 against the enriched diagram doc and
#: ≈0.37 against the standalone logo's bare description — both clear
#: the mock-calibrated gate (0.30), the FTS leg corroborates).
ANCHOR_QUESTION = (
"quote the image reference — what is shown in the homelab network diagram?"
)
#: The mock's two deterministic answers for the marker (the exact
#: branch texts — the phase-125 pins key off them).
ANSWER_WITH_LINE = (
f"The notes reference the image thus: {CONTEXT_LINE} "
"(Deterministic mock answer for E2E.)"
)
ANSWER_STANDALONE = (
"The image stands alone — the notes carry no reference line for it. "
"(Deterministic mock answer for E2E.)"
)
#: The honest-deflection opening (the gate must NOT fire on the
#: marker question — a deflected prompt carries no ``<documents>``
#: block, so the mock would never see the image doc).
DEFLECT_OPENING = "I haven't done anything like that"
UPLOAD_TIMEOUT_MS = 30_000
SYNC_TIMEOUT_MS = 60_000
SYNCED_LABEL = re.compile(r"^Synced \d{1,2}:\d{2}$")
#: A real 1×1 transparent PNG (the phase-122 fixture — the pipeline
#: is content-agnostic, the bytes just need to be plausible image
#: bytes).
PNG_1X1 = base64.b64decode(
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR4nGP4DwQACfsD/fteaysAAAAASUVORK5CYII="
)
# ---------------------------------------------------------------------------
# Module app: images ON (the story) — the conftest env pattern with the
# phase-122/125 knobs + leak guards.
# ---------------------------------------------------------------------------
def _app_env(mock_llm: int, app_port: int, scratch: Path) -> dict[str, str]:
"""The conftest app env (leak guards included) with the phase-122
knobs: ``BOR_IMAGES`` pinned EXPLICITLY true (the story) and
``BOR_ANSWER_IMAGES`` pinned EXPLICITLY false (the default — the
auto-cite/figure behavior is phase 124's story, not this phase's,
and an operator's local gitignored ``.env`` cannot leak it in:
both toggles are the contract under test's neighbors), the
image/upload homes in the suite's scratch dir (the app under test
must not write copies into the owner's real ``~/bor-sources``)."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{mock_llm}/v1"
)
# Mock-calibrated gate (conftest pattern) — the marker question
# grounds on these values (see ANCHOR_QUESTION).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
# Phase 67: instant retry waits + the code-default budget.
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
# Leak guards (conftest pattern).
env["BOR_GIT_SOURCES"] = ""
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(Settings.model_fields["suggestions"].default)
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
# Phase 122/124 toggles + the suite-private homes.
env["BOR_IMAGES"] = "true"
env["BOR_ANSWER_IMAGES"] = "false"
env["BOR_UPLOAD_DIR"] = str(scratch / "uploads")
env["BOR_SOURCES_DIR"] = str(scratch / "checkouts")
env["BOR_IMAGE_DIR"] = str(scratch / "images")
return env
@pytest.fixture(scope="module")
def app_server(mock_llm: int, tmp_path_factory: pytest.TempPathFactory) -> Iterator[str]:
"""The story app — ``BOR_IMAGES=true`` (the module env override;
the conftest session app is never started in this isolated run,
so no port clash)."""
scratch = tmp_path_factory.mktemp("bor_anchor")
port = int(os.environ.get("E2E_APP_PORT_ANCHOR", "8160"))
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(port), "--log-level", "warning"],
cwd=REPO,
env=_app_env(mock_llm, port, scratch=scratch),
)
try:
_wait_http(f"http://127.0.0.1:{port}/api/health")
yield f"http://127.0.0.1:{port}"
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
# ---------------------------------------------------------------------------
# Fixture archives
# ---------------------------------------------------------------------------
@pytest.fixture(scope="module")
def anchor_zip(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Phase A's archive: the markdown that references the image in
the SAME source (``docs/guide.md`` → ``docs/diagram.png``)."""
root = tmp_path_factory.mktemp("bor_anchor_zip")
docs = root / "docs"
docs.mkdir()
(docs / "guide.md").write_text(GUIDE_MD, encoding="utf-8")
(docs / "diagram.png").write_bytes(PNG_1X1)
archive = root / f"{SOURCE_NAME}.zip"
with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as zf:
zf.write(docs / "guide.md", arcname="docs/guide.md")
zf.write(docs / "diagram.png", arcname="docs/diagram.png")
return archive
@pytest.fixture(scope="module")
def standalone_zip(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Phase B's archive: ONE standalone PNG at the source root — no
markdown in the source references it (the byte-identical
pre-phase case)."""
root = tmp_path_factory.mktemp("bor_anchor_standalone_zip")
(root / LOGO_PATH).write_bytes(PNG_1X1)
archive = root / f"{STANDALONE_SOURCE_NAME}.zip"
with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as zf:
zf.write(root / LOGO_PATH, arcname=LOGO_PATH)
return archive
# ---------------------------------------------------------------------------
# DB + UI helpers
# ---------------------------------------------------------------------------
def _truncate_all() -> None:
"""Fresh KB + registry per phase (the E2E isolation pattern): the
suites share one Postgres, and a leftover row would corrupt the
counts. ``sources_meta`` (the KB generation counter) resets with
the KB — the sync re-creates the row."""
with SessionLocal() as db:
db.execute(text(
"TRUNCATE chunks, documents, query_log, kb_overview, "
"git_sources, sources_meta, folder_summaries"
))
db.commit()
def _health_db(app_url: str) -> bool:
try:
return httpx.get(f"{app_url}/api/health", timeout=5).json()["db"] == "up"
except Exception: # noqa: BLE001 — unreachable is the skip case
return False
def _upload_zip(page: Page, app_url: str, archive: Path, source: str) -> None:
"""The real admin upload flow: form login → the git-sources page →
pick the archive → submit → the terminal result line (phase 90:
"Uploaded <source> — press Sync sources to import it.")."""
login(page, app_url, next=GIT_SOURCES_URL)
expect(page).to_have_url(app_url + GIT_SOURCES_URL, timeout=30_000)
expect(page.locator("#sign-out-btn")).to_be_visible(timeout=15_000)
page.set_input_files("#archive-upload-file", str(archive))
page.click("#archive-upload-btn")
result = page.locator("#archive-upload-result")
expect(result).to_be_visible(timeout=UPLOAD_TIMEOUT_MS)
expect(result).to_have_text(
f"Uploaded {source} — press Sync sources to import it.",
timeout=UPLOAD_TIMEOUT_MS,
)
def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
"""The RAG page's "Sync sources" button (the scan — phase 90 A3):
click → "Syncing…" → the terminal "Synced HH:MM" label + the fresh
counts in #sync-result (the never-stale contract)."""
page.goto(app_url + "/sources.html")
btn = page.locator("#sync-btn")
expect(btn).to_be_visible(timeout=30_000)
expect(page.locator("#sync-label")).to_have_text(
re.compile(r"^Sync sources$|^Synced \d{1,2}:\d{2}$")
)
btn.click()
expect(btn).to_be_disabled()
expect(page.locator("#sync-label")).to_have_text("Syncing…")
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
expect(btn).to_be_enabled()
expect(page.locator("#sync-result")).to_have_text(expected_result)
def _captured_systems(mock_llm: int) -> list[str]:
"""The system prompts of the captured chat requests (the mock's
ring buffer — the wire-level lens on the EXACT prompt the app
built; only ``/v1/chat/completions`` bodies are captured, so after
a reset + one grounded zero-read turn there is exactly one)."""
bodies = httpx.get(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured", timeout=10).json()
systems: list[str] = []
for body in bodies:
for msg in body.get("messages", []):
if msg.get("role") == "system" and isinstance(msg.get("content"), str):
systems.append(msg["content"])
return systems
def _reset_capture(mock_llm: int) -> None:
httpx.post(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured/reset", timeout=10)
def _ask(page: Page, app_url: str, question: str) -> None:
"""The chat user path: login → the question → Send → the final
brain bubble (the Send button re-enabled = the turn is over)."""
login(page, app_url, next="/")
expect(page.locator("#kb-banner")).to_be_hidden()
page.fill("#message-input", question)
page.click("#send-btn")
bubble = page.locator(".msg.brain .bubble").last
expect(page.locator("#send-btn")).to_be_enabled(timeout=60_000)
expect(page.locator("#send-label")).to_have_text("Send")
expect(bubble).to_contain_text("Deterministic mock answer", timeout=30_000)
# ---------------------------------------------------------------------------
# The module seeds: phase A (the anchored archive) and phase B (the
# standalone archive) — each owns its truncate/re-import window.
# ---------------------------------------------------------------------------
@pytest.fixture(scope="module")
def seeded(
app_url: str,
browser: Browser,
anchor_zip: Path,
) -> Iterator[dict[str, str]]:
"""Phase A: upload the anchored archive through the admin UI, then
scan it with the RAG page's Sync button (the mock's vision model
describes the image in-process, with the markdown context). The
module app's ``/api/config`` flags are asserted first — the env
overrides must have reached the app under test, or the whole
premise of the suite is void."""
if not _health_db(app_url):
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
cfg = httpx.get(f"{app_url}/api/config", timeout=5).json()
assert cfg["images"] is True, "the module app must run with BOR_IMAGES=true"
_truncate_all()
page = browser.new_page(viewport={"width": 1280, "height": 800})
try:
_upload_zip(page, app_url, anchor_zip, SOURCE_NAME)
_sync_via_ui(page, app_url, expected_result="2 added")
with SessionLocal() as db:
doc = db.scalar(
select(Document).where(
Document.source == SOURCE_NAME, Document.path == DOC_PATH
)
)
assert doc is not None, "the uploaded image must become a document"
assert doc.is_image is True and doc.image_path is not None
finally:
page.close()
yield {"image_url": f"/api/documents/{doc.id}/image"}
_truncate_all()
@pytest.fixture(scope="module")
def standalone(
app_url: str,
browser: Browser,
standalone_zip: Path,
) -> Iterator[dict[str, str]]:
"""Phase B: a FRESH KB (the phase-A rows are truncated) seeded
with the standalone archive — a single PNG no markdown in the
source references (the byte-identical pre-phase case)."""
_truncate_all()
page = browser.new_page(viewport={"width": 1280, "height": 800})
try:
_upload_zip(page, app_url, standalone_zip, STANDALONE_SOURCE_NAME)
_sync_via_ui(page, app_url, expected_result="1 added")
with SessionLocal() as db:
doc = db.scalar(
select(Document).where(
Document.source == STANDALONE_SOURCE_NAME, Document.path == LOGO_PATH
)
)
assert doc is not None, "the uploaded image must become a document"
assert doc.is_image is True and doc.image_path is not None
finally:
page.close()
yield {"image_url": f"/api/documents/{doc.id}/image"}
_truncate_all()
# ---------------------------------------------------------------------------
# 1. Sync: the image doc is enriched with the markdown context
# ---------------------------------------------------------------------------
def test_sync_enriches_the_image_doc_with_markdown_context(
page: Page, app_url: str, seeded: dict[str, str]
) -> None:
"""LOCKED A1/A2 end-to-end: the markdown in the SAME source
references the image → after the sync the image doc's
``image_context`` is the ``Referenced in`` block, its ``content``
(AND ``summary`` — the phase-122 convention) is
``<context>\\n\n<description>`` (context first — for retrieval —
then the vision description), the content chunk is embedded from
the enriched text (the packer's single-newline form) and the
``is_summary`` chunk mirrors the raw enriched text. The referring
markdown's OWN ``image_context`` stays NULL: the image was indexed
BEFORE the markdown (``docs/diagram.png`` sorts first), so its
reference names an existing image doc — the task-03 path owns that
case, no pending record is written. The Sources page counts the
KB: 2 docs, 4 chunks (one content + one summary chunk per doc)."""
with SessionLocal() as db:
doc = db.scalar(
select(Document).where(
Document.source == SOURCE_NAME, Document.path == DOC_PATH
)
)
assert doc is not None
# The stored context (task 05's display source).
assert doc.image_context == BLOCK
# The enriched content — context FIRST, then the description —
# IS the summary (the phase-122 image convention).
assert doc.content == ENRICHED
assert doc.summary == ENRICHED
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
content_chunks = [c for c in chunks if not c.is_summary]
assert [c.content for c in content_chunks] == [PACKED], (
"the content chunk is embedded from the ENRICHED text"
)
summary_chunks = [c for c in chunks if c.is_summary]
assert len(summary_chunks) == 1 and summary_chunks[0].position == -1
assert summary_chunks[0].content == ENRICHED
for c in chunks:
assert c.embedding is not None and len(c.embedding) == 768
# The referring markdown — the image doc existed at its
# indexing (sort order), so NO pending record was written.
md = db.scalar(
select(Document).where(
Document.source == SOURCE_NAME, Document.path == MD_PATH
)
)
assert md is not None and md.is_image is False
assert md.image_context is None, "existing image: the record stays NULL"
# The KB on the Sources page: 2 docs (the markdown + the image),
# 4 chunks (one content + one is_summary per doc).
login(page, app_url)
expect(page.locator("#stat-docs")).to_have_text("2")
expect(page.locator("#stat-chunks")).to_have_text("4")
# ---------------------------------------------------------------------------
# 2. Chat: the enriched image doc's <document> block leads with the
# context line — the answer AND the wire prompt prove it
# ---------------------------------------------------------------------------
def test_chat_answer_quotes_the_context_line(
page: Page, app_url: str, mock_llm: int, seeded: dict[str, str]
) -> None:
"""LOCKED A4 end-to-end: a grounded turn whose suggested tier
carries the ENRICHED image doc → the ``<document>`` block LEADS
with the ONE-LINE context line (the task-05 rendering of the
stored block: the referring markdown's identity + the reference's
alt, BEFORE the summary — for an enriched image doc the summary
is the enriched content, so the stored block rides in the body
too, but the header is the one-line rendering, never the stored
multi-line header). The mock's marker branch QUOTES THE EXACT
LINE of the block — so the rendered answer proves the one-line
rendering was in the prompt (the quoted alt sits on the header
line — the stored block's shape carries it on the next line as a
bullet), and the captured wire prompt pins the exact block shape:
line, blank line, enriched summary."""
_reset_capture(mock_llm)
_ask(page, app_url, ANCHOR_QUESTION)
bubble = page.locator(".msg.brain .bubble").last
# The mock's enriched answer — the EXACT context line, quoted.
expect(bubble).to_contain_text(ANSWER_WITH_LINE, timeout=30_000)
# Grounded (not deflected) — a deflected prompt would carry no
# <documents> block at all.
expect(bubble).not_to_contain_text(DEFLECT_OPENING)
# The wire prompt: exactly one chat request (grounded, zero tool
# calls), and the image doc's <document> block leads with the
# ONE-LINE context line — then a blank line, then the summary
# (for an enriched image doc the summary IS the enriched content —
# the phase-122 ``summary = content`` convention — so the stored
# block rides along in the body, but the block HEADS with the
# one-line rendering, never the stored header line).
systems = _captured_systems(mock_llm)
assert len(systems) == 1, "one grounded zero-read turn → one chat request"
system = systems[0]
start = system.index(f'<document source="{SOURCE_NAME}" path="{DOC_PATH}"')
body_start = system.index(">\n", start) + 2
body_end = system.index("\n</document>", body_start)
block_body = system[body_start:body_end]
# The exact block shape: the one-line context line (the alt QUOTED
# on the same line — the stored block's shape carries the alt on
# the NEXT line as a bullet, so this pins the task-05 rendering),
# a blank line, then the enriched summary.
assert block_body == f"{CONTEXT_LINE}\n\n{ENRICHED}"
# The line is inside <documents>, before the summary's block.
assert system.index("<documents>") < system.index(CONTEXT_LINE)
assert system.index(CONTEXT_LINE) < system.index("\n" + BLOCK)
# ---------------------------------------------------------------------------
# 3. Standalone image: the byte-identical pre-phase row
# ---------------------------------------------------------------------------
def test_standalone_image_keeps_null_context_and_bare_description(
page: Page, app_url: str, standalone: dict[str, str]
) -> None:
"""LOCKED A4 (the byte-identity criterion) on the ROW: a standalone
image (no markdown in the source references it) → ``image_context``
NULL, ``content`` (AND ``summary``) the BARE vision description,
every chunk the bare description — exactly the phase-122 shape.
The Sources page counts 1 doc / 2 chunks (one content + one
summary)."""
with SessionLocal() as db:
doc = db.scalar(
select(Document).where(
Document.source == STANDALONE_SOURCE_NAME, Document.path == LOGO_PATH
)
)
assert doc is not None
assert doc.image_context is None, "standalone: the context stays NULL"
assert doc.content == IMAGE_DESCRIPTION_ANSWER, "standalone: the bare description"
assert doc.summary == IMAGE_DESCRIPTION_ANSWER
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
assert len(chunks) == 2
assert all(c.content == IMAGE_DESCRIPTION_ANSWER for c in chunks)
for c in chunks:
assert c.embedding is not None and len(c.embedding) == 768
login(page, app_url)
expect(page.locator("#stat-docs")).to_have_text("1")
expect(page.locator("#stat-chunks")).to_have_text("2")
# ---------------------------------------------------------------------------
# 4. Standalone image: the <document> block is byte-identical in the
# prompt — no context line, the bare description alone
# ---------------------------------------------------------------------------
def test_standalone_answer_carries_no_context_line(
page: Page, app_url: str, mock_llm: int, standalone: dict[str, str]
) -> None:
"""LOCKED A4 (the byte-identity criterion) in the PROMPT: the SAME
marker question against the standalone KB → the image doc's
``<document>`` block body is EXACTLY the bare description (no
context line, no stray blank — the pre-phase block), and the mock
— which quotes the block's first line when it is a ``Referenced
in`` line — answers that the image carries no reference line: the
deterministic witness that the line was NOT in the prompt.
``Referenced in`` does not appear ANYWHERE in the system prompt."""
_reset_capture(mock_llm)
_ask(page, app_url, ANCHOR_QUESTION)
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(ANSWER_STANDALONE, timeout=30_000)
expect(bubble).not_to_contain_text(DEFLECT_OPENING)
systems = _captured_systems(mock_llm)
assert len(systems) == 1, "one grounded zero-read turn → one chat request"
system = systems[0]
start = system.index(
f'<document source="{STANDALONE_SOURCE_NAME}" path="{LOGO_PATH}"'
)
body_start = system.index(">\n", start) + 2
body_end = system.index("\n</document>", body_start)
block_body = system[body_start:body_end]
# The pre-phase block body: the description alone, verbatim.
assert block_body == IMAGE_DESCRIPTION_ANSWER
assert "Referenced in" not in system
+99
View File
@@ -2790,3 +2790,102 @@ def test_zero_read_text_only_turn_still_chips_nothing(
assert done["sources"] == [] # phase-119 read-docs-only rule intact
for ref in done["sources"] + done["related"]:
assert "image_url" not in ref
# ---------------------------------------------------------------------------
# Phase 125, task 05 — retrieval display: the suggested tier's ENRICHED
# image doc (non-NULL image_context, the task-03/04 block shape) leads
# its <document> block with the one-line context line (the referring
# markdown's identity + the reference's alt) before the description;
# the non-enriched image doc (NULL image_context) stays byte-identical
# to pre-phase (LOCKED A4).
# ---------------------------------------------------------------------------
def test_grounded_turn_suggested_enriched_image_doc_carries_context_line(
client, db, seeded_kb: FakeRagLLM
) -> None:
"""LOCKED A4 end-to-end: a grounded turn whose suggested tier
contains an enriched image doc (the task-03/04 ``Referenced in``
block stored on ``image_context``) → the system prompt's
``<document>`` block for that doc leads with the short context
line, a blank line, then the description — and the ``Context:``
paragraphs of the stored block never reach the prompt (the line is
the locator + purpose, not a duplicate of the paragraphs)."""
doc = _seed_image_doc_with_chunk(db)
doc.image_context = (
"Referenced in docs/homelab/kubernetes.md:\n"
'- Alt text: "Cluster layout"\n'
" Context: The diagram shows the three Talos nodes.\n"
"It is referenced again from the runbook."
)
db.commit()
scripted = FakeRagLLM() # zero-read grounded turn (the image seed block IS its content)
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
assert frames[-1]["deflected"] is False
system = scripted.seen_messages[0][0]["content"]
# The doc is in the suggested tier (retrieval is unchanged by the
# phase — the line rides on a doc the retrieval already picked).
block_start = system.index(f'<document source="docs" path="{doc.path}"')
block = system[
block_start : system.index("</document>", block_start) + len("</document>")
]
line = 'Referenced in docs/homelab/kubernetes.md: "Cluster layout"'
# The exact block shape: identity attrs (phase 106 date), the line,
# a blank line, the description (the phase-122 content == summary
# == vision description invariant).
assert block == (
f'<document source="docs" path="pic.png" title="pic" '
f'date="{_FIXTURE_CREATED_AT:%Y-%m-%d}">\n'
f"{line}\n"
"\n"
f"{doc.summary}\n"
"</document>"
)
# The line is the FIRST thing in the block body (before the
# description), inside <documents>.
assert system.index("<documents>") < system.index(line)
assert system.index(line) < system.index(doc.summary or "")
# The Context: body stays out of the whole prompt.
assert "The diagram shows the three Talos nodes." not in system
assert "referenced again from the runbook" not in system
assert "Context:" not in system
def test_grounded_turn_suggested_plain_image_doc_block_is_byte_identical(
client, db, seeded_kb: FakeRagLLM
) -> None:
"""LOCKED A4 (the byte-identity criterion) end-to-end: the SAME
flow with ``image_context`` left NULL (the standalone / pre-phase
image doc — the phase's default) → the ``<document>`` block is
EXACTLY the pre-phase block (the summary alone, no context line,
no stray blank) and the marker string does not appear anywhere in
the system prompt."""
doc = _seed_image_doc_with_chunk(db)
assert doc.image_context is None # the seed leaves it NULL (standalone)
scripted = FakeRagLLM()
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
try:
_, _, frames = _stream_chat(client, QUESTION)
finally:
fastapi_app.dependency_overrides.clear()
assert frames[-1]["deflected"] is False
system = scripted.seen_messages[0][0]["content"]
block_start = system.index(f'<document source="docs" path="{doc.path}"')
block = system[
block_start : system.index("</document>", block_start) + len("</document>")
]
# The pre-phase block: identity attrs + the description, verbatim.
assert block == (
f'<document source="docs" path="pic.png" title="pic" '
f'date="{_FIXTURE_CREATED_AT:%Y-%m-%d}">\n'
f"{doc.summary}\n"
"</document>"
)
assert "Referenced in" not in system
+6 -2
View File
@@ -869,9 +869,13 @@ def _image_llm[
def _patch_description(monkeypatch, description) -> None:
"""Pin the task-02 seam (``rag_importer._describe_or_skip``) to
*description*. Task 03 fills the seam with the CHAT model's vision
call — these end-to-end mechanics do not change with it."""
call — these end-to-end mechanics do not change with it. Phase 125
(task 03) widened the seam with the optional ``context`` kwarg (the
markdown reference context) — the fake accepts (and ignores) it:
these fixtures carry no image references, so the seam gets
``context=None``."""
async def _fake(llm, *, data, source, rel, full_path):
async def _fake(llm, *, data, source, rel, full_path, context=None):
return description
monkeypatch.setattr(rag_importer, "_describe_or_skip", _fake)
+446
View File
@@ -0,0 +1,446 @@
"""Integration: phase 125, task 03 — the image-anchor enrichment through
the REAL import pipeline (real compose Postgres, the real
``_describe_or_skip`` → ``describe_image`` seam, the mock vision
client — NO seam patch).
A markdown file referencing an image in the SAME source → the vision
call receives the markdown context (appended after the locked
``DESCRIBE_PROMPT``), the image doc's ``image_context`` is set, its
``content``/``summary`` is ``<context>\\n\\n<description>``, and its
chunks are embedded from the enriched content. A standalone image (no
markdown reference) → ``image_context`` NULL, ``content`` the bare
description, the bare locked prompt — byte-identical to pre-phase-125.
Phase 125, task 04 (the markdown-FIRST case) — a markdown indexed while
its referenced image has no document yet stores the reference on the
markdown doc's ``image_context`` (the ``IMAGE_CONTEXT:`` record); the
image that appears later picks the context up at index time (the
task-03 disk scan) or, when the markdown file is gone from disk by
then, through the post-index enrichment pass (the stored doc rows —
record as the index, stored content as the source). Pinned here: the
record-then-image sync pair (the record stored, not cleared), the
pass-only path (markdown file deleted from disk, ``prune=False``), and
the existing-image case (the markdown's record stays NULL — the
task-03 path owns that case).
The unit-level mechanics (the prefix shape, the scan's combined string,
the seam forwarding, the changed-image re-scan, the record formatter +
parser, the pass's idempotency/fail-soft) are pinned in
``tests/unit/test_image_anchor_context.py``; the task-06 story suite
pins the E2E surface.
"""
from __future__ import annotations
import asyncio
import base64
from sqlalchemy import select
from app.config import Settings
from app.models import Chunk, Document
from app.rag.importer import import_sources
from app.rag.summarizer import DESCRIBE_PROMPT
from tests.fakes import FakeEmbedder
#: A real 1×1 transparent PNG (the phase-122 fixture — the importer is
#: content-agnostic, the bytes just need to be plausible image bytes).
PNG_1X1 = base64.b64decode(
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR4nGP4DwQACfsD/fteaysAAAAASUVORK5CYII="
)
DESCRIPTION = (
"A network diagram of the homelab VLANs: the core switch, the "
"router, and three labeled subnets."
)
#: The exact context prefix the task-03 extension appends to the locked
#: prompt (``app.rag.summarizer.describe_image``).
CONTEXT_PREFIX = "\n\nThis image is referenced in documentation with the following context:\n"
class _MockVisionLLM(FakeEmbedder):
"""The phase-122 mock VISION client: the CHAT model answers the
multimodal describe call with the fixed :data:`DESCRIPTION` (the
text part is recorded — ``text_parts``); text (``lite``) calls keep
the ``FakeEmbedder`` behaviour. The REAL
``import_sources`` → ``_describe_or_skip`` → ``describe_image``
chain runs against it (no seam patch)."""
DESCRIPTION = DESCRIPTION
def __init__(self) -> None:
super().__init__()
self.chat_models: list[str | None] = []
self.text_parts: list[str] = []
async def chat(self, messages, model=None):
self.chat_calls.append(list(messages))
self.chat_models.append(model)
user = next((m["content"] for m in messages if m.get("role") == "user"), "")
if isinstance(user, list):
self.text_parts.append(user[0]["text"])
return self.DESCRIPTION
first = user.split()
return "Summary of " + (first[0] if first else "<empty>")
def _image_llm[
LLM: FakeEmbedder
](tmp_path, llm_cls: type[LLM] = _MockVisionLLM, **kwargs) -> LLM:
"""A fake LLM with the phase-122 image knobs (toggle ON; the image
dir defaults under *tmp_path*). *llm_cls* may be the mock vision
client (the default — ``text_parts``/``chat_models`` state)."""
kwargs.setdefault("_env_file", None)
kwargs.setdefault("images", True)
kwargs.setdefault("image_dir", str(tmp_path / "images"))
llm = llm_cls()
llm.settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
return llm
def _cleanup_source(db, source: str) -> None:
for doc in db.scalars(select(Document).where(Document.source == source)).all():
db.delete(doc)
db.commit()
#: A markdown file that references ``pic.png`` on line 3 — one context
#: paragraph before and one after (the task-02 window).
GUIDE_MD = (
"The restore order diagram explains the backup procedure.\n"
"\n"
"![Restore order](pic.png)\n"
"\n"
"Only run the restore script after the checks pass.\n"
)
def _expected_block(source: str) -> str:
"""The task-03-shape combined string *GUIDE_MD* yields for
``<source>/guide.md`` (the disk scan AND the task-04 pass build
exactly this)."""
return (
f"Referenced in {source}/guide.md:\n"
'- Alt text: "Restore order"\n'
" Context: The restore order diagram explains the backup procedure.\n"
"\n"
"Only run the restore script after the checks pass."
)
#: The ``IMAGE_CONTEXT:`` record the markdown-side bookkeeping (phase
#: 125, task 04, work 1) stores for *GUIDE_MD* — the reference on line
#: 3, 2 context paragraphs.
RECORD = (
"IMAGE_CONTEXT:\n"
'- [pic.png] (line 3): "Restore order" — 2 paragraphs of context'
)
def test_markdown_referenced_image_gets_context_and_enriched_chunks(
db, tmp_path
) -> None:
"""A markdown file referencing an image in the same source → sync
with ``images=True`` (the REAL seam — no patch) → the vision call
carries the context after the locked prompt, the image doc's
``image_context`` is set, its ``content`` includes the context
(context first, then the description), and its chunks are embedded
from the enriched content (the ``is_summary`` chunk mirrors it)."""
source_root = tmp_path / "anchoredkb"
source_root.mkdir()
(source_root / "guide.md").write_text(
"The backup procedure diagram below shows the restore order.\n"
"\n"
"![Backup procedure diagram](diagram.png)\n"
"\n"
"Run the restore script only after the diagram's checks pass.\n",
encoding="utf-8",
)
(source_root / "diagram.png").write_bytes(PNG_1X1)
llm = _image_llm(tmp_path)
try:
summary = asyncio.run(import_sources([source_root], llm, session=db))
assert (summary.files, summary.added, summary.images_failed) == (2, 2, 0)
doc = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "diagram.png"
)
)
assert doc is not None and doc.is_image is True and doc.image_path is not None
expected_context = (
f"Referenced in {source_root.name}/guide.md:\n"
'- Alt text: "Backup procedure diagram"\n'
" Context: The backup procedure diagram below shows the restore order.\n"
"\n"
"Run the restore script only after the diagram's checks pass."
)
# The vision model received the context — appended AFTER the
# locked prompt (the ``IMAGE_DESCRIPTION_MODE`` marker is
# untouched). The describe call (the ONLY multimodal one) went
# to the CHAT model (``chat_models`` also carries the markdown
# doc's ``lite`` summary call — the phase-30 path).
assert llm.chat_models[0] == llm.settings.llm_chat_model
assert llm.text_parts == [f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{expected_context}"]
# ``image_context`` stores the combined context (task 05's
# retrieval display reads from here).
assert doc.image_context == expected_context
# The enriched content — context FIRST, then the description —
# IS the summary (the phase-122 convention) and the chunks are
# embedded from it (768 dims, the summary chunk mirrors it). The
# plain-text packer joins the enriched paragraphs with single
# newlines (the chunker's normalization, PLAN §5), so the content
# chunk carries the packed form and the summary chunk the raw
# text.
expected_content = f"{expected_context}\n\n{DESCRIPTION}"
assert doc.content == expected_content
assert doc.summary == expected_content
packed_content = "\n".join(expected_content.split("\n\n"))
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
content_chunks = [c for c in chunks if not c.is_summary]
assert [c.content for c in content_chunks] == [packed_content]
summary_chunks = [c for c in chunks if c.is_summary]
assert len(summary_chunks) == 1 and summary_chunks[0].position == -1
assert summary_chunks[0].content == expected_content
# Both enriched forms reached the embedder (the markdown doc's
# two texts complete the four embedded strings).
embedded = [t for batch in llm.calls for t in batch]
assert len(embedded) == 4
assert packed_content in embedded and expected_content in embedded
for c in chunks:
assert c.embedding is not None and len(c.embedding) == 768
# The referencing markdown stays a plain text doc (the
# ``image_context`` column is an image-doc field — NULL).
md = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "guide.md"
)
)
assert md is not None and md.is_image is False and md.image_context is None
finally:
_cleanup_source(db, source_root.name)
def test_standalone_image_keeps_null_context_and_bare_description(db, tmp_path) -> None:
"""A standalone image (no markdown reference in the source — a
sibling markdown exists but references nothing) → ``image_context``
NULL, ``content`` the bare description, the vision message the bare
locked prompt — byte-identical to pre-phase-125 (the chunk set and
the embedded text are exactly the phase-122 shape)."""
source_root = tmp_path / "standalonekb"
source_root.mkdir()
(source_root / "notes.md").write_text("# Notes\n\nBody text.\n", encoding="utf-8")
(source_root / "diagram.png").write_bytes(PNG_1X1)
llm = _image_llm(tmp_path)
try:
summary = asyncio.run(import_sources([source_root], llm, session=db))
assert (summary.files, summary.added, summary.images_failed) == (2, 2, 0)
doc = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "diagram.png"
)
)
assert doc is not None and doc.is_image is True
assert doc.image_context is None, "standalone: the context stays NULL"
assert doc.content == DESCRIPTION, "standalone: the bare description"
assert doc.summary == DESCRIPTION
assert llm.text_parts == [DESCRIBE_PROMPT], "standalone: the bare prompt"
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
assert all(c.content == DESCRIPTION for c in chunks)
for c in chunks:
assert c.embedding is not None and len(c.embedding) == 768
finally:
_cleanup_source(db, source_root.name)
# ---------------------------------------------------------------------------
# Phase 125, task 04 — the markdown-FIRST case: the ``IMAGE_CONTEXT:``
# record on the markdown doc + the post-index enrichment pass.
# ---------------------------------------------------------------------------
def test_markdown_indexed_before_image_stores_record_and_image_picks_it_up(
db, tmp_path
) -> None:
"""Task 04's primary scenario — a markdown is indexed while the
image it references does not yet exist: the reference is stored on
the markdown doc's ``image_context`` (the ``IMAGE_CONTEXT:``
record). The image file then appears and is synced: the image doc
is enriched with the markdown's context (the task-03 disk scan —
the markdown is still on disk), and the markdown's record is NOT
cleared (ASSUMPTION 4)."""
source_root = tmp_path / "beforekb"
source_root.mkdir()
(source_root / "guide.md").write_text(GUIDE_MD, encoding="utf-8")
llm = _image_llm(tmp_path)
try:
# Run 1 — the markdown alone: the record is stored, no image
# doc exists.
s1 = asyncio.run(import_sources([source_root], llm, session=db))
assert (s1.files, s1.added) == (1, 1)
md = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "guide.md"
)
)
assert md is not None and md.image_context == RECORD
assert (
db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "pic.png"
)
)
is None
)
# Run 2 — the image appears: enriched at index time (the
# task-03 disk scan sees the on-disk markdown), the record
# survives.
(source_root / "pic.png").write_bytes(PNG_1X1)
s2 = asyncio.run(import_sources([source_root], llm, session=db))
assert (s2.added, s2.unchanged) == (1, 1)
block = _expected_block(source_root.name)
doc = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "pic.png"
)
)
assert doc is not None and doc.is_image is True
assert doc.image_context == block
expected_content = f"{block}\n\n{DESCRIPTION}"
assert doc.content == expected_content
assert doc.summary == expected_content
# The describe call carried the context after the locked prompt.
assert llm.text_parts == [f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{block}"]
md2 = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "guide.md"
)
)
assert md2 is not None and md2.image_context == RECORD, "the record is NOT cleared"
finally:
_cleanup_source(db, source_root.name)
def test_enrichment_pass_enriches_when_markdown_file_is_gone_from_disk(
db, tmp_path
) -> None:
"""The post-index enrichment pass (task 04, work 3) — the case the
task-03 disk scan CANNOT cover: the markdown is indexed first (the
record lands on its doc row), its file is then removed from disk
(``prune=False`` keeps the row), and only then does the image
appear. The image is indexed BARE (the disk scan finds no
reference), and the pass enriches it from the markdown doc's
STORED rows: the record identifies the referencing markdown, the
reference is re-extracted from the stored content, and the
re-describe goes out WITH the context (a second vision call — the
first, at index time, was bare)."""
source_root = tmp_path / "passkb"
source_root.mkdir()
(source_root / "guide.md").write_text(GUIDE_MD, encoding="utf-8")
llm = _image_llm(tmp_path)
try:
s1 = asyncio.run(import_sources([source_root], llm, session=db))
assert s1.added == 1
md = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "guide.md"
)
)
assert md is not None and md.image_context == RECORD
# The markdown file disappears from disk (``prune=False`` keeps
# the row); the image appears.
(source_root / "guide.md").unlink()
(source_root / "pic.png").write_bytes(PNG_1X1)
s2 = asyncio.run(import_sources([source_root], llm, session=db))
# The image is added (bare) and then enriched by the pass
# (updated) — both in this run.
assert (s2.files, s2.added, s2.updated) == (1, 1, 1)
block = _expected_block(source_root.name)
doc = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "pic.png"
)
)
assert doc is not None and doc.is_image is True
assert doc.image_context == block
expected_content = f"{block}\n\n{DESCRIPTION}"
assert doc.content == expected_content
assert doc.summary == expected_content
# The re-embedded pipeline (the pass replaced the bare chunks).
packed = "\n".join(expected_content.split("\n\n"))
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
assert [c.content for c in chunks if not c.is_summary] == [packed]
assert [c.content for c in chunks if c.is_summary] == [expected_content]
# Two vision calls: the bare index-time describe (the disk scan
# saw no reference) and the pass's context-carrying re-describe.
assert llm.text_parts == [DESCRIBE_PROMPT, f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{block}"]
# The markdown doc's record survives (ASSUMPTION 4).
md2 = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "guide.md"
)
)
assert md2 is not None and md2.image_context == RECORD
finally:
_cleanup_source(db, source_root.name)
def test_markdown_referencing_existing_image_keeps_null_record(db, tmp_path) -> None:
"""An EXISTING image doc (indexed before the markdown ever
existed): the markdown's record stays NULL when it is indexed (the
task-03 path owns that case — enrichment at IMAGE-index time), the
unchanged image is not re-scanned in that run, and the image picks
up the context on its NEXT re-index (bytes changed) via the
task-03 disk scan — the markdown's record is NULL throughout."""
source_root = tmp_path / "existingkb"
source_root.mkdir()
(source_root / "pic.png").write_bytes(PNG_1X1)
llm = _image_llm(tmp_path)
try:
s1 = asyncio.run(import_sources([source_root], llm, session=db))
assert s1.added == 1
# Run 2 — the markdown appears (the image is unchanged).
(source_root / "guide.md").write_text(GUIDE_MD, encoding="utf-8")
s2 = asyncio.run(import_sources([source_root], llm, session=db))
assert (s2.added, s2.unchanged) == (1, 1)
md = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "guide.md"
)
)
assert md is not None and md.image_context is None
doc = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "pic.png"
)
)
assert doc is not None and doc.image_context is None and doc.content == DESCRIPTION
# Run 3 — the image changes: the re-index re-scans the on-disk
# markdown (task 03) and enriches; the record is still NULL.
(source_root / "pic.png").write_bytes(PNG_1X1 + b"changed")
s3 = asyncio.run(import_sources([source_root], llm, session=db))
assert (s3.added, s3.updated, s3.unchanged) == (0, 1, 1)
block = _expected_block(source_root.name)
doc3 = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "pic.png"
)
)
assert doc3 is not None and doc3.image_context == block
assert doc3.content == f"{block}\n\n{DESCRIPTION}"
md3 = db.scalar(
select(Document).where(
Document.source == source_root.name, Document.path == "guide.md"
)
)
assert md3 is not None and md3.image_context is None
finally:
_cleanup_source(db, source_root.name)
+335
View File
@@ -0,0 +1,335 @@
"""Integration: migration 0023 (documents.image_context) schema contract
(phase 125, task 01).
Drives the **real Alembic engine** against the live dev database
(``podman compose up -d db``), mirroring the house pattern of
``test_migration_0022.py`` (information_schema assertions on the state
the migration must leave). The tests target the 0022 → 0023 step
explicitly so later migrations cannot break the pins:
* upgrade 0022 → 0023 → ``image_context`` exists with the full
contract — TEXT, NULLABLE, no server default (NULL is the
standalone-image / pre-phase value, LOCKED A1/A4) — while the 0022
``documents`` schema (``is_image`` + ``image_path`` from phase 122,
``content``/``content_hash`` NOT NULL, the (source, path) unique
constraint) survives;
* a ``documents`` row inserted while the DB is at 0022 (a pre-phase
image doc) reads ``image_context IS NULL`` after the upgrade — the
pre-phase round-trip pin;
* the ORM contract agrees: a freshly inserted image ``Document`` WITH
``image_context`` round-trips it through a fresh session, and one
WITHOUT (standalone image / text doc) reads ``image_context is None``;
* downgrade to 0022 → the column is GONE (A13) while the row and its
0022 columns survive; upgrade back to 0023 → the column is back
(round-trip).
The ``alembic`` fixture guarantees the DB ends at head even if a test
fails or the process is interrupted.
"""
from __future__ import annotations
import uuid
from collections.abc import Iterator
from typing import Any
import pytest
from alembic.config import Config
from sqlalchemy import text
from sqlalchemy.orm import Session
from alembic import command
from app.db import SessionLocal, db_available
from app.models import Document
SOURCE = "mig0023"
PATH_TEXT = "notes/readme.md"
PATH_IMAGE = "notes/diagram.png"
@pytest.fixture()
def alembic(db: Session) -> Iterator[Config]:
"""Real Alembic config bound to the dev DB (URL from app settings).
Starts at head (repairs an interrupted earlier run); teardown
upgrades to head no matter what happened, so the dev DB is never
left below head.
"""
if not db_available():
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
cfg = Config() # no alembic.ini file — env.py gets the URL from app config
cfg.set_main_option("script_location", "alembic")
command.upgrade(cfg, "head")
try:
yield cfg
finally:
# Release the test session's open transaction BEFORE the repair
# DDL: an idle-in-transaction SELECT holds an ACCESS SHARE lock
# on ``documents``, which would deadlock the repair's
# ``ALTER TABLE`` (0023) forever.
db.rollback()
command.upgrade(cfg, "head")
def _version(db: Session) -> str | None:
return db.execute(text("SELECT version_num FROM alembic_version")).scalar()
def _column(db: Session, column: str) -> tuple[Any, ...] | None:
"""(data_type, is_nullable, column_default) for one documents
column."""
row = db.execute(
text(
"SELECT data_type, is_nullable, column_default"
" FROM information_schema.columns"
" WHERE table_name = 'documents' AND column_name = :c"
),
{"c": column},
).fetchone()
return tuple(row) if row is not None else None
def _insert_sql(db: Session, path: str) -> uuid.UUID:
"""Insert one documents row with the PRE-0023 column set (the 0022
shape — ``image_context``, when present, is omitted so its NULL
backfill is what the row reads)."""
row_id = uuid.uuid4()
db.execute(
text(
"INSERT INTO documents (id, source, path, full_path, title,"
" content, content_hash)"
" VALUES (:id, :s, :p, :f, :t, :c, :h)"
),
{
"id": row_id,
"s": SOURCE,
"p": path,
"f": f"/tmp/{SOURCE}/{path}",
"t": path.rsplit("/", 1)[-1],
"c": "content",
"h": "0" * 64,
},
)
db.commit()
return row_id
def _delete(db: Session, row_id: uuid.UUID) -> None:
db.execute(text("DELETE FROM documents WHERE id = :id"), {"id": row_id})
db.commit()
def _unique_column_sets(db: Session) -> set[tuple[str, ...]]:
"""The column tuples of every UNIQUE constraint on ``documents``.
Column-based (not name-based): the integration suite's table
self-heal (``tests/integration/conftest.py``) rewrites bloated
tables via ``CREATE TABLE (LIKE …)``, which renames copied
constraints (PG auto-names them) — the (source, path) uniqueness
contract is what must hold, not the original name.
"""
rows = db.execute(
text(
"SELECT (SELECT string_agg(a.attname, ',' ORDER BY k.ord)"
" FROM unnest(c.conkey) WITH ORDINALITY k(attnum, ord)"
" JOIN pg_attribute a"
" ON a.attrelid = c.conrelid AND a.attnum = k.attnum)"
" FROM pg_constraint c"
" WHERE c.contype = 'u' AND c.conrelid = 'documents'::regclass"
)
).fetchall()
return {tuple(r[0].split(",")) for r in rows}
def test_upgrade_to_0023_adds_image_context(db: Session, alembic: Config) -> None:
"""Upgrade 0022 → 0023: ``image_context`` exists with the full
contract (TEXT, NULLABLE, no server default — NULL is the
standalone-image / pre-phase value, LOCKED A1) and is ABSENT at
0022; a pre-0023 row (a phase-122 image doc) backfills to NULL; and
the 0022 table contract survives the additive upgrade."""
command.downgrade(alembic, "0022") # start from the pre-0023 state
assert _version(db) == "0022"
assert _column(db, "image_context") is None, (
"image_context must be absent at 0022"
)
pre_id = _insert_sql(db, PATH_IMAGE) # the 0022 column set (a pre-125 row)
try:
command.upgrade(alembic, "0023")
assert _version(db) == "0023", "alembic_version must be at 0023"
context = _column(db, "image_context")
assert context is not None, "documents.image_context is missing"
assert context[0] == "text", "image_context must be TEXT"
assert context[1] == "YES", "image_context must be NULLABLE"
assert context[2] is None, (
"image_context must carry NO server default — NULL is the"
" standalone-image / pre-phase value (A1, A4)"
)
# The pre-0023 row (inserted with the 0022 column set) reads
# NULL — every pre-phase row keeps its byte-identical state.
row = db.execute(
text("SELECT image_context FROM documents WHERE id = :id"),
{"id": pre_id},
).fetchone()
assert row is not None, "the pre-0023 row must survive the upgrade"
assert row[0] is None, (
"a pre-phase image doc must backfill image_context to NULL"
)
# A row written without the column reads the same (an omitted
# image_context inserts NULL — the text-doc / standalone value).
new_id = _insert_sql(db, PATH_TEXT)
try:
backfilled = db.execute(
text("SELECT image_context FROM documents WHERE id = :id"),
{"id": new_id},
).fetchone()
assert backfilled == (None,), (
"an omitted image_context must read NULL"
)
finally:
_delete(db, new_id)
# The 0022 schema survives the additive upgrade.
is_image = _column(db, "is_image")
assert (
is_image is not None
and is_image[0] == "boolean"
and is_image[1] == "NO"
and is_image[2] == "false"
), "documents.is_image (0022) must keep its 0022 contract"
image_path = _column(db, "image_path")
assert (
image_path is not None
and image_path[0] == "text"
and image_path[1] == "YES"
), "documents.image_path (0022) must keep its 0022 contract"
content = _column(db, "content")
assert content is not None and content[0] == "text" and content[1] == "NO", (
"documents.content (0001) must keep its 0022 contract"
)
assert ("source", "path") in _unique_column_sets(db), (
"the (source, path) unique constraint must survive the upgrade"
)
finally:
_delete(db, pre_id)
def test_orm_image_context_round_trip(db: Session, alembic: Config) -> None:
"""The ORM contract agrees with the column contract: a freshly
inserted image ``Document`` WITH ``image_context`` round-trips the
string through a FRESH session, and one WITHOUT it (a standalone
image, LOCKED A1/A4) reads ``image_context is None``."""
command.upgrade(alembic, "head")
enriched = Document(
source=SOURCE,
path=PATH_IMAGE,
full_path=f"/tmp/{SOURCE}/{PATH_IMAGE}",
title="diagram",
content="A description of the diagram.",
content_hash="1" * 64,
is_image=True,
image_path="/srv/bor-images/diagram.png",
image_context=(
"![Deployment diagram](diagram.png)\n\nThe diagram\n"
"shows the cluster layout and how the VMs connect."
),
)
standalone = Document(
source=SOURCE,
path="standalone.png",
full_path=f"/tmp/{SOURCE}/standalone.png",
title="standalone",
content="A description of a standalone image.",
content_hash="2" * 64,
is_image=True,
image_path="/srv/bor-images/standalone.png",
)
db.add(enriched)
db.add(standalone)
db.commit()
try:
with SessionLocal() as fresh:
reloaded_enriched = fresh.get(Document, enriched.id)
assert reloaded_enriched is not None, "the image row must be readable"
assert reloaded_enriched.image_context == enriched.image_context, (
"image_context must round-trip the exact context text"
)
reloaded_standalone = fresh.get(Document, standalone.id)
assert reloaded_standalone is not None
assert reloaded_standalone.image_context is None, (
"an omitted image_context must read NULL (standalone)"
)
finally:
_delete(db, enriched.id)
_delete(db, standalone.id)
def test_downgrade_to_0022_drops_the_column(db: Session, alembic: Config) -> None:
"""Downgrade 0023 → 0022: the ``image_context`` column is gone
(A13 — fully reversible) while the row, its 0022 columns (including
the phase-122 image pair) and the table's unique constraint
survive."""
command.upgrade(alembic, "head")
row = Document(
source=SOURCE,
path=PATH_IMAGE,
full_path=f"/tmp/{SOURCE}/{PATH_IMAGE}",
title="diagram",
content="A description of the diagram.",
content_hash="3" * 64,
is_image=True,
image_path="/srv/bor-images/diagram.png",
image_context="![alt](diagram.png)\n\nSurrounding paragraph.",
)
db.add(row)
db.commit()
try:
command.downgrade(alembic, "0022")
assert _version(db) == "0022"
assert _column(db, "image_context") is None, (
"image_context must be dropped"
)
surviving = db.execute(
text(
"SELECT source, path, content, content_hash, is_image,"
" image_path FROM documents WHERE id = :id"
),
{"id": row.id},
).fetchone()
assert surviving is not None, "the row must survive the column drop"
assert surviving[0] == SOURCE and surviving[1] == PATH_IMAGE
assert surviving[2] == "A description of the diagram."
assert surviving[3] == "3" * 64
assert surviving[4] is True, "is_image (0022) must survive the drop"
assert surviving[5] == "/srv/bor-images/diagram.png"
assert ("source", "path") in _unique_column_sets(db), (
"the (source, path) unique constraint must survive the downgrade"
)
finally:
_delete(db, row.id)
# Repair: the fixture teardown re-upgrades to head.
def test_upgrade_round_trip_restores_the_column(db: Session, alembic: Config) -> None:
"""Downgrade to 0022, then upgrade back to 0023: the column is back
with the full contract (TEXT, NULLABLE, no server default) — a
clean upgrade → downgrade → upgrade round-trip (A13)."""
command.downgrade(alembic, "0022")
command.upgrade(alembic, "head")
assert _version(db) == "0023", (
"round-trip upgrade must land at head (0023)"
)
context = _column(db, "image_context")
assert context is not None, "documents.image_context must be back"
assert context[0] == "text", "image_context must be TEXT after the round-trip"
assert context[1] == "YES", (
"image_context must be NULLABLE after the round-trip"
)
assert context[2] is None, (
"image_context must still carry NO server default"
)
File diff suppressed because it is too large Load Diff
+19 -3
View File
@@ -283,11 +283,21 @@ def _patch_description(
"""Pin the task-02 seam (``importer._describe_or_skip``) to
*description* and record the BYTES it is handed (the digest-rule
pin). Task 03 replaces the seam's body — the importer-side
mechanics under test here do not change with it."""
mechanics under test here do not change with it. Phase 125 (task
03) widened the seam with the optional ``context`` kwarg (the
markdown reference context) — the fake accepts (and ignores) it:
these fixtures carry no image references, so the seam gets
``context=None``."""
handed: list[bytes] = []
async def _fake(
llm: importer.Embedder, *, data: bytes, source: str, rel: str, full_path: Path
llm: importer.Embedder,
*,
data: bytes,
source: str,
rel: str,
full_path: Path,
context: str | None = None,
) -> str | None:
handed.append(data)
return description
@@ -524,7 +534,13 @@ def test_changed_image_deletes_the_stale_copy_before_replacing(
state = {"desc": "v1 description"}
async def _fake(
llm: importer.Embedder, *, data: bytes, source: str, rel: str, full_path: Path
llm: importer.Embedder,
*,
data: bytes,
source: str,
rel: str,
full_path: Path,
context: str | None = None,
) -> str | None:
handed.append(data)
return state["desc"]
+275
View File
@@ -0,0 +1,275 @@
"""Unit tests: markdown image-reference extraction (phase 125, task 02).
``extract_image_refs`` is a pure function (LOCKED A1 — the storage
content of ``Document.image_context``), so the tests pin the exact
contract the enrichment pipeline (tasks 03/04) will rely on:
* one :class:`ImageRef` per ``![alt](path)`` reference outside code
fences (multiple per line extracted separately, same window);
* the context window — up to *N* paragraphs before and AFTER the
reference paragraph, joined with blank lines, the reference
paragraph itself excluded (the reference is already ``alt_text`` +
``path``);
* fence semantics identical to the chunker (PLAN §5) — fences are
ATOMIC paragraphs (one context unit, blank lines inside don't split
it) and references inside a fence are literal text — skipped;
* edge cases: empty alt, start/end of file, headings, paths with
spaces, nested parens (accepted miss), empty content, no refs,
1-based ``line_number``.
"""
from __future__ import annotations
import pytest
from app.rag.md_images import CONTEXT_PARAGRAPHS, ImageRef, extract_image_refs
def test_default_constant() -> None:
assert CONTEXT_PARAGRAPHS == 2
doc = "P1.\n\nP2.\n\nP3.\n\n![a](a.png)\n\nQ1.\n\nQ2.\n\nQ3.\n"
assert extract_image_refs(doc) == extract_image_refs(doc, context_paragraphs=2)
def test_single_reference_with_surrounding_paragraphs() -> None:
doc = (
"# Title\n"
"\n"
"First paragraph.\n"
"\n"
"Second paragraph.\n"
"\n"
"![alt](image.png)\n"
"\n"
"Third paragraph.\n"
"\n"
"Fourth paragraph.\n"
)
refs = extract_image_refs(doc)
assert refs == [
ImageRef(
path="image.png",
alt_text="alt",
context=(
"First paragraph.\n\n"
"Second paragraph.\n\n"
"Third paragraph.\n\n"
"Fourth paragraph."
),
line_number=7,
)
]
def test_multiple_references_on_different_lines() -> None:
doc = (
"Intro.\n"
"\n"
"Para A.\n"
"\n"
"![one](one.png)\n"
"\n"
"Para B.\n"
"\n"
"Para C.\n"
"\n"
"Para D.\n"
"\n"
"![two](two.png)\n"
"\n"
"Outro.\n"
)
refs = extract_image_refs(doc)
assert [r.path for r in refs] == ["one.png", "two.png"]
assert [r.alt_text for r in refs] == ["one", "two"]
assert [r.line_number for r in refs] == [5, 13]
# Each gets its OWN window (N=2 around its own paragraph — spaced
# apart so the windows don't contain the other reference line).
assert refs[0].context == "Intro.\n\nPara A.\n\nPara B.\n\nPara C."
assert refs[1].context == "Para C.\n\nPara D.\n\nOutro."
def test_multiple_references_on_same_line_share_window() -> None:
doc = "Before.\n\nSee ![a](a.png) and ![b](b.png) here.\n\nAfter.\n"
refs = extract_image_refs(doc)
assert [r.path for r in refs] == ["a.png", "b.png"]
assert [r.alt_text for r in refs] == ["a", "b"]
assert [r.line_number for r in refs] == [3, 3]
# Same reference paragraph → same context, reference line excluded.
assert refs[0].context == "Before.\n\nAfter."
assert refs[1].context == "Before.\n\nAfter."
def test_reference_in_code_fence_is_skipped() -> None:
doc = (
"Intro.\n"
"\n"
"```\n"
"![not real](fenced.png)\n"
"```\n"
"\n"
"![real](real.png)\n"
)
refs = extract_image_refs(doc)
assert [r.path for r in refs] == ["real.png"]
# The fence is an ATOMIC paragraph and counts as one context unit
# inside the N=2 window (chunker semantics, PLAN §5).
assert refs[0].context == "Intro.\n\n```\n![not real](fenced.png)\n```"
def test_fence_with_blank_lines_inside_is_one_atomic_block() -> None:
doc = (
"P1.\n"
"\n"
"```\n"
"line one\n"
"\n"
"line three\n"
"```\n"
"\n"
"![a](a.png)\n"
)
refs = extract_image_refs(doc)
assert len(refs) == 1
# The fence (with its inner blank line) is ONE atomic context unit,
# occupying a single slot of the N=2 window before the reference
# (if it split on the inner blank line, the window would hold only
# "P1" + the fence's first lines instead of P1 + the whole fence).
assert refs[0].context == "P1.\n\n```\nline one\n\nline three\n```"
def test_tilde_fence_and_long_fence_markers() -> None:
doc = "~~~\n![skip](skip.png)\n~~~\n\n![keep](keep.png)\n"
assert [r.path for r in extract_image_refs(doc)] == ["keep.png"]
# Four-backtick fence (marker = first three chars), inner blank line.
doc = "````\n\n![skip](skip2.png)\n````\n\n![keep](keep2.png)\n"
assert [r.path for r in extract_image_refs(doc)] == ["keep2.png"]
def test_unclosed_fence_skips_rest_of_file() -> None:
doc = "```\n![skip](a.png)\nmore text after an unclosed fence\n"
assert extract_image_refs(doc) == []
def test_reference_with_no_alt_text() -> None:
refs = extract_image_refs("![](image.png)")
assert refs == [
ImageRef(path="image.png", alt_text="", context="", line_number=1)
]
def test_reference_at_start_of_file() -> None:
doc = "![first](first.png)\n\nP1.\n\nP2.\n\nP3.\n"
refs = extract_image_refs(doc)
assert len(refs) == 1
# Only paragraphs after exist; the N=2 cap still applies (P3 left out).
assert refs[0].context == "P1.\n\nP2."
assert refs[0].line_number == 1
def test_reference_at_end_of_file() -> None:
doc = "P1.\n\nP2.\n\nP3.\n\nP4.\n\n![last](last.png)\n"
refs = extract_image_refs(doc)
assert len(refs) == 1
# Only paragraphs before exist; the N=2 window is the two paragraphs
# immediately before the reference (P1 and P2 left out).
assert refs[0].context == "P3.\n\nP4."
assert refs[0].line_number == 9
def test_reference_in_heading_line() -> None:
doc = (
"Intro paragraph.\n"
"\n"
"## ![alt](image.png)\n"
"\n"
"Outro paragraph.\n"
)
refs = extract_image_refs(doc)
assert len(refs) == 1
assert refs[0].alt_text == "alt"
assert refs[0].path == "image.png"
assert refs[0].line_number == 3
# The heading line is the reference paragraph — excluded from the
# context; only the surrounding text remains.
assert refs[0].context == "Intro paragraph.\n\nOutro paragraph."
assert "##" not in refs[0].context
def test_context_paragraphs_zero_and_custom() -> None:
doc = "P1.\n\nP2.\n\nP3.\n\n![a](a.png)\n\nQ1.\n\nQ2.\n\nQ3.\n"
assert extract_image_refs(doc, context_paragraphs=0)[0].context == ""
assert extract_image_refs(doc, context_paragraphs=1)[0].context == "P3.\n\nQ1."
assert (
extract_image_refs(doc, context_paragraphs=3)[0].context
== "P1.\n\nP2.\n\nP3.\n\nQ1.\n\nQ2.\n\nQ3."
)
# A window LARGER than the file gets whatever is available.
assert (
extract_image_refs(doc, context_paragraphs=10)[0].context
== "P1.\n\nP2.\n\nP3.\n\nQ1.\n\nQ2.\n\nQ3."
)
def test_negative_context_paragraphs_raises() -> None:
with pytest.raises(ValueError, match="context_paragraphs must be >= 0"):
extract_image_refs("![a](a.png)", context_paragraphs=-1)
def test_empty_and_reference_free_content() -> None:
assert extract_image_refs("") == []
assert extract_image_refs("\n\n \n") == []
assert extract_image_refs("# No images here\n\nJust plain text.\n") == []
# A link is not an image reference; an empty target is not a ref.
assert extract_image_refs("[a link](http://x.example/) ![empty]()\n") == []
def test_image_nested_in_link() -> None:
refs = extract_image_refs("![alt](img.png) wrapped: [![alt](img2.png)](http://x/)\n")
assert [r.path for r in refs] == ["img.png", "img2.png"]
def test_alt_text_with_nested_brackets() -> None:
# The alt field is ``[^]]*`` — it may OPEN a bracket (``[``) but
# the first CLOSING bracket (``]``) ends it: an alt with a nested
# ``[`` (``a [1``) still captures the path, while an alt that
# itself CARRIES a ``]`` (``a [1] v2``) breaks the whole match
# (the ``]`` ends the alt field, and no ``(`` follows) — an
# accepted miss, the locked regex's limitation (task 02), the
# alt-side twin of the nested-parens path miss above.
refs = extract_image_refs("![a [1](img.png)\n")
assert refs == [ImageRef(path="img.png", alt_text="a [1", context="", line_number=1)]
assert extract_image_refs("![a [1] v2](img.png)\n") == []
def test_paths_with_spaces_and_parens() -> None:
# Spaces are fine; surrounding whitespace in the target is stripped.
refs = extract_image_refs("![a](diagram v2.png) and ![b]( x.png )\n")
assert [r.path for r in refs] == ["diagram v2.png", "x.png"]
# Nested parens: the regex stops at the first ')' — an accepted
# miss (rare in practice, phase 125 task 02).
refs = extract_image_refs("![a](a(b).png)\n")
assert refs[0].path == "a(b"
def test_line_numbers_are_one_based() -> None:
doc = "\n\n\n# Head\n\n\n![a](a.png)\n"
refs = extract_image_refs(doc)
# Three leading blanks + "# Head" + two blanks = lines 1..6; the
# reference is on line 7.
assert refs[0].line_number == 7
def test_multi_line_reference_paragraph_contributes_nothing() -> None:
# The reference sits in the MIDDLE of a paragraph (no blank lines):
# the whole reference paragraph is excluded from the context.
doc = (
"Before.\n"
"\n"
"Line one.\n"
"Line two with ![alt](img.png)\n"
"Line three.\n"
"\n"
"After.\n"
)
refs = extract_image_refs(doc)
assert refs[0].context == "Before.\n\nAfter."
+48
View File
@@ -69,6 +69,54 @@ def test_chunks_reference_documents_cascade() -> None:
assert fkc.ondelete == "CASCADE"
def test_documents_image_context_column_contract() -> None:
"""Phase 125 (task 01, LOCKED A1): ``documents.image_context`` —
TEXT, NULLABLE, NO server default. NULL is the standalone-image /
pre-phase-125 value (no backfill — A4 keeps those rows
byte-identical)."""
from sqlalchemy import Text as SAText
documents = Base.metadata.tables["documents"]
col = documents.c["image_context"]
assert col.nullable is True, "image_context must be NULLABLE (pre-phase rows)"
assert isinstance(col.type, SAText), "image_context must be TEXT"
assert col.server_default is None, (
"image_context must carry NO server default — NULL is the"
" standalone/pre-phase value"
)
assert col.default is None, (
"image_context must carry NO ORM default clause either (an"
" omitted column inserts NULL, the text-doc value)"
)
def test_document_image_context_orm_parses() -> None:
"""A freshly constructed ``Document`` (no DB) accepts the field in
both states: a set context string round-trips as-is, and
``None`` (and the omitted kwarg) read NULL — the standalone-image
/ pre-phase state."""
from app.models import Document
base = dict(
source="s",
path="p.md",
full_path="/tmp/s/p.md",
title="p",
content="c",
content_hash="1" * 64,
)
enriched = Document(**base, image_context="some context")
assert enriched.image_context == "some context"
explicit_null = Document(**base, image_context=None)
assert explicit_null.image_context is None
omitted = Document(**base)
assert omitted.image_context is None, (
"an omitted image_context must read NULL (the text-doc value)"
)
def test_documents_unique_source_path() -> None:
documents = Base.metadata.tables["documents"]
uq = [
+306
View File
@@ -48,6 +48,8 @@ from app.rag.prompts import (
SUGGEST_INTRO,
TOOLS_SECTION,
_base,
_document_body,
_extract_context_line,
build_deflect_prompt,
build_high_prompt,
build_kb_section,
@@ -74,6 +76,9 @@ def _doc(
content: str,
title: str,
summary: str | None = None,
*,
is_image: bool = False,
image_context: str | None = None,
) -> Document:
return Document(
id=uuid.uuid4(),
@@ -85,6 +90,9 @@ def _doc(
content_hash="0" * 64,
created_at=_FIXTURE_CREATED_AT,
summary=summary,
is_image=is_image,
image_path="/tmp/pic.png" if is_image else None,
image_context=image_context,
)
@@ -1141,3 +1149,301 @@ def test_show_image_clause_is_last_for_every_high_shape() -> None:
assert on.index(section) < on.index("<tools>")
assert on.index("K8s deploy notes.") < on.index("<tools>") # the summary still seeds
assert SHOW_IMAGE_CLAUSE not in off # the toggle-off twin carries none of it
# ---------- phase 125 (task 05, LOCKED A4): the image context line ----------
# An enriched image doc (tasks 03/04: non-NULL ``image_context`` in the
# "Referenced in …" block shape) leads its ``<document>`` suggestion
# block with ONE short context line — the referring markdown's identity
# + the reference's alt text — before the description, so the LLM knows
# WHY the image exists, not only what it shows. Every other doc (every
# text doc, every non-enriched image doc) is byte-identical to the
# pre-phase body.
#: The exact stored block shape (tasks 03/04): one "Referenced in"
#: block per referring markdown file, blank-line joined; the
#: ``Context:`` body may span many lines (the surrounding paragraphs).
CTX_SINGLE = (
"Referenced in Homelab/homelab/kubernetes.md:\n"
'- Alt text: "Cluster layout"\n'
" Context: The diagram shows the three Talos nodes and Cilium.\n"
"It is referenced again from the runbook."
)
CTX_TWO_BLOCKS = CTX_SINGLE + "\n\nReferenced in Homelab/ops/runbook.md:\n- Alt text: \"\""
CTX_ALT_EMPTY = "Referenced in Homelab/ops/runbook.md:\n- Alt text: \"\""
#: The task-04 pending record — stored on MARKDOWN (text) docs, with no
#: ``Referenced in`` header at all.
MD_PENDING_RECORD = (
'IMAGE_CONTEXT:\n- [image.png] (line 42): "Alt text here" — 2 paragraphs of context'
)
def _image_doc(
image_context: str | None = None, summary: str | None = "A diagram of the cluster."
) -> Document:
return _doc(
"cluster-diagram.png",
"A diagram of the cluster.",
"cluster-diagram",
summary=summary,
is_image=True,
image_context=image_context,
)
# ---------- _extract_context_line: the parser contract ----------
def test_extract_context_line_single_block_with_alt() -> None:
"""One block with an alt → the short one-line locator: the stored
identity (``<source>/<path>`` exactly as the header carries it) +
the reference's alt between quotes."""
assert _extract_context_line(CTX_SINGLE) == (
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
)
def test_extract_context_line_context_body_never_renders() -> None:
"""The ``Context:`` paragraphs (LOCKED A3: they already reached the
vision model at enrichment time) stay out of the line — the line is
the locator + purpose, short enough not to bloat the prompt."""
line = _extract_context_line(CTX_SINGLE)
assert line is not None
assert "The diagram shows" not in line
assert "referenced again" not in line
assert "Context:" not in line
assert line.count("\n") == 0 # genuinely ONE line
def test_extract_context_line_empty_alt_renders_the_bare_identity() -> None:
"""A reference without an alt (``![](path)``) → the clause is the
bare identity — nothing to quote after the colon."""
assert _extract_context_line(CTX_ALT_EMPTY) == "Referenced in Homelab/ops/runbook.md"
def test_extract_context_line_two_blocks_join_with_semicolon() -> None:
"""One clause per referring file after a single prefix — the second
block's empty alt renders bare."""
assert _extract_context_line(CTX_TWO_BLOCKS) == (
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"; '
"Homelab/ops/runbook.md"
)
def test_extract_context_line_uses_the_first_reference_only() -> None:
"""A block with two references (two ``- Alt text:`` bullets) → the
FIRST reference's alt is the purpose; the second stays out."""
text = (
"Referenced in Homelab/a.md:\n"
'- Alt text: "first purpose"\n'
" Context: some paragraph.\n"
'- Alt text: "second purpose"'
)
assert _extract_context_line(text) == 'Referenced in Homelab/a.md: "first purpose"'
assert "second purpose" not in (_extract_context_line(text) or "")
def test_extract_context_line_multiline_context_body_is_ignored() -> None:
"""The ``Context:`` value is paragraph-joined (blank lines inside)
— the parser walks it without breaking the block."""
text = (
"Referenced in Homelab/a.md:\n"
'- Alt text: "purpose"\n'
" Context: para one.\n"
"\n"
"para two, and it contains a line that\n"
"looks like context continuation."
)
assert _extract_context_line(text) == 'Referenced in Homelab/a.md: "purpose"'
def test_extract_context_line_alt_with_inner_quotes_survives() -> None:
"""The block writer does not escape quotes: the greedy alt capture
(between the opening quote and the line's final quote) keeps an
alt with inner ``"`` intact."""
text = 'Referenced in Homelab/x.md:\n- Alt text: "He said "hi""'
assert _extract_context_line(text) == 'Referenced in Homelab/x.md: "He said "hi""'
def test_extract_context_line_spurious_header_is_dropped() -> None:
"""A look-alike ``Referenced in …:`` line WITHOUT its ``- Alt text:``
bullet (context text that merely looks like a header) parses to
``None`` — never a clause from the stray line."""
assert _extract_context_line(
"Referenced in Homelab/fake.md:\nsome context line that is not a bullet"
) is None
# A header with NO following line at all (end of value) is spurious too.
assert _extract_context_line("Referenced in Homelab/fake.md:") is None
# Spurious then REAL: only the real block's clause survives.
mixed = (
"Referenced in Homelab/fake.md:\nsome context line\n\n"
+ CTX_SINGLE
)
assert _extract_context_line(mixed) == (
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
)
def test_extract_context_line_header_after_header_drops_the_bulletless_one() -> None:
"""Two headers in a row: the first (never followed by its bullet) is
dropped and the second block parses normally."""
text = "Referenced in Homelab/fake.md:\nReferenced in Homelab/real.md:\n- Alt text: \"alt\""
assert _extract_context_line(text) == 'Referenced in Homelab/real.md: "alt"'
def test_extract_context_line_markdown_pending_record_is_none() -> None:
"""The task-04 ``IMAGE_CONTEXT:`` record (the MARKDOWN doc's stored
``image_context``) has no ``Referenced in`` header — it parses to
``None`` (and it rides a text doc, which the ``is_image`` guard
excludes from the line anyway)."""
assert _extract_context_line(MD_PENDING_RECORD) is None
def test_extract_context_line_blank_and_empty_are_none() -> None:
assert _extract_context_line("") is None
assert _extract_context_line(" \n \t ") is None
# Garbage that contains the keyword somewhere is not a block.
assert _extract_context_line("I was referenced in Homelab/a.md: yesterday") is None
# ---------- _document_body: the block-body contract ----------
def test_document_body_enriched_image_doc_leads_with_the_context_line() -> None:
"""The completion criterion at the body level: the context line
leads, a blank line, then the description — the exact two-part
shape."""
body = _document_body(_image_doc(image_context=CTX_SINGLE))
assert body == (
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n\n'
"A diagram of the cluster."
)
def test_document_body_image_doc_null_context_is_byte_identical_to_pre_phase() -> None:
"""LOCKED A4 (the byte-identity criterion): a standalone / pre-phase
image doc (NULL ``image_context``) → the body is EXACTLY the
stripped summary — no line, no blank, no drift."""
doc = _image_doc(image_context=None)
assert _document_body(doc) == "A diagram of the cluster."
# And the built prompt matches a text twin (same fields, not an
# image) to the byte.
twin = _doc("cluster-diagram.png", "A diagram of the cluster.", "cluster-diagram")
assert build_high_prompt([doc]) == build_high_prompt([twin])
def test_document_body_text_doc_with_image_context_is_unchanged() -> None:
"""The guard is ``is_image``: a TEXT doc whose ``image_context``
carries the task-04 pending record (the markdown case) keeps the
pre-phase body — the record never renders."""
doc = _doc("guide.md", "CONTENT", "Guide", summary="Guide summary.",
image_context=MD_PENDING_RECORD)
assert _document_body(doc) == "Guide summary."
assert "IMAGE_CONTEXT" not in build_high_prompt([doc])
assert "Referenced in" not in build_high_prompt([doc])
def test_document_body_image_doc_unparsable_context_is_unchanged() -> None:
"""Graceful degradation: an image doc whose ``image_context`` parses
to nothing (owner-edited / unknown shape) → the description only,
exactly pre-phase — the prompt build never raises."""
for junk in ("", " ", "referenced in lowercase.md: no header shape", MD_PENDING_RECORD):
assert _document_body(_image_doc(image_context=junk)) == "A diagram of the cluster."
def test_document_body_enriched_image_doc_without_summary_precedes_the_preview() -> None:
"""The line leads the fallback body too (the preview + marker of
LOCKED A5) — the "context before the description" contract holds on
both body shapes."""
doc = _image_doc(image_context=CTX_SINGLE, summary=None)
doc.content = "A" * 400 + "B" * 600 # past the 400-char preview cap
body = _document_body(doc)
line = 'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
assert body.startswith(line + "\n\n")
assert body[len(line) + 2 :] == "A" * 400 + "\n" + TRUNCATION_MARKER
assert "B" * 40 not in body # beyond the cut, as pre-phase
def test_enriched_image_doc_with_summary_makes_no_settings_call(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""House pattern (the LOCKED A5 preview-cap contract): the
context-line path for a summary-bearing doc is pure — a dead
``get_settings`` must not break such a build (extraction is pure;
the cap is read on the fallback path only)."""
from app.rag import prompts as prompts_mod
doc = _image_doc(image_context=CTX_SINGLE)
def _dead() -> None: # pragma: no cover - must never be called
raise AssertionError("get_settings() called for a summarized doc")
monkeypatch.setattr(prompts_mod, "get_settings", _dead)
assert _document_body(doc) == (
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n\n'
"A diagram of the cluster."
)
# ---------- build_high_prompt: the line in the built block ----------
def test_high_prompt_enriched_image_doc_block_exact_shape() -> None:
"""The full block in the built prompt: identity attributes
unchanged (phase 106 date appended after title), then the context
line, a blank line, the description — the pre-phase block shape
with the line spliced in."""
doc = _image_doc(image_context=CTX_SINGLE)
prompt = build_high_prompt([doc])
assert (
'<document source="Homelab" path="cluster-diagram.png" '
'title="cluster-diagram" date="2024-06-15">\n'
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n'
"\n"
"A diagram of the cluster.\n"
"</document>"
) in prompt
# The line is the FIRST thing in the block body (before the
# description), inside <documents>.
line = 'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
block_start = prompt.index('<document source="Homelab"')
assert prompt.index("<documents>") < prompt.index(line)
assert prompt.index(line) < prompt.index("A diagram of the cluster.")
assert block_start < prompt.index(line) < prompt.index("</document>")
# The Context: body stays out of the whole prompt.
assert "The diagram shows the three Talos nodes" not in prompt
def test_high_prompt_enriched_image_doc_mixed_with_text_docs() -> None:
"""A suggested tier with both text docs and one enriched image doc:
the line rides ONLY the image doc's block; the text blocks are
byte-identical to their summaries (no line, no drift)."""
text_doc = _doc("kubernetes.md", "CONTENT", "K8s", summary="K8s summary.")
img = _image_doc(image_context=CTX_SINGLE)
prompt = build_high_prompt([text_doc, img])
assert prompt.count('Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"') == 1
# The text block is the bare summary, exactly as pre-phase.
assert (
'<document source="Homelab" path="kubernetes.md" '
'title="K8s" date="2024-06-15">\nK8s summary.\n</document>'
) in prompt
# The image block carries the line + description.
assert (
'title="cluster-diagram" date="2024-06-15">\n'
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n\n'
"A diagram of the cluster.\n</document>"
) in prompt
# Ordering unchanged: the text block (first doc) still precedes the
# image block (second doc) — the line never re-orders the tier.
assert prompt.index('path="kubernetes.md"') < prompt.index('path="cluster-diagram.png"')
def test_high_prompt_image_doc_without_context_is_byte_identical_to_pre_phase() -> None:
"""LOCKED A4 at the prompt level: an image doc with NULL
``image_context`` builds the EXACT pre-phase prompt (the text-twin
equality — no context line anywhere, not even a stray blank)."""
img = _image_doc(image_context=None)
twin = _doc("cluster-diagram.png", "A diagram of the cluster.", "cluster-diagram")
assert build_high_prompt([img]) == build_high_prompt([twin])
assert "Referenced in" not in build_high_prompt([img])