phase: 125_image_anchor_context
All completion criteria verified. The repository is left functional with all changes in the working tree (no commits made, per the harness protocol). ## Final report — phase 125 `image_anchor_context` (verification pass) All 6 tasks were already complete; this pass verified every criterion and ran the full gates: - **Migration 0023** `documents.image_context` (TEXT nullable, reversible) — applied (`alembic current` = 0023 head); round-trip test (4 passed) exercises real upgrade → downgrade 0022 → upgrade. - **Criterion 1** (referenced image enriched + context line in retrieval): integration `test_markdown_referenced_image_gets_context_and_enriched_chunks` + E2E `test_sync_enriches_the_image_doc_with_markdown_context`, `test_chat_answer_quotes_the_context_line` (wire prompt pinned: `<document>` block leads with the one-line `Referenced in …: "alt"` line) — verified present and passing. - **Criterion 2** (standalone image byte-identical): E2E `test_standalone_*` (NULL `image_context`, block body exactly the bare description, no `Referenced in` anywhere in the prompt) + unit `test_describe_image_without_context_is_byte_identical` — passing. - **Criterion 3** (markdown-first: record stored on markdown doc, image picks up context later): integration `test_markdown_indexed_before_image_stores_record_and_image_picks_it_up` + post-index enrichment pass test (markdown file gone from disk) — passing. - **Gates:** `uv run pytest` → 2938 passed, exit 0 · `uv run pytest --cov=app --cov-report=term-missing` → exit 0, TOTAL **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings. - **Phase E2E in isolation:** `uv run pytest tests/e2e/test_image_anchor_context.py -v --no-cov` → 4 passed. - **Mock-LLM change regression check:** neighboring image E2E suites green in isolation — `test_smoke` (3), `test_image_documents` (16), `test_chat_image_questions` (17), `test_answer_images` (17). - No defects found; no code changes needed. No live-infrastructure changes (local dev DB only). No commit made (harness commits). - Next pending phase: check `.agents/phases/todo/` after the harness moves this phase.
This commit is contained in:
@@ -517,6 +517,28 @@ Implements just enough of the aipi surface:
|
||||
marker question always gets the echo whatever the honesty gate
|
||||
says. Verified 2026-09-26: no existing E2E question or fixture
|
||||
file contains the phrase, so every other suite is unaffected.
|
||||
- user message containing ``quote the image reference``
|
||||
(``CONTEXT_LINE_TRIGGER``, phase 125 task 06 — the image-anchor
|
||||
context suite ``tests/e2e/test_image_anchor_context.py``) -> the
|
||||
deterministic ZERO-READ answer that QUOTES THE CONTEXT LINE of the
|
||||
seeded image doc's ``<document>`` block (the same image-doc pick
|
||||
as ``DIAGRAM_TRIGGER``): an ENRICHED image doc's block (phase 125
|
||||
task 05, LOCKED A4) leads with the one-line
|
||||
``Referenced in <source>/<path>: "<alt>"`` context line BEFORE
|
||||
the description, so the mock's quote — the block's exact first
|
||||
line — proves the line WAS in the prompt (and that its text is
|
||||
the task-05 one-line rendering, not the stored multi-line block).
|
||||
A non-enriched (standalone) image doc's block is the bare
|
||||
description (byte-identical pre-phase): its first line never
|
||||
starts with ``Referenced in `` (the mock's fixed description
|
||||
cannot begin that way), so the mock answers that the image
|
||||
carries no reference line — the byte-identity criterion's
|
||||
deterministic witness. Checked BEFORE the ``DEFLECT_MODE``
|
||||
branch (the marker lives in the user message — the
|
||||
``TABLE_TRIGGER``/``HISTORY_TRIGGER`` convention), so a marker
|
||||
question always gets the echo whatever the honesty gate says.
|
||||
Verified 2026-09-27: no existing E2E question or fixture file
|
||||
contains the phrase, so every other suite is unaffected.
|
||||
|
||||
Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic
|
||||
dead-endpoint behavior for the retry E2E suite (``tests/e2e/
|
||||
@@ -1843,6 +1865,26 @@ SHOW_IMAGE_TRIGGER = "show me the picture of"
|
||||
#: says.
|
||||
DIAGRAM_TRIGGER = "which diagram is in your notes"
|
||||
|
||||
#: The phase-125 (task 06, image-anchor context, LOCKED A4) trigger
|
||||
#: (case-insensitive): the mock QUOTES THE CONTEXT LINE of the seeded
|
||||
#: image doc's ``<document>`` block (the same image-doc pick as
|
||||
#: :data:`DIAGRAM_TRIGGER`). For an ENRICHED image doc the block
|
||||
#: leads with the one-line ``Referenced in <source>/<path>: "<alt>"``
|
||||
#: line (the task-05 rendering of the stored ``image_context``,
|
||||
#: BEFORE the description) — quoting the block's first line proves
|
||||
#: the line reached the prompt; for a non-enriched (standalone) doc
|
||||
#: the block is the bare description (byte-identical pre-phase) and
|
||||
#: its first line never starts with ``Referenced in `` (the mock's
|
||||
#: fixed description cannot begin that way) — the mock then answers
|
||||
#: that the image carries no reference line. Checked BEFORE the
|
||||
#: ``DEFLECT_MODE`` branch in :func:`compose_answer` (the marker
|
||||
#: lives in the user message — the TABLE/HISTORY convention), so a
|
||||
#: marker question always gets the echo whatever the honesty gate
|
||||
#: says. Disjoint from every other trigger phrase; verified
|
||||
#: 2026-09-27: no existing E2E question or fixture file contains the
|
||||
#: phrase, so every other suite is unaffected.
|
||||
CONTEXT_LINE_TRIGGER = "quote the image reference"
|
||||
|
||||
#: The agent's ``show_image`` SUCCESS result's confirming first line
|
||||
#: (``app.rag.agent``'s ``show_image`` branch — the line ``read``'s
|
||||
#: result never carries): the SHOW_IMAGE flow's follow-up request
|
||||
@@ -1878,6 +1920,26 @@ def _seeded_image_doc(system: str) -> tuple[str, str, str] | None:
|
||||
return None
|
||||
|
||||
|
||||
def _seed_context_line(block_content: str) -> str | None:
|
||||
"""The phase-125 context line of a seeded image doc's block
|
||||
(task 06, LOCKED A4) — or ``None``.
|
||||
|
||||
An ENRICHED image doc's ``<document>`` block leads with the
|
||||
one-line ``Referenced in …`` rendering (the task-05
|
||||
:func:`app.rag.prompts._extract_context_line` output, BEFORE the
|
||||
description): its first line, verbatim, is the line the mock
|
||||
quotes (the prompt-injection echo convention). A non-enriched
|
||||
block is the bare description — its first line never starts with
|
||||
``Referenced in `` (the mock's fixed
|
||||
:data:`IMAGE_DESCRIPTION_ANSWER` cannot begin that way), so
|
||||
``None`` pins the byte-identical pre-phase shape.
|
||||
"""
|
||||
first = block_content.split("\n", 1)[0].strip()
|
||||
if first.startswith("Referenced in "):
|
||||
return first
|
||||
return None
|
||||
|
||||
|
||||
def _show_image_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
|
||||
"""Classify a phase-124 SHOW_IMAGE request (see the module
|
||||
docstring). Stateless over the messages, like the other marker
|
||||
@@ -2467,6 +2529,32 @@ def compose_answer(body: dict[str, Any]) -> str:
|
||||
else "Your notes don't carry that diagram — there is nothing to show. "
|
||||
"(Deterministic mock answer for E2E.)"
|
||||
)
|
||||
elif CONTEXT_LINE_TRIGGER in user.lower():
|
||||
# Phase 125 (task 06, image-anchor context, LOCKED A4): the
|
||||
# ZERO-READ context-line echo — the mock QUOTES THE CONTEXT
|
||||
# LINE of the seeded image doc's <document> block (the block's
|
||||
# FIRST line — the task-05 one-line rendering for an enriched
|
||||
# doc; a standalone doc's block is the bare description, whose
|
||||
# first line never matches — the "no reference line" answer
|
||||
# is the byte-identity witness). Checked BEFORE the
|
||||
# DEFLECT_MODE branch (the marker lives in the user message —
|
||||
# the TABLE/HISTORY convention), so a marker question always
|
||||
# gets the echo whatever the honesty gate says.
|
||||
image_doc = _seeded_image_doc(system)
|
||||
if image_doc is None:
|
||||
answer = (
|
||||
"Your notes don't carry that image — there is nothing to reference. "
|
||||
"(Deterministic mock answer for E2E.)"
|
||||
)
|
||||
else:
|
||||
context_line = _seed_context_line(image_doc[2])
|
||||
answer = (
|
||||
f"The notes reference the image thus: {context_line} "
|
||||
"(Deterministic mock answer for E2E.)"
|
||||
if context_line is not None
|
||||
else "The image stands alone — the notes carry no reference line for it. "
|
||||
"(Deterministic mock answer for E2E.)"
|
||||
)
|
||||
elif "DEFLECT_MODE" in system:
|
||||
answer = (
|
||||
"Ah — I haven't done anything like that, so I don't want to make stuff up! "
|
||||
|
||||
@@ -0,0 +1,623 @@
|
||||
"""Phase 125 E2E (Playwright): image-anchor context — the user path.
|
||||
|
||||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
uv run pytest tests/e2e/test_image_anchor_context.py -v --no-cov
|
||||
|
||||
The suite's app instance runs with ``BOR_IMAGES=true`` and
|
||||
``BOR_ANSWER_IMAGES=false`` (module env override — the conftest
|
||||
per-suite-app pattern, leak guards included): the fixture archive
|
||||
carries ``docs/guide.md`` — a markdown file whose ````
|
||||
reference NAMES ``docs/diagram.png`` in the SAME source — so the real
|
||||
upload → sync flow drives the whole phase-125 pipeline (LOCKED
|
||||
A1/A2): the task-03 disk scan finds the reference at image-index time,
|
||||
the vision model receives the context after the locked describe prompt,
|
||||
and the image doc's ``image_context`` (and so its ``content``,
|
||||
``summary`` and embedded chunks) becomes
|
||||
``<context>\\n\n<description>``.
|
||||
|
||||
* the sync leaves the image doc ENRICHED (the ``Referenced in`` block
|
||||
on ``image_context``, the enriched content + chunks, the markdown
|
||||
doc's record NULL — the image was indexed first, the task-03 path
|
||||
owns that case);
|
||||
* a grounded chat turn whose suggested tier carries the enriched
|
||||
image doc gets the task-05 context line: the mock (the
|
||||
``CONTEXT_LINE_TRIGGER`` branch — the prompt parser extended for
|
||||
this phase) QUOTES THE EXACT LINE of the image doc's ``<document>``
|
||||
block, so the rendered answer AND the captured wire prompt prove
|
||||
the block LEADS with the one-line ``Referenced in
|
||||
<source>/<path>: "<alt>"`` rendering before the summary (the stored
|
||||
block's shape carries the alt on the next line as a bullet — the
|
||||
quote pins the task-05 rendering, not the stored block);
|
||||
* a STANDALONE image (a second phase of the same app — the KB is
|
||||
re-seeded with a single PNG no markdown references) stays
|
||||
byte-identical to pre-phase: ``image_context`` NULL, the bare
|
||||
description everywhere, the ``<document>`` block the description
|
||||
alone, and the mock's "no reference line" answer (the deterministic
|
||||
witness that no context line was in the prompt).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import zipfile
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from playwright.sync_api import Browser, Page, expect
|
||||
from sqlalchemy import select, text
|
||||
|
||||
from app.config import Settings
|
||||
from app.db import SessionLocal
|
||||
from app.models import Chunk, Document
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
||||
from e2e.mock_llm import IMAGE_DESCRIPTION_ANSWER
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
GIT_SOURCES_URL = "/git-sources.html"
|
||||
|
||||
#: Phase A — the anchored source: the markdown references the image
|
||||
#: from the SAME folder (``diagram.png`` next to ``guide.md`` → the
|
||||
#: resolved in-source path ``docs/diagram.png``).
|
||||
SOURCE_NAME = "e2e-anchor" # the archive stem (archive_source_name)
|
||||
MD_PATH = "docs/guide.md"
|
||||
DOC_PATH = "docs/diagram.png"
|
||||
|
||||
#: Phase B — the standalone source: a single PNG no markdown names.
|
||||
STANDALONE_SOURCE_NAME = "e2e-anchor-standalone"
|
||||
LOGO_PATH = "logo.png"
|
||||
|
||||
#: The fixture markdown — the reference on line 5, two context
|
||||
#: paragraphs (the task-02 default window).
|
||||
GUIDE_MD = (
|
||||
"# Backup runbook\n"
|
||||
"\n"
|
||||
"The restore order diagram explains the backup procedure.\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"Only run the restore script after the checks pass.\n"
|
||||
)
|
||||
|
||||
#: The task-03-shape combined context the same-source disk scan builds
|
||||
#: for ``docs/diagram.png`` (one referring markdown, one reference,
|
||||
#: the task-02 window as the ``Context:`` body — the ``# Backup
|
||||
#: runbook`` heading is the first of the two paragraphs before the
|
||||
#: reference; the N=2 window keeps it).
|
||||
BLOCK = (
|
||||
"Referenced in e2e-anchor/docs/guide.md:\n"
|
||||
'- Alt text: "Restore order"\n'
|
||||
" Context: # Backup runbook\n"
|
||||
"\n"
|
||||
"The restore order diagram explains the backup procedure.\n"
|
||||
"\n"
|
||||
"Only run the restore script after the checks pass."
|
||||
)
|
||||
|
||||
#: The enriched image doc's stored ``content`` (AND ``summary`` — the
|
||||
#: phase-122 ``summary = content`` convention): the context FIRST
|
||||
#: (for retrieval), then the vision description.
|
||||
ENRICHED = f"{BLOCK}\n\n{IMAGE_DESCRIPTION_ANSWER}"
|
||||
|
||||
#: The content chunk's stored text — the plain-text packer joins the
|
||||
#: enriched paragraphs with single newlines (the chunker's
|
||||
#: normalization, PLAN §5); the ``is_summary`` chunk keeps the raw
|
||||
#: enriched text.
|
||||
PACKED = "\n".join(ENRICHED.split("\n\n"))
|
||||
|
||||
#: The task-05 ONE-LINE rendering of :data:`BLOCK` (the context line
|
||||
#: that leads the enriched image doc's ``<document>`` block body):
|
||||
#: the referring markdown's identity + the reference's alt, the
|
||||
#: ``Context:`` body never rendered.
|
||||
CONTEXT_LINE = 'Referenced in e2e-anchor/docs/guide.md: "Restore order"'
|
||||
|
||||
#: The grounded question — the mock's ``CONTEXT_LINE_TRIGGER`` marker
|
||||
#: (the zero-read context-line echo) + the phase-122 grounding phrase
|
||||
#: (token-overlap cosine ≈0.47 against the enriched diagram doc and
|
||||
#: ≈0.37 against the standalone logo's bare description — both clear
|
||||
#: the mock-calibrated gate (0.30), the FTS leg corroborates).
|
||||
ANCHOR_QUESTION = (
|
||||
"quote the image reference — what is shown in the homelab network diagram?"
|
||||
)
|
||||
|
||||
#: The mock's two deterministic answers for the marker (the exact
|
||||
#: branch texts — the phase-125 pins key off them).
|
||||
ANSWER_WITH_LINE = (
|
||||
f"The notes reference the image thus: {CONTEXT_LINE} "
|
||||
"(Deterministic mock answer for E2E.)"
|
||||
)
|
||||
ANSWER_STANDALONE = (
|
||||
"The image stands alone — the notes carry no reference line for it. "
|
||||
"(Deterministic mock answer for E2E.)"
|
||||
)
|
||||
|
||||
#: The honest-deflection opening (the gate must NOT fire on the
|
||||
#: marker question — a deflected prompt carries no ``<documents>``
|
||||
#: block, so the mock would never see the image doc).
|
||||
DEFLECT_OPENING = "I haven't done anything like that"
|
||||
|
||||
UPLOAD_TIMEOUT_MS = 30_000
|
||||
SYNC_TIMEOUT_MS = 60_000
|
||||
SYNCED_LABEL = re.compile(r"^Synced \d{1,2}:\d{2}$")
|
||||
|
||||
#: A real 1×1 transparent PNG (the phase-122 fixture — the pipeline
|
||||
#: is content-agnostic, the bytes just need to be plausible image
|
||||
#: bytes).
|
||||
PNG_1X1 = base64.b64decode(
|
||||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR4nGP4DwQACfsD/fteaysAAAAASUVORK5CYII="
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Module app: images ON (the story) — the conftest env pattern with the
|
||||
# phase-122/125 knobs + leak guards.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _app_env(mock_llm: int, app_port: int, scratch: Path) -> dict[str, str]:
|
||||
"""The conftest app env (leak guards included) with the phase-122
|
||||
knobs: ``BOR_IMAGES`` pinned EXPLICITLY true (the story) and
|
||||
``BOR_ANSWER_IMAGES`` pinned EXPLICITLY false (the default — the
|
||||
auto-cite/figure behavior is phase 124's story, not this phase's,
|
||||
and an operator's local gitignored ``.env`` cannot leak it in:
|
||||
both toggles are the contract under test's neighbors), the
|
||||
image/upload homes in the suite's scratch dir (the app under test
|
||||
must not write copies into the owner's real ``~/bor-sources``)."""
|
||||
env = dict(os.environ)
|
||||
env.pop("DEBUGPY", None)
|
||||
env["BOR_ENVIRONMENT"] = "e2e"
|
||||
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
||||
env["BOR_LLM_BASE_URL"] = (
|
||||
"https://aipi.reeseapps.com/v1"
|
||||
if USE_REAL_LLM
|
||||
else f"http://127.0.0.1:{mock_llm}/v1"
|
||||
)
|
||||
# Mock-calibrated gate (conftest pattern) — the marker question
|
||||
# grounds on these values (see ANCHOR_QUESTION).
|
||||
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
||||
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
|
||||
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
|
||||
# Phase 67: instant retry waits + the code-default budget.
|
||||
env["BOR_LLM_RETRY_DELAY"] = "0"
|
||||
env["BOR_LLM_RETRIES"] = str(Settings.model_fields["llm_retries"].default)
|
||||
env.setdefault(
|
||||
"BOR_DATABASE_URL",
|
||||
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
||||
)
|
||||
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
||||
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
||||
# Leak guards (conftest pattern).
|
||||
env["BOR_GIT_SOURCES"] = ""
|
||||
env["BOR_DOCS_REPO"] = ""
|
||||
env["BOR_SUGGESTIONS"] = json.dumps(Settings.model_fields["suggestions"].default)
|
||||
env["BOR_INPUT_PLACEHOLDER"] = Settings.model_fields["input_placeholder"].default
|
||||
env["BOR_FOOTER_TEXT"] = Settings.model_fields["footer_text"].default
|
||||
# Phase 122/124 toggles + the suite-private homes.
|
||||
env["BOR_IMAGES"] = "true"
|
||||
env["BOR_ANSWER_IMAGES"] = "false"
|
||||
env["BOR_UPLOAD_DIR"] = str(scratch / "uploads")
|
||||
env["BOR_SOURCES_DIR"] = str(scratch / "checkouts")
|
||||
env["BOR_IMAGE_DIR"] = str(scratch / "images")
|
||||
return env
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_server(mock_llm: int, tmp_path_factory: pytest.TempPathFactory) -> Iterator[str]:
|
||||
"""The story app — ``BOR_IMAGES=true`` (the module env override;
|
||||
the conftest session app is never started in this isolated run,
|
||||
so no port clash)."""
|
||||
scratch = tmp_path_factory.mktemp("bor_anchor")
|
||||
port = int(os.environ.get("E2E_APP_PORT_ANCHOR", "8160"))
|
||||
proc = subprocess.Popen(
|
||||
[sys.executable, "-m", "uvicorn", "app.main:app",
|
||||
"--host", "127.0.0.1", "--port", str(port), "--log-level", "warning"],
|
||||
cwd=REPO,
|
||||
env=_app_env(mock_llm, port, scratch=scratch),
|
||||
)
|
||||
try:
|
||||
_wait_http(f"http://127.0.0.1:{port}/api/health")
|
||||
yield f"http://127.0.0.1:{port}"
|
||||
finally:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_url(app_server: str) -> str:
|
||||
return app_server
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixture archives
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def anchor_zip(tmp_path_factory: pytest.TempPathFactory) -> Path:
|
||||
"""Phase A's archive: the markdown that references the image in
|
||||
the SAME source (``docs/guide.md`` → ``docs/diagram.png``)."""
|
||||
root = tmp_path_factory.mktemp("bor_anchor_zip")
|
||||
docs = root / "docs"
|
||||
docs.mkdir()
|
||||
(docs / "guide.md").write_text(GUIDE_MD, encoding="utf-8")
|
||||
(docs / "diagram.png").write_bytes(PNG_1X1)
|
||||
archive = root / f"{SOURCE_NAME}.zip"
|
||||
with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
zf.write(docs / "guide.md", arcname="docs/guide.md")
|
||||
zf.write(docs / "diagram.png", arcname="docs/diagram.png")
|
||||
return archive
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def standalone_zip(tmp_path_factory: pytest.TempPathFactory) -> Path:
|
||||
"""Phase B's archive: ONE standalone PNG at the source root — no
|
||||
markdown in the source references it (the byte-identical
|
||||
pre-phase case)."""
|
||||
root = tmp_path_factory.mktemp("bor_anchor_standalone_zip")
|
||||
(root / LOGO_PATH).write_bytes(PNG_1X1)
|
||||
archive = root / f"{STANDALONE_SOURCE_NAME}.zip"
|
||||
with zipfile.ZipFile(archive, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
zf.write(root / LOGO_PATH, arcname=LOGO_PATH)
|
||||
return archive
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DB + UI helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _truncate_all() -> None:
|
||||
"""Fresh KB + registry per phase (the E2E isolation pattern): the
|
||||
suites share one Postgres, and a leftover row would corrupt the
|
||||
counts. ``sources_meta`` (the KB generation counter) resets with
|
||||
the KB — the sync re-creates the row."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(text(
|
||||
"TRUNCATE chunks, documents, query_log, kb_overview, "
|
||||
"git_sources, sources_meta, folder_summaries"
|
||||
))
|
||||
db.commit()
|
||||
|
||||
|
||||
def _health_db(app_url: str) -> bool:
|
||||
try:
|
||||
return httpx.get(f"{app_url}/api/health", timeout=5).json()["db"] == "up"
|
||||
except Exception: # noqa: BLE001 — unreachable is the skip case
|
||||
return False
|
||||
|
||||
|
||||
def _upload_zip(page: Page, app_url: str, archive: Path, source: str) -> None:
|
||||
"""The real admin upload flow: form login → the git-sources page →
|
||||
pick the archive → submit → the terminal result line (phase 90:
|
||||
"Uploaded <source> — press Sync sources to import it.")."""
|
||||
login(page, app_url, next=GIT_SOURCES_URL)
|
||||
expect(page).to_have_url(app_url + GIT_SOURCES_URL, timeout=30_000)
|
||||
expect(page.locator("#sign-out-btn")).to_be_visible(timeout=15_000)
|
||||
page.set_input_files("#archive-upload-file", str(archive))
|
||||
page.click("#archive-upload-btn")
|
||||
result = page.locator("#archive-upload-result")
|
||||
expect(result).to_be_visible(timeout=UPLOAD_TIMEOUT_MS)
|
||||
expect(result).to_have_text(
|
||||
f"Uploaded {source} — press Sync sources to import it.",
|
||||
timeout=UPLOAD_TIMEOUT_MS,
|
||||
)
|
||||
|
||||
|
||||
def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
|
||||
"""The RAG page's "Sync sources" button (the scan — phase 90 A3):
|
||||
click → "Syncing…" → the terminal "Synced HH:MM" label + the fresh
|
||||
counts in #sync-result (the never-stale contract)."""
|
||||
page.goto(app_url + "/sources.html")
|
||||
btn = page.locator("#sync-btn")
|
||||
expect(btn).to_be_visible(timeout=30_000)
|
||||
expect(page.locator("#sync-label")).to_have_text(
|
||||
re.compile(r"^Sync sources$|^Synced \d{1,2}:\d{2}$")
|
||||
)
|
||||
btn.click()
|
||||
expect(btn).to_be_disabled()
|
||||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(page.locator("#sync-result")).to_have_text(expected_result)
|
||||
|
||||
|
||||
def _captured_systems(mock_llm: int) -> list[str]:
|
||||
"""The system prompts of the captured chat requests (the mock's
|
||||
ring buffer — the wire-level lens on the EXACT prompt the app
|
||||
built; only ``/v1/chat/completions`` bodies are captured, so after
|
||||
a reset + one grounded zero-read turn there is exactly one)."""
|
||||
bodies = httpx.get(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured", timeout=10).json()
|
||||
systems: list[str] = []
|
||||
for body in bodies:
|
||||
for msg in body.get("messages", []):
|
||||
if msg.get("role") == "system" and isinstance(msg.get("content"), str):
|
||||
systems.append(msg["content"])
|
||||
return systems
|
||||
|
||||
|
||||
def _reset_capture(mock_llm: int) -> None:
|
||||
httpx.post(f"http://127.0.0.1:{mock_llm}/v1/e2e/captured/reset", timeout=10)
|
||||
|
||||
|
||||
def _ask(page: Page, app_url: str, question: str) -> None:
|
||||
"""The chat user path: login → the question → Send → the final
|
||||
brain bubble (the Send button re-enabled = the turn is over)."""
|
||||
login(page, app_url, next="/")
|
||||
expect(page.locator("#kb-banner")).to_be_hidden()
|
||||
page.fill("#message-input", question)
|
||||
page.click("#send-btn")
|
||||
bubble = page.locator(".msg.brain .bubble").last
|
||||
expect(page.locator("#send-btn")).to_be_enabled(timeout=60_000)
|
||||
expect(page.locator("#send-label")).to_have_text("Send")
|
||||
expect(bubble).to_contain_text("Deterministic mock answer", timeout=30_000)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The module seeds: phase A (the anchored archive) and phase B (the
|
||||
# standalone archive) — each owns its truncate/re-import window.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def seeded(
|
||||
app_url: str,
|
||||
browser: Browser,
|
||||
anchor_zip: Path,
|
||||
) -> Iterator[dict[str, str]]:
|
||||
"""Phase A: upload the anchored archive through the admin UI, then
|
||||
scan it with the RAG page's Sync button (the mock's vision model
|
||||
describes the image in-process, with the markdown context). The
|
||||
module app's ``/api/config`` flags are asserted first — the env
|
||||
overrides must have reached the app under test, or the whole
|
||||
premise of the suite is void."""
|
||||
if not _health_db(app_url):
|
||||
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
||||
cfg = httpx.get(f"{app_url}/api/config", timeout=5).json()
|
||||
assert cfg["images"] is True, "the module app must run with BOR_IMAGES=true"
|
||||
|
||||
_truncate_all()
|
||||
page = browser.new_page(viewport={"width": 1280, "height": 800})
|
||||
try:
|
||||
_upload_zip(page, app_url, anchor_zip, SOURCE_NAME)
|
||||
_sync_via_ui(page, app_url, expected_result="2 added")
|
||||
with SessionLocal() as db:
|
||||
doc = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == SOURCE_NAME, Document.path == DOC_PATH
|
||||
)
|
||||
)
|
||||
assert doc is not None, "the uploaded image must become a document"
|
||||
assert doc.is_image is True and doc.image_path is not None
|
||||
finally:
|
||||
page.close()
|
||||
yield {"image_url": f"/api/documents/{doc.id}/image"}
|
||||
_truncate_all()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def standalone(
|
||||
app_url: str,
|
||||
browser: Browser,
|
||||
standalone_zip: Path,
|
||||
) -> Iterator[dict[str, str]]:
|
||||
"""Phase B: a FRESH KB (the phase-A rows are truncated) seeded
|
||||
with the standalone archive — a single PNG no markdown in the
|
||||
source references (the byte-identical pre-phase case)."""
|
||||
_truncate_all()
|
||||
page = browser.new_page(viewport={"width": 1280, "height": 800})
|
||||
try:
|
||||
_upload_zip(page, app_url, standalone_zip, STANDALONE_SOURCE_NAME)
|
||||
_sync_via_ui(page, app_url, expected_result="1 added")
|
||||
with SessionLocal() as db:
|
||||
doc = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == STANDALONE_SOURCE_NAME, Document.path == LOGO_PATH
|
||||
)
|
||||
)
|
||||
assert doc is not None, "the uploaded image must become a document"
|
||||
assert doc.is_image is True and doc.image_path is not None
|
||||
finally:
|
||||
page.close()
|
||||
yield {"image_url": f"/api/documents/{doc.id}/image"}
|
||||
_truncate_all()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Sync: the image doc is enriched with the markdown context
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_sync_enriches_the_image_doc_with_markdown_context(
|
||||
page: Page, app_url: str, seeded: dict[str, str]
|
||||
) -> None:
|
||||
"""LOCKED A1/A2 end-to-end: the markdown in the SAME source
|
||||
references the image → after the sync the image doc's
|
||||
``image_context`` is the ``Referenced in`` block, its ``content``
|
||||
(AND ``summary`` — the phase-122 convention) is
|
||||
``<context>\\n\n<description>`` (context first — for retrieval —
|
||||
then the vision description), the content chunk is embedded from
|
||||
the enriched text (the packer's single-newline form) and the
|
||||
``is_summary`` chunk mirrors the raw enriched text. The referring
|
||||
markdown's OWN ``image_context`` stays NULL: the image was indexed
|
||||
BEFORE the markdown (``docs/diagram.png`` sorts first), so its
|
||||
reference names an existing image doc — the task-03 path owns that
|
||||
case, no pending record is written. The Sources page counts the
|
||||
KB: 2 docs, 4 chunks (one content + one summary chunk per doc)."""
|
||||
with SessionLocal() as db:
|
||||
doc = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == SOURCE_NAME, Document.path == DOC_PATH
|
||||
)
|
||||
)
|
||||
assert doc is not None
|
||||
# The stored context (task 05's display source).
|
||||
assert doc.image_context == BLOCK
|
||||
# The enriched content — context FIRST, then the description —
|
||||
# IS the summary (the phase-122 image convention).
|
||||
assert doc.content == ENRICHED
|
||||
assert doc.summary == ENRICHED
|
||||
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
|
||||
content_chunks = [c for c in chunks if not c.is_summary]
|
||||
assert [c.content for c in content_chunks] == [PACKED], (
|
||||
"the content chunk is embedded from the ENRICHED text"
|
||||
)
|
||||
summary_chunks = [c for c in chunks if c.is_summary]
|
||||
assert len(summary_chunks) == 1 and summary_chunks[0].position == -1
|
||||
assert summary_chunks[0].content == ENRICHED
|
||||
for c in chunks:
|
||||
assert c.embedding is not None and len(c.embedding) == 768
|
||||
|
||||
# The referring markdown — the image doc existed at its
|
||||
# indexing (sort order), so NO pending record was written.
|
||||
md = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == SOURCE_NAME, Document.path == MD_PATH
|
||||
)
|
||||
)
|
||||
assert md is not None and md.is_image is False
|
||||
assert md.image_context is None, "existing image: the record stays NULL"
|
||||
|
||||
# The KB on the Sources page: 2 docs (the markdown + the image),
|
||||
# 4 chunks (one content + one is_summary per doc).
|
||||
login(page, app_url)
|
||||
expect(page.locator("#stat-docs")).to_have_text("2")
|
||||
expect(page.locator("#stat-chunks")).to_have_text("4")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Chat: the enriched image doc's <document> block leads with the
|
||||
# context line — the answer AND the wire prompt prove it
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_chat_answer_quotes_the_context_line(
|
||||
page: Page, app_url: str, mock_llm: int, seeded: dict[str, str]
|
||||
) -> None:
|
||||
"""LOCKED A4 end-to-end: a grounded turn whose suggested tier
|
||||
carries the ENRICHED image doc → the ``<document>`` block LEADS
|
||||
with the ONE-LINE context line (the task-05 rendering of the
|
||||
stored block: the referring markdown's identity + the reference's
|
||||
alt, BEFORE the summary — for an enriched image doc the summary
|
||||
is the enriched content, so the stored block rides in the body
|
||||
too, but the header is the one-line rendering, never the stored
|
||||
multi-line header). The mock's marker branch QUOTES THE EXACT
|
||||
LINE of the block — so the rendered answer proves the one-line
|
||||
rendering was in the prompt (the quoted alt sits on the header
|
||||
line — the stored block's shape carries it on the next line as a
|
||||
bullet), and the captured wire prompt pins the exact block shape:
|
||||
line, blank line, enriched summary."""
|
||||
_reset_capture(mock_llm)
|
||||
_ask(page, app_url, ANCHOR_QUESTION)
|
||||
|
||||
bubble = page.locator(".msg.brain .bubble").last
|
||||
# The mock's enriched answer — the EXACT context line, quoted.
|
||||
expect(bubble).to_contain_text(ANSWER_WITH_LINE, timeout=30_000)
|
||||
# Grounded (not deflected) — a deflected prompt would carry no
|
||||
# <documents> block at all.
|
||||
expect(bubble).not_to_contain_text(DEFLECT_OPENING)
|
||||
|
||||
# The wire prompt: exactly one chat request (grounded, zero tool
|
||||
# calls), and the image doc's <document> block leads with the
|
||||
# ONE-LINE context line — then a blank line, then the summary
|
||||
# (for an enriched image doc the summary IS the enriched content —
|
||||
# the phase-122 ``summary = content`` convention — so the stored
|
||||
# block rides along in the body, but the block HEADS with the
|
||||
# one-line rendering, never the stored header line).
|
||||
systems = _captured_systems(mock_llm)
|
||||
assert len(systems) == 1, "one grounded zero-read turn → one chat request"
|
||||
system = systems[0]
|
||||
start = system.index(f'<document source="{SOURCE_NAME}" path="{DOC_PATH}"')
|
||||
body_start = system.index(">\n", start) + 2
|
||||
body_end = system.index("\n</document>", body_start)
|
||||
block_body = system[body_start:body_end]
|
||||
# The exact block shape: the one-line context line (the alt QUOTED
|
||||
# on the same line — the stored block's shape carries the alt on
|
||||
# the NEXT line as a bullet, so this pins the task-05 rendering),
|
||||
# a blank line, then the enriched summary.
|
||||
assert block_body == f"{CONTEXT_LINE}\n\n{ENRICHED}"
|
||||
# The line is inside <documents>, before the summary's block.
|
||||
assert system.index("<documents>") < system.index(CONTEXT_LINE)
|
||||
assert system.index(CONTEXT_LINE) < system.index("\n" + BLOCK)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Standalone image: the byte-identical pre-phase row
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_standalone_image_keeps_null_context_and_bare_description(
|
||||
page: Page, app_url: str, standalone: dict[str, str]
|
||||
) -> None:
|
||||
"""LOCKED A4 (the byte-identity criterion) on the ROW: a standalone
|
||||
image (no markdown in the source references it) → ``image_context``
|
||||
NULL, ``content`` (AND ``summary``) the BARE vision description,
|
||||
every chunk the bare description — exactly the phase-122 shape.
|
||||
The Sources page counts 1 doc / 2 chunks (one content + one
|
||||
summary)."""
|
||||
with SessionLocal() as db:
|
||||
doc = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == STANDALONE_SOURCE_NAME, Document.path == LOGO_PATH
|
||||
)
|
||||
)
|
||||
assert doc is not None
|
||||
assert doc.image_context is None, "standalone: the context stays NULL"
|
||||
assert doc.content == IMAGE_DESCRIPTION_ANSWER, "standalone: the bare description"
|
||||
assert doc.summary == IMAGE_DESCRIPTION_ANSWER
|
||||
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
|
||||
assert len(chunks) == 2
|
||||
assert all(c.content == IMAGE_DESCRIPTION_ANSWER for c in chunks)
|
||||
for c in chunks:
|
||||
assert c.embedding is not None and len(c.embedding) == 768
|
||||
|
||||
login(page, app_url)
|
||||
expect(page.locator("#stat-docs")).to_have_text("1")
|
||||
expect(page.locator("#stat-chunks")).to_have_text("2")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. Standalone image: the <document> block is byte-identical in the
|
||||
# prompt — no context line, the bare description alone
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_standalone_answer_carries_no_context_line(
|
||||
page: Page, app_url: str, mock_llm: int, standalone: dict[str, str]
|
||||
) -> None:
|
||||
"""LOCKED A4 (the byte-identity criterion) in the PROMPT: the SAME
|
||||
marker question against the standalone KB → the image doc's
|
||||
``<document>`` block body is EXACTLY the bare description (no
|
||||
context line, no stray blank — the pre-phase block), and the mock
|
||||
— which quotes the block's first line when it is a ``Referenced
|
||||
in`` line — answers that the image carries no reference line: the
|
||||
deterministic witness that the line was NOT in the prompt.
|
||||
``Referenced in`` does not appear ANYWHERE in the system prompt."""
|
||||
_reset_capture(mock_llm)
|
||||
_ask(page, app_url, ANCHOR_QUESTION)
|
||||
|
||||
bubble = page.locator(".msg.brain .bubble").last
|
||||
expect(bubble).to_contain_text(ANSWER_STANDALONE, timeout=30_000)
|
||||
expect(bubble).not_to_contain_text(DEFLECT_OPENING)
|
||||
|
||||
systems = _captured_systems(mock_llm)
|
||||
assert len(systems) == 1, "one grounded zero-read turn → one chat request"
|
||||
system = systems[0]
|
||||
start = system.index(
|
||||
f'<document source="{STANDALONE_SOURCE_NAME}" path="{LOGO_PATH}"'
|
||||
)
|
||||
body_start = system.index(">\n", start) + 2
|
||||
body_end = system.index("\n</document>", body_start)
|
||||
block_body = system[body_start:body_end]
|
||||
# The pre-phase block body: the description alone, verbatim.
|
||||
assert block_body == IMAGE_DESCRIPTION_ANSWER
|
||||
assert "Referenced in" not in system
|
||||
@@ -2790,3 +2790,102 @@ def test_zero_read_text_only_turn_still_chips_nothing(
|
||||
assert done["sources"] == [] # phase-119 read-docs-only rule intact
|
||||
for ref in done["sources"] + done["related"]:
|
||||
assert "image_url" not in ref
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 125, task 05 — retrieval display: the suggested tier's ENRICHED
|
||||
# image doc (non-NULL image_context, the task-03/04 block shape) leads
|
||||
# its <document> block with the one-line context line (the referring
|
||||
# markdown's identity + the reference's alt) before the description;
|
||||
# the non-enriched image doc (NULL image_context) stays byte-identical
|
||||
# to pre-phase (LOCKED A4).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_grounded_turn_suggested_enriched_image_doc_carries_context_line(
|
||||
client, db, seeded_kb: FakeRagLLM
|
||||
) -> None:
|
||||
"""LOCKED A4 end-to-end: a grounded turn whose suggested tier
|
||||
contains an enriched image doc (the task-03/04 ``Referenced in``
|
||||
block stored on ``image_context``) → the system prompt's
|
||||
``<document>`` block for that doc leads with the short context
|
||||
line, a blank line, then the description — and the ``Context:``
|
||||
paragraphs of the stored block never reach the prompt (the line is
|
||||
the locator + purpose, not a duplicate of the paragraphs)."""
|
||||
doc = _seed_image_doc_with_chunk(db)
|
||||
doc.image_context = (
|
||||
"Referenced in docs/homelab/kubernetes.md:\n"
|
||||
'- Alt text: "Cluster layout"\n'
|
||||
" Context: The diagram shows the three Talos nodes.\n"
|
||||
"It is referenced again from the runbook."
|
||||
)
|
||||
db.commit()
|
||||
scripted = FakeRagLLM() # zero-read grounded turn (the image seed block IS its content)
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
|
||||
try:
|
||||
_, _, frames = _stream_chat(client, QUESTION)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
assert frames[-1]["deflected"] is False
|
||||
system = scripted.seen_messages[0][0]["content"]
|
||||
# The doc is in the suggested tier (retrieval is unchanged by the
|
||||
# phase — the line rides on a doc the retrieval already picked).
|
||||
block_start = system.index(f'<document source="docs" path="{doc.path}"')
|
||||
block = system[
|
||||
block_start : system.index("</document>", block_start) + len("</document>")
|
||||
]
|
||||
line = 'Referenced in docs/homelab/kubernetes.md: "Cluster layout"'
|
||||
# The exact block shape: identity attrs (phase 106 date), the line,
|
||||
# a blank line, the description (the phase-122 content == summary
|
||||
# == vision description invariant).
|
||||
assert block == (
|
||||
f'<document source="docs" path="pic.png" title="pic" '
|
||||
f'date="{_FIXTURE_CREATED_AT:%Y-%m-%d}">\n'
|
||||
f"{line}\n"
|
||||
"\n"
|
||||
f"{doc.summary}\n"
|
||||
"</document>"
|
||||
)
|
||||
# The line is the FIRST thing in the block body (before the
|
||||
# description), inside <documents>.
|
||||
assert system.index("<documents>") < system.index(line)
|
||||
assert system.index(line) < system.index(doc.summary or "")
|
||||
# The Context: body stays out of the whole prompt.
|
||||
assert "The diagram shows the three Talos nodes." not in system
|
||||
assert "referenced again from the runbook" not in system
|
||||
assert "Context:" not in system
|
||||
|
||||
|
||||
def test_grounded_turn_suggested_plain_image_doc_block_is_byte_identical(
|
||||
client, db, seeded_kb: FakeRagLLM
|
||||
) -> None:
|
||||
"""LOCKED A4 (the byte-identity criterion) end-to-end: the SAME
|
||||
flow with ``image_context`` left NULL (the standalone / pre-phase
|
||||
image doc — the phase's default) → the ``<document>`` block is
|
||||
EXACTLY the pre-phase block (the summary alone, no context line,
|
||||
no stray blank) and the marker string does not appear anywhere in
|
||||
the system prompt."""
|
||||
doc = _seed_image_doc_with_chunk(db)
|
||||
assert doc.image_context is None # the seed leaves it NULL (standalone)
|
||||
scripted = FakeRagLLM()
|
||||
fastapi_app.dependency_overrides[chat_api.get_llm] = lambda: scripted
|
||||
try:
|
||||
_, _, frames = _stream_chat(client, QUESTION)
|
||||
finally:
|
||||
fastapi_app.dependency_overrides.clear()
|
||||
|
||||
assert frames[-1]["deflected"] is False
|
||||
system = scripted.seen_messages[0][0]["content"]
|
||||
block_start = system.index(f'<document source="docs" path="{doc.path}"')
|
||||
block = system[
|
||||
block_start : system.index("</document>", block_start) + len("</document>")
|
||||
]
|
||||
# The pre-phase block: identity attrs + the description, verbatim.
|
||||
assert block == (
|
||||
f'<document source="docs" path="pic.png" title="pic" '
|
||||
f'date="{_FIXTURE_CREATED_AT:%Y-%m-%d}">\n'
|
||||
f"{doc.summary}\n"
|
||||
"</document>"
|
||||
)
|
||||
assert "Referenced in" not in system
|
||||
|
||||
@@ -869,9 +869,13 @@ def _image_llm[
|
||||
def _patch_description(monkeypatch, description) -> None:
|
||||
"""Pin the task-02 seam (``rag_importer._describe_or_skip``) to
|
||||
*description*. Task 03 fills the seam with the CHAT model's vision
|
||||
call — these end-to-end mechanics do not change with it."""
|
||||
call — these end-to-end mechanics do not change with it. Phase 125
|
||||
(task 03) widened the seam with the optional ``context`` kwarg (the
|
||||
markdown reference context) — the fake accepts (and ignores) it:
|
||||
these fixtures carry no image references, so the seam gets
|
||||
``context=None``."""
|
||||
|
||||
async def _fake(llm, *, data, source, rel, full_path):
|
||||
async def _fake(llm, *, data, source, rel, full_path, context=None):
|
||||
return description
|
||||
|
||||
monkeypatch.setattr(rag_importer, "_describe_or_skip", _fake)
|
||||
|
||||
@@ -0,0 +1,446 @@
|
||||
"""Integration: phase 125, task 03 — the image-anchor enrichment through
|
||||
the REAL import pipeline (real compose Postgres, the real
|
||||
``_describe_or_skip`` → ``describe_image`` seam, the mock vision
|
||||
client — NO seam patch).
|
||||
|
||||
A markdown file referencing an image in the SAME source → the vision
|
||||
call receives the markdown context (appended after the locked
|
||||
``DESCRIBE_PROMPT``), the image doc's ``image_context`` is set, its
|
||||
``content``/``summary`` is ``<context>\\n\\n<description>``, and its
|
||||
chunks are embedded from the enriched content. A standalone image (no
|
||||
markdown reference) → ``image_context`` NULL, ``content`` the bare
|
||||
description, the bare locked prompt — byte-identical to pre-phase-125.
|
||||
|
||||
Phase 125, task 04 (the markdown-FIRST case) — a markdown indexed while
|
||||
its referenced image has no document yet stores the reference on the
|
||||
markdown doc's ``image_context`` (the ``IMAGE_CONTEXT:`` record); the
|
||||
image that appears later picks the context up at index time (the
|
||||
task-03 disk scan) or, when the markdown file is gone from disk by
|
||||
then, through the post-index enrichment pass (the stored doc rows —
|
||||
record as the index, stored content as the source). Pinned here: the
|
||||
record-then-image sync pair (the record stored, not cleared), the
|
||||
pass-only path (markdown file deleted from disk, ``prune=False``), and
|
||||
the existing-image case (the markdown's record stays NULL — the
|
||||
task-03 path owns that case).
|
||||
|
||||
The unit-level mechanics (the prefix shape, the scan's combined string,
|
||||
the seam forwarding, the changed-image re-scan, the record formatter +
|
||||
parser, the pass's idempotency/fail-soft) are pinned in
|
||||
``tests/unit/test_image_anchor_context.py``; the task-06 story suite
|
||||
pins the E2E surface.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.config import Settings
|
||||
from app.models import Chunk, Document
|
||||
from app.rag.importer import import_sources
|
||||
from app.rag.summarizer import DESCRIBE_PROMPT
|
||||
from tests.fakes import FakeEmbedder
|
||||
|
||||
#: A real 1×1 transparent PNG (the phase-122 fixture — the importer is
|
||||
#: content-agnostic, the bytes just need to be plausible image bytes).
|
||||
PNG_1X1 = base64.b64decode(
|
||||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR4nGP4DwQACfsD/fteaysAAAAASUVORK5CYII="
|
||||
)
|
||||
|
||||
DESCRIPTION = (
|
||||
"A network diagram of the homelab VLANs: the core switch, the "
|
||||
"router, and three labeled subnets."
|
||||
)
|
||||
|
||||
#: The exact context prefix the task-03 extension appends to the locked
|
||||
#: prompt (``app.rag.summarizer.describe_image``).
|
||||
CONTEXT_PREFIX = "\n\nThis image is referenced in documentation with the following context:\n"
|
||||
|
||||
|
||||
class _MockVisionLLM(FakeEmbedder):
|
||||
"""The phase-122 mock VISION client: the CHAT model answers the
|
||||
multimodal describe call with the fixed :data:`DESCRIPTION` (the
|
||||
text part is recorded — ``text_parts``); text (``lite``) calls keep
|
||||
the ``FakeEmbedder`` behaviour. The REAL
|
||||
``import_sources`` → ``_describe_or_skip`` → ``describe_image``
|
||||
chain runs against it (no seam patch)."""
|
||||
|
||||
DESCRIPTION = DESCRIPTION
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.chat_models: list[str | None] = []
|
||||
self.text_parts: list[str] = []
|
||||
|
||||
async def chat(self, messages, model=None):
|
||||
self.chat_calls.append(list(messages))
|
||||
self.chat_models.append(model)
|
||||
user = next((m["content"] for m in messages if m.get("role") == "user"), "")
|
||||
if isinstance(user, list):
|
||||
self.text_parts.append(user[0]["text"])
|
||||
return self.DESCRIPTION
|
||||
first = user.split()
|
||||
return "Summary of " + (first[0] if first else "<empty>")
|
||||
|
||||
|
||||
def _image_llm[
|
||||
LLM: FakeEmbedder
|
||||
](tmp_path, llm_cls: type[LLM] = _MockVisionLLM, **kwargs) -> LLM:
|
||||
"""A fake LLM with the phase-122 image knobs (toggle ON; the image
|
||||
dir defaults under *tmp_path*). *llm_cls* may be the mock vision
|
||||
client (the default — ``text_parts``/``chat_models`` state)."""
|
||||
kwargs.setdefault("_env_file", None)
|
||||
kwargs.setdefault("images", True)
|
||||
kwargs.setdefault("image_dir", str(tmp_path / "images"))
|
||||
llm = llm_cls()
|
||||
llm.settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
|
||||
return llm
|
||||
|
||||
|
||||
def _cleanup_source(db, source: str) -> None:
|
||||
for doc in db.scalars(select(Document).where(Document.source == source)).all():
|
||||
db.delete(doc)
|
||||
db.commit()
|
||||
|
||||
|
||||
#: A markdown file that references ``pic.png`` on line 3 — one context
|
||||
#: paragraph before and one after (the task-02 window).
|
||||
GUIDE_MD = (
|
||||
"The restore order diagram explains the backup procedure.\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"Only run the restore script after the checks pass.\n"
|
||||
)
|
||||
|
||||
|
||||
def _expected_block(source: str) -> str:
|
||||
"""The task-03-shape combined string *GUIDE_MD* yields for
|
||||
``<source>/guide.md`` (the disk scan AND the task-04 pass build
|
||||
exactly this)."""
|
||||
return (
|
||||
f"Referenced in {source}/guide.md:\n"
|
||||
'- Alt text: "Restore order"\n'
|
||||
" Context: The restore order diagram explains the backup procedure.\n"
|
||||
"\n"
|
||||
"Only run the restore script after the checks pass."
|
||||
)
|
||||
|
||||
|
||||
#: The ``IMAGE_CONTEXT:`` record the markdown-side bookkeeping (phase
|
||||
#: 125, task 04, work 1) stores for *GUIDE_MD* — the reference on line
|
||||
#: 3, 2 context paragraphs.
|
||||
RECORD = (
|
||||
"IMAGE_CONTEXT:\n"
|
||||
'- [pic.png] (line 3): "Restore order" — 2 paragraphs of context'
|
||||
)
|
||||
|
||||
|
||||
def test_markdown_referenced_image_gets_context_and_enriched_chunks(
|
||||
db, tmp_path
|
||||
) -> None:
|
||||
"""A markdown file referencing an image in the same source → sync
|
||||
with ``images=True`` (the REAL seam — no patch) → the vision call
|
||||
carries the context after the locked prompt, the image doc's
|
||||
``image_context`` is set, its ``content`` includes the context
|
||||
(context first, then the description), and its chunks are embedded
|
||||
from the enriched content (the ``is_summary`` chunk mirrors it)."""
|
||||
source_root = tmp_path / "anchoredkb"
|
||||
source_root.mkdir()
|
||||
(source_root / "guide.md").write_text(
|
||||
"The backup procedure diagram below shows the restore order.\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"Run the restore script only after the diagram's checks pass.\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(source_root / "diagram.png").write_bytes(PNG_1X1)
|
||||
llm = _image_llm(tmp_path)
|
||||
try:
|
||||
summary = asyncio.run(import_sources([source_root], llm, session=db))
|
||||
assert (summary.files, summary.added, summary.images_failed) == (2, 2, 0)
|
||||
|
||||
doc = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "diagram.png"
|
||||
)
|
||||
)
|
||||
assert doc is not None and doc.is_image is True and doc.image_path is not None
|
||||
|
||||
expected_context = (
|
||||
f"Referenced in {source_root.name}/guide.md:\n"
|
||||
'- Alt text: "Backup procedure diagram"\n'
|
||||
" Context: The backup procedure diagram below shows the restore order.\n"
|
||||
"\n"
|
||||
"Run the restore script only after the diagram's checks pass."
|
||||
)
|
||||
# The vision model received the context — appended AFTER the
|
||||
# locked prompt (the ``IMAGE_DESCRIPTION_MODE`` marker is
|
||||
# untouched). The describe call (the ONLY multimodal one) went
|
||||
# to the CHAT model (``chat_models`` also carries the markdown
|
||||
# doc's ``lite`` summary call — the phase-30 path).
|
||||
assert llm.chat_models[0] == llm.settings.llm_chat_model
|
||||
assert llm.text_parts == [f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{expected_context}"]
|
||||
|
||||
# ``image_context`` stores the combined context (task 05's
|
||||
# retrieval display reads from here).
|
||||
assert doc.image_context == expected_context
|
||||
# The enriched content — context FIRST, then the description —
|
||||
# IS the summary (the phase-122 convention) and the chunks are
|
||||
# embedded from it (768 dims, the summary chunk mirrors it). The
|
||||
# plain-text packer joins the enriched paragraphs with single
|
||||
# newlines (the chunker's normalization, PLAN §5), so the content
|
||||
# chunk carries the packed form and the summary chunk the raw
|
||||
# text.
|
||||
expected_content = f"{expected_context}\n\n{DESCRIPTION}"
|
||||
assert doc.content == expected_content
|
||||
assert doc.summary == expected_content
|
||||
packed_content = "\n".join(expected_content.split("\n\n"))
|
||||
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
|
||||
content_chunks = [c for c in chunks if not c.is_summary]
|
||||
assert [c.content for c in content_chunks] == [packed_content]
|
||||
summary_chunks = [c for c in chunks if c.is_summary]
|
||||
assert len(summary_chunks) == 1 and summary_chunks[0].position == -1
|
||||
assert summary_chunks[0].content == expected_content
|
||||
# Both enriched forms reached the embedder (the markdown doc's
|
||||
# two texts complete the four embedded strings).
|
||||
embedded = [t for batch in llm.calls for t in batch]
|
||||
assert len(embedded) == 4
|
||||
assert packed_content in embedded and expected_content in embedded
|
||||
for c in chunks:
|
||||
assert c.embedding is not None and len(c.embedding) == 768
|
||||
|
||||
# The referencing markdown stays a plain text doc (the
|
||||
# ``image_context`` column is an image-doc field — NULL).
|
||||
md = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "guide.md"
|
||||
)
|
||||
)
|
||||
assert md is not None and md.is_image is False and md.image_context is None
|
||||
finally:
|
||||
_cleanup_source(db, source_root.name)
|
||||
|
||||
|
||||
def test_standalone_image_keeps_null_context_and_bare_description(db, tmp_path) -> None:
|
||||
"""A standalone image (no markdown reference in the source — a
|
||||
sibling markdown exists but references nothing) → ``image_context``
|
||||
NULL, ``content`` the bare description, the vision message the bare
|
||||
locked prompt — byte-identical to pre-phase-125 (the chunk set and
|
||||
the embedded text are exactly the phase-122 shape)."""
|
||||
source_root = tmp_path / "standalonekb"
|
||||
source_root.mkdir()
|
||||
(source_root / "notes.md").write_text("# Notes\n\nBody text.\n", encoding="utf-8")
|
||||
(source_root / "diagram.png").write_bytes(PNG_1X1)
|
||||
llm = _image_llm(tmp_path)
|
||||
try:
|
||||
summary = asyncio.run(import_sources([source_root], llm, session=db))
|
||||
assert (summary.files, summary.added, summary.images_failed) == (2, 2, 0)
|
||||
|
||||
doc = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "diagram.png"
|
||||
)
|
||||
)
|
||||
assert doc is not None and doc.is_image is True
|
||||
assert doc.image_context is None, "standalone: the context stays NULL"
|
||||
assert doc.content == DESCRIPTION, "standalone: the bare description"
|
||||
assert doc.summary == DESCRIPTION
|
||||
assert llm.text_parts == [DESCRIBE_PROMPT], "standalone: the bare prompt"
|
||||
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
|
||||
assert all(c.content == DESCRIPTION for c in chunks)
|
||||
for c in chunks:
|
||||
assert c.embedding is not None and len(c.embedding) == 768
|
||||
finally:
|
||||
_cleanup_source(db, source_root.name)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 125, task 04 — the markdown-FIRST case: the ``IMAGE_CONTEXT:``
|
||||
# record on the markdown doc + the post-index enrichment pass.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_markdown_indexed_before_image_stores_record_and_image_picks_it_up(
|
||||
db, tmp_path
|
||||
) -> None:
|
||||
"""Task 04's primary scenario — a markdown is indexed while the
|
||||
image it references does not yet exist: the reference is stored on
|
||||
the markdown doc's ``image_context`` (the ``IMAGE_CONTEXT:``
|
||||
record). The image file then appears and is synced: the image doc
|
||||
is enriched with the markdown's context (the task-03 disk scan —
|
||||
the markdown is still on disk), and the markdown's record is NOT
|
||||
cleared (ASSUMPTION 4)."""
|
||||
source_root = tmp_path / "beforekb"
|
||||
source_root.mkdir()
|
||||
(source_root / "guide.md").write_text(GUIDE_MD, encoding="utf-8")
|
||||
llm = _image_llm(tmp_path)
|
||||
try:
|
||||
# Run 1 — the markdown alone: the record is stored, no image
|
||||
# doc exists.
|
||||
s1 = asyncio.run(import_sources([source_root], llm, session=db))
|
||||
assert (s1.files, s1.added) == (1, 1)
|
||||
md = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "guide.md"
|
||||
)
|
||||
)
|
||||
assert md is not None and md.image_context == RECORD
|
||||
assert (
|
||||
db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "pic.png"
|
||||
)
|
||||
)
|
||||
is None
|
||||
)
|
||||
|
||||
# Run 2 — the image appears: enriched at index time (the
|
||||
# task-03 disk scan sees the on-disk markdown), the record
|
||||
# survives.
|
||||
(source_root / "pic.png").write_bytes(PNG_1X1)
|
||||
s2 = asyncio.run(import_sources([source_root], llm, session=db))
|
||||
assert (s2.added, s2.unchanged) == (1, 1)
|
||||
block = _expected_block(source_root.name)
|
||||
doc = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "pic.png"
|
||||
)
|
||||
)
|
||||
assert doc is not None and doc.is_image is True
|
||||
assert doc.image_context == block
|
||||
expected_content = f"{block}\n\n{DESCRIPTION}"
|
||||
assert doc.content == expected_content
|
||||
assert doc.summary == expected_content
|
||||
# The describe call carried the context after the locked prompt.
|
||||
assert llm.text_parts == [f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{block}"]
|
||||
md2 = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "guide.md"
|
||||
)
|
||||
)
|
||||
assert md2 is not None and md2.image_context == RECORD, "the record is NOT cleared"
|
||||
finally:
|
||||
_cleanup_source(db, source_root.name)
|
||||
|
||||
|
||||
def test_enrichment_pass_enriches_when_markdown_file_is_gone_from_disk(
|
||||
db, tmp_path
|
||||
) -> None:
|
||||
"""The post-index enrichment pass (task 04, work 3) — the case the
|
||||
task-03 disk scan CANNOT cover: the markdown is indexed first (the
|
||||
record lands on its doc row), its file is then removed from disk
|
||||
(``prune=False`` keeps the row), and only then does the image
|
||||
appear. The image is indexed BARE (the disk scan finds no
|
||||
reference), and the pass enriches it from the markdown doc's
|
||||
STORED rows: the record identifies the referencing markdown, the
|
||||
reference is re-extracted from the stored content, and the
|
||||
re-describe goes out WITH the context (a second vision call — the
|
||||
first, at index time, was bare)."""
|
||||
source_root = tmp_path / "passkb"
|
||||
source_root.mkdir()
|
||||
(source_root / "guide.md").write_text(GUIDE_MD, encoding="utf-8")
|
||||
llm = _image_llm(tmp_path)
|
||||
try:
|
||||
s1 = asyncio.run(import_sources([source_root], llm, session=db))
|
||||
assert s1.added == 1
|
||||
md = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "guide.md"
|
||||
)
|
||||
)
|
||||
assert md is not None and md.image_context == RECORD
|
||||
|
||||
# The markdown file disappears from disk (``prune=False`` keeps
|
||||
# the row); the image appears.
|
||||
(source_root / "guide.md").unlink()
|
||||
(source_root / "pic.png").write_bytes(PNG_1X1)
|
||||
s2 = asyncio.run(import_sources([source_root], llm, session=db))
|
||||
# The image is added (bare) and then enriched by the pass
|
||||
# (updated) — both in this run.
|
||||
assert (s2.files, s2.added, s2.updated) == (1, 1, 1)
|
||||
block = _expected_block(source_root.name)
|
||||
doc = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "pic.png"
|
||||
)
|
||||
)
|
||||
assert doc is not None and doc.is_image is True
|
||||
assert doc.image_context == block
|
||||
expected_content = f"{block}\n\n{DESCRIPTION}"
|
||||
assert doc.content == expected_content
|
||||
assert doc.summary == expected_content
|
||||
# The re-embedded pipeline (the pass replaced the bare chunks).
|
||||
packed = "\n".join(expected_content.split("\n\n"))
|
||||
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
|
||||
assert [c.content for c in chunks if not c.is_summary] == [packed]
|
||||
assert [c.content for c in chunks if c.is_summary] == [expected_content]
|
||||
# Two vision calls: the bare index-time describe (the disk scan
|
||||
# saw no reference) and the pass's context-carrying re-describe.
|
||||
assert llm.text_parts == [DESCRIBE_PROMPT, f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{block}"]
|
||||
# The markdown doc's record survives (ASSUMPTION 4).
|
||||
md2 = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "guide.md"
|
||||
)
|
||||
)
|
||||
assert md2 is not None and md2.image_context == RECORD
|
||||
finally:
|
||||
_cleanup_source(db, source_root.name)
|
||||
|
||||
|
||||
def test_markdown_referencing_existing_image_keeps_null_record(db, tmp_path) -> None:
|
||||
"""An EXISTING image doc (indexed before the markdown ever
|
||||
existed): the markdown's record stays NULL when it is indexed (the
|
||||
task-03 path owns that case — enrichment at IMAGE-index time), the
|
||||
unchanged image is not re-scanned in that run, and the image picks
|
||||
up the context on its NEXT re-index (bytes changed) via the
|
||||
task-03 disk scan — the markdown's record is NULL throughout."""
|
||||
source_root = tmp_path / "existingkb"
|
||||
source_root.mkdir()
|
||||
(source_root / "pic.png").write_bytes(PNG_1X1)
|
||||
llm = _image_llm(tmp_path)
|
||||
try:
|
||||
s1 = asyncio.run(import_sources([source_root], llm, session=db))
|
||||
assert s1.added == 1
|
||||
|
||||
# Run 2 — the markdown appears (the image is unchanged).
|
||||
(source_root / "guide.md").write_text(GUIDE_MD, encoding="utf-8")
|
||||
s2 = asyncio.run(import_sources([source_root], llm, session=db))
|
||||
assert (s2.added, s2.unchanged) == (1, 1)
|
||||
md = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "guide.md"
|
||||
)
|
||||
)
|
||||
assert md is not None and md.image_context is None
|
||||
doc = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "pic.png"
|
||||
)
|
||||
)
|
||||
assert doc is not None and doc.image_context is None and doc.content == DESCRIPTION
|
||||
|
||||
# Run 3 — the image changes: the re-index re-scans the on-disk
|
||||
# markdown (task 03) and enriches; the record is still NULL.
|
||||
(source_root / "pic.png").write_bytes(PNG_1X1 + b"changed")
|
||||
s3 = asyncio.run(import_sources([source_root], llm, session=db))
|
||||
assert (s3.added, s3.updated, s3.unchanged) == (0, 1, 1)
|
||||
block = _expected_block(source_root.name)
|
||||
doc3 = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "pic.png"
|
||||
)
|
||||
)
|
||||
assert doc3 is not None and doc3.image_context == block
|
||||
assert doc3.content == f"{block}\n\n{DESCRIPTION}"
|
||||
md3 = db.scalar(
|
||||
select(Document).where(
|
||||
Document.source == source_root.name, Document.path == "guide.md"
|
||||
)
|
||||
)
|
||||
assert md3 is not None and md3.image_context is None
|
||||
finally:
|
||||
_cleanup_source(db, source_root.name)
|
||||
@@ -0,0 +1,335 @@
|
||||
"""Integration: migration 0023 (documents.image_context) schema contract
|
||||
(phase 125, task 01).
|
||||
|
||||
Drives the **real Alembic engine** against the live dev database
|
||||
(``podman compose up -d db``), mirroring the house pattern of
|
||||
``test_migration_0022.py`` (information_schema assertions on the state
|
||||
the migration must leave). The tests target the 0022 → 0023 step
|
||||
explicitly so later migrations cannot break the pins:
|
||||
|
||||
* upgrade 0022 → 0023 → ``image_context`` exists with the full
|
||||
contract — TEXT, NULLABLE, no server default (NULL is the
|
||||
standalone-image / pre-phase value, LOCKED A1/A4) — while the 0022
|
||||
``documents`` schema (``is_image`` + ``image_path`` from phase 122,
|
||||
``content``/``content_hash`` NOT NULL, the (source, path) unique
|
||||
constraint) survives;
|
||||
* a ``documents`` row inserted while the DB is at 0022 (a pre-phase
|
||||
image doc) reads ``image_context IS NULL`` after the upgrade — the
|
||||
pre-phase round-trip pin;
|
||||
* the ORM contract agrees: a freshly inserted image ``Document`` WITH
|
||||
``image_context`` round-trips it through a fresh session, and one
|
||||
WITHOUT (standalone image / text doc) reads ``image_context is None``;
|
||||
* downgrade to 0022 → the column is GONE (A13) while the row and its
|
||||
0022 columns survive; upgrade back to 0023 → the column is back
|
||||
(round-trip).
|
||||
|
||||
The ``alembic`` fixture guarantees the DB ends at head even if a test
|
||||
fails or the process is interrupted.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from collections.abc import Iterator
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from alembic.config import Config
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from alembic import command
|
||||
from app.db import SessionLocal, db_available
|
||||
from app.models import Document
|
||||
|
||||
SOURCE = "mig0023"
|
||||
PATH_TEXT = "notes/readme.md"
|
||||
PATH_IMAGE = "notes/diagram.png"
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def alembic(db: Session) -> Iterator[Config]:
|
||||
"""Real Alembic config bound to the dev DB (URL from app settings).
|
||||
|
||||
Starts at head (repairs an interrupted earlier run); teardown
|
||||
upgrades to head no matter what happened, so the dev DB is never
|
||||
left below head.
|
||||
"""
|
||||
if not db_available():
|
||||
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
||||
cfg = Config() # no alembic.ini file — env.py gets the URL from app config
|
||||
cfg.set_main_option("script_location", "alembic")
|
||||
command.upgrade(cfg, "head")
|
||||
try:
|
||||
yield cfg
|
||||
finally:
|
||||
# Release the test session's open transaction BEFORE the repair
|
||||
# DDL: an idle-in-transaction SELECT holds an ACCESS SHARE lock
|
||||
# on ``documents``, which would deadlock the repair's
|
||||
# ``ALTER TABLE`` (0023) forever.
|
||||
db.rollback()
|
||||
command.upgrade(cfg, "head")
|
||||
|
||||
|
||||
def _version(db: Session) -> str | None:
|
||||
return db.execute(text("SELECT version_num FROM alembic_version")).scalar()
|
||||
|
||||
|
||||
def _column(db: Session, column: str) -> tuple[Any, ...] | None:
|
||||
"""(data_type, is_nullable, column_default) for one documents
|
||||
column."""
|
||||
row = db.execute(
|
||||
text(
|
||||
"SELECT data_type, is_nullable, column_default"
|
||||
" FROM information_schema.columns"
|
||||
" WHERE table_name = 'documents' AND column_name = :c"
|
||||
),
|
||||
{"c": column},
|
||||
).fetchone()
|
||||
return tuple(row) if row is not None else None
|
||||
|
||||
|
||||
def _insert_sql(db: Session, path: str) -> uuid.UUID:
|
||||
"""Insert one documents row with the PRE-0023 column set (the 0022
|
||||
shape — ``image_context``, when present, is omitted so its NULL
|
||||
backfill is what the row reads)."""
|
||||
row_id = uuid.uuid4()
|
||||
db.execute(
|
||||
text(
|
||||
"INSERT INTO documents (id, source, path, full_path, title,"
|
||||
" content, content_hash)"
|
||||
" VALUES (:id, :s, :p, :f, :t, :c, :h)"
|
||||
),
|
||||
{
|
||||
"id": row_id,
|
||||
"s": SOURCE,
|
||||
"p": path,
|
||||
"f": f"/tmp/{SOURCE}/{path}",
|
||||
"t": path.rsplit("/", 1)[-1],
|
||||
"c": "content",
|
||||
"h": "0" * 64,
|
||||
},
|
||||
)
|
||||
db.commit()
|
||||
return row_id
|
||||
|
||||
|
||||
def _delete(db: Session, row_id: uuid.UUID) -> None:
|
||||
db.execute(text("DELETE FROM documents WHERE id = :id"), {"id": row_id})
|
||||
db.commit()
|
||||
|
||||
|
||||
def _unique_column_sets(db: Session) -> set[tuple[str, ...]]:
|
||||
"""The column tuples of every UNIQUE constraint on ``documents``.
|
||||
|
||||
Column-based (not name-based): the integration suite's table
|
||||
self-heal (``tests/integration/conftest.py``) rewrites bloated
|
||||
tables via ``CREATE TABLE (LIKE …)``, which renames copied
|
||||
constraints (PG auto-names them) — the (source, path) uniqueness
|
||||
contract is what must hold, not the original name.
|
||||
"""
|
||||
rows = db.execute(
|
||||
text(
|
||||
"SELECT (SELECT string_agg(a.attname, ',' ORDER BY k.ord)"
|
||||
" FROM unnest(c.conkey) WITH ORDINALITY k(attnum, ord)"
|
||||
" JOIN pg_attribute a"
|
||||
" ON a.attrelid = c.conrelid AND a.attnum = k.attnum)"
|
||||
" FROM pg_constraint c"
|
||||
" WHERE c.contype = 'u' AND c.conrelid = 'documents'::regclass"
|
||||
)
|
||||
).fetchall()
|
||||
return {tuple(r[0].split(",")) for r in rows}
|
||||
|
||||
|
||||
def test_upgrade_to_0023_adds_image_context(db: Session, alembic: Config) -> None:
|
||||
"""Upgrade 0022 → 0023: ``image_context`` exists with the full
|
||||
contract (TEXT, NULLABLE, no server default — NULL is the
|
||||
standalone-image / pre-phase value, LOCKED A1) and is ABSENT at
|
||||
0022; a pre-0023 row (a phase-122 image doc) backfills to NULL; and
|
||||
the 0022 table contract survives the additive upgrade."""
|
||||
command.downgrade(alembic, "0022") # start from the pre-0023 state
|
||||
assert _version(db) == "0022"
|
||||
assert _column(db, "image_context") is None, (
|
||||
"image_context must be absent at 0022"
|
||||
)
|
||||
|
||||
pre_id = _insert_sql(db, PATH_IMAGE) # the 0022 column set (a pre-125 row)
|
||||
try:
|
||||
command.upgrade(alembic, "0023")
|
||||
assert _version(db) == "0023", "alembic_version must be at 0023"
|
||||
|
||||
context = _column(db, "image_context")
|
||||
assert context is not None, "documents.image_context is missing"
|
||||
assert context[0] == "text", "image_context must be TEXT"
|
||||
assert context[1] == "YES", "image_context must be NULLABLE"
|
||||
assert context[2] is None, (
|
||||
"image_context must carry NO server default — NULL is the"
|
||||
" standalone-image / pre-phase value (A1, A4)"
|
||||
)
|
||||
|
||||
# The pre-0023 row (inserted with the 0022 column set) reads
|
||||
# NULL — every pre-phase row keeps its byte-identical state.
|
||||
row = db.execute(
|
||||
text("SELECT image_context FROM documents WHERE id = :id"),
|
||||
{"id": pre_id},
|
||||
).fetchone()
|
||||
assert row is not None, "the pre-0023 row must survive the upgrade"
|
||||
assert row[0] is None, (
|
||||
"a pre-phase image doc must backfill image_context to NULL"
|
||||
)
|
||||
|
||||
# A row written without the column reads the same (an omitted
|
||||
# image_context inserts NULL — the text-doc / standalone value).
|
||||
new_id = _insert_sql(db, PATH_TEXT)
|
||||
try:
|
||||
backfilled = db.execute(
|
||||
text("SELECT image_context FROM documents WHERE id = :id"),
|
||||
{"id": new_id},
|
||||
).fetchone()
|
||||
assert backfilled == (None,), (
|
||||
"an omitted image_context must read NULL"
|
||||
)
|
||||
finally:
|
||||
_delete(db, new_id)
|
||||
|
||||
# The 0022 schema survives the additive upgrade.
|
||||
is_image = _column(db, "is_image")
|
||||
assert (
|
||||
is_image is not None
|
||||
and is_image[0] == "boolean"
|
||||
and is_image[1] == "NO"
|
||||
and is_image[2] == "false"
|
||||
), "documents.is_image (0022) must keep its 0022 contract"
|
||||
image_path = _column(db, "image_path")
|
||||
assert (
|
||||
image_path is not None
|
||||
and image_path[0] == "text"
|
||||
and image_path[1] == "YES"
|
||||
), "documents.image_path (0022) must keep its 0022 contract"
|
||||
content = _column(db, "content")
|
||||
assert content is not None and content[0] == "text" and content[1] == "NO", (
|
||||
"documents.content (0001) must keep its 0022 contract"
|
||||
)
|
||||
assert ("source", "path") in _unique_column_sets(db), (
|
||||
"the (source, path) unique constraint must survive the upgrade"
|
||||
)
|
||||
finally:
|
||||
_delete(db, pre_id)
|
||||
|
||||
|
||||
def test_orm_image_context_round_trip(db: Session, alembic: Config) -> None:
|
||||
"""The ORM contract agrees with the column contract: a freshly
|
||||
inserted image ``Document`` WITH ``image_context`` round-trips the
|
||||
string through a FRESH session, and one WITHOUT it (a standalone
|
||||
image, LOCKED A1/A4) reads ``image_context is None``."""
|
||||
command.upgrade(alembic, "head")
|
||||
enriched = Document(
|
||||
source=SOURCE,
|
||||
path=PATH_IMAGE,
|
||||
full_path=f"/tmp/{SOURCE}/{PATH_IMAGE}",
|
||||
title="diagram",
|
||||
content="A description of the diagram.",
|
||||
content_hash="1" * 64,
|
||||
is_image=True,
|
||||
image_path="/srv/bor-images/diagram.png",
|
||||
image_context=(
|
||||
"\n\nThe diagram\n"
|
||||
"shows the cluster layout and how the VMs connect."
|
||||
),
|
||||
)
|
||||
standalone = Document(
|
||||
source=SOURCE,
|
||||
path="standalone.png",
|
||||
full_path=f"/tmp/{SOURCE}/standalone.png",
|
||||
title="standalone",
|
||||
content="A description of a standalone image.",
|
||||
content_hash="2" * 64,
|
||||
is_image=True,
|
||||
image_path="/srv/bor-images/standalone.png",
|
||||
)
|
||||
db.add(enriched)
|
||||
db.add(standalone)
|
||||
db.commit()
|
||||
try:
|
||||
with SessionLocal() as fresh:
|
||||
reloaded_enriched = fresh.get(Document, enriched.id)
|
||||
assert reloaded_enriched is not None, "the image row must be readable"
|
||||
assert reloaded_enriched.image_context == enriched.image_context, (
|
||||
"image_context must round-trip the exact context text"
|
||||
)
|
||||
reloaded_standalone = fresh.get(Document, standalone.id)
|
||||
assert reloaded_standalone is not None
|
||||
assert reloaded_standalone.image_context is None, (
|
||||
"an omitted image_context must read NULL (standalone)"
|
||||
)
|
||||
finally:
|
||||
_delete(db, enriched.id)
|
||||
_delete(db, standalone.id)
|
||||
|
||||
|
||||
def test_downgrade_to_0022_drops_the_column(db: Session, alembic: Config) -> None:
|
||||
"""Downgrade 0023 → 0022: the ``image_context`` column is gone
|
||||
(A13 — fully reversible) while the row, its 0022 columns (including
|
||||
the phase-122 image pair) and the table's unique constraint
|
||||
survive."""
|
||||
command.upgrade(alembic, "head")
|
||||
row = Document(
|
||||
source=SOURCE,
|
||||
path=PATH_IMAGE,
|
||||
full_path=f"/tmp/{SOURCE}/{PATH_IMAGE}",
|
||||
title="diagram",
|
||||
content="A description of the diagram.",
|
||||
content_hash="3" * 64,
|
||||
is_image=True,
|
||||
image_path="/srv/bor-images/diagram.png",
|
||||
image_context="\n\nSurrounding paragraph.",
|
||||
)
|
||||
db.add(row)
|
||||
db.commit()
|
||||
try:
|
||||
command.downgrade(alembic, "0022")
|
||||
assert _version(db) == "0022"
|
||||
assert _column(db, "image_context") is None, (
|
||||
"image_context must be dropped"
|
||||
)
|
||||
|
||||
surviving = db.execute(
|
||||
text(
|
||||
"SELECT source, path, content, content_hash, is_image,"
|
||||
" image_path FROM documents WHERE id = :id"
|
||||
),
|
||||
{"id": row.id},
|
||||
).fetchone()
|
||||
assert surviving is not None, "the row must survive the column drop"
|
||||
assert surviving[0] == SOURCE and surviving[1] == PATH_IMAGE
|
||||
assert surviving[2] == "A description of the diagram."
|
||||
assert surviving[3] == "3" * 64
|
||||
assert surviving[4] is True, "is_image (0022) must survive the drop"
|
||||
assert surviving[5] == "/srv/bor-images/diagram.png"
|
||||
|
||||
assert ("source", "path") in _unique_column_sets(db), (
|
||||
"the (source, path) unique constraint must survive the downgrade"
|
||||
)
|
||||
finally:
|
||||
_delete(db, row.id)
|
||||
# Repair: the fixture teardown re-upgrades to head.
|
||||
|
||||
|
||||
def test_upgrade_round_trip_restores_the_column(db: Session, alembic: Config) -> None:
|
||||
"""Downgrade to 0022, then upgrade back to 0023: the column is back
|
||||
with the full contract (TEXT, NULLABLE, no server default) — a
|
||||
clean upgrade → downgrade → upgrade round-trip (A13)."""
|
||||
command.downgrade(alembic, "0022")
|
||||
command.upgrade(alembic, "head")
|
||||
assert _version(db) == "0023", (
|
||||
"round-trip upgrade must land at head (0023)"
|
||||
)
|
||||
|
||||
context = _column(db, "image_context")
|
||||
assert context is not None, "documents.image_context must be back"
|
||||
assert context[0] == "text", "image_context must be TEXT after the round-trip"
|
||||
assert context[1] == "YES", (
|
||||
"image_context must be NULLABLE after the round-trip"
|
||||
)
|
||||
assert context[2] is None, (
|
||||
"image_context must still carry NO server default"
|
||||
)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -283,11 +283,21 @@ def _patch_description(
|
||||
"""Pin the task-02 seam (``importer._describe_or_skip``) to
|
||||
*description* and record the BYTES it is handed (the digest-rule
|
||||
pin). Task 03 replaces the seam's body — the importer-side
|
||||
mechanics under test here do not change with it."""
|
||||
mechanics under test here do not change with it. Phase 125 (task
|
||||
03) widened the seam with the optional ``context`` kwarg (the
|
||||
markdown reference context) — the fake accepts (and ignores) it:
|
||||
these fixtures carry no image references, so the seam gets
|
||||
``context=None``."""
|
||||
handed: list[bytes] = []
|
||||
|
||||
async def _fake(
|
||||
llm: importer.Embedder, *, data: bytes, source: str, rel: str, full_path: Path
|
||||
llm: importer.Embedder,
|
||||
*,
|
||||
data: bytes,
|
||||
source: str,
|
||||
rel: str,
|
||||
full_path: Path,
|
||||
context: str | None = None,
|
||||
) -> str | None:
|
||||
handed.append(data)
|
||||
return description
|
||||
@@ -524,7 +534,13 @@ def test_changed_image_deletes_the_stale_copy_before_replacing(
|
||||
state = {"desc": "v1 description"}
|
||||
|
||||
async def _fake(
|
||||
llm: importer.Embedder, *, data: bytes, source: str, rel: str, full_path: Path
|
||||
llm: importer.Embedder,
|
||||
*,
|
||||
data: bytes,
|
||||
source: str,
|
||||
rel: str,
|
||||
full_path: Path,
|
||||
context: str | None = None,
|
||||
) -> str | None:
|
||||
handed.append(data)
|
||||
return state["desc"]
|
||||
|
||||
@@ -0,0 +1,275 @@
|
||||
"""Unit tests: markdown image-reference extraction (phase 125, task 02).
|
||||
|
||||
``extract_image_refs`` is a pure function (LOCKED A1 — the storage
|
||||
content of ``Document.image_context``), so the tests pin the exact
|
||||
contract the enrichment pipeline (tasks 03/04) will rely on:
|
||||
|
||||
* one :class:`ImageRef` per ```` reference outside code
|
||||
fences (multiple per line extracted separately, same window);
|
||||
* the context window — up to *N* paragraphs before and AFTER the
|
||||
reference paragraph, joined with blank lines, the reference
|
||||
paragraph itself excluded (the reference is already ``alt_text`` +
|
||||
``path``);
|
||||
* fence semantics identical to the chunker (PLAN §5) — fences are
|
||||
ATOMIC paragraphs (one context unit, blank lines inside don't split
|
||||
it) and references inside a fence are literal text — skipped;
|
||||
* edge cases: empty alt, start/end of file, headings, paths with
|
||||
spaces, nested parens (accepted miss), empty content, no refs,
|
||||
1-based ``line_number``.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from app.rag.md_images import CONTEXT_PARAGRAPHS, ImageRef, extract_image_refs
|
||||
|
||||
|
||||
def test_default_constant() -> None:
|
||||
assert CONTEXT_PARAGRAPHS == 2
|
||||
doc = "P1.\n\nP2.\n\nP3.\n\n\n\nQ1.\n\nQ2.\n\nQ3.\n"
|
||||
assert extract_image_refs(doc) == extract_image_refs(doc, context_paragraphs=2)
|
||||
|
||||
|
||||
def test_single_reference_with_surrounding_paragraphs() -> None:
|
||||
doc = (
|
||||
"# Title\n"
|
||||
"\n"
|
||||
"First paragraph.\n"
|
||||
"\n"
|
||||
"Second paragraph.\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"Third paragraph.\n"
|
||||
"\n"
|
||||
"Fourth paragraph.\n"
|
||||
)
|
||||
refs = extract_image_refs(doc)
|
||||
assert refs == [
|
||||
ImageRef(
|
||||
path="image.png",
|
||||
alt_text="alt",
|
||||
context=(
|
||||
"First paragraph.\n\n"
|
||||
"Second paragraph.\n\n"
|
||||
"Third paragraph.\n\n"
|
||||
"Fourth paragraph."
|
||||
),
|
||||
line_number=7,
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def test_multiple_references_on_different_lines() -> None:
|
||||
doc = (
|
||||
"Intro.\n"
|
||||
"\n"
|
||||
"Para A.\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"Para B.\n"
|
||||
"\n"
|
||||
"Para C.\n"
|
||||
"\n"
|
||||
"Para D.\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"\n"
|
||||
"Outro.\n"
|
||||
)
|
||||
refs = extract_image_refs(doc)
|
||||
assert [r.path for r in refs] == ["one.png", "two.png"]
|
||||
assert [r.alt_text for r in refs] == ["one", "two"]
|
||||
assert [r.line_number for r in refs] == [5, 13]
|
||||
# Each gets its OWN window (N=2 around its own paragraph — spaced
|
||||
# apart so the windows don't contain the other reference line).
|
||||
assert refs[0].context == "Intro.\n\nPara A.\n\nPara B.\n\nPara C."
|
||||
assert refs[1].context == "Para C.\n\nPara D.\n\nOutro."
|
||||
|
||||
|
||||
def test_multiple_references_on_same_line_share_window() -> None:
|
||||
doc = "Before.\n\nSee  and  here.\n\nAfter.\n"
|
||||
refs = extract_image_refs(doc)
|
||||
assert [r.path for r in refs] == ["a.png", "b.png"]
|
||||
assert [r.alt_text for r in refs] == ["a", "b"]
|
||||
assert [r.line_number for r in refs] == [3, 3]
|
||||
# Same reference paragraph → same context, reference line excluded.
|
||||
assert refs[0].context == "Before.\n\nAfter."
|
||||
assert refs[1].context == "Before.\n\nAfter."
|
||||
|
||||
|
||||
def test_reference_in_code_fence_is_skipped() -> None:
|
||||
doc = (
|
||||
"Intro.\n"
|
||||
"\n"
|
||||
"```\n"
|
||||
"\n"
|
||||
"```\n"
|
||||
"\n"
|
||||
"\n"
|
||||
)
|
||||
refs = extract_image_refs(doc)
|
||||
assert [r.path for r in refs] == ["real.png"]
|
||||
# The fence is an ATOMIC paragraph and counts as one context unit
|
||||
# inside the N=2 window (chunker semantics, PLAN §5).
|
||||
assert refs[0].context == "Intro.\n\n```\n\n```"
|
||||
|
||||
|
||||
def test_fence_with_blank_lines_inside_is_one_atomic_block() -> None:
|
||||
doc = (
|
||||
"P1.\n"
|
||||
"\n"
|
||||
"```\n"
|
||||
"line one\n"
|
||||
"\n"
|
||||
"line three\n"
|
||||
"```\n"
|
||||
"\n"
|
||||
"\n"
|
||||
)
|
||||
refs = extract_image_refs(doc)
|
||||
assert len(refs) == 1
|
||||
# The fence (with its inner blank line) is ONE atomic context unit,
|
||||
# occupying a single slot of the N=2 window before the reference
|
||||
# (if it split on the inner blank line, the window would hold only
|
||||
# "P1" + the fence's first lines instead of P1 + the whole fence).
|
||||
assert refs[0].context == "P1.\n\n```\nline one\n\nline three\n```"
|
||||
|
||||
|
||||
def test_tilde_fence_and_long_fence_markers() -> None:
|
||||
doc = "~~~\n\n~~~\n\n\n"
|
||||
assert [r.path for r in extract_image_refs(doc)] == ["keep.png"]
|
||||
# Four-backtick fence (marker = first three chars), inner blank line.
|
||||
doc = "````\n\n\n````\n\n\n"
|
||||
assert [r.path for r in extract_image_refs(doc)] == ["keep2.png"]
|
||||
|
||||
|
||||
def test_unclosed_fence_skips_rest_of_file() -> None:
|
||||
doc = "```\n\nmore text after an unclosed fence\n"
|
||||
assert extract_image_refs(doc) == []
|
||||
|
||||
|
||||
def test_reference_with_no_alt_text() -> None:
|
||||
refs = extract_image_refs("")
|
||||
assert refs == [
|
||||
ImageRef(path="image.png", alt_text="", context="", line_number=1)
|
||||
]
|
||||
|
||||
|
||||
def test_reference_at_start_of_file() -> None:
|
||||
doc = "\n\nP1.\n\nP2.\n\nP3.\n"
|
||||
refs = extract_image_refs(doc)
|
||||
assert len(refs) == 1
|
||||
# Only paragraphs after exist; the N=2 cap still applies (P3 left out).
|
||||
assert refs[0].context == "P1.\n\nP2."
|
||||
assert refs[0].line_number == 1
|
||||
|
||||
|
||||
def test_reference_at_end_of_file() -> None:
|
||||
doc = "P1.\n\nP2.\n\nP3.\n\nP4.\n\n\n"
|
||||
refs = extract_image_refs(doc)
|
||||
assert len(refs) == 1
|
||||
# Only paragraphs before exist; the N=2 window is the two paragraphs
|
||||
# immediately before the reference (P1 and P2 left out).
|
||||
assert refs[0].context == "P3.\n\nP4."
|
||||
assert refs[0].line_number == 9
|
||||
|
||||
|
||||
def test_reference_in_heading_line() -> None:
|
||||
doc = (
|
||||
"Intro paragraph.\n"
|
||||
"\n"
|
||||
"## \n"
|
||||
"\n"
|
||||
"Outro paragraph.\n"
|
||||
)
|
||||
refs = extract_image_refs(doc)
|
||||
assert len(refs) == 1
|
||||
assert refs[0].alt_text == "alt"
|
||||
assert refs[0].path == "image.png"
|
||||
assert refs[0].line_number == 3
|
||||
# The heading line is the reference paragraph — excluded from the
|
||||
# context; only the surrounding text remains.
|
||||
assert refs[0].context == "Intro paragraph.\n\nOutro paragraph."
|
||||
assert "##" not in refs[0].context
|
||||
|
||||
|
||||
def test_context_paragraphs_zero_and_custom() -> None:
|
||||
doc = "P1.\n\nP2.\n\nP3.\n\n\n\nQ1.\n\nQ2.\n\nQ3.\n"
|
||||
assert extract_image_refs(doc, context_paragraphs=0)[0].context == ""
|
||||
assert extract_image_refs(doc, context_paragraphs=1)[0].context == "P3.\n\nQ1."
|
||||
assert (
|
||||
extract_image_refs(doc, context_paragraphs=3)[0].context
|
||||
== "P1.\n\nP2.\n\nP3.\n\nQ1.\n\nQ2.\n\nQ3."
|
||||
)
|
||||
# A window LARGER than the file gets whatever is available.
|
||||
assert (
|
||||
extract_image_refs(doc, context_paragraphs=10)[0].context
|
||||
== "P1.\n\nP2.\n\nP3.\n\nQ1.\n\nQ2.\n\nQ3."
|
||||
)
|
||||
|
||||
|
||||
def test_negative_context_paragraphs_raises() -> None:
|
||||
with pytest.raises(ValueError, match="context_paragraphs must be >= 0"):
|
||||
extract_image_refs("", context_paragraphs=-1)
|
||||
|
||||
|
||||
def test_empty_and_reference_free_content() -> None:
|
||||
assert extract_image_refs("") == []
|
||||
assert extract_image_refs("\n\n \n") == []
|
||||
assert extract_image_refs("# No images here\n\nJust plain text.\n") == []
|
||||
# A link is not an image reference; an empty target is not a ref.
|
||||
assert extract_image_refs("[a link](http://x.example/) ![empty]()\n") == []
|
||||
|
||||
|
||||
def test_image_nested_in_link() -> None:
|
||||
refs = extract_image_refs(" wrapped: [](http://x/)\n")
|
||||
assert [r.path for r in refs] == ["img.png", "img2.png"]
|
||||
|
||||
|
||||
def test_alt_text_with_nested_brackets() -> None:
|
||||
# The alt field is ``[^]]*`` — it may OPEN a bracket (``[``) but
|
||||
# the first CLOSING bracket (``]``) ends it: an alt with a nested
|
||||
# ``[`` (``a [1``) still captures the path, while an alt that
|
||||
# itself CARRIES a ``]`` (``a [1] v2``) breaks the whole match
|
||||
# (the ``]`` ends the alt field, and no ``(`` follows) — an
|
||||
# accepted miss, the locked regex's limitation (task 02), the
|
||||
# alt-side twin of the nested-parens path miss above.
|
||||
refs = extract_image_refs("\n")
|
||||
assert refs == [ImageRef(path="img.png", alt_text="a [1", context="", line_number=1)]
|
||||
assert extract_image_refs("![a [1] v2](img.png)\n") == []
|
||||
|
||||
|
||||
def test_paths_with_spaces_and_parens() -> None:
|
||||
# Spaces are fine; surrounding whitespace in the target is stripped.
|
||||
refs = extract_image_refs(" and \n")
|
||||
assert [r.path for r in refs] == ["diagram v2.png", "x.png"]
|
||||
# Nested parens: the regex stops at the first ')' — an accepted
|
||||
# miss (rare in practice, phase 125 task 02).
|
||||
refs = extract_image_refs(".png)\n")
|
||||
assert refs[0].path == "a(b"
|
||||
|
||||
|
||||
def test_line_numbers_are_one_based() -> None:
|
||||
doc = "\n\n\n# Head\n\n\n\n"
|
||||
refs = extract_image_refs(doc)
|
||||
# Three leading blanks + "# Head" + two blanks = lines 1..6; the
|
||||
# reference is on line 7.
|
||||
assert refs[0].line_number == 7
|
||||
|
||||
|
||||
def test_multi_line_reference_paragraph_contributes_nothing() -> None:
|
||||
# The reference sits in the MIDDLE of a paragraph (no blank lines):
|
||||
# the whole reference paragraph is excluded from the context.
|
||||
doc = (
|
||||
"Before.\n"
|
||||
"\n"
|
||||
"Line one.\n"
|
||||
"Line two with \n"
|
||||
"Line three.\n"
|
||||
"\n"
|
||||
"After.\n"
|
||||
)
|
||||
refs = extract_image_refs(doc)
|
||||
assert refs[0].context == "Before.\n\nAfter."
|
||||
@@ -69,6 +69,54 @@ def test_chunks_reference_documents_cascade() -> None:
|
||||
assert fkc.ondelete == "CASCADE"
|
||||
|
||||
|
||||
def test_documents_image_context_column_contract() -> None:
|
||||
"""Phase 125 (task 01, LOCKED A1): ``documents.image_context`` —
|
||||
TEXT, NULLABLE, NO server default. NULL is the standalone-image /
|
||||
pre-phase-125 value (no backfill — A4 keeps those rows
|
||||
byte-identical)."""
|
||||
from sqlalchemy import Text as SAText
|
||||
|
||||
documents = Base.metadata.tables["documents"]
|
||||
col = documents.c["image_context"]
|
||||
assert col.nullable is True, "image_context must be NULLABLE (pre-phase rows)"
|
||||
assert isinstance(col.type, SAText), "image_context must be TEXT"
|
||||
assert col.server_default is None, (
|
||||
"image_context must carry NO server default — NULL is the"
|
||||
" standalone/pre-phase value"
|
||||
)
|
||||
assert col.default is None, (
|
||||
"image_context must carry NO ORM default clause either (an"
|
||||
" omitted column inserts NULL, the text-doc value)"
|
||||
)
|
||||
|
||||
|
||||
def test_document_image_context_orm_parses() -> None:
|
||||
"""A freshly constructed ``Document`` (no DB) accepts the field in
|
||||
both states: a set context string round-trips as-is, and
|
||||
``None`` (and the omitted kwarg) read NULL — the standalone-image
|
||||
/ pre-phase state."""
|
||||
from app.models import Document
|
||||
|
||||
base = dict(
|
||||
source="s",
|
||||
path="p.md",
|
||||
full_path="/tmp/s/p.md",
|
||||
title="p",
|
||||
content="c",
|
||||
content_hash="1" * 64,
|
||||
)
|
||||
enriched = Document(**base, image_context="some context")
|
||||
assert enriched.image_context == "some context"
|
||||
|
||||
explicit_null = Document(**base, image_context=None)
|
||||
assert explicit_null.image_context is None
|
||||
|
||||
omitted = Document(**base)
|
||||
assert omitted.image_context is None, (
|
||||
"an omitted image_context must read NULL (the text-doc value)"
|
||||
)
|
||||
|
||||
|
||||
def test_documents_unique_source_path() -> None:
|
||||
documents = Base.metadata.tables["documents"]
|
||||
uq = [
|
||||
|
||||
@@ -48,6 +48,8 @@ from app.rag.prompts import (
|
||||
SUGGEST_INTRO,
|
||||
TOOLS_SECTION,
|
||||
_base,
|
||||
_document_body,
|
||||
_extract_context_line,
|
||||
build_deflect_prompt,
|
||||
build_high_prompt,
|
||||
build_kb_section,
|
||||
@@ -74,6 +76,9 @@ def _doc(
|
||||
content: str,
|
||||
title: str,
|
||||
summary: str | None = None,
|
||||
*,
|
||||
is_image: bool = False,
|
||||
image_context: str | None = None,
|
||||
) -> Document:
|
||||
return Document(
|
||||
id=uuid.uuid4(),
|
||||
@@ -85,6 +90,9 @@ def _doc(
|
||||
content_hash="0" * 64,
|
||||
created_at=_FIXTURE_CREATED_AT,
|
||||
summary=summary,
|
||||
is_image=is_image,
|
||||
image_path="/tmp/pic.png" if is_image else None,
|
||||
image_context=image_context,
|
||||
)
|
||||
|
||||
|
||||
@@ -1141,3 +1149,301 @@ def test_show_image_clause_is_last_for_every_high_shape() -> None:
|
||||
assert on.index(section) < on.index("<tools>")
|
||||
assert on.index("K8s deploy notes.") < on.index("<tools>") # the summary still seeds
|
||||
assert SHOW_IMAGE_CLAUSE not in off # the toggle-off twin carries none of it
|
||||
|
||||
|
||||
# ---------- phase 125 (task 05, LOCKED A4): the image context line ----------
|
||||
# An enriched image doc (tasks 03/04: non-NULL ``image_context`` in the
|
||||
# "Referenced in …" block shape) leads its ``<document>`` suggestion
|
||||
# block with ONE short context line — the referring markdown's identity
|
||||
# + the reference's alt text — before the description, so the LLM knows
|
||||
# WHY the image exists, not only what it shows. Every other doc (every
|
||||
# text doc, every non-enriched image doc) is byte-identical to the
|
||||
# pre-phase body.
|
||||
|
||||
#: The exact stored block shape (tasks 03/04): one "Referenced in"
|
||||
#: block per referring markdown file, blank-line joined; the
|
||||
#: ``Context:`` body may span many lines (the surrounding paragraphs).
|
||||
CTX_SINGLE = (
|
||||
"Referenced in Homelab/homelab/kubernetes.md:\n"
|
||||
'- Alt text: "Cluster layout"\n'
|
||||
" Context: The diagram shows the three Talos nodes and Cilium.\n"
|
||||
"It is referenced again from the runbook."
|
||||
)
|
||||
CTX_TWO_BLOCKS = CTX_SINGLE + "\n\nReferenced in Homelab/ops/runbook.md:\n- Alt text: \"\""
|
||||
CTX_ALT_EMPTY = "Referenced in Homelab/ops/runbook.md:\n- Alt text: \"\""
|
||||
|
||||
#: The task-04 pending record — stored on MARKDOWN (text) docs, with no
|
||||
#: ``Referenced in`` header at all.
|
||||
MD_PENDING_RECORD = (
|
||||
'IMAGE_CONTEXT:\n- [image.png] (line 42): "Alt text here" — 2 paragraphs of context'
|
||||
)
|
||||
|
||||
|
||||
def _image_doc(
|
||||
image_context: str | None = None, summary: str | None = "A diagram of the cluster."
|
||||
) -> Document:
|
||||
return _doc(
|
||||
"cluster-diagram.png",
|
||||
"A diagram of the cluster.",
|
||||
"cluster-diagram",
|
||||
summary=summary,
|
||||
is_image=True,
|
||||
image_context=image_context,
|
||||
)
|
||||
|
||||
|
||||
# ---------- _extract_context_line: the parser contract ----------
|
||||
|
||||
|
||||
def test_extract_context_line_single_block_with_alt() -> None:
|
||||
"""One block with an alt → the short one-line locator: the stored
|
||||
identity (``<source>/<path>`` exactly as the header carries it) +
|
||||
the reference's alt between quotes."""
|
||||
assert _extract_context_line(CTX_SINGLE) == (
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
|
||||
)
|
||||
|
||||
|
||||
def test_extract_context_line_context_body_never_renders() -> None:
|
||||
"""The ``Context:`` paragraphs (LOCKED A3: they already reached the
|
||||
vision model at enrichment time) stay out of the line — the line is
|
||||
the locator + purpose, short enough not to bloat the prompt."""
|
||||
line = _extract_context_line(CTX_SINGLE)
|
||||
assert line is not None
|
||||
assert "The diagram shows" not in line
|
||||
assert "referenced again" not in line
|
||||
assert "Context:" not in line
|
||||
assert line.count("\n") == 0 # genuinely ONE line
|
||||
|
||||
|
||||
def test_extract_context_line_empty_alt_renders_the_bare_identity() -> None:
|
||||
"""A reference without an alt (````) → the clause is the
|
||||
bare identity — nothing to quote after the colon."""
|
||||
assert _extract_context_line(CTX_ALT_EMPTY) == "Referenced in Homelab/ops/runbook.md"
|
||||
|
||||
|
||||
def test_extract_context_line_two_blocks_join_with_semicolon() -> None:
|
||||
"""One clause per referring file after a single prefix — the second
|
||||
block's empty alt renders bare."""
|
||||
assert _extract_context_line(CTX_TWO_BLOCKS) == (
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"; '
|
||||
"Homelab/ops/runbook.md"
|
||||
)
|
||||
|
||||
|
||||
def test_extract_context_line_uses_the_first_reference_only() -> None:
|
||||
"""A block with two references (two ``- Alt text:`` bullets) → the
|
||||
FIRST reference's alt is the purpose; the second stays out."""
|
||||
text = (
|
||||
"Referenced in Homelab/a.md:\n"
|
||||
'- Alt text: "first purpose"\n'
|
||||
" Context: some paragraph.\n"
|
||||
'- Alt text: "second purpose"'
|
||||
)
|
||||
assert _extract_context_line(text) == 'Referenced in Homelab/a.md: "first purpose"'
|
||||
assert "second purpose" not in (_extract_context_line(text) or "")
|
||||
|
||||
|
||||
def test_extract_context_line_multiline_context_body_is_ignored() -> None:
|
||||
"""The ``Context:`` value is paragraph-joined (blank lines inside)
|
||||
— the parser walks it without breaking the block."""
|
||||
text = (
|
||||
"Referenced in Homelab/a.md:\n"
|
||||
'- Alt text: "purpose"\n'
|
||||
" Context: para one.\n"
|
||||
"\n"
|
||||
"para two, and it contains a line that\n"
|
||||
"looks like context continuation."
|
||||
)
|
||||
assert _extract_context_line(text) == 'Referenced in Homelab/a.md: "purpose"'
|
||||
|
||||
|
||||
def test_extract_context_line_alt_with_inner_quotes_survives() -> None:
|
||||
"""The block writer does not escape quotes: the greedy alt capture
|
||||
(between the opening quote and the line's final quote) keeps an
|
||||
alt with inner ``"`` intact."""
|
||||
text = 'Referenced in Homelab/x.md:\n- Alt text: "He said "hi""'
|
||||
assert _extract_context_line(text) == 'Referenced in Homelab/x.md: "He said "hi""'
|
||||
|
||||
|
||||
def test_extract_context_line_spurious_header_is_dropped() -> None:
|
||||
"""A look-alike ``Referenced in …:`` line WITHOUT its ``- Alt text:``
|
||||
bullet (context text that merely looks like a header) parses to
|
||||
``None`` — never a clause from the stray line."""
|
||||
assert _extract_context_line(
|
||||
"Referenced in Homelab/fake.md:\nsome context line that is not a bullet"
|
||||
) is None
|
||||
# A header with NO following line at all (end of value) is spurious too.
|
||||
assert _extract_context_line("Referenced in Homelab/fake.md:") is None
|
||||
# Spurious then REAL: only the real block's clause survives.
|
||||
mixed = (
|
||||
"Referenced in Homelab/fake.md:\nsome context line\n\n"
|
||||
+ CTX_SINGLE
|
||||
)
|
||||
assert _extract_context_line(mixed) == (
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
|
||||
)
|
||||
|
||||
|
||||
def test_extract_context_line_header_after_header_drops_the_bulletless_one() -> None:
|
||||
"""Two headers in a row: the first (never followed by its bullet) is
|
||||
dropped and the second block parses normally."""
|
||||
text = "Referenced in Homelab/fake.md:\nReferenced in Homelab/real.md:\n- Alt text: \"alt\""
|
||||
assert _extract_context_line(text) == 'Referenced in Homelab/real.md: "alt"'
|
||||
|
||||
|
||||
def test_extract_context_line_markdown_pending_record_is_none() -> None:
|
||||
"""The task-04 ``IMAGE_CONTEXT:`` record (the MARKDOWN doc's stored
|
||||
``image_context``) has no ``Referenced in`` header — it parses to
|
||||
``None`` (and it rides a text doc, which the ``is_image`` guard
|
||||
excludes from the line anyway)."""
|
||||
assert _extract_context_line(MD_PENDING_RECORD) is None
|
||||
|
||||
|
||||
def test_extract_context_line_blank_and_empty_are_none() -> None:
|
||||
assert _extract_context_line("") is None
|
||||
assert _extract_context_line(" \n \t ") is None
|
||||
# Garbage that contains the keyword somewhere is not a block.
|
||||
assert _extract_context_line("I was referenced in Homelab/a.md: yesterday") is None
|
||||
|
||||
|
||||
# ---------- _document_body: the block-body contract ----------
|
||||
|
||||
|
||||
def test_document_body_enriched_image_doc_leads_with_the_context_line() -> None:
|
||||
"""The completion criterion at the body level: the context line
|
||||
leads, a blank line, then the description — the exact two-part
|
||||
shape."""
|
||||
body = _document_body(_image_doc(image_context=CTX_SINGLE))
|
||||
assert body == (
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n\n'
|
||||
"A diagram of the cluster."
|
||||
)
|
||||
|
||||
|
||||
def test_document_body_image_doc_null_context_is_byte_identical_to_pre_phase() -> None:
|
||||
"""LOCKED A4 (the byte-identity criterion): a standalone / pre-phase
|
||||
image doc (NULL ``image_context``) → the body is EXACTLY the
|
||||
stripped summary — no line, no blank, no drift."""
|
||||
doc = _image_doc(image_context=None)
|
||||
assert _document_body(doc) == "A diagram of the cluster."
|
||||
# And the built prompt matches a text twin (same fields, not an
|
||||
# image) to the byte.
|
||||
twin = _doc("cluster-diagram.png", "A diagram of the cluster.", "cluster-diagram")
|
||||
assert build_high_prompt([doc]) == build_high_prompt([twin])
|
||||
|
||||
|
||||
def test_document_body_text_doc_with_image_context_is_unchanged() -> None:
|
||||
"""The guard is ``is_image``: a TEXT doc whose ``image_context``
|
||||
carries the task-04 pending record (the markdown case) keeps the
|
||||
pre-phase body — the record never renders."""
|
||||
doc = _doc("guide.md", "CONTENT", "Guide", summary="Guide summary.",
|
||||
image_context=MD_PENDING_RECORD)
|
||||
assert _document_body(doc) == "Guide summary."
|
||||
assert "IMAGE_CONTEXT" not in build_high_prompt([doc])
|
||||
assert "Referenced in" not in build_high_prompt([doc])
|
||||
|
||||
|
||||
def test_document_body_image_doc_unparsable_context_is_unchanged() -> None:
|
||||
"""Graceful degradation: an image doc whose ``image_context`` parses
|
||||
to nothing (owner-edited / unknown shape) → the description only,
|
||||
exactly pre-phase — the prompt build never raises."""
|
||||
for junk in ("", " ", "referenced in lowercase.md: no header shape", MD_PENDING_RECORD):
|
||||
assert _document_body(_image_doc(image_context=junk)) == "A diagram of the cluster."
|
||||
|
||||
|
||||
def test_document_body_enriched_image_doc_without_summary_precedes_the_preview() -> None:
|
||||
"""The line leads the fallback body too (the preview + marker of
|
||||
LOCKED A5) — the "context before the description" contract holds on
|
||||
both body shapes."""
|
||||
doc = _image_doc(image_context=CTX_SINGLE, summary=None)
|
||||
doc.content = "A" * 400 + "B" * 600 # past the 400-char preview cap
|
||||
body = _document_body(doc)
|
||||
line = 'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
|
||||
assert body.startswith(line + "\n\n")
|
||||
assert body[len(line) + 2 :] == "A" * 400 + "\n" + TRUNCATION_MARKER
|
||||
assert "B" * 40 not in body # beyond the cut, as pre-phase
|
||||
|
||||
|
||||
def test_enriched_image_doc_with_summary_makes_no_settings_call(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""House pattern (the LOCKED A5 preview-cap contract): the
|
||||
context-line path for a summary-bearing doc is pure — a dead
|
||||
``get_settings`` must not break such a build (extraction is pure;
|
||||
the cap is read on the fallback path only)."""
|
||||
from app.rag import prompts as prompts_mod
|
||||
|
||||
doc = _image_doc(image_context=CTX_SINGLE)
|
||||
|
||||
def _dead() -> None: # pragma: no cover - must never be called
|
||||
raise AssertionError("get_settings() called for a summarized doc")
|
||||
|
||||
monkeypatch.setattr(prompts_mod, "get_settings", _dead)
|
||||
assert _document_body(doc) == (
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n\n'
|
||||
"A diagram of the cluster."
|
||||
)
|
||||
|
||||
|
||||
# ---------- build_high_prompt: the line in the built block ----------
|
||||
|
||||
|
||||
def test_high_prompt_enriched_image_doc_block_exact_shape() -> None:
|
||||
"""The full block in the built prompt: identity attributes
|
||||
unchanged (phase 106 date appended after title), then the context
|
||||
line, a blank line, the description — the pre-phase block shape
|
||||
with the line spliced in."""
|
||||
doc = _image_doc(image_context=CTX_SINGLE)
|
||||
prompt = build_high_prompt([doc])
|
||||
assert (
|
||||
'<document source="Homelab" path="cluster-diagram.png" '
|
||||
'title="cluster-diagram" date="2024-06-15">\n'
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n'
|
||||
"\n"
|
||||
"A diagram of the cluster.\n"
|
||||
"</document>"
|
||||
) in prompt
|
||||
# The line is the FIRST thing in the block body (before the
|
||||
# description), inside <documents>.
|
||||
line = 'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"'
|
||||
block_start = prompt.index('<document source="Homelab"')
|
||||
assert prompt.index("<documents>") < prompt.index(line)
|
||||
assert prompt.index(line) < prompt.index("A diagram of the cluster.")
|
||||
assert block_start < prompt.index(line) < prompt.index("</document>")
|
||||
# The Context: body stays out of the whole prompt.
|
||||
assert "The diagram shows the three Talos nodes" not in prompt
|
||||
|
||||
|
||||
def test_high_prompt_enriched_image_doc_mixed_with_text_docs() -> None:
|
||||
"""A suggested tier with both text docs and one enriched image doc:
|
||||
the line rides ONLY the image doc's block; the text blocks are
|
||||
byte-identical to their summaries (no line, no drift)."""
|
||||
text_doc = _doc("kubernetes.md", "CONTENT", "K8s", summary="K8s summary.")
|
||||
img = _image_doc(image_context=CTX_SINGLE)
|
||||
prompt = build_high_prompt([text_doc, img])
|
||||
assert prompt.count('Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"') == 1
|
||||
# The text block is the bare summary, exactly as pre-phase.
|
||||
assert (
|
||||
'<document source="Homelab" path="kubernetes.md" '
|
||||
'title="K8s" date="2024-06-15">\nK8s summary.\n</document>'
|
||||
) in prompt
|
||||
# The image block carries the line + description.
|
||||
assert (
|
||||
'title="cluster-diagram" date="2024-06-15">\n'
|
||||
'Referenced in Homelab/homelab/kubernetes.md: "Cluster layout"\n\n'
|
||||
"A diagram of the cluster.\n</document>"
|
||||
) in prompt
|
||||
# Ordering unchanged: the text block (first doc) still precedes the
|
||||
# image block (second doc) — the line never re-orders the tier.
|
||||
assert prompt.index('path="kubernetes.md"') < prompt.index('path="cluster-diagram.png"')
|
||||
|
||||
|
||||
def test_high_prompt_image_doc_without_context_is_byte_identical_to_pre_phase() -> None:
|
||||
"""LOCKED A4 at the prompt level: an image doc with NULL
|
||||
``image_context`` builds the EXACT pre-phase prompt (the text-twin
|
||||
equality — no context line anywhere, not even a stray blank)."""
|
||||
img = _image_doc(image_context=None)
|
||||
twin = _doc("cluster-diagram.png", "A diagram of the cluster.", "cluster-diagram")
|
||||
assert build_high_prompt([img]) == build_high_prompt([twin])
|
||||
assert "Referenced in" not in build_high_prompt([img])
|
||||
|
||||
Reference in New Issue
Block a user