From f50d4c8bc37f6ab13c1459069ce5a7ccfe015601 Mon Sep 17 00:00:00 2001 From: ducoterra Date: Sun, 27 Sep 2026 16:05:37 -0400 Subject: [PATCH] chore(agent): phase roadmap from TODO.md, 1 phase (137) Convert the PDF-embedded-image document-context item (TODO.md L3) into phase 137_pdf_image_context (Protocol B append, 5 tasks): per-page text extraction, the forward context path into the vision prompt + image_context, the idempotent per-sync backfill for existing image docs, the From-loc display line, and the dedicated E2E suite. TODO.md cleared (items now live in .agents/phases/). LLM-Generated: true --- .../todo/137_pdf_image_context/00_phase.md | 48 +++++++++++++ .../01_pdf_page_texts.md | 32 +++++++++ .../02_embedded_image_document_context.md | 72 +++++++++++++++++++ .../03_backfill_image_context.md | 45 ++++++++++++ .../04_retrieval_display_line.md | 36 ++++++++++ .../05_e2e_pdf_image_context.md | 58 +++++++++++++++ 6 files changed, 291 insertions(+) create mode 100644 .agents/phases/todo/137_pdf_image_context/00_phase.md create mode 100644 .agents/phases/todo/137_pdf_image_context/01_pdf_page_texts.md create mode 100644 .agents/phases/todo/137_pdf_image_context/02_embedded_image_document_context.md create mode 100644 .agents/phases/todo/137_pdf_image_context/03_backfill_image_context.md create mode 100644 .agents/phases/todo/137_pdf_image_context/04_retrieval_display_line.md create mode 100644 .agents/phases/todo/137_pdf_image_context/05_e2e_pdf_image_context.md diff --git a/.agents/phases/todo/137_pdf_image_context/00_phase.md b/.agents/phases/todo/137_pdf_image_context/00_phase.md new file mode 100644 index 0000000..cda19d5 --- /dev/null +++ b/.agents/phases/todo/137_pdf_image_context/00_phase.md @@ -0,0 +1,48 @@ +# Phase 137 — PDF-embedded images always carry the document context in their summary + +**Source:** `TODO.md` L3 — "Images extracted from PDFs still don't have any relevant context in their summaries provided by the PDF. They're still just getting summarized 'as-is'. … Image summaries should always include the document context if the images come from a document (this includes markdown files) so the embedder can more effectively retrieve image sources." +**Story:** n/a (TODO-derived phase, Protocol B append). +**Context:** `app/rag/pdf.py` (phase 126 — `extract_pdf_text` (L87, per-page `get_text("text")` with `---PAGE-N---` separators, fail-soft `""`), `extract_pdf_images` (L145, one `PdfImage` per occurrence: `page` 1-indexed, `index` 0-based, `data`, `mime`)); `app/rag/importer.py` — `_index_pdf_embedded_images` (L1091, the per-PDF embedded-image pass after its document commit), `_index_one_embedded_image` (L1182, the phase-122 pipeline inline: mime gate → synthetic path `#page=N#img=I.` → byte-digest skip → `_markdown_image_context` (L1403, the phase-125 reference scan) → `_describe_or_skip` (L1742, the single describe seam) → copy write → chunk + embed → `_store_summary`), `_markdown_image_context` (L1403) + `_md_ref_block` (L1436, the `Referenced in /:` block writer), `_enrich_pending_images` (L1627, the phase-125 task-04 post-index pass — the shape of the new backfill pass), `ImportSummary` (L212, stats counters + `log()` — stats are log-only, the sync `detail` API shape never changes); `app/rag/summarizer.py` — `describe_image` (L208, ONE chat-model vision call, `context` appended after the locked `DESCRIBE_PROMPT` as a `\n\nThis image is referenced in documentation with the following context:\n` prefix — the wording phase 137 generalizes), `DESCRIBE_PROMPT` / `IMAGE_DESCRIPTION_MODE` (L78/L57 — the mock's key; byte-identity criteria below); `app/rag/prompts.py` — `_extract_context_line` (L470, renders only `Referenced in …:` blocks, everything else degrades to `None`), `_image_context_line` (L536), `_document_body` (L548, the phase-125 context-line lead); `app/models.py` — `Document.image_context` (L170, TEXT nullable, migration 0023); `tests/e2e/mock_llm.py` — the `IMAGE_DESCRIPTION_MODE` canned vision reply (L2472 branch, `IMAGE_DESCRIPTION_ANSWER` L865) and the phase-125 `CONTEXT_LINE_TRIGGER` pattern (the model for this phase's mock trigger); `tests/e2e/test_pdf_support.py` (the synthetic PDF fixture recipe: PyMuPDF build, `page2.insert_image(…, stream=PNG_1X1)`, images-on/off module apps); `test_sources/players_handbook.pdf` (293 pages, 586 embedded images — the real multi-page fixture), `test_sources/hnb_2026_guide.tar.gz` (the owner's example — stock-photo descriptions with no document context). + +## Objective +Every image that comes from a document carries the document context in the text that gets embedded: the vision prompt for a PDF-embedded image includes WHERE it came from (source + PDF path + page) and the surrounding page text, so the model writes a description that names the document and what the image illustrates; the context is stored on `image_context` (idempotent, re-scanned on re-index) and rendered as a short locator line in the chat suggestion block; and every image document already in the KB is backfilled on the next sync. Markdown-referenced images (phase 125) keep working byte-for-byte; PDFs are the only importable format that embeds images, so this closes the TODO's gap completely. + +## Dependencies +- `136_force_push_reset` (todo) — pipeline order only. +- Code: the completed phase-126 PDF pipeline + the phase-122 vision seam + the phase-125 context machinery (all on `main`); and pending `133_pdf_screenshot_fallback` — its `is_usable_pdf_text` gate (task 02 of that phase) runs BEFORE this phase in the pipeline and is reused to suppress garbage page text in the context (ASSUMPTION A7). + +## Design (shared by all tasks — the executor reads this, not the chat) +- **The context block (task 02, ASSUMPTIONS A2/A3):** one block per embedded image, built from the PDF's own text — + ``` + Extracted from / (page ): + + ``` + The origin line is ALWAYS present (even when the page has no text — the owner's core complaint is "no mention of the document it came from"); the page-text line is omitted for a text-less page. If phase 133's gate exists (`app.rag.pdf.is_usable_pdf_text`) and says the PDF's text layer is unusable, the page text is dropped (origin line only — ASSUMPTION A7). When the image ALSO has phase-125 markdown reference blocks, the stored/passed context is the document block FIRST, then the markdown blocks (provenance → purpose — ASSUMPTION A2); the combining helper is one shared function used by the forward path AND the backfill (task 03) so the two can never drift. +- **The seam (task 02, ASSUMPTION A5):** `describe_image`'s `context` prefix wording generalizes from "This image is referenced in documentation with the following context:" to neutral text that covers both provenance and reference (e.g. "This image is part of a document. The following context describes where it comes from and what it is used for:"). `DESCRIBE_PROMPT` (marker + locked instruction) stays byte-identical; `context=None` (standalone image) sends the bare prompt — byte-identical to today, so the phase-122 standalone path and every pre-existing E2E stay green. +- **The forward path (task 02):** `_index_pdf_embedded_images` calls the new `extract_pdf_page_texts` ONCE per PDF (a `dict[int, str]` page → stripped text — task 01) and passes it to `_index_one_embedded_image`, which builds the combined context, passes it as `context` to `_describe_or_skip`, and stores the SAME value on `image_context` (the phase-125 task-03 rule: context follows the current source state on re-index — a re-indexed image re-scans; the digest-skip for unchanged images is UNCHANGED: an unchanged image keeps its (now context-carrying) content). +- **The backfill (task 03, ASSUMPTION A4):** a post-index pass in `import_sources` next to `_enrich_pending_images` — the phase-118 A2 "best-effort until it sticks" shape. Candidates: `is_image=True` docs whose `path` matches the embedded-image pattern (`*.pdf#page=#img=.`) AND whose `image_context` does NOT start with `Extracted from ` (NULL or markdown-only) AND whose `image_path` copy still exists on disk. Per candidate: bytes from the copy, page text re-extracted from the parent PDF if it is still in the source (parent gone → origin line only, parsed from the doc's own synthetic path — no disk dependency), re-describe through the SAME seam, re-chunk + re-embed (`_replace_content_chunks`, one transaction per image), store the combined context. Idempotent by the stored-context gate: a success writes a context starting with `Extracted from ` → the doc never qualifies again. Fail-soft per image (a `None` description leaves the row untouched, counts `images_failed`, sync continues). New `ImportSummary.image_context_backfilled: int = 0` counter + `log()` field ONLY (the stats log-only convention — the sync `detail` API shape is untouched). One loud log line per backfilled image (PLAN §9). +- **The display line (task 04, ASSUMPTION A6):** `_extract_context_line` learns the `Extracted from / (page ):` header (regex; the page-text body is NEVER rendered — same rule as the `Context:` body) and renders it as `From / (page )`. A value carrying BOTH block kinds renders the clauses in stored order (`From …; Referenced in …`). A value with no parsable block degrades to `None` exactly as today (pre-phase rows, markdown `IMAGE_CONTEXT:` records on text docs — the `is_image` guard already excludes those). +- **The E2E (task 05):** the mock gains ONE new trigger, following the phase-125 `CONTEXT_LINE_TRIGGER` pattern exactly: when the `IMAGE_DESCRIPTION_MODE` message's text part contains `Extracted from ` (the new provenance header), the canned reply becomes a context-aware description (fixed string, e.g. naming the document + page) instead of `IMAGE_DESCRIPTION_ANSWER` — the deterministic observable that the context REACHED the vision model. The suite uploads a synthetic PDF (text on page 1, one embedded PNG on page 2 — the `test_pdf_support.py` recipe) through the images-on module app, syncs, and asserts: the image doc's content/summary is the context-aware canned reply; its `image_context` starts with `Extracted from /report.pdf (page 2)`; a chat question that retrieves the image shows the `From /report.pdf (page 2)` line in the suggestion. A markdown-referencing case (a `.md` in the same archive referencing the synthetic path) asserts BOTH blocks, document first. + +## Tasks +1. `01_pdf_page_texts.md` — `extract_pdf_page_texts` in `app/rag/pdf.py` (the shared per-page iteration, fail-soft) + unit tests on the real fixture PDFs. +2. `02_embedded_image_document_context.md` — the document-context block builder + the forward path (build once per PDF, combine with markdown blocks, pass through the seam, store on `image_context`) + the generalized seam prefix + unit tests with a capturing fake LLM. +3. `03_backfill_image_context.md` — the per-sync idempotent backfill pass + the `image_context_backfilled` counter + integration tests (existing row backfilled, idempotent no-op, parent-gone origin-only, standalone never touched). +4. `04_retrieval_display_line.md` — `_extract_context_line` renders the `Extracted from` block as the one-line `From … (page N)` locator + unit tests (both-blocks ordering, degradation, pre-phase byte-identity). +5. `05_e2e_pdf_image_context.md` — the mock's `Extracted from` trigger + the dedicated Playwright suite `tests/e2e/test_pdf_image_context.py`, run in isolation. + +## Testing & Quality +- Unit: task 01 — `extract_pdf_page_texts` on `test_sources/players_handbook.pdf` (293 pages, spot-check known page text, page count, 1-indexed keys) and `pdf_horse_test.pdf`; an unreadable path → `{}` (fail-soft); `extract_pdf_text` output byte-identical after the shared-iteration refactor (the phase-126 regression pin). Task 02 — the block builder (page text capped at 6000, text-less page → origin line only, garbage gate drops the page text when 133's gate exists); the combining order (document first, markdown second; markdown-only and document-only cases); the seam message (context present → neutral prefix + combined text AFTER the byte-identical `DESCRIBE_PROMPT`; context=None → byte-identical pre-137 message); the forward path via a capturing fake LLM (the stored `image_context` == the context passed; the title/unchanged-skip rules untouched). Task 04 — the renderer (`Extracted from` alone, both blocks in stored order, unparseable → `None`, the `is_image` guard). Task 05 — the mock trigger (context-aware reply only when `Extracted from ` is in the text part; the bare `IMAGE_DESCRIPTION_ANSWER` otherwise — the phase-122/126 suites stay green without edits). +- Integration: task 03 — a seeded PDF-embedded image row without context is backfilled through the real `import_sources` (the fake LLM records the context-bearing describe call; content/chunks/`image_context` updated); a second sync is a NO-OP (zero describe calls — the idempotency gate); a row whose parent PDF was deleted from the source gets the origin-line-only context; a standalone image (no `#page=`) is NEVER touched; a markdown-enriched row (context `Referenced in …`) gets the combined context (document block prepended). +- E2E: one dedicated Playwright file, run in isolation (AGENTS.md §4/§9), mock-only. +- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`). + +## Completion Criteria +- [ ] a (re)indexed PDF's embedded images are described with the document context: the vision message carries `Extracted from / (page N):` + the capped page text (the unit/integration proofs above), and the stored `image_context`/summary carry it too +- [ ] a markdown-referenced PDF image carries BOTH blocks, document first — and a standalone image's describe message is byte-identical to pre-phase (the phase-122 regression pin) +- [ ] an existing context-free image doc (e.g. one of the HNB-guide stock photos) is re-described WITH context on the next sync, and the following sync burns zero vision calls for it (the idempotency test) +- [ ] the chat suggestion block for a backfilled/forward image leads with `From / (page N)` +- [ ] `uv run pytest --cov=app --cov-report=term-missing` green, `app/` coverage >90%; `uv run pytest tests/e2e/test_pdf_image_context.py -v --no-cov` green in isolation; the pre-existing `test_pdf_support.py`, `test_image_anchor_context.py`, `test_image_documents.py` suites stay green UNEDITED +- [ ] no behavior change in completed phases (the phase-122 standalone path, the phase-125 markdown enrichment, the phase-126 PDF text path, the sync API shape) +- [ ] `uv run ruff check . && uv run pyright` green +- [ ] one atomic Conventional Commits commit for the phase (`feat(rag): …`), `--no-gpg-sign` diff --git a/.agents/phases/todo/137_pdf_image_context/01_pdf_page_texts.md b/.agents/phases/todo/137_pdf_image_context/01_pdf_page_texts.md new file mode 100644 index 0000000..3fa24eb --- /dev/null +++ b/.agents/phases/todo/137_pdf_image_context/01_pdf_page_texts.md @@ -0,0 +1,32 @@ +# Task 01 — `extract_pdf_page_texts`: per-page text map for image context + +**Source:** `TODO.md` L3 — "Image summaries should always include the document context if the images come from a document … so the embedder can more effectively retrieve image sources." (This task builds the per-page text source the context block is made from; the item itself is implemented across tasks 01–05.) + +## Objective +`app/rag/pdf.py` gains `extract_pdf_page_texts(pdf_path: Path) -> dict[int, str]` — one entry per page that has text, key = 1-indexed page number, value = the page's stripped text (the same `page.get_text("text")` read `extract_pdf_text` uses). Pages without text contribute nothing (no key). This is the raw material task 02 feeds the vision prompt: for one embedded image on page N, the document context is page N's text. The existing `extract_pdf_text` output stays **byte-identical** (the phase-126 contract — the `---PAGE-N---` blocks, blank-line join, fail-soft `""`). + +## Work +1. Read `app/rag/pdf.py` in full (phase 126). Note `extract_pdf_text` (L87): open → per-page `get_text("text")` → strip → skip empties → `PAGE_SEPARATOR` prefix → `"\n\n"` join; fail-soft per page and per file. +2. Refactor the shared page iteration into one private helper, e.g. `_iter_page_texts(doc) -> Iterator[tuple[int, str]]` yielding `(page_1_indexed, stripped_text)` for pages with non-empty text (the per-page try/except + warning logging moves into the helper, unchanged in behavior). `extract_pdf_text` becomes: open (fail-soft `""`) → consume the helper → build the separator-joined string. Its signature, output bytes, and log lines are UNCHANGED. +3. Add the public function: + ```python + def extract_pdf_page_texts(pdf_path: Path) -> dict[int, str]: + """One entry per page of *pdf_path* that has text (1-indexed page → stripped text). …""" + ``` + Same open/fail-soft contract as `extract_pdf_text` (unreadable file → `{}`, one warning — the `pdf:` logger prefix house style). +4. Update the module docstring's function list (the file documents every public function at the top). +5. Unit tests in `tests/unit/test_pdf.py` (the file that owns `app/rag/pdf` — extend it): + - `test_sources/players_handbook.pdf` (293 pages): 293 or fewer entries, keys 1-indexed, first key `1`, every value non-empty and stripped; spot-check one page's text equals the corresponding `extract_pdf_text` block body (parse a `---PAGE-N---` block and compare — the two functions must agree page-for-page). + - `test_sources/pdf_horse_test.pdf`: 1 page → exactly one entry (or `{}` if that page is text-less — assert against `extract_pdf_text`'s output for the same file, whatever it is). + - A missing path → `{}` (fail-soft, no raise). + - **Regression pin:** `extract_pdf_text` on both fixture files is byte-identical before/after the refactor (capture the current output NOW, in the test, by asserting the separator structure + a known substring from page 1 — do not hardcode a 300-page string). +6. Run the phase-126 pdf unit tests + `uv run ruff check . && uv run pyright` — green before moving on. + +## Testing & Quality +- The tests above ARE the task's tests; all in the existing `app/rag/pdf` unit-test file (no new file unless the house layout says otherwise — check `tests/unit/` first). +- Coverage: `app/rag/pdf.py` stays >90% (the helper's per-page failure branch: reuse the existing unreadable-page test pattern if one exists; otherwise a temp file that fitz opens but fails mid-read is not worth fabricating — the existing coverage of that branch via `extract_pdf_text`'s tests carries over since both call sites share the helper). + +## Completion Criteria +- [ ] `extract_pdf_page_texts` exists, is documented, fail-soft, and returns 1-indexed page → stripped text +- [ ] `extract_pdf_text` output is byte-identical (the regression pin green on both fixture PDFs) +- [ ] `uv run pytest tests/unit -k pdf -v --no-cov` green; `uv run ruff check . && uv run pyright` green diff --git a/.agents/phases/todo/137_pdf_image_context/02_embedded_image_document_context.md b/.agents/phases/todo/137_pdf_image_context/02_embedded_image_document_context.md new file mode 100644 index 0000000..2aa5602 --- /dev/null +++ b/.agents/phases/todo/137_pdf_image_context/02_embedded_image_document_context.md @@ -0,0 +1,72 @@ +# Task 02 — The forward path: document context into the vision prompt + `image_context` + +**Source:** `TODO.md` L3 — "Images extracted from PDFs still don't have any relevant context in their summaries … there are lots of stock photos in test_sources/hnb_2026_guide.tar.gz and they get summaries like 'A smiling woman with dark curly hair sits holding a grey laptop on her lap...' which has no mention of the document it came from or what it might have been helping to illustrate. … Image summaries should always include the document context if the images come from a document." + +## Objective +When a PDF is (re)indexed, every embedded image's vision call and stored `image_context` carry a document-context block — `Extracted from / (page ):` plus the page's text (capped) — combined with any phase-125 markdown reference blocks (document block first). The vision model therefore writes a description that names the document and the page, and the stored `content`/`summary`/`image_context` carry the same context the embedder needs. Standalone images and the phase-125 markdown-only shape are untouched. + +## Work +1. Read, in full: `app/rag/importer.py` — `_index_pdf_embedded_images` (L1091), `_index_one_embedded_image` (L1182), `_markdown_image_context` (L1403), `_md_ref_block` (L1436), `_describe_or_skip` (L1742); `app/rag/summarizer.py` — `describe_image` (L208, esp. the `context_prefix` build near the message construction). +2. **The block builder** in `app/rag/importer.py` (next to `_markdown_image_context`): + ```python + PDF_IMAGE_CONTEXT_MAX_CHARS = 6000 # ASSUMPTION A3 — module constant, no env var + + def _pdf_image_context_block( + source: str, pdf_rel: str, page: int, page_texts: dict[int, str], + text_usable: bool = True, + ) -> str: + """The document-context block for one embedded image (phase 137, A2). + + Always non-empty: the origin line is present even for a text-less + page. The page text is stripped, capped at + :data:`PDF_IMAGE_CONTEXT_MAX_CHARS` (overflow cut at the cap, + then ``TRUNCATION_MARKER`` on its own line — the summarizer's + existing truncation convention), and OMITTED entirely when the + page has no text OR when the phase-133 gate (see step 3) marks + the PDF's text layer unusable. + """ + ``` + Shape (exactly): + ``` + Extracted from / (page ): + + ``` + `TRUNCATION_MARKER` lives in `app.rag.retriever` (import it there-from, the same way `app/rag/summarizer.py` does — L49). + (header line `Extracted from {source}/{pdf_rel} (page {page}):` — `page` is the 1-indexed page number (the forward path passes `img.page`; the task-03 backfill passes the page parsed from the synthetic path — taking a plain `int`, NOT a `PdfImage`, is deliberate so the backfill needs no fake `PdfImage`); NO "of M" total — the total page count is not on `PdfImage` and the origin line must stay a stable single line for the task-04 renderer's regex.) +3. **The garbage-text guard (ASSUMPTION A7):** the phase-133 gate `is_usable_pdf_text` is added by pending `133_pdf_screenshot_fallback` (runs BEFORE this phase in the pipeline). In `_index_pdf_embedded_images`, import it defensively — module-level `from app.rag.pdf import extract_pdf_images, extract_pdf_page_texts` and, for the gate, `is_usable_pdf_text = getattr(_pdf_module, "is_usable_pdf_text", None)` (a small import-time probe or a try/except ImportError fallback) — and pass a `text_usable: bool` (default `True` when the gate is absent) down to `_index_one_embedded_image` → `_pdf_image_context_block`: `text_usable=False` → the block is the origin line only. Never import the gate unconditionally (the phase must run in isolation if 133 is skipped). +4. **`_index_pdf_embedded_images`:** after the `settings.images` gate, call `extract_pdf_page_texts(full_path)` ONCE (task 01's function) and pass the resulting `page_texts` + `text_usable` to `_index_one_embedded_image` (add keyword params — the function signature is internal, no other callers: verify with `rg "_index_one_embedded_image"`). +5. **`_index_one_embedded_image`:** where it currently computes `image_context = _markdown_image_context(...)` (the phase-125 task-03 scan), build the combined context with a NEW shared helper (task 03 reuses it — define it once, near the block builder): + ```python + def _combine_image_contexts(doc_block: str, md_blocks: str | None) -> str: + """Document block first, markdown reference block(s) second (A2). + + ``md_blocks`` is ``None``/empty → the doc block alone; the doc + block is ALWAYS present for embedded images (it is the provenance + — never empty). Markdown-only values (no doc block) are produced + by the standalone path only and never pass through this helper. + """ + ``` + Set `image_context = _combine_image_contexts(block, md)` — everything downstream is unchanged (the `context=` arg of `_describe_or_skip`, the `content = f"{image_context}\n\n{description}"` enrichment, the row write, `_store_summary`). +6. **The seam prefix (ASSUMPTION A5):** in `app/rag/summarizer.py`'s `describe_image`, replace the context prefix + `"\n\nThis image is referenced in documentation with the following context:\n{context}"` + with neutral wording covering both provenance and reference, e.g. + `"\n\nThis image is part of a document. The following context describes where it comes from and what it is used for:\n{context}"`. + `DESCRIBE_PROMPT` (marker + locked instruction) is UNTOUCHED; `context=None` still yields the bare prompt (byte-identical — assert it). Update `describe_image`'s docstring (`*context*` no longer means "markdown reference context" only — it means "the document/reference context built by the importer") and `_describe_or_skip`'s docstring likewise. +7. Unit tests (extend the importer's existing image tests — `rg "_markdown_image_context|_index_one_embedded_image" tests/unit`): + - **Block builder:** page with text → header + text; text capped at exactly 6000 + `TRUNCATION_MARKER` (build a page of ~7000 chars); text-less page → header line only; `text_usable=False` → header line only even with text present. + - **Combining:** doc-only; doc + markdown (doc block first, blank-line join — assert the exact joined string); markdown shape never lost (the `Referenced in …` block byte-identical inside the combined value). + - **Forward path with a capturing fake LLM** (the importer's existing fake pattern — `chat` records the message list): a PDF with page text + one embedded image → the `describe_image` message's text part == `DESCRIBE_PROMPT` + neutral prefix + combined context (assert the prefix bytes explicitly); the stored `Document.image_context` == the same combined value; `content`/`summary` == `context + "\n\n" + description`. + - **Regression pins:** standalone image (no markdown, no PDF) → message byte-identical to pre-phase (capture the current message in a test by asserting `DESCRIBE_PROMPT` is the entire text part — `context=None`); markdown-only reference for a STANDALONE image (the phase-125 path) → still the markdown block with the (now neutral) prefix — the stored value is byte-identical (only the prompt prefix wording moved, not the stored data). + - The phase-126 unchanged-image digest skip is untouched (an unchanged embedded image still re-describes nothing — add/keep the existing test). +8. `uv run pytest tests/unit -k "pdf or image" -v --no-cov`, then the full unit suite; `uv run ruff check . && uv run pyright` — green before moving on. + +## Testing & Quality +- The tests in step 7; every one deterministic (fake LLM, fixture PDFs — no network, no real model). +- Coverage: new/modified lines in `app/rag/importer.py` + `app/rag/summarizer.py` covered by these tests (the cap boundary, both guard branches, both combining branches). + +## Completion Criteria +- [ ] a (re)indexed PDF's embedded images reach the vision model with `Extracted from / (page N):` + the (capped) page text, and the stored `image_context`/`content`/`summary` carry the same value +- [ ] a markdown-referenced embedded image carries BOTH blocks, document first (exact-string test) +- [ ] the standalone image's describe message is byte-identical to pre-phase; `DESCRIBE_PROMPT` unchanged +- [ ] garbage-text guard: when the phase-133 gate exists and fails, the block is the origin line only; when the gate is absent, the block behaves normally (the phase runs in isolation) +- [ ] `uv run pytest --no-cov` green (full suite); `uv run ruff check . && uv run pyright` green diff --git a/.agents/phases/todo/137_pdf_image_context/03_backfill_image_context.md b/.agents/phases/todo/137_pdf_image_context/03_backfill_image_context.md new file mode 100644 index 0000000..027f338 --- /dev/null +++ b/.agents/phases/todo/137_pdf_image_context/03_backfill_image_context.md @@ -0,0 +1,45 @@ +# Task 03 — Backfill: existing PDF-embedded image docs gain the document context on the next sync + +**Source:** `TODO.md` L3 — "Images extracted from PDFs still don't have any relevant context in their summaries … That context is critical in the image summaries. Image summaries should always include the document context if the images come from a document …" (The forward path of task 02 only fixes images that get (re)indexed; this task fixes the images ALREADY in the KB — e.g. the HNB-guide stock photos the owner cited — without waiting for a byte change.) + +## Objective +A per-sync, idempotent, best-effort pass (the phase-118 A2 "until it sticks" shape, next to the phase-125 `_enrich_pending_images` post-index pass) re-describes every PDF-embedded image document that is missing the document-context block: bytes come from the `image_dir` copy, the page text is re-extracted from the parent PDF when it is still in the source (parent gone → origin line only), the new context-aware description goes through the SAME seam, and the row's `content`/`summary`/`image_context`/chunks are replaced. After a success the doc never qualifies again; a failure leaves the row untouched and retries next sync. + +## Work +1. Read, in full: `_enrich_pending_images` (L1627 — the shape to mirror: driven by committed doc rows, idempotent, fail-soft, one transaction per image, deletes nothing), the per-source loop in `import_sources` (L525–590: `source = root.name`, the `_enrich_pending_images` call site at L587), `ImportSummary` (L212 + its `log()` method — the stats log-only convention: new field + log field, the sync `detail` API shape never changes), `_store_summary` (the image-aware summary store — the description IS the summary, stored verbatim), `_replace_content_chunks` (the per-image re-chunk + re-embed), `_pdf_image_context_block` / `_combine_image_contexts` / `extract_pdf_page_texts` (tasks 01–02). +2. **The candidate query + gate** in a new function `async def _backfill_image_contexts(session, *, source: str, root: Path, llm: Embedder, summary: ImportSummary) -> None` in `app/rag/importer.py`: + - Select this source's `Document` rows with `is_image.is_(True)`. + - A doc is a CANDIDATE iff ALL hold: + 1. its `path` parses as the phase-126 embedded-image synthetic path `#page=#img=.` (module-level compiled regex, e.g. `^.+\.pdf#page=(?P\d+)#img=(?P\d+)(?P\.\w+)$` — parse out `pdf_rel`, page, index, ext; non-matches — standalone images, screenshot-fallback PDFs are `is_image=False` anyway — are NEVER touched); + 2. its `image_context` is `NULL` OR does not start with `"Extracted from "` (the idempotency gate — a markdown-only `"Referenced in …"` value STILL qualifies: the doc block is missing); + 3. its `image_path` names a file that exists on disk (the `image_dir` copy — the bytes source; missing → debug log, skip, the doc stays as-is). + - If no candidates → return (the common case after the one-time backfill — zero cost per sync). +3. **Per candidate** (fail-soft, one transaction per image, mirroring `_enrich_pending_images`'s error handling): + - `page_texts`: call `extract_pdf_page_texts(root / pdf_rel)` ONCE per unique `pdf_rel` across the candidates (a local cache dict in the pass — one parent may own many images); parent missing/unreadable → `page_texts = {}` AND `text_usable = True` (origin line only — the origin is still valuable and needs no disk); parent present → apply the same phase-133 gate probe as task 02 (shared helper if task 02 factored one out — reuse it, do not duplicate the probe). + - Build `doc_block = _pdf_image_context_block(source, pdf_rel, page, page_texts, text_usable=text_usable)` (`page` from the regex — the builder takes a plain `int`, task 02), then `combined = _combine_image_contexts(doc_block, existing_md_blocks)` — where `existing_md_blocks` = the doc's current `image_context` if it starts with `"Referenced in "` (the markdown blocks are kept verbatim — re-scanning the disk for them is NOT the pass's job: the phase-125 pass owns the markdown context and ran first this sync), else `None`. + - `data = Path(doc.image_path).read_bytes()` (OSError → skip, debug log). + - `description = await _describe_or_skip(llm, data=data, source=source, rel=doc.path, full_path=Path(doc.image_path), context=combined)` — the SAME seam (the mime comes from the copy's extension, exactly like the forward path). `None` → the row is UNTOUCHED (no rollback of anything — nothing was written), `summary.images_failed += 1`, warning, move on (the next sync retries — the gate still matches). + - Write: `content = f"{combined}\n\n{description}"`; `image_context = combined`; `indexed_at = now`; then `_replace_content_chunks(...)` (same call shape as the forward path, L1311) in the image's own transaction (an embed failure rolls back THIS image only — `summary.errors += 1`, error line, the copy stays, move on); then `_store_summary(...)` (the image convention — the description is the summary, verbatim); commit; `summary.image_context_backfilled += 1`; one loud log line (PLAN §9): `import: backfilled image context source= path= chunks=`. + - Deliberately UNCHANGED on the row: `content_hash` (the byte digest — the image identity is untouched), `created_at`/`created_at_manual` (the SOURCE did not change — the phase-106 date rules do not apply), `title`, `image_path`. +4. **The counter:** `ImportSummary` gains `image_context_backfilled: int = 0` (docstring: "PDF-embedded image docs whose stored `image_context` was backfilled with the document-context block (phase 137, task 03) — the one-time best-effort pass; a success never repeats, a failure retries next sync") + its field in `ImportSummary.log()` (the per-turn/per-sync log line — append it next to `images_failed`; the sync `detail` API response shape is NOT changed — the stats log-only convention, phase 133). +5. **The call site:** in `import_sources`'s per-source loop, immediately AFTER the existing `await _enrich_pending_images(...)` (L587 — enrichment first: it owns the markdown-FIRST case and may set `image_context`; the backfill then prepends the document block to whatever is there). The pass is driven by committed rows (like the enrichment pass), so it is safe under `limit`. +6. **Integration tests** (extend the importer's existing integration file — `rg "_enrich_pending_images" tests/integration`): + - **Backfill happy path:** seed the DB with a PDF-embedded image row (the phase-126 synthetic path, `image_context=NULL`, `content`/`summary` = a bare description, an `image_path` pointing at a temp file with the image bytes) + the parent PDF on disk with page text; run the REAL `import_sources` over the source (fake LLM that records describe calls and returns a fixed context-aware reply); assert: exactly ONE describe call whose context carries `Extracted from / (page N):` + the page text; the row's `image_context` starts with `Extracted from `; `content`/`summary` = combined context + new description; chunks replaced; `image_context_backfilled == 1`. + - **Idempotency:** run `import_sources` a second time over the same source → ZERO describe calls, the row byte-identical (the gate no longer matches), `image_context_backfilled == 0`. + - **Parent gone:** the parent PDF is deleted from the source dir → the describe call's context is the origin line ONLY (no page-text line); the row is still backfilled (the gate matches again next sync iff it failed — here it succeeds once). + - **Markdown-only row:** `image_context = "Referenced in /doc.md:\n- Alt text: \"x\""` (the phase-125 shape) → the backfilled `image_context` = doc block + blank line + the markdown block VERBATIM (document first). + - **Standalone image never touched:** an `is_image` row whose `path` has no `#page=` (a plain `pic.png`) with `image_context=NULL` → zero describe calls for it, across two runs. + - **Fail-soft:** a candidate whose describe call returns `None` (the fake fails once) → the row untouched, `images_failed == 1`; a second run with a healthy fake → backfilled. +7. `uv run pytest tests/integration -k "image or import" -v --no-cov`, then the full unit + integration suites; `uv run ruff check . && uv run pyright` — green before moving on. + +## Testing & Quality +- The tests in step 6 (real `import_sources`, fake LLM, temp dirs + seeded session — the house integration pattern; no network, no real model). +- Coverage: every gate branch (pattern non-match, gate match, missing copy, parent gone), the happy path, idempotency, the fail-soft paths — all new lines covered. + +## Completion Criteria +- [ ] an existing context-free PDF-embedded image doc is re-described WITH the document context on the next sync (integration test), counted in the new log-only `image_context_backfilled` counter +- [ ] the pass is idempotent: the following sync burns zero vision calls for a backfilled doc (integration test) +- [ ] a parent-gone image gets the origin-line-only context; a markdown-enriched row keeps its markdown block verbatim, document block prepended +- [ ] standalone images and non-embedded rows are never touched; a describe failure leaves the row untouched and retries next sync +- [ ] the sync `detail` API shape is unchanged (the new counter is log-only — verify the sync response schema test still passes unedited) +- [ ] `uv run pytest --no-cov` green; `uv run ruff check . && uv run pyright` green diff --git a/.agents/phases/todo/137_pdf_image_context/04_retrieval_display_line.md b/.agents/phases/todo/137_pdf_image_context/04_retrieval_display_line.md new file mode 100644 index 0000000..7152021 --- /dev/null +++ b/.agents/phases/todo/137_pdf_image_context/04_retrieval_display_line.md @@ -0,0 +1,36 @@ +# Task 04 — The suggestion-block display line: `From / (page N)` + +**Source:** `TODO.md` L3 — "…so the embedder can more effectively retrieve image sources." (The summary itself now carries the context via tasks 02/03; this task makes the context VISIBLE in the chat suggestion block the way phase 125 did for markdown references — a short locator line, never the page-text body.) + +## Objective +`_extract_context_line` in `app/rag/prompts.py` renders the phase-137 document-context block — `Extracted from / (page ):` — as the one-line clause `From / (page )`, so a PDF-embedded image's `` suggestion body leads with where the image came from (before its description), exactly the way an enriched markdown image leads with `Referenced in …`. A context value carrying BOTH block kinds renders both clauses in stored order. Everything that parses to nothing degrades to `None` exactly as today (pre-phase rows, the `IMAGE_CONTEXT:` pending record, unparseable text). + +## Work +1. Read, in full: `_extract_context_line` (L470), `_image_ctx_clause` (L459), `_image_context_line` (L536), `_document_body` (L548) in `app/rag/prompts.py` — the phase-125 task-05 renderer: header regex `_IMAGE_CTX_HEADER_RE` (`^Referenced in (?P.+):$`), the alt-bullet regex, the clause join (`"Referenced in " + "; ".join(clauses)`), and the `expecting_alt` state machine. +2. Add the header regex for the new block (next to `_IMAGE_CTX_HEADER_RE`): `_PDF_CTX_HEADER_RE = re.compile(r"^Extracted from (?P.+) \(page (?P\d+)\):$")` — `loc` is the `/` locator (greedy to the last ` (page N):` — the pdf path may itself contain spaces). +3. Extend the state machine in `_extract_context_line` to handle the new header: + - A `Extracted from …` header STARTS a pdf clause: it needs NO following bullet (unlike the `Referenced in` header, whose next line MUST be the alt bullet or the block is dropped — the pdf block's next line is page-text body or the next header, and none of it changes the clause). + - Its clause text: `From {loc} (page {page})`. + - Clause JOINING: the current code prefixes `"Referenced in "` to the joined clauses — that prefix belongs to the REFERENCED clauses only. Restructure so each clause carries its own prefix: referenced clauses stay `Referenced in : ""` (byte-identical output for markdown-only values — the phase-125 regression pin), pdf clauses are `From (page N)`; the join is still `"; "` between clauses IN STORED ORDER (document block first per ASSUMPTION A2 → a both-blocks value renders `From a.pdf (page 2); Referenced in docs/a.md: "alt"`). + - A pdf clause followed by page-text body lines: the body lines are skipped by the existing "any other line is context body" fall-through — verify the state machine does not mis-read a page-text line that LOOKS like a header (e.g. a line of page text starting with `Referenced in ` — the existing spurious-header rule already handles the referenced case by demanding the alt bullet; a spurious `Extracted from … (page N):` line inside page text would emit a clause — accept that: it is the stored context the importer wrote, not model text; document it in the docstring). + - A value with NO parsable header of either kind → `None` (unchanged). + - Update `_extract_context_line`'s docstring (the stored value can now carry the `Extracted from` block first — tasks 02/03 write it — plus the markdown blocks) and the module-level comment block above the regexes (the phase-137 note). +4. `_image_context_line` / `_document_body` need NO changes (the `is_image` guard + the renderer call already do the right thing) — verify with a read, and note it in the commit message if so. +5. Unit tests (extend the prompts test file — `rg "_extract_context_line|_document_body" tests/unit`): + - pdf block alone → `"From /guide.pdf (page 2)"` exactly. + - pdf block + page-text body (multi-line, including a blank line and a line starting with `Referenced in ` without the alt bullet) → the same single clause (the spurious referenced header is dropped by the existing rule). + - markdown block alone → byte-identical to the pre-task output (the phase-125 pin: `Referenced in docs/a.md: "alt a"; docs/b.md` from the existing test's shape). + - BOTH blocks in stored order (pdf first, markdown second — the tasks 02/03 combined shape) → `From /guide.pdf (page 2); Referenced in docs/a.md: "alt"`. + - garbage/unparseable value → `None` (unchanged); the `IMAGE_CONTEXT:` pending record → `None` (unchanged). + - `_document_body` end-to-end: an `is_image` doc with the combined context → body leads with the one-line rendering, then the summary; a non-image doc with a non-NULL `image_context` → byte-identical body (the `is_image` guard). +6. `uv run pytest tests/unit -k prompt -v --no-cov`, then the full unit suite; `uv run ruff check . && uv run pyright` — green before moving on. + +## Testing & Quality +- The tests in step 5; the markdown-only byte-identity pin is the load-bearing one (phase 125's display contract must not move). +- Coverage: the new header branch, both orderings, the degradation paths — all covered by step 5. + +## Completion Criteria +- [ ] a PDF-embedded image doc's suggestion `` body leads with `From / (page N)` (unit test), never rendering the page-text body +- [ ] a both-blocks context renders `From …; Referenced in …` in stored order; a markdown-only context renders byte-identically to pre-phase (the phase-125 pin green UNEDITED) +- [ ] unparseable values and the `IMAGE_CONTEXT:` record degrade to `None` as before; non-image docs are byte-identical +- [ ] `uv run pytest --no-cov` green; `uv run ruff check . && uv run pyright` green diff --git a/.agents/phases/todo/137_pdf_image_context/05_e2e_pdf_image_context.md b/.agents/phases/todo/137_pdf_image_context/05_e2e_pdf_image_context.md new file mode 100644 index 0000000..9aa5d35 --- /dev/null +++ b/.agents/phases/todo/137_pdf_image_context/05_e2e_pdf_image_context.md @@ -0,0 +1,58 @@ +# Task 05 — The dedicated E2E: `tests/e2e/test_pdf_image_context.py` (mock trigger + the story suite) + +**Source:** `TODO.md` L3 — the whole item, end to end: "Images extracted from PDFs still don't have any relevant context in their summaries … Image summaries should always include the document context if the images come from a document (this includes markdown files) so the embedder can more effectively retrieve image sources." (AGENTS.md §4/§9: every phase carries its own Playwright E2E file, run in isolation; this is this phase's gate.) + +## Objective +One new Playwright suite proving the story through the REAL UI + REAL pipeline with the deterministic mock LLM: (a) an uploaded PDF's embedded image is described WITH the document context (the context-aware canned reply is the witness that the context reached the vision model), stored on `image_context`/`content`/`summary`; (b) the chat suggestion for that image leads with the `From / (page N)` line; (c) a markdown reference in the same source COMBINES with the document context, document block first. Plus the one mock-LLM change that makes (a) observable. + +## Work +1. Read, in full: `tests/e2e/test_pdf_support.py` (the module-app pattern: `_app_env` with the port/env overrides, `zip_path` fixture — the PyMuPDF PDF build: `doc.new_page().insert_text((72, 72), PAGE_ONE)`, `page2.insert_image(fitz.Rect(72, 150, 73, 151), stream=PNG_1X1)`, `doc.set_metadata({"title": PDF_TITLE})`, zip to `.zip`; the `_truncate_all`, `_upload_zip` (admin login → git-sources page → `#archive-upload-file`/`#archive-upload-btn` → result line), `_sync_via_ui` (RAG page `#sync-btn` → `#sync-label` "Syncing…" → synced label + `#sync-result`), and `seeded` module fixture), `tests/e2e/test_image_anchor_context.py` (the `ANCHOR_QUESTION = "quote the image reference — what is shown in the homelab network diagram?"` pattern + the mock-calibrated gate env values + the standalone-doc "no reference line" witness), `tests/e2e/mock_llm.py` — the `IMAGE_DESCRIPTION_MODE` branch (L2472), `IMAGE_DESCRIPTION_ANSWER` (L865), `CONTEXT_LINE_TRIGGER` (L1886) + its branch (L2532, `_seeded_image_doc` / `_seed_context_line`), and the docstring's behavior list at the top (every new mock behavior is documented there). +2. **The mock change** (the ONLY edit to `tests/e2e/mock_llm.py`): + - New constant next to `IMAGE_DESCRIPTION_ANSWER`: + ```python + #: The phase-137 witness — the mock's vision reply when the + #: ``IMAGE_DESCRIPTION_MODE`` message's text part carries the + #: document-context block (its ``Extracted from …`` header). It + #: names the document + page (the context reached the model — the + #: deterministic proof) and keeps the diagram wording of + #: :data:`IMAGE_DESCRIPTION_ANSWER` so the story question's + #: token-overlap retrieval still grounds on the image doc. + IMAGE_DESCRIPTION_CONTEXT_ANSWER = ( + "From report.pdf, page 2: a network diagram of the homelab " + "server room: a core router on top, a core switch in the " + "middle, and three labeled subnets at the bottom — VLAN 10 " + "office, VLAN 20 lab, and VLAN 30 storage — with a legend of " + "cable runs. Title: Homelab Network Map." + ) + ``` + - In the `IMAGE_DESCRIPTION_MODE` branch: `answer = IMAGE_DESCRIPTION_CONTEXT_ANSWER if "Extracted from " in user else IMAGE_DESCRIPTION_ANSWER` — the `Extracted from ` probe is the phase-137 provenance header (the importer's block builder — task 02); the bare path (standalone images, the pre-137 embedded shape, every existing suite's PDFs) stays on `IMAGE_DESCRIPTION_ANSWER` — byte-identical. + - Document the new branch behavior in the mock's top docstring (the house rule — every behavior is listed). + - NO other mock change: the `CONTEXT_LINE_TRIGGER` branch already echoes the seeded image doc's block FIRST line — which is now the task-04 `From …` rendering for PDF images (zero mock work for the display assertion). +3. **The suite** `tests/e2e/test_pdf_image_context.py` (run in isolation: `uv run pytest tests/e2e/test_pdf_image_context.py -v --no-cov`): + - Module app: ONE images-ON app, the `test_pdf_support.py` pattern — new env port (`E2E_APP_PORT_PDF_IMG_CTX`, default 8154 — check `8152/8153` are taken by test_pdf_support and pick the next free), scratch dirs via `tmp_path_factory`, the mock's port from the session `mock_llm` fixture; assert `/api/config` `images` is True in the seed (the suite is void otherwise). + - **Fixture archive 1** (`zip_path`): ONE PDF at the root, source `e2e-pdf-imgctx`, `report.pdf` — page 1 = `PAGE_ONE` text, page 2 = `PAGE_TWO` text + the 1×1 PNG embedded (the phase-126 recipe; define `PAGE_ONE`/`PAGE_TWO`/`PDF_TITLE`/`PNG_1X1` locally — do NOT import from test_pdf_support; the suites must stay standalone). + - **Seed 1** (module): `_truncate_all` → `_upload_zip` → `_sync_via_ui` (expected result line: the 2-added counts — the PDF doc + the image doc). + - **Test 1 — the forward path (the TODO's core complaint):** DB assertions (`SessionLocal`, like test_pdf_support's `seeded`): + - the image doc exists at `e2e-pdf-imgctx/report.pdf#page=2#img=0.png`, `is_image=True`; + - its `image_context` STARTS WITH exactly `Extracted from e2e-pdf-imgctx/report.pdf (page 2):` and CONTAINS the `PAGE_TWO` text (the document context reached the store — the owner's "no mention of the document it came from" is gone); + - its `content` == `image_context + "\n\n" + IMAGE_DESCRIPTION_CONTEXT_ANSWER` and `summary` == `content` (the image convention) — the CONTEXT-AWARE reply is the witness the describe call carried the context (the bare `IMAGE_DESCRIPTION_ANSWER` string must NOT appear in `content`). + - **Test 2 — the display line in chat:** the `ANCHOR_QUESTION` pattern (`"quote the image reference — what is shown in the homelab network diagram?"` — the diagram wording is in `IMAGE_DESCRIPTION_CONTEXT_ANSWER`, so the same retrieval grounds), the mock-calibrated gate env (the test_image_anchor_context values); assert the final answer contains `From e2e-pdf-imgctx/report.pdf (page 2)` (the mock's `CONTEXT_LINE_TRIGGER` echo of the block's first line — the task-04 rendering) and does NOT contain `stands alone` (the standalone witness is absent). + - **Fixture archive 2** (`zip_md_path`): the SAME PDF + `notes.md` at the root containing `![network diagram](report.pdf#page=2#img=0.png)` plus a paragraph of surrounding context (the phase-125 reference shape) — source `e2e-pdf-imgctx-md`. + - **Seed 2** (module): `_truncate_all` → upload archive 2 → sync (expected counts: PDF + image + the markdown doc). + - **Test 3 — markdown + document combined, document first:** the image doc's `image_context` == `Extracted from e2e-pdf-imgctx-md/report.pdf (page 2):` + `PAGE_TWO` text + `"\n\n"` + the markdown block starting `Referenced in e2e-pdf-imgctx-md/notes.md:` (assert the STARTS-WITH document block AND the `Referenced in` block's presence AND the document block's index < the `Referenced in` block's index — document first, ASSUMPTION A2); the chat display answer (the same anchor question) contains BOTH `From e2e-pdf-imgctx-md/report.pdf (page 2)` and `Referenced in e2e-pdf-imgctx-md/notes.md`. + - **Test 4 — standalone parity witness:** a plain standalone image upload in the SAME seeded source is overkill; instead assert against seed 1's KB: NO other `is_image` doc exists (the suite's KB is exactly the PDF + its image), and the mock's bare branch is pinned by test_pdf_support running green unedited (do not duplicate it). +4. **Regression run (the load-bearing gate):** `uv run pytest tests/e2e/test_pdf_support.py tests/e2e/test_image_anchor_context.py tests/e2e/test_image_documents.py -v --no-cov` — all three green UNEDITED (the mock change must not move their canned replies — the bare describe path is byte-identical; the `CONTEXT_LINE_TRIGGER` echo of a markdown-enriched doc is byte-identical). +5. `uv run ruff check . && uv run pyright` — green. (The suite itself is linted like its siblings; no type-check burden on the mock constants.) + +## Testing & Quality +- The suite in step 3 IS the phase's E2E deliverable; deterministic end to end (mock LLM, fixture PDFs built in tmp dirs — `test_sources/` is never touched, the house rule from phase 133's fixture handling). +- The mock edit is covered by the suite's tests 1–3 (the trigger) + the regression run step 4 (the non-trigger). +- Coverage gate (app/) is unaffected by test files; the phase's >90% comes from tasks 01–04. + +## Completion Criteria +- [ ] `tests/e2e/test_pdf_image_context.py` exists and passes IN ISOLATION: `uv run pytest tests/e2e/test_pdf_image_context.py -v --no-cov` +- [ ] the embedded image's stored summary names the document + page (the context-aware canned reply — test 1); `image_context` leads with `Extracted from /report.pdf (page 2):` + the page text +- [ ] the chat suggestion for the image leads with `From /report.pdf (page 2)` (test 2, via the existing `CONTEXT_LINE_TRIGGER` echo — no new display mock branch) +- [ ] a markdown-referenced PDF image shows BOTH the document and the reference context, document first (test 3) +- [ ] `test_pdf_support.py`, `test_image_anchor_context.py`, `test_image_documents.py` green UNEDITED after the mock change +- [ ] `uv run ruff check . && uv run pyright` green