phase: 137_pdf_image_context

All verification complete — everything is 100% green. Final report:

**Phase 137 final verification pass — all tasks already complete, all criteria verified**

- **Full suite:** `uv run pytest --cov=app --cov-report=term-missing` → 3272 passed, exit 0; `app/` coverage **99%** (>90% ✓)
- **Phase E2E in isolation:** `uv run pytest tests/e2e/test_pdf_image_context.py -v --no-cov` → **4 passed** (forward context, display line, standalone parity, both-blocks markdown case)
- **Pre-existing suites in isolation:** `test_pdf_support.py` + `test_image_anchor_context.py` + `test_image_documents.py` → **14 passed**; ripple-edited `test_garbage_pdf.py` → **3 passed**
- **Lint/types:** `uv run ruff check .` → all passed; `uv run pyright` → 0 errors

**Completion criteria:** (1) forward-path context in vision msg + stored `image_context`/summary — ✓ (unit builder/seam + integration fake-LLM + E2E); (2) both blocks document-first + standalone describe byte-identical — ✓ (combiner + standalone pins); (3) backfill with context + zero-vision idempotent re-sync + parent-gone origin-only — ✓ (6 integration tests); (4) suggestion block leads with `From <source>/<pdf> (page N)` — ✓ (renderer units + E2E); (5) all gates green — ✓ above; (6) no behavior change in phases 122/125/126 + sync `detail` shape untouched — ✓ (verified in code: new counter is log-only); (7) ruff+pyright — ✓; (8) commit — left to the harness per pipeline rules (working tree untouched by me).

**Documentation:** README.md audited in full — the new phase-137 bullet verified against code (6 000-char cap, log-only `image_context_backfilled=N`, `<document>`-body locator placement), all env tables/defaults cross-checked vs `app/config.py` and `.env.example` — consistent, no changes needed. AGENTS.md: every rule/command verified — consistent, no changes needed.

**Notable deviations (pre-existing, documented in task-05 report, forced by design):** `test_pdf_support.py`/`test_garbage_pdf.py` got minimal ripple edits (the "UNEDITED" pin was unsatisfiable once describe calls carry context by design); the unit `test_image_anchor_context.py` CONTEXT_PREFIX pin generalized per ASSUMPTION A5; one extra mock line (`_seed_context_line` accepts `From …`).

**Next pending phase:** none — `todo/` holds only this phase.

tests/integration/test_importer.py

@@ -42,6 +42,7 @@ from sqlalchemy import select
 from app.config import Settings
 from app.models import Chunk, Document
 from app.rag.importer import import_sources
+from app.rag.llm import EmbeddingError
 from app.rag.summarizer import DESCRIBE_PROMPT
 from tests.fakes import FakeEmbedder
 
@@ -57,8 +58,14 @@ DESCRIPTION = (
 )
 
 #: The exact context prefix the task-03 extension appends to the locked
-#: prompt (``app.rag.summarizer.describe_image``).
-CONTEXT_PREFIX = "\n\nThis image is referenced in documentation with the following context:\n"
+#: prompt (``app.rag.summarizer.describe_image``). Phase 137 (task 02,
+#: ASSUMPTION A5) generalized the wording — it now covers BOTH the
+#: provenance's document context and the phase-125 markdown reference
+#: blocks.
+CONTEXT_PREFIX = (
+    "\n\nThis image is part of a document. The following context "
+    "describes where it comes from and what it is used for:\n"
+)
 
 
 class _MockVisionLLM(FakeEmbedder):
@@ -500,9 +507,11 @@ def test_pdf_becomes_document_with_text_chunks_embedding_and_summary(
     title), page-anchored chunks with 768-dim embeddings, and the
     phase-30/118 lite summary. With ``images=True`` (A3) the embedded
     image joins the pipeline through the REAL seam — one vision call on
-    the CHAT model with the bare locked prompt (no markdown reference
-    exists — ``image_context`` stays NULL), the copy in ``image_dir``,
-    and the description as the image doc's content AND summary."""
+    the CHAT model with the locked prompt + the phase-137 document
+    context (the provenance's origin line + page 2's text — no markdown
+    reference exists, so no ``Referenced in`` block), the copy in
+    ``image_dir``, and the description (enriched with the context)
+    as the image doc's content AND summary."""
     source_root = tmp_path / "pdfkb"
     source_root.mkdir()
     png = _png_bytes(40, 30)
@@ -547,19 +556,28 @@ def test_pdf_becomes_document_with_text_chunks_embedding_and_summary(
                 Document.source == source_root.name, Document.path == image_rel
             )
         )
+        # Phase 137 (task 02): the provenance's document context — the
+        # origin line + page 2's text (no markdown reference → no
+        # ``Referenced in`` block) — is stored on ``image_context`` and
+        # enriches the content (context first, then the description).
+        expected_context = (
+            f"Extracted from {source_root.name}/report.pdf (page 2):\n{PAGE_TWO}"
+        )
         assert img is not None and img.is_image is True
-        assert img.image_context is None, "no markdown reference → bare prompt"
-        assert img.content == DESCRIPTION and img.summary == DESCRIPTION
+        assert img.image_context == expected_context
+        expected_content = f"{expected_context}\n\n{DESCRIPTION}"
+        assert img.content == expected_content and img.summary == expected_content
         copy = Path(img.image_path)
         assert copy.parent == Path(llm.settings.image_dir).expanduser()
         assert copy.name == f"{img.id}.png"
         assert copy.read_bytes() == png
-        # The vision call went to the CHAT model with the BARE locked
-        # prompt (the PDF's lite summary is the other chat call).
+        # The vision call went to the CHAT model with the locked prompt
+        # + the generalized prefix + the document context (the PDF's
+        # lite summary is the other chat call).
         assert llm.chat_models == [
             llm.settings.llm_summary_model, llm.settings.llm_chat_model
         ]
-        assert llm.text_parts == [DESCRIBE_PROMPT]
+        assert llm.text_parts == [DESCRIBE_PROMPT + CONTEXT_PREFIX + expected_context]
     finally:
         _cleanup_source(db, source_root.name)
 
@@ -804,6 +822,12 @@ def test_pdf_embedded_image_referenced_by_markdown_markdown_first(
         assert (summary.files, summary.added, summary.images_failed) == (2, 3, 0)
 
         expected_block = _pdf_expected_block(source_root.name, "guide.md", "report.pdf")
+        # Phase 137 (task 02, A2): the provenance's document block is
+        # FIRST — the combined context the vision call, the stored
+        # ``image_context``, and the enriched content all carry.
+        expected_context = (
+            f"Extracted from {source_root.name}/report.pdf (page 2):\n{PAGE_TWO}"
+        ) + "\n\n" + expected_block
 
         # Work 1 — the markdown doc STORES the reference (the record —
         # line 3 is the reference, 2 context paragraphs). It is the
@@ -830,14 +854,14 @@ def test_pdf_embedded_image_referenced_by_markdown_markdown_first(
             )
         )
         assert img is not None and img.is_image is True
-        assert img.image_context == expected_block
-        expected_content = f"{expected_block}\n\n{DESCRIPTION}"
+        assert img.image_context == expected_context
+        expected_content = f"{expected_context}\n\n{DESCRIPTION}"
         assert img.content == expected_content
         assert img.summary == expected_content
-        # The vision call carried the context AFTER the locked prompt
-        # (the multimodal part list's text part — the describe seam's
-        # phase-125 extension).
-        assert llm.text_parts == [f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{expected_block}"]
+        # The vision call carried the combined context AFTER the locked
+        # prompt (the multimodal part list's text part — the describe
+        # seam's phase-125 extension, the phase-137 generalized prefix).
+        assert llm.text_parts == [f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{expected_context}"]
         # Chat call order (walk order): guide.md's ``lite`` summary,
         # the PDF's ``lite`` summary, then the image's CHAT-model
         # description (the ONLY multimodal call).
@@ -887,6 +911,12 @@ def test_pdf_embedded_image_referenced_by_markdown_pdf_first(
         assert (summary.files, summary.added, summary.images_failed) == (2, 3, 0)
 
         expected_block = _pdf_expected_block(source_root.name, "zguide.md", "alpha.pdf")
+        # Phase 137 (task 02, A2): the provenance's document block is
+        # FIRST (the same combined context as the markdown-first final
+        # state — the enrichment never depends on the walk order).
+        expected_context = (
+            f"Extracted from {source_root.name}/alpha.pdf (page 2):\n{PAGE_TWO}"
+        ) + "\n\n" + expected_block
 
         # The task-03 path owns this case: the image doc existed when
         # the markdown was indexed → NO record (the field is the
@@ -906,11 +936,11 @@ def test_pdf_embedded_image_referenced_by_markdown_pdf_first(
                 Document.source == source_root.name, Document.path == image_rel
             )
         )
-        assert img is not None and img.image_context == expected_block
-        expected_content = f"{expected_block}\n\n{DESCRIPTION}"
+        assert img is not None and img.image_context == expected_context
+        expected_content = f"{expected_context}\n\n{DESCRIPTION}"
         assert img.content == expected_content
         assert img.summary == expected_content
-        assert llm.text_parts == [f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{expected_block}"]
+        assert llm.text_parts == [f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{expected_context}"]
         # Chat call order (walk order): the PDF's ``lite`` summary, the
         # image's CHAT-model description, then zguide.md's summary.
         assert llm.chat_models == [
@@ -920,3 +950,559 @@ def test_pdf_embedded_image_referenced_by_markdown_pdf_first(
         ]
     finally:
         _cleanup_source(db, source_root.name)
+
+
+# ---------------------------------------------------------------------------
+# Phase 137, task 03 — the document-context backfill through the REAL
+# import pipeline (seeded pre-phase-137 image rows, the REAL
+# ``_backfill_image_contexts`` pass inside ``import_sources``, the REAL
+# ``_describe_or_skip`` → ``describe_image`` seam — NO seam patch).
+# ---------------------------------------------------------------------------
+
+#: The seeded embedded image's phase-126 synthetic path (the parent
+#: ``report.pdf``, page 2, index 0, ``.png`` — the upsert key the
+#: forward path would have built).
+BACKFILL_IMAGE_REL = "report.pdf#page=2#img=0.png"
+
+
+def _seed_backfill_row(
+    db,
+    *,
+    source: str,
+    image_rel: str,
+    image_path: str,
+    context: str | None = None,
+    title: str = "Embedded image from report page 2",
+) -> Document:
+    """One pre-phase-137 PDF-embedded image doc row — the shape a
+    phase-126 sync left behind: ``image_context`` NULL (or the
+    markdown-only phase-125 value), ``content``/``summary`` the bare
+    description (or the phase-125 enriched shape), one content chunk +
+    the ``is_summary`` chunk, and ``image_path`` naming the persistent
+    ``image_dir`` copy (the caller controls whether the file exists).
+    """
+    content = f"{context}\n\n{DESCRIPTION}" if context is not None else DESCRIPTION
+    doc = Document(
+        source=source,
+        path=image_rel,
+        full_path=image_path,
+        title=title,
+        content=content,
+        content_hash=hashlib.sha256(PNG_1X1).hexdigest(),
+        is_image=True,
+        image_path=image_path,
+        image_context=context,
+        summary=content,
+    )
+    db.add(doc)
+    db.flush()
+    packed = "\n".join(content.split("\n\n"))
+    db.add(
+        Chunk(
+            document_id=doc.id, position=0, content=packed,
+            embedding=[0.0] * 768,
+        )
+    )
+    db.add(
+        Chunk(
+            document_id=doc.id, position=-1, content=content, is_summary=True,
+            embedding=[0.0] * 768,
+        )
+    )
+    db.commit()
+    return doc
+
+
+def test_backfill_re_describes_context_free_embedded_image_with_document_context(
+    db, tmp_path
+) -> None:
+    """A pre-phase-137 PDF-embedded image doc (``image_context`` NULL,
+    the bare description, the persistent copy) + the parent PDF still
+    in the source (with page text) → the next REAL sync backfills:
+    exactly ONE describe call whose context carries the document block
+    (the origin line + page 2's text); the row's ``image_context``
+    starts with ``Extracted from ``, its ``content``/``summary`` is the
+    combined context + the fresh description; the chunks are replaced;
+    the row's identity fields stay; the log-only counter is 1 (the
+    backfill NEVER counts added/updated — the source did not change)."""
+    source_root = tmp_path / "bkfillkb"
+    source_root.mkdir()
+    _build_pdf(source_root / "report.pdf", [(PAGE_ONE, []), (PAGE_TWO, [])])
+    copy = tmp_path / "copies" / "seed.png"
+    copy.parent.mkdir(parents=True)
+    copy.write_bytes(PNG_1X1)
+    _seed_backfill_row(
+        db, source=source_root.name, image_rel=BACKFILL_IMAGE_REL,
+        image_path=str(copy),
+    )
+    llm = _image_llm(tmp_path, import_extensions="md,txt,pdf")
+    try:
+        summary = asyncio.run(import_sources([source_root], llm, session=db))
+        assert summary.image_context_backfilled == 1
+        assert summary.images_failed == 0 and summary.errors == 0
+        # The parent PDF is the only WALKED file (added); the backfilled
+        # row counts on its own log-only counter, never added/updated.
+        assert (summary.files, summary.added, summary.updated) == (1, 1, 0)
+
+        expected_context = (
+            f"Extracted from {source_root.name}/report.pdf (page 2):\n{PAGE_TWO}"
+        )
+        # The ONE describe call carries the document context (the locked
+        # prompt + the generalized prefix); the parent's ``lite`` summary
+        # is the other chat call, not a describe.
+        assert llm.text_parts == [
+            f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{expected_context}"
+        ]
+        assert llm.chat_models == [
+            llm.settings.llm_summary_model, llm.settings.llm_chat_model
+        ]
+
+        img = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == BACKFILL_IMAGE_REL,
+            )
+        )
+        assert img is not None and img.is_image is True
+        assert img.image_context == expected_context
+        assert img.image_context.startswith("Extracted from ")
+        expected_content = f"{expected_context}\n\n{DESCRIPTION}"
+        assert img.content == expected_content
+        assert img.summary == expected_content
+        # The chunks are REPLACED (the bare-description chunk is gone;
+        # the enriched content chunk + the mirroring summary chunk are
+        # freshly embedded).
+        packed = "\n".join(expected_content.split("\n\n"))
+        chunks = db.scalars(
+            select(Chunk).where(Chunk.document_id == img.id).order_by(Chunk.position)
+        ).all()
+        assert [c.content for c in chunks if not c.is_summary] == [packed]
+        summary_chunks = [c for c in chunks if c.is_summary]
+        assert len(summary_chunks) == 1 and summary_chunks[0].position == -1
+        assert summary_chunks[0].content == expected_content
+        for c in chunks:
+            assert c.embedding is not None and len(c.embedding) == 768
+        # Deliberately UNCHANGED on the row: the byte digest (the image
+        # identity), the copy, the title (the source did not change —
+        # the phase-106 date rules do not apply).
+        assert img.content_hash == hashlib.sha256(PNG_1X1).hexdigest()
+        assert img.image_path == str(copy)
+        assert img.title == "Embedded image from report page 2"
+        assert copy.read_bytes() == PNG_1X1
+    finally:
+        _cleanup_source(db, source_root.name)
+
+
+def test_backfill_is_idempotent_the_next_sync_burns_no_vision_calls(
+    db, tmp_path
+) -> None:
+    """A backfilled doc never qualifies again: the next REAL sync over
+    the same source burns ZERO describe calls (the gate no longer
+    matches — ``image_context`` starts with ``Extracted from ``), the
+    row is byte-identical, and the counter is 0."""
+    source_root = tmp_path / "bkfillidempotent"
+    source_root.mkdir()
+    _build_pdf(source_root / "report.pdf", [(PAGE_ONE, []), (PAGE_TWO, [])])
+    copy = tmp_path / "copies" / "seed.png"
+    copy.parent.mkdir(parents=True)
+    copy.write_bytes(PNG_1X1)
+    _seed_backfill_row(
+        db, source=source_root.name, image_rel=BACKFILL_IMAGE_REL,
+        image_path=str(copy),
+    )
+    llm = _image_llm(tmp_path, import_extensions="md,txt,pdf")
+    try:
+        s1 = asyncio.run(import_sources([source_root], llm, session=db))
+        assert s1.image_context_backfilled == 1
+        img1 = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == BACKFILL_IMAGE_REL,
+            )
+        )
+        assert img1 is not None
+        snapshot = (
+            img1.content, img1.summary, img1.image_context,
+            img1.content_hash, img1.indexed_at, img1.created_at,
+            img1.title, img1.image_path,
+        )
+        describes = len(llm.text_parts)
+        assert describes == 1
+
+        # The next sync: the parent PDF is unchanged, the image doc no
+        # longer matches the gate.
+        s2 = asyncio.run(import_sources([source_root], llm, session=db))
+        assert s2.image_context_backfilled == 0
+        assert (s2.added, s2.updated, s2.unchanged) == (0, 0, 1)
+        assert len(llm.text_parts) == describes  # zero NEW describe calls
+        assert s2.images_failed == 0 and s2.errors == 0
+
+        img2 = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == BACKFILL_IMAGE_REL,
+            )
+        )
+        assert img2 is not None
+        assert (
+            img2.content, img2.summary, img2.image_context,
+            img2.content_hash, img2.indexed_at, img2.created_at,
+            img2.title, img2.image_path,
+        ) == snapshot
+    finally:
+        _cleanup_source(db, source_root.name)
+
+
+def test_backfill_parent_gone_gets_the_origin_line_only(db, tmp_path) -> None:
+    """The parent PDF is gone from the source dir (no disk dependency):
+    the describe call's context is the origin line ONLY (no page-text
+    line), and the row is still backfilled — it succeeds once, so the
+    gate matches again next sync iff it had failed."""
+    source_root = tmp_path / "bkfillgone"
+    source_root.mkdir()  # the parent PDF is already gone
+    copy = tmp_path / "copies" / "seed.png"
+    copy.parent.mkdir(parents=True)
+    copy.write_bytes(PNG_1X1)
+    _seed_backfill_row(
+        db, source=source_root.name, image_rel=BACKFILL_IMAGE_REL,
+        image_path=str(copy),
+    )
+    llm = _image_llm(tmp_path, import_extensions="md,txt,pdf")
+    try:
+        summary = asyncio.run(import_sources([source_root], llm, session=db))
+        assert summary.image_context_backfilled == 1
+        assert summary.images_failed == 0 and summary.errors == 0
+        expected_context = (
+            f"Extracted from {source_root.name}/report.pdf (page 2):"
+        )
+        assert llm.text_parts == [
+            f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{expected_context}"
+        ]
+        img = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == BACKFILL_IMAGE_REL,
+            )
+        )
+        assert img is not None
+        assert img.image_context == expected_context  # origin line only
+        assert img.content == f"{expected_context}\n\n{DESCRIPTION}"
+        assert img.summary == img.content
+    finally:
+        _cleanup_source(db, source_root.name)
+
+
+def test_backfill_keeps_the_markdown_block_verbatim_document_first(
+    db, tmp_path
+) -> None:
+    """A markdown-enriched row (the phase-125 ``Referenced in …``
+    value — the document block missing) STILL qualifies: the backfilled
+    ``image_context`` is the document block + blank line + the stored
+    markdown block VERBATIM (document first — re-scanning the disk for
+    the markdown context is NOT this pass's job: the phase-125 pass
+    owns it and ran first this sync)."""
+    source_root = tmp_path / "bkfillmd"
+    source_root.mkdir()
+    _build_pdf(source_root / "report.pdf", [(PAGE_ONE, []), (PAGE_TWO, [])])
+    copy = tmp_path / "copies" / "seed.png"
+    copy.parent.mkdir(parents=True)
+    copy.write_bytes(PNG_1X1)
+    md_block = (
+        f"Referenced in {source_root.name}/doc.md:\n"
+        '- Alt text: "x"'
+    )
+    _seed_backfill_row(
+        db, source=source_root.name, image_rel=BACKFILL_IMAGE_REL,
+        image_path=str(copy), context=md_block,
+    )
+    llm = _image_llm(tmp_path, import_extensions="md,txt,pdf")
+    try:
+        summary = asyncio.run(import_sources([source_root], llm, session=db))
+        assert summary.image_context_backfilled == 1
+        doc_block = (
+            f"Extracted from {source_root.name}/report.pdf (page 2):\n{PAGE_TWO}"
+        )
+        expected_context = f"{doc_block}\n\n{md_block}"
+        assert llm.text_parts == [
+            f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{expected_context}"
+        ]
+        img = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == BACKFILL_IMAGE_REL,
+            )
+        )
+        assert img is not None
+        # Document block FIRST, the markdown block VERBATIM after it.
+        assert img.image_context == expected_context
+        assert img.content == f"{expected_context}\n\n{DESCRIPTION}"
+        assert img.summary == img.content
+    finally:
+        _cleanup_source(db, source_root.name)
+
+
+def test_backfill_never_touches_standalone_and_copyless_rows(db, tmp_path) -> None:
+    """A standalone image row (a plain path — no ``#page=``) is NEVER a
+    candidate (the pattern gate) even with a live copy + NULL context,
+    and an embedded-pattern row whose ``image_dir`` copy is gone is
+    skipped (the disk gate): both stay byte-identical across TWO REAL
+    syncs, with zero describe calls."""
+    source_root = tmp_path / "bkfillstandalone"
+    source_root.mkdir()
+    copy = tmp_path / "copies" / "pic.png"
+    copy.parent.mkdir(parents=True)
+    copy.write_bytes(PNG_1X1)
+    standalone = _seed_backfill_row(
+        db, source=source_root.name, image_rel="pic.png",
+        image_path=str(copy), title="pic",
+    )
+    missing_copy = _seed_backfill_row(
+        db, source=source_root.name,
+        image_rel="notes.pdf#page=1#img=0.png",
+        image_path=str(tmp_path / "copies" / "gone.png"),
+        title="Embedded image from notes page 1",
+    )
+    llm = _image_llm(tmp_path, import_extensions="md,txt,pdf")
+    try:
+        s1 = asyncio.run(import_sources([source_root], llm, session=db))
+        s2 = asyncio.run(import_sources([source_root], llm, session=db))
+        for s in (s1, s2):
+            assert s.image_context_backfilled == 0
+            assert s.images_failed == 0 and s.errors == 0
+        assert llm.text_parts == []  # zero describe calls, both runs
+        db.expire(standalone)
+        db.expire(missing_copy)
+        assert standalone.image_context is None
+        assert standalone.content == DESCRIPTION
+        assert missing_copy.image_context is None
+        assert missing_copy.content == DESCRIPTION
+    finally:
+        _cleanup_source(db, source_root.name)
+
+
+def test_backfill_unreadable_copy_is_skipped_without_touching_the_row(
+    db, tmp_path
+) -> None:
+    """A candidate whose copy passes the existence gate but is
+    UNREADABLE (a ``read_bytes`` OSError between the gate and the read)
+    is skipped — the row stays as-is (the next sync retries it), nothing
+    is counted as a failure, and no describe call is made."""
+    source_root = tmp_path / "bkfillunreadable"
+    source_root.mkdir()
+    _build_pdf(source_root / "report.pdf", [(PAGE_ONE, []), (PAGE_TWO, [])])
+    copy = tmp_path / "copies" / "seed.png"
+    copy.parent.mkdir(parents=True)
+    copy.write_bytes(PNG_1X1)
+    _seed_backfill_row(
+        db, source=source_root.name, image_rel=BACKFILL_IMAGE_REL,
+        image_path=str(copy),
+    )
+    llm = _image_llm(tmp_path, import_extensions="md,txt,pdf")
+    try:
+        copy.chmod(0o000)  # exists but unreadable (the suite runs non-root)
+        try:
+            summary = asyncio.run(import_sources([source_root], llm, session=db))
+        finally:
+            copy.chmod(0o644)
+        assert summary.image_context_backfilled == 0
+        assert summary.images_failed == 0 and summary.errors == 0
+        assert llm.text_parts == []  # no describe attempt
+        img = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == BACKFILL_IMAGE_REL,
+            )
+        )
+        assert img is not None
+        assert img.image_context is None  # untouched
+        assert img.content == DESCRIPTION
+    finally:
+        _cleanup_source(db, source_root.name)
+
+
+def test_backfill_recognizes_uppercase_pdf_extension(db, tmp_path) -> None:
+    """The walk matches extensions case-insensitively, so an on-disk
+    ``REPORT.PDF`` produces a synthetic path with the uppercase suffix —
+    the backfill's pattern gate must still recognize it (and the parent
+    lookup must hit the uppercase file)."""
+    source_root = tmp_path / "bkfillupper"
+    source_root.mkdir()
+    _build_pdf(source_root / "REPORT.PDF", [(PAGE_ONE, []), (PAGE_TWO, [])])
+    copy = tmp_path / "copies" / "seed.png"
+    copy.parent.mkdir(parents=True)
+    copy.write_bytes(PNG_1X1)
+    _seed_backfill_row(
+        db, source=source_root.name, image_rel="REPORT.PDF#page=2#img=0.png",
+        image_path=str(copy),
+    )
+    llm = _image_llm(tmp_path, import_extensions="md,txt,pdf")
+    try:
+        summary = asyncio.run(import_sources([source_root], llm, session=db))
+        assert summary.image_context_backfilled == 1
+        expected_context = (
+            f"Extracted from {source_root.name}/REPORT.PDF (page 2):\n{PAGE_TWO}"
+        )
+        assert llm.text_parts == [
+            f"{DESCRIBE_PROMPT}{CONTEXT_PREFIX}{expected_context}"
+        ]
+        img = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == "REPORT.PDF#page=2#img=0.png",
+            )
+        )
+        assert img is not None and img.image_context == expected_context
+    finally:
+        _cleanup_source(db, source_root.name)
+
+
+class _FirstDescribeFailsLLM(_MockVisionLLM):
+    """The first multimodal describe reply is EMPTY (``describe_image``
+    → ``None`` — the seam's fail-soft input); later calls answer
+    normally (the retry-next-sync proof)."""
+
+    def __init__(self) -> None:
+        super().__init__()
+        self._failures_left = 1
+
+    async def chat(self, messages, model=None):
+        user = next((m["content"] for m in messages if m.get("role") == "user"), "")
+        if isinstance(user, list) and self._failures_left:
+            self._failures_left -= 1
+            saved = self.DESCRIPTION
+            self.DESCRIPTION = ""  # the empty reply → the seam's None
+            try:
+                return await super().chat(messages, model)
+            finally:
+                self.DESCRIPTION = saved
+        return await super().chat(messages, model)
+
+
+def test_backfill_describe_failure_leaves_row_untouched_and_retries_next_sync(
+    db, tmp_path
+) -> None:
+    """A candidate whose describe call returns ``None`` (an empty vision
+    reply) leaves the row UNTOUCHED — nothing was written, so there is
+    nothing to roll back — ``images_failed`` + the warning, and the next
+    REAL sync (the healthy fake) retries and backfills: the fail-soft
+    contract."""
+    source_root = tmp_path / "bkfillflaky"
+    source_root.mkdir()
+    _build_pdf(source_root / "report.pdf", [(PAGE_ONE, []), (PAGE_TWO, [])])
+    copy = tmp_path / "copies" / "seed.png"
+    copy.parent.mkdir(parents=True)
+    copy.write_bytes(PNG_1X1)
+    _seed_backfill_row(
+        db, source=source_root.name, image_rel=BACKFILL_IMAGE_REL,
+        image_path=str(copy),
+    )
+    llm = _image_llm(
+        tmp_path, llm_cls=_FirstDescribeFailsLLM, import_extensions="md,txt,pdf"
+    )
+    try:
+        s1 = asyncio.run(import_sources([source_root], llm, session=db))
+        assert s1.images_failed == 1
+        assert s1.image_context_backfilled == 0 and s1.errors == 0
+        # The failed call itself carried the right context.
+        assert len(llm.text_parts) == 1
+        assert "Extracted from" in llm.text_parts[0]
+
+        img = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == BACKFILL_IMAGE_REL,
+            )
+        )
+        assert img is not None
+        assert img.image_context is None  # untouched
+        assert img.content == DESCRIPTION
+        assert img.summary == DESCRIPTION
+        assert img.content_hash == hashlib.sha256(PNG_1X1).hexdigest()
+
+        # The next sync (the fake is healthy now): the gate still
+        # matches — the retry succeeds.
+        s2 = asyncio.run(import_sources([source_root], llm, session=db))
+        assert s2.image_context_backfilled == 1 and s2.images_failed == 0
+        img2 = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == BACKFILL_IMAGE_REL,
+            )
+        )
+        assert img2 is not None and img2.image_context is not None
+        assert img2.image_context.startswith("Extracted from ")
+        assert img2.content == f"{img2.image_context}\n\n{DESCRIPTION}"
+    finally:
+        _cleanup_source(db, source_root.name)
+
+
+class _BackfillEmbedFailsLLM(_MockVisionLLM):
+    """The first embed of the backfilled content (a text starting with
+    the ``Extracted from`` header) raises :class:`EmbeddingError`; later
+    embeds succeed (the retry-next-sync proof for the embed path)."""
+
+    def __init__(self) -> None:
+        super().__init__()
+        self._failures_left = 1
+
+    async def embed(self, texts, *, on_progress=None):
+        if self._failures_left and any(
+            t.startswith("Extracted from") for t in texts
+        ):
+            self._failures_left -= 1
+            raise EmbeddingError("embed boom (backfill test)")
+        return await super().embed(texts, on_progress=on_progress)
+
+
+def test_backfill_embed_failure_rolls_back_the_image_and_retries_next_sync(
+    db, tmp_path
+) -> None:
+    """A candidate whose re-chunk/re-embed gives up (an
+    ``EmbeddingError``) rolls back THIS image only — the row is
+    untouched, the copy STAYS, ``errors`` + the error line — and the
+    next REAL sync (the healthy fake) retries and backfills."""
+    source_root = tmp_path / "bkfillembedfail"
+    source_root.mkdir()
+    _build_pdf(source_root / "report.pdf", [(PAGE_ONE, []), (PAGE_TWO, [])])
+    copy = tmp_path / "copies" / "seed.png"
+    copy.parent.mkdir(parents=True)
+    copy.write_bytes(PNG_1X1)
+    _seed_backfill_row(
+        db, source=source_root.name, image_rel=BACKFILL_IMAGE_REL,
+        image_path=str(copy),
+    )
+    llm = _image_llm(
+        tmp_path, llm_cls=_BackfillEmbedFailsLLM, import_extensions="md,txt,pdf"
+    )
+    try:
+        s1 = asyncio.run(import_sources([source_root], llm, session=db))
+        assert s1.errors == 1
+        assert s1.image_context_backfilled == 0 and s1.images_failed == 0
+        # The describe SUCCEEDED (the embed is what failed) — one
+        # context-bearing call.
+        assert len(llm.text_parts) == 1
+
+        img = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == BACKFILL_IMAGE_REL,
+            )
+        )
+        assert img is not None
+        assert img.image_context is None  # rolled back
+        assert img.content == DESCRIPTION
+        assert img.content_hash == hashlib.sha256(PNG_1X1).hexdigest()
+        assert copy.read_bytes() == PNG_1X1  # the copy stays
+
+        s2 = asyncio.run(import_sources([source_root], llm, session=db))
+        assert s2.image_context_backfilled == 1 and s2.errors == 0
+        img2 = db.scalar(
+            select(Document).where(
+                Document.source == source_root.name,
+                Document.path == BACKFILL_IMAGE_REL,
+            )
+        )
+        assert img2 is not None and img2.image_context is not None
+        assert img2.image_context.startswith("Extracted from ")
+    finally:
+        _cleanup_source(db, source_root.name)

tests/integration/test_pdf_hnb_guide_regression.py

@@ -64,6 +64,7 @@ import asyncio
 import re
 import tarfile
 import time
+from collections import Counter
 from collections.abc import Iterator
 from pathlib import Path
 from typing import Any
@@ -74,8 +75,9 @@ from sqlalchemy.orm import Session
 
 from app.config import Settings
 from app.models import Chunk, Document
-from app.rag.importer import ImportSummary, import_sources
-from app.rag.pdf import PAGE_SEPARATOR
+from app.rag.importer import PDF_IMAGE_CONTEXT_MAX_CHARS, ImportSummary, import_sources
+from app.rag.pdf import PAGE_SEPARATOR, extract_pdf_page_texts
+from app.rag.retriever import TRUNCATION_MARKER
 from app.rag.summarizer import DESCRIBE_PROMPT
 from tests.fakes import FakeEmbedder
 
@@ -125,6 +127,33 @@ pytestmark = pytest.mark.skipif(
 )
 
 
+#: The phase-137 (task 02, ASSUMPTION A5) generalized context prefix
+#: (pinned here AND in the summarizer's code) — it covers BOTH the
+#: provenance's document context and the phase-125 markdown reference
+#: blocks (this PDF-only source has the former alone).
+DESCRIBE_CONTEXT_PREFIX = (
+    "\n\nThis image is part of a document. The following context "
+    "describes where it comes from and what it is used for:\n"
+)
+
+
+def _expected_context_block(
+    pdf_rel: str, page: int, page_texts: dict[int, str]
+) -> str:
+    """The phase-137 (task 02) document-context block the importer
+    builds for an embedded image on *page* of *pdf_rel* (the guide's
+    CLEAN text layer: the origin line + the capped page text — the A7
+    origin-line-only shape belongs to the garbage layer, never this
+    fixture; a text-less page is the origin line alone)."""
+    header = f"Extracted from {pdf_rel} (page {page}):"
+    text = page_texts.get(page, "").strip()
+    if not text:
+        return header
+    if len(text) > PDF_IMAGE_CONTEXT_MAX_CHARS:
+        text = text[:PDF_IMAGE_CONTEXT_MAX_CHARS] + "\n" + TRUNCATION_MARKER
+    return f"{header}\n{text}"
+
+
 class GuideVisionLLM(FakeEmbedder):
     """The task-03-style vision mock the ``fakes.py`` comment
     anticipates: the base ``chat`` is string-only — the MULTIMODAL user
@@ -194,7 +223,7 @@ def _nonprint_frac(content: str) -> float:
 @pytest.fixture()
 def guide_kb(
     db: Session, tmp_path: Path
-) -> Iterator[tuple[GuideVisionLLM, ImportSummary, float]]:
+) -> Iterator[tuple[GuideVisionLLM, ImportSummary, float, Path]]:
     """The real guide, imported ONCE per test through the REAL importer
     (the house KB pattern — ``test_read_line_ranges_handbook.py``):
     ``chunks``/``documents``/``folder_summaries`` are truncated around
@@ -202,7 +231,7 @@ def guide_kb(
     source dir, and the deterministic in-process
     :class:`GuideVisionLLM` stands in for the embed/summary/vision
     calls (the whole 48-page index build stays fast and offline).
-    Yields ``(llm, summary, elapsed_s)``."""
+    Yields ``(llm, summary, elapsed_s, pdf_path)``."""
     db.execute(text("TRUNCATE chunks, documents, folder_summaries"))
     db.commit()
     root = tmp_path / SOURCE
@@ -214,7 +243,7 @@ def guide_kb(
     started = time.monotonic()
     summary = asyncio.run(import_sources([root], llm, session=db))
     elapsed = time.monotonic() - started
-    yield llm, summary, elapsed
+    yield llm, summary, elapsed, root / PDF_NAME
     db.execute(text("TRUNCATE chunks, documents, folder_summaries"))
     db.commit()
     db.expire_all()
@@ -222,15 +251,17 @@ def guide_kb(
 
 def test_hnb_guide_imports_clean_through_the_real_pipeline(
     db: Session,
-    guide_kb: tuple[GuideVisionLLM, ImportSummary, float],
+    guide_kb: tuple[GuideVisionLLM, ImportSummary, float, Path],
 ) -> None:
     """The owner's exact file lands CLEAN through the real pipeline:
     the walk admits the one file (``pdf`` token — the PDF branch, never
     the text branch), the stored content is clean + carries the real
     first-page text, the page-anchored chunks are embedded, the ``lite``
     summary is stored, and every embedded-image doc got the fake vision
-    description (the multimodal ``chat`` was called)."""
-    llm, summary, elapsed = guide_kb
+    description (the multimodal ``chat`` was called) enriched with the
+    phase-137 (task 02) document context (the provenance block first —
+    the owner's stock photos finally name their document)."""
+    llm, summary, elapsed, pdf_path = guide_kb
 
     # The task's wall-time guard — the suite must stay fast.
     assert elapsed < WALL_TIME_BAR_S, (
@@ -291,7 +322,19 @@ def test_hnb_guide_imports_clean_through_the_real_pipeline(
     # The embedded-image docs (the phase-126 A3 ``#page=…#img=…``
     # rows): every one of them got the FAKE vision description — the
     # multimodal ``chat`` was called (the task's literal assertion),
-    # exactly once per image doc, with the locked bare prompt.
+    # exactly once per image doc — and phase 137 (task 02) enriched it
+    # with the document context: the provenance block (the origin line
+    # + the page's own text, capped — the guide's layer is CLEAN, so
+    # the gate leaves the page texts IN; the A7 origin-line-only shape
+    # is the garbage layer's, pinned in the unit suite) FIRST, then
+    # the description.
+    page_texts = extract_pdf_page_texts(pdf_path)
+
+    def _page_of(path: str) -> int:
+        m = re.search(r"#page=(\d+)", path)
+        assert m is not None, f"unparseable embedded-image path {path!r}"
+        return int(m.group(1))
+
     image_docs = db.scalars(
         select(Document).where(
             Document.source == SOURCE,
@@ -301,7 +344,9 @@ def test_hnb_guide_imports_clean_through_the_real_pipeline(
     ).all()
     assert image_docs, "the guide's embedded images did not index"
     for img in image_docs:
-        assert img.content == GuideVisionLLM.VISION_REPLY
+        context = _expected_context_block(PDF_REL, _page_of(img.path), page_texts)
+        assert img.image_context == context
+        assert img.content == f"{context}\n\n{GuideVisionLLM.VISION_REPLY}"
         assert img.summary == img.content  # the image convention
         assert img.image_path is not None
         copy = Path(img.image_path)
@@ -318,9 +363,16 @@ def test_hnb_guide_imports_clean_through_the_real_pipeline(
     assert len(llm.vision_texts) == len(image_docs), (
         "one vision call per embedded image doc (no failures)"
     )
-    # The bare locked prompt on every call (no markdown reference
-    # exists in this PDF-only source — the phase-126 A3 shape).
-    assert all(t == DESCRIBE_PROMPT for t in llm.vision_texts)
+    # The locked prompt + the generalized prefix + the document context
+    # on EVERY call (no markdown reference exists in this PDF-only
+    # source — the phase-137 A2 shape). The multiset is the
+    # order-independent exact pin (the rows come back unsorted).
+    expected_calls = Counter(
+        DESCRIBE_PROMPT + DESCRIBE_CONTEXT_PREFIX
+        + _expected_context_block(PDF_REL, _page_of(img.path), page_texts)
+        for img in image_docs
+    )
+    assert Counter(llm.vision_texts) == expected_calls
     assert summary.images_failed == 0
 
     # The added accounting (see the module docstring): the walked file