Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 125_image_anchor_context
Build and Push Containers / build-and-push-app (push) Successful in 2m8s
Details
Build and Push Containers / build-and-push-db (push) Failing after 1m38s
Details

Browse Source 
All completion criteria verified. The repository is left functional with all changes in the working tree (no commits made, per the harness protocol).

## Final report — phase 125 `image_anchor_context` (verification pass)

All 6 tasks were already complete; this pass verified every criterion and ran the full gates:

- **Migration 0023** `documents.image_context` (TEXT nullable, reversible) — applied (`alembic current` = 0023 head); round-trip test (4 passed) exercises real upgrade → downgrade 0022 → upgrade.
- **Criterion 1** (referenced image enriched + context line in retrieval): integration `test_markdown_referenced_image_gets_context_and_enriched_chunks` + E2E `test_sync_enriches_the_image_doc_with_markdown_context`, `test_chat_answer_quotes_the_context_line` (wire prompt pinned: `<document>` block leads with the one-line `Referenced in …: "alt"` line) — verified present and passing.
- **Criterion 2** (standalone image byte-identical): E2E `test_standalone_*` (NULL `image_context`, block body exactly the bare description, no `Referenced in` anywhere in the prompt) + unit `test_describe_image_without_context_is_byte_identical` — passing.
- **Criterion 3** (markdown-first: record stored on markdown doc, image picks up context later): integration `test_markdown_indexed_before_image_stores_record_and_image_picks_it_up` + post-index enrichment pass test (markdown file gone from disk) — passing.
- **Gates:** `uv run pytest` → 2938 passed, exit 0 · `uv run pytest --cov=app --cov-report=term-missing` → exit 0, TOTAL **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings.
- **Phase E2E in isolation:** `uv run pytest tests/e2e/test_image_anchor_context.py -v --no-cov` → 4 passed.
- **Mock-LLM change regression check:** neighboring image E2E suites green in isolation — `test_smoke` (3), `test_image_documents` (16), `test_chat_image_questions` (17), `test_answer_images` (17).
- No defects found; no code changes needed. No live-infrastructure changes (local dev DB only). No commit made (harness commits).
- Next pending phase: check `.agents/phases/todo/` after the harness moves this phase.
This commit is contained in:
ducoterra
2026-09-27 05:53:12 -04:00
parent 62e4f6b572
commit e0307fe3ff
60 changed files with 6583 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/rag/importer.py
+592 -64
View File
@@ -85,6 +85,40 @@ summary (stored verbatim, no ``lite`` call, no pointer line), so the
``is_summary`` position −1 chunk mirrors ``Document.summary``, which
equals ``Document.content``.
Image-anchor context (phase 125, task 03, LOCKED A1/A2/A3): BEFORE the
description, the same source's importable markdown files are scanned for
``![alt](path)`` references that name the image (the task-02
:func:`app.rag.md_images.extract_image_refs`); when found, the combined
context string (``Referenced in <source>/<md>:`` + alt text + surrounding
paragraphs, see :func:`_markdown_image_context`) is (a) passed to the
vision call as ``context`` (appended AFTER the locked describe prompt —
the model learns what the image is for) and (b) stored on the doc's
``image_context`` (the retrieval display's context line, task 05), and
the stored ``content`` is ``<context>\n\n<description>`` (context first
for retrieval; ``summary`` mirrors it). A standalone image (no markdown
reference) gets ``image_context = NULL`` and the bare description —
byte-identical to pre-phase-125.
Markdown-FIRST image context (phase 125, task 04, LOCKED A1): a
markdown file (re)indexed while a reference's image has NO document in
the source yet records those references on ITS OWN ``image_context``
(the ``IMAGE_CONTEXT:`` record — :func:`_format_missing_image_refs`;
references are resolved exactly as the task-03 scan, and a reference
escaping the source is not recorded). A reference that DOES name an
existing image doc records nothing (the task-03 path owns that case —
enrichment at image-index time — and the markdown doc's field stays
NULL). After ALL of a source's files are indexed, the post-index
enrichment pass (:func:`_enrich_pending_images`) enriches the still-
``image_context IS NULL`` image docs that a markdown doc's record
references — the case where the markdown was indexed first and its
on-disk file is gone by the time the image is indexed (the task-03
disk scan cannot see it) — with the task-03 mechanics (re-describe
with context, ``content = <context>\n\n<description>``, the context
stored, re-chunk + re-embed). The pass is idempotent (an image doc
with ``image_context`` set is never touched; a markdown doc's record
is NOT cleared on success — it is the retrieval display's record
also, task 05).
``import_sources`` accepts an optional per-file ``progress`` callback
(phase 64, task 01) reporting the file being processed right now.
"""
@@ -92,11 +126,13 @@ from __future__ import annotations
import hashlib
import logging
import posixpath
import re
import uuid
from collections.abc import Callable
from collections.abc import Callable, Sequence
from dataclasses import dataclass, field
from datetime import UTC, datetime
from pathlib import Path
from pathlib import Path, PurePosixPath
from typing import Any, Protocol
from sqlalchemy import select
@@ -108,6 +144,7 @@ from app.models import Chunk, Document
from app.rag.chunker import chunk_document, extract_title
from app.rag.doc_dates import file_mtime_datetime, normalize_doc_date
from app.rag.llm import EmbeddingError, LLMError
from app.rag.md_images import ImageRef, extract_image_refs
from app.rag.summarizer import (
IMAGE_FALLBACK_MIME,
IMAGE_MIMES,
@@ -122,6 +159,12 @@ EXCLUDED_DIRS: frozenset[str] = frozenset(
{".venv", "node_modules", ".git", "__pycache__", ".pytest_cache", "dist", "build"}
)
#: Markdown extensions that can carry ``![alt](path)`` image references
#: (phase 125, task 03) — the two markdown tokens of the A9 family. The
#: scan (see :func:`_markdown_image_context`) only reads these, and only
#: among the source's importable files.
_MARKDOWN_EXTENSIONS: frozenset[str] = frozenset({".md", ".markdown"})
class Embedder(Protocol):
"""Everything the importer needs from the LLM client (duck-typed for tests)."""
@@ -409,6 +452,18 @@ async def import_sources(
file normally (row + ``image_dir`` copy); with the toggle OFF the
prune skips ``is_image`` docs (the prune guard — the image is
invisible to the walk, not a deleted file).
Markdown-FIRST image context (phase 125, task 04): after ALL of a
source's files are indexed, the post-index enrichment pass
(:func:`_enrich_pending_images`) runs for that source — it enriches
the still-unenriched image docs (``image_context IS NULL``) that a
markdown doc's ``IMAGE_CONTEXT:`` record references (the case where
the markdown was indexed first and its on-disk file is gone by the
time the image is indexed — the task-03 disk scan cannot see it).
The pass is driven by committed doc rows, so it is safe under
``limit`` too (it deletes nothing), and a per-image failure
(missing copy, failed re-description, embed error) keeps that image
as-is and moves on.
"""
if limit is not None and limit <= 0:
raise ValueError("limit must be >= 1")
@@ -496,6 +551,14 @@ async def import_sources(
session.rollback()
summary.errors += 1
logger.error("import: error source=%s path=%s — %s", source, rel, e)
# Phase 125 (task 04): the post-index enrichment pass — the
# markdown-FIRST case (a markdown indexed while its
# referenced image had no document yet recorded the
# reference on its doc row; the image was just indexed, but
# the task-03 disk scan could not see the reference — the
# markdown file is no longer on disk). Idempotent +
# fail-soft (see the function).
await _enrich_pending_images(session, source=source, llm=llm, summary=summary)
if prune:
if limit is not None:
logger.warning("import: --prune ignored because --limit was given")
@@ -511,6 +574,50 @@ async def import_sources(
session.close()
async def _replace_content_chunks(
session: Session, *, doc: Document, llm: Embedder, content: str, rel: str
) -> int:
"""Phase 1+2 on *content*: chunk it (format-aware, PLAN §5), replace
the doc's content chunk rows (embeddings NULL), and embed the new
chunks in batches — returning the chunk count. The caller owns the
transaction (it commits, or rolls back on the :class:`EmbeddingError`
that propagates when the token-cap retry gives up).
The endpoint token-cap retry (halve this file's chunk target, floor
400 — URL-dense paragraphs tokenize at ~1.1 chars/token; the global
policy stays intact for the rest of the KB) is the house rule, shared
verbatim by the text path (:func:`_index_file`), the image path
(task 03), and the task-04 enrichment pass — one copy, three call
sites.
"""
settings = llm.settings
target = max(400, settings.chunk_target_chars)
chunks_text: list[str] = []
while True:
chunks_text = chunk_document(content, rel, target, settings.chunk_overlap_chars)
doc.chunks = [
Chunk(document_id=doc.id, position=i, content=c) for i, c in enumerate(chunks_text)
]
session.flush() # delete-orphan cascade drops the previous rows
if not doc.chunks:
break
try:
vectors = await llm.embed([c.content for c in doc.chunks])
for row, vec in zip(doc.chunks, vectors, strict=True):
row.embedding = vec
break
except EmbeddingError as e:
if "token cap" not in str(e) or target <= 400:
raise
logger.info(
"import: re-chunking at %d chars after endpoint token cap: %s",
target // 2,
rel,
)
target //= 2
return len(chunks_text)
async def _index_file(
session: Session,
*,
@@ -627,42 +734,34 @@ async def _index_file(
session.flush() # guarantees doc.id even for brand-new rows
# Phase 125 (task 04, LOCKED A1): a markdown file's image-reference
# bookkeeping — references whose image has NO document yet in this
# source are recorded on the doc's ``image_context`` (the
# ``IMAGE_CONTEXT:`` record) so the post-index enrichment pass picks
# them up when the image is later indexed; references that DO name an
# existing image doc record nothing (the task-03 path owns that case
# — the field stays NULL). The record rides the file's own
# transaction (committed by the ``session.commit()`` below).
if full_path.suffix.lower() in _MARKDOWN_EXTENSIONS:
doc.image_context = _missing_image_context(
session, source=source, md_rel=rel, content=content
)
# Phase 1+2 — chunk, replace the chunk rows (embeddings NULL), embed,
# and commit the whole file atomically. If the endpoint rejects a chunk
# as over its input token cap (URL-dense paragraphs tokenize at ~1.1
# chars/token), halve this file's chunk target and retry — the global
# policy stays intact for the rest of the KB.
target = max(400, settings.chunk_target_chars)
while True:
chunks_text = chunk_document(content, rel, target, settings.chunk_overlap_chars)
doc.chunks = [
Chunk(document_id=doc.id, position=i, content=c) for i, c in enumerate(chunks_text)
]
session.flush() # delete-orphan cascade drops the previous rows
if not doc.chunks:
break
try:
vectors = await llm.embed([c.content for c in doc.chunks])
for row, vec in zip(doc.chunks, vectors, strict=True):
row.embedding = vec
break
except EmbeddingError as e:
if "token cap" not in str(e) or target <= 400:
raise
logger.info(
"import: re-chunking at %d chars after endpoint token cap: %s",
target // 2,
rel,
)
target //= 2
# and commit the whole file atomically (the endpoint token-cap retry
# lives in the shared helper — the global policy stays intact for the
# rest of the KB).
chunks_n = await _replace_content_chunks(
session, doc=doc, llm=llm, content=content, rel=rel
)
session.commit()
if verb == "added":
summary.added += 1
else:
summary.updated += 1
summary.chunks += len(chunks_text)
logger.info("import: %s source=%s path=%s chunks=%d", verb, source, rel, len(chunks_text))
summary.chunks += chunks_n
logger.info("import: %s source=%s path=%s chunks=%d", verb, source, rel, chunks_n)
# Phase 30; phase 118 (A2, 2026-09-15): EVERY new/changed document
# gets a ``lite``-model summary — markdown included. Phase 30's
@@ -756,8 +855,409 @@ async def _store_summary(
logger.error("import: summary failed source=%s path=%s — %s", source, rel, e)
def _source_root_for(full_path: Path, rel: str) -> Path:
"""The source root of one walked file (phase 125, task 03).
The walk's invariant is ``full_path == root / rel`` (``rel`` is the
source-relative POSIX path, ``full_path`` the absolute path), so the
root is *full_path*'s parent walked UP by the number of DIRECTORY
components of *rel* (a top-level file has none — its own directory
IS the root).
"""
root = full_path.parent
for _ in PurePosixPath(rel).parts[:-1]:
root = root.parent
return root
def _resolve_ref_path(ref_path: str, md_rel: str) -> str | None:
"""One ``![alt](path)`` reference path → the image's source-relative
POSIX path, or ``None`` when the reference names no in-source file
(phase 125 — the single resolution rule shared by the task-03
context scan, the task-04 markdown bookkeeping, and the task-04
post-index enrichment pass).
A leading ``/`` = the source root (a root-absolute doc-style
reference); otherwise the path is resolved against the markdown
file's directory (``md_rel`` is the markdown's source-relative
POSIX path — its ``dirname``, so a top-level markdown joins against
the root). The result is normalized; a normalized path that still
escapes (``..``-prefixed — e.g. ``../../other/x.png``) or names no
file (``.``) matches no image document in this source, ever.
"""
if ref_path.startswith("/"):
resolved = ref_path.lstrip("/")
else:
resolved = posixpath.join(posixpath.dirname(md_rel), ref_path)
normalized = posixpath.normpath(resolved)
if normalized in ("", ".") or normalized.startswith(".."):
return None
return normalized
def _ref_points_at(ref: ImageRef, md_rel: str, image_rel: str) -> bool:
"""Does *ref* point at the image at *image_rel* (phase 125, task 03)?
The reference path is resolved against the markdown file's directory
(``md_rel`` is the markdown's source-relative POSIX path) — or,
when it starts with ``/``, against the SOURCE ROOT (a root-absolute
doc-style reference) — then normalized and compared for equality
with the image's source-relative path. Handles both the bare
relative reference (``pic.png`` next to the markdown) and the
folder-carrying one (``folder/pic.png`` from the root); a reference
to a different file that shares the filename is NOT a match.
"""
resolved = _resolve_ref_path(ref.path, md_rel)
return resolved is not None and resolved == image_rel
def _markdown_image_context(
root: Path,
llm: Embedder,
*,
source: str,
image_rel: str,
) -> str | None:
"""The combined markdown reference context for one image (phase 125,
task 03, LOCKED A1/A2) — or ``None`` when no markdown file in the
source references it (the standalone-image case: the whole phase-125
machinery is a no-op, byte-identical to pre-phase).
Scans the source's importable MARKDOWN files (``.md``/``.markdown``
under the source's import-extension set — hidden/excluded-dir rules
and the extension filter apply exactly as to the walk) and keeps
every :func:`~app.rag.md_images.extract_image_refs` reference that
POINTS AT this image (:func:`_md_ref_block` resolves each reference
via :func:`_resolve_ref_path` — ``pic.png`` from ``guide.md`` →
``pic.png``; ``../pic.png`` from ``sub/guide.md`` → ``pic.png``;
``folder/pic.png`` from ``guide.md`` → ``folder/pic.png``), one
block per referencing markdown file in the walk's sorted order.
The result is both the *context* argument of the vision call (the
model learns WHAT the image is for) and the stored
``image_context`` (the retrieval display's context line, task 05).
"""
blocks: list[str] = []
for md_path in iter_importable_files(root, llm.settings.import_extension_set):
if md_path.suffix.lower() not in _MARKDOWN_EXTENSIONS:
continue
try:
content = md_path.read_text(encoding="utf-8", errors="replace").replace("\x00", "")
except OSError:
continue # unreadable source file — no context from it
block = _md_ref_block(source, md_path.relative_to(root).as_posix(), content, image_rel)
if block is not None:
blocks.append(block)
return "\n\n".join(blocks) if blocks else None
def _md_ref_block(source: str, md_rel: str, content: str, image_rel: str) -> str | None:
"""One ``Referenced in <source>/<md>:`` block for one markdown's
content (phase 125) — the shared builder of the task-03 disk scan
(:func:`_markdown_image_context`) and the task-04 pass
(:func:`_pending_image_context`).
Keeps every image reference in *content* that POINTS AT *image_rel*
(the resolution rule: :func:`_resolve_ref_path` — a reference to a
different file that merely shares the filename never matches,
LOCKED A1), in line order, rendered as::
Referenced in <source>/<markdown_path>:
- Alt text: "<alt_text>"
Context: <context_paragraphs>
(the ``Context:`` line is omitted for a reference with no
surrounding paragraphs). ``None`` when *content* has no reference
to this image.
"""
refs = [ref for ref in extract_image_refs(content) if _ref_points_at(ref, md_rel, image_rel)]
if not refs:
return None
lines = [f"Referenced in {source}/{md_rel}:"]
for ref in refs:
lines.append(f'- Alt text: "{ref.alt_text}"')
if ref.context:
lines.append(f" Context: {ref.context}")
return "\n".join(lines)
#: Header line of a markdown doc's pending-image record (phase 125,
#: task 04, LOCKED A1) — the stored ``image_context`` of a markdown
#: file that references images with no document yet in the source.
#: :func:`_parse_missing_image_refs` is its parse target (the record is
#: the post-index enrichment pass's reference index; the header is
#: ignored by the parser — the bullets are the data).
IMAGE_CONTEXT_HEADER = "IMAGE_CONTEXT:"
def _format_missing_image_refs(refs: Sequence[ImageRef], source: str) -> str:
"""One markdown's missing-image references → the ``IMAGE_CONTEXT:``
record stored on the markdown doc's ``image_context`` (phase 125,
task 04, LOCKED A1) — the durable record that "this markdown
referenced these images, which have no document yet in this
source":
.. code-block:: none
IMAGE_CONTEXT:
- [image.png] (line 42): "Alt text here" — 2 paragraphs of context
- [diagram.jpg] (line 100): "" — 1 paragraph of context
One bullet per reference, in reference order: the RESOLVED
source-relative image path in the brackets (the caller resolves via
:func:`_resolve_ref_path` — the post-index pass matches it against
the image doc's ``path``, so the resolution must agree with the
task-03 scan's), the 1-based line number, the alt text between the
quotes (``""`` for ``![](path)``), and the count of the reference's
context paragraphs (the task-02 window — the paragraphs themselves
live on the markdown doc's ``content``; the pass re-extracts them:
the record is the index, the content is the source). The shape is
exactly what :func:`_parse_missing_image_refs` parses back (the
"task 04b" parse target). *source* stays on the signature so the
call site reads like the record's home (the record rides a doc row
whose source is already known — display context, not data).
Empty *refs* → ``""`` (the caller stores ``None`` for "no record"
— an empty record would be a non-NULL empty string, and the pass's
``IS NOT NULL`` gate must see "no record" as NULL, not ``""``).
"""
if not refs:
return ""
lines = [IMAGE_CONTEXT_HEADER]
for ref in refs:
n = len(ref.context.split("\n\n")) if ref.context else 0
unit = "paragraph" if n == 1 else "paragraphs"
lines.append(
f'- [{ref.path}] (line {ref.line_number}): "{ref.alt_text}"'
f" — {n} {unit} of context"
)
return "\n".join(lines)
#: One bullet of the ``IMAGE_CONTEXT:`` record — the exact output shape
#: of :func:`_format_missing_image_refs`: the resolved image path in
#: the brackets, the 1-based line number, the alt text between the
#: quotes, the context-paragraph count.
_MISSING_REF_LINE_RE = re.compile(
r'^- \[(?P<path>[^\]]+)\] \(line (?P<line>\d+)\): "(?P<alt>.*)" — '
r"(?P<count>\d+) paragraph(?:s)? of context$"
)
def _parse_missing_image_refs(text: str) -> list[tuple[str, int, str]]:
"""Parse a markdown doc's ``image_context`` record back into
``(image path, line number, alt text)`` triples (phase 125, task 04
— the enrichment pass's reference index).
Only bullets in :func:`_format_missing_image_refs`' exact shape
parse (the greedy alt field still pins an alt that itself contains
``"``, as long as it does not contain the ``" — N paragraph…"``
tail): the header line, an owner-edited value, or a future shape
yields no references and the pass simply finds no match (fail-soft
— a malformed record must never raise into the sync).
"""
refs: list[tuple[str, int, str]] = []
for line in text.splitlines():
m = _MISSING_REF_LINE_RE.match(line)
if m is not None:
refs.append((m.group("path"), int(m.group("line")), m.group("alt")))
return refs
def _missing_image_context(
session: Session, *, source: str, md_rel: str, content: str
) -> str | None:
"""The markdown doc's ``image_context`` for one (re)indexed markdown
file (phase 125, task 04, LOCKED A1): the ``IMAGE_CONTEXT:`` record
of every ``![alt](path)`` reference whose image has NO ``is_image``
document yet in this source, or ``None`` when every in-source
reference names an existing image doc (the task-03 path owns those
— enrichment at image-index time — and the field stays NULL) or the
file has no references at all.
Each reference is resolved exactly as the task-03 scan
(:func:`_resolve_ref_path`); a reference that escapes the source
(e.g. ``../../other/x.png``) names no in-source image document, ever,
and is not recorded (the field is the pending-enrichment index, not
a general reference dump). A re-indexed markdown recomputes the
field from its CURRENT content — the image now exists (or the
reference was dropped) → the field drops back to ``None``.
"""
refs = extract_image_refs(content)
if not refs:
return None
existing = set(
session.scalars(
select(Document.path).where(
Document.source == source, Document.is_image.is_(True)
)
).all()
)
missing: list[ImageRef] = []
for ref in refs:
resolved = _resolve_ref_path(ref.path, md_rel)
if resolved is None or resolved in existing:
continue
missing.append(
ImageRef(
path=resolved,
alt_text=ref.alt_text,
context=ref.context,
line_number=ref.line_number,
)
)
if not missing:
return None
return _format_missing_image_refs(missing, source)
def _pending_image_context(
source: str, image_rel: str, md_docs: Sequence[Document]
) -> str | None:
"""The task-04 pass's context for one pending image doc: the SAME
combined string the task-03 scan builds (``Referenced in …`` blocks
joined with a blank line) — but sourced from the markdown DOC ROWS
instead of the on-disk files: a markdown doc contributes only when
its stored ``image_context`` record lists the image's source-relative
path (the durable "this markdown referenced this image" record,
work 1), and the reference (alt text + surrounding paragraphs) is
re-extracted from the doc's stored ``content`` (the full text is on
the row — the record is the index, the content is the source).
``None`` when no markdown doc references the image.
"""
blocks: list[str] = []
for md in md_docs:
listed = _parse_missing_image_refs(md.image_context or "")
if image_rel not in {path for path, _, _ in listed}:
continue
block = _md_ref_block(source, md.path, md.content, image_rel)
if block is not None:
blocks.append(block)
return "\n\n".join(blocks) if blocks else None
async def _enrich_pending_images(
session: Session, *, source: str, llm: Embedder, summary: ImportSummary
) -> None:
"""The phase-125 task-04 post-index enrichment pass — once per
source, after ALL of its files are indexed (:func:`import_sources`).
Closes the markdown-FIRST case: a markdown file indexed while its
referenced image had no document yet stores the reference on its
``image_context`` (work 1 — the ``IMAGE_CONTEXT:`` record). When the
image is later indexed but the task-03 disk scan could not see the
reference (the markdown file no longer on disk — a deleted / moved
source file whose doc row survives a ``prune=False`` run — or any
other disk-vs-DB drift), the image doc is still ``image_context IS
NULL`` and this pass enriches it from the markdown doc's STORED rows
(see :func:`_pending_image_context`).
The enrichment is the task-03 mechanics on the existing doc: a fresh
context-aware description (``describe_image`` with the context — the
description remains the primary content, LOCKED A3), ``content`` =
``<context>\\n\\n<description>`` (and so the ``summary`` — the
phase-122 ``summary = content`` convention), the context stored on
``image_context``, and the re-chunked + re-embedded pipeline (the
re-describe reads the image's PERSISTENT copy — the source file may
be the one that is gone).
Idempotent by contract (ASSUMPTION 4): an image doc with
``image_context`` SET is never touched (already enriched — the
task-03 path or an earlier pass), and a markdown doc's
``image_context`` is NOT cleared on success (it stays the retrieval
display's record, task 05). Fail-soft: a missing persistent copy, a
failed/empty re-description, or an embed error keeps the image doc
as-is (warned / counted) and the next sync retries it; one bad image
never blocks the others. A success counts ``updated`` + ``chunks``
(the doc's content really changed — the caller's KB-change gate and
the sources-version bump both fire on ``added + updated``, which is
correct here) and a dedicated ``image-context-enriched`` log line
(PLAN §9).
"""
pending = session.scalars(
select(Document).where(
Document.source == source,
Document.is_image.is_(True),
Document.image_context.is_(None),
)
).all()
if not pending:
return
md_docs = session.scalars(
select(Document).where(
Document.source == source,
Document.is_image.is_(False),
Document.image_context.is_not(None),
)
).all()
if not md_docs:
return
for image_doc in pending:
context = _pending_image_context(source, image_doc.path, md_docs)
if context is None:
continue # no markdown doc records a reference to this image
if not image_doc.image_path or not Path(image_doc.image_path).is_file():
# No persistent bytes to re-describe — keep the doc as-is
# (the next sync retries; a copy gone for good leaves an
# unenrichable doc, and the warning is the signal).
summary.images_failed += 1
logger.warning(
"import: image-context enrichment skipped (no persistent copy) "
"source=%s path=%s",
source, image_doc.path,
)
continue
copy_path = Path(image_doc.image_path)
description = await _describe_or_skip(
llm,
data=copy_path.read_bytes(),
source=source,
rel=image_doc.path,
full_path=copy_path,
context=context,
)
if description is None:
summary.images_failed += 1
logger.warning(
"import: image re-description failed source=%s path=%s",
source, image_doc.path,
)
continue
try:
image_doc.image_context = context
image_doc.content = f"{context}\n\n{description}"
chunks_n = await _replace_content_chunks(
session, doc=image_doc, llm=llm,
content=image_doc.content, rel=image_doc.path,
)
session.commit()
except EmbeddingError as e:
session.rollback()
summary.errors += 1
logger.error(
"import: image-context enrichment failed source=%s path=%s — %s",
source, image_doc.path, e,
)
continue
summary.updated += 1
summary.chunks += chunks_n
logger.info(
"import: image-context-enriched source=%s path=%s chunks=%d",
source, image_doc.path, chunks_n,
)
await _store_summary(
session, doc=image_doc, source=source, rel=image_doc.path,
content=image_doc.content, llm=llm, summary=summary,
)
async def _describe_or_skip(
llm: Embedder, *, data: bytes, source: str, rel: str, full_path: Path
llm: Embedder,
*,
data: bytes,
source: str,
rel: str,
full_path: Path,
context: str | None = None,
) -> str | None:
"""Phase 122 — the SINGLE seam for the image description (task 03).
@@ -777,9 +1277,17 @@ async def _describe_or_skip(
description error). ``source``/``rel`` stay on the signature so the
caller (and the patch) reads like the document being described;
the failure's doc identity is logged by the caller's warning.
``context`` (phase 125, task 03) is the optional markdown reference
context (:func:`_markdown_image_context`), forwarded verbatim to
:func:`app.rag.summarizer.describe_image` — appended AFTER the
locked describe prompt so the vision model can write a
context-aware description (LOCKED A3: the description remains the
primary content). ``None`` (standalone image) keeps the message
byte-identical to pre-phase-125.
"""
mime = IMAGE_MIMES.get(full_path.suffix.lower(), IMAGE_FALLBACK_MIME)
return await describe_image(llm, data=data, mime=mime)
return await describe_image(llm, data=data, mime=mime, context=context)
def _delete_image_copy(image_path: str | None) -> None:
@@ -833,6 +1341,20 @@ async def _index_image_file(
Fail-soft (LOCKED A3): a failed/empty description SKIPS the doc
entirely (no row, no copy) — ``summary.images_failed`` + a warning,
the sync continues.
Phase 125 (task 03, LOCKED A1/A2/A3): a NEW/CHANGED image first
scans the same source's markdown files for references to it (see
:func:`_markdown_image_context`). When one or more exist, the
combined context string goes to the vision call (``context`` — the
model writes a context-aware description; the description remains
the primary content) and is stored on the doc's ``image_context``;
the doc's ``content`` becomes ``<context>\n\n<description>`` (and so
does ``summary`` — the phase-122 ``summary = content`` convention —
as do the embedded chunks). No reference → ``image_context`` stays
NULL and ``content`` is the bare description (byte-identical to
pre-phase). The UNCHANGED path never re-scans (the stored context
is part of the row's committed content; a changed source state
re-indexes the file through this path).
"""
settings = llm.settings
data = full_path.read_bytes()
@@ -880,12 +1402,24 @@ async def _index_image_file(
return
verb = "updated" if doc is not None else "added"
# Phase 125 (task 03, LOCKED A1/A2): BEFORE the description — the
# markdown reference context for this image (the alt text +
# surrounding paragraphs of the ``![alt](path)`` references that
# name it in the same source), or ``None`` (standalone image — the
# whole phase-125 enrichment below is a no-op, byte-identical to
# pre-phase). It goes to the vision model (a context-aware
# description, LOCKED A3) AND is stored on the doc (``image_context``
# — the retrieval display's context line, task 05).
image_context = _markdown_image_context(
_source_root_for(full_path, rel), llm, source=source, image_rel=rel
)
# The description is the doc's content (task 03: and its summary) —
# it is generated BEFORE anything is written, so a failure skips the
# doc with no row and no copy (the copy is only made after a
# successful description — a failure never leaves an orphan).
content = await _describe_or_skip(
llm, data=data, source=source, rel=rel, full_path=full_path
llm, data=data, source=source, rel=rel, full_path=full_path,
context=image_context,
)
if content is None:
# LOCKED A3 fail-soft: an undescribed image is unsearchable
@@ -893,6 +1427,13 @@ async def _index_image_file(
summary.images_failed += 1
logger.warning("import: image description failed source=%s path=%s", source, rel)
return
if image_context is not None:
# Phase 125 (task 03): the enriched content — the reference
# context FIRST (for retrieval), then the description. The
# ``summary`` field mirrors this (the phase-122 image
# convention: ``summary = content``), and the chunk pipeline
# below embeds it.
content = f"{image_context}\n\n{content}"
# The persistent copy: uploads are replaced on every upload, git
# checkouts are re-cloned, local dirs are user-edited — the served
@@ -925,6 +1466,10 @@ async def _index_image_file(
created_at=normalize_doc_date(raw_date),
is_image=True,
image_path=str(copy_path),
# Phase 125 (task 03, LOCKED A1): the referencing markdown's
# alt text + surrounding paragraphs (``None`` for a
# standalone image — the pre-phase row shape).
image_context=image_context,
)
session.add(doc)
else:
@@ -940,45 +1485,28 @@ async def _index_image_file(
doc.created_at_manual = False
doc.is_image = True
doc.image_path = str(copy_path)
# Phase 125 (task 03): a re-indexed image re-scans its markdown
# references — the context follows the current source state
# (a deleted referencing markdown drops it back to ``None``).
doc.image_context = image_context
session.flush() # guarantees doc.id even for brand-new rows
# Phase 1+2 — the UNCHANGED pipeline on the description: chunk,
# replace the chunk rows (embeddings NULL), embed, and commit the
# whole file atomically (one transaction per file). The token-cap
# retry loop is copied from the text path; a description is short,
# so it never fires in practice.
target = max(400, settings.chunk_target_chars)
while True:
chunks_text = chunk_document(content, rel, target, settings.chunk_overlap_chars)
doc.chunks = [
Chunk(document_id=doc.id, position=i, content=c) for i, c in enumerate(chunks_text)
]
session.flush() # delete-orphan cascade drops the previous rows
if not doc.chunks:
break
try:
vectors = await llm.embed([c.content for c in doc.chunks])
for row, vec in zip(doc.chunks, vectors, strict=True):
row.embedding = vec
break
except EmbeddingError as e:
if "token cap" not in str(e) or target <= 400:
raise
logger.info(
"import: re-chunking at %d chars after endpoint token cap: %s",
target // 2,
rel,
)
target //= 2
# Phase 1+2 — the same pipeline on the description (chunk, replace
# the chunk rows (embeddings NULL), embed, commit — one transaction
# per file; the endpoint token-cap retry lives in the shared helper,
# and a description is short, so it never fires in practice).
chunks_n = await _replace_content_chunks(
session, doc=doc, llm=llm, content=content, rel=rel
)
session.commit()
if verb == "added":
summary.added += 1
else:
summary.updated += 1
summary.chunks += len(chunks_text)
logger.info("import: %s source=%s path=%s chunks=%d", verb, source, rel, len(chunks_text))
summary.chunks += chunks_n
logger.info("import: %s source=%s path=%s chunks=%d", verb, source, rel, chunks_n)
# Phase 30 shape on the description — image-aware (task 03, LOCKED
# A3): the description IS the summary (stored verbatim, no