brain-of-reese/.agents/phases/complete/122_image_documents/02_image_ingest.md

Task 02 — Image ingest: model fields, walk filter, binary index path, prune guard

Phase: 122_image_documents · Source: TODO.md:6 — "Images uploaded as part of documents or as standalone images should be read, summarized, and retrieved like any other document."

Objective

When BOR_IMAGES=true, standalone image files in ANY ingest path (direct upload, uploaded archive, git/local source walk) become Document rows — bytes persisted to image_dir, is_image/image_path set, content = the vision description (task 03) — and an images-off sync never prunes existing image docs.

Work

1. app/models.py — Document (L102): add, with house docstrings (the summary L140 / created_at_manual precedent):
   • is_image: Mapped[bool] = mapped_column(Boolean, default=False, server_default=text("false"), nullable=False) — True iff the doc's content is a vision description of an image (phase 122); the image bytes live at image_path.
   • image_path: Mapped[str | None] = mapped_column(Text, default=None) — absolute path of the persistent copy in settings.image_dir; NULL for text docs.
2. alembic/versions/ — new revision: both columns (is_image NOT NULL server_default 'false'; image_path nullable) + downgrade.
3. app/rag/importer.py:
   • iter_importable_files (L235) / the walk in import_sources (L279): when llm.settings.images, accept a file iff its extension matches import_extension_set OR the image frozenset (task 01) — pass the image set in (the function takes explicit extension sets; the image set is NOT merged into import_extension_set).
   • _index_file (L435): image branch FIRST (before the read_text at L453) — if the path's extension is in the image set: data = full_path.read_bytes(), digest = sha256(data), and on new/changed: copy data to image_dir/<doc-id or uuid4>.<ext> (dir created with mkdir(parents=True, exist_ok=True)), set is_image=True + image_path on the Document row, content = the description (task 03's describe_image — this task wires the call; the function lands in task 03, so for THIS task store content = "" placeholder ONLY if task 03 is not yet merged — the phases run task-ordered, so in practice task 03's function exists; wire it directly and let task 03 implement it. If implementing strictly per task: this task stores content via a _describe_or_skip hook that task 03 fills — keep the seam single and commented).
     • A CHANGED image (hash differs) deletes the stale image_path copy before replacing it.
     • The unchanged/hash path (L470+) works unmodified for images (byte digest); the phase-118 summary-backfill branch (L480) re-describes an image doc whose summary is NULL on the next sync (same fail-soft).
   • _prune (L649): the prune guard (LOCKED derived decision) — when settings.images is FALSE, skip every is_image doc (invisible to the walk ≠ deleted); toggle TRUE → normal prune + delete the image_path copy of each pruned image doc (also on the normal prune path when the file is gone).
   • ImportSummary (L100): new images_failed: int = 0 counter + its slot in format_counts/log (L135–151) — task 03 increments it; add it now so the log shape is stable.
4. ASSUMPTION (A4 re-stated): only standalone image FILES are ingested — no archive-of-documents extraction, no PDF/DOCX embedded-image pulls (the archive unpacker already places image members on disk; the walk now just accepts them).

Testing & Quality

• Unit: tests/unit/test_image_documents.py (task 06) — the walk accepts .png only when images=True (off → ignored, the byte-identical default), the binary branch (digest over bytes, copy made, fields set, read_text never called for an image), the changed-image stale-copy delete, the prune guard (off → image doc survives; on + file gone → pruned + copy deleted), images_failed in the log line.
• Integration: tests/integration/test_docs_api.py (task 06) — the import_sources end-to-end cases.
• Coverage: >90% on the touched modules.

Completion Criteria

• alembic upgrade head applies; a seeded image walk with images=True creates the doc row + image_dir copy; images=False ignores the file entirely.
• A sync with images=False leaves a pre-existing image doc untouched (prune guard).
• uv run pytest green; uv run ruff check . && uv run pyright clean.