phase: 122_image_documents
Build and Push Containers / build-and-push-app (push) Successful in 1m57s
Build and Push Containers / build-and-push-db (push) Failing after 13s

**Phase 122 (image documents) — final verification pass: all green. No code changes were needed; defects found: none.**

**Verified (implementation already complete in working tree, reviewed end-to-end):**
- Toggle (`BOR_IMAGES`/`BOR_IMAGE_EXTENSIONS`/`BOR_IMAGE_DIR`, off by default) + `GET /api/config` `images` flag
- Ingest: bytes digest, `image_dir` persistent copy, `content = summary = vision description` (chat-model call; only text embedded), fail-soft skip + `images_failed` counter
- Serve/display: `/api/documents/{id}/image` route (404 matrix), viewer `<img>` + description, Sources 48px lazy thumbnails, chat inline source figure (alt = summary), agent `read` marker
- Prune guard: images-off syncs never prune `is_image` docs

**Test / lint / coverage (exact commands & outcomes):**
- `uv run pytest` → exit 0 (green; note: pytest 9.1.1 `-q` omits the final count line in output — exit code authoritative)
- `uv run pytest --cov=app --cov-report=term-missing` → **2715 passed, exit 0, TOTAL 99%** (>90% gate)
- `uv run ruff check . && uv run pyright` → "All checks passed!" / "0 errors, 0 warnings, 0 informations"
- `uv run pytest tests/e2e/test_image_documents.py -v --no-cov` → **4 passed, exit 0** (isolation)

**Completion criteria:** (1) images=true → described/embedded/displayed docs: ✅ (E2E + integration) · (2) images=false byte-identical + image docs survive sync: ✅ (E2E negative app + unit/integration) · (3) viewer + chat rendering with alt text; failed description skips + logs, sync completes: ✅ · (4) test/lint/coverage gates: ✅ · (5) commit + phase move: deferred to harness per this pass's rules (working tree left uncommitted).

**Notable deviation (pre-existing, documented in code):** image route uses `require_user` (phase-79 posture, same gate as the document content endpoint) rather than the phase text's "public" parenthetical — matches the endpoint it mirrors.

**Next pending phase:** `123_chat_image_questions`.
This commit is contained in:
2026-09-25 01:54:23 -04:00
parent 0f77e9a876
commit a19d78d284
63 changed files with 5484 additions and 111 deletions
@@ -0,0 +1,30 @@
# Task 02 — Image ingest: model fields, walk filter, binary index path, prune guard
**Phase:** `122_image_documents` · **Source:** `TODO.md:6` — "Images uploaded as part of documents or as standalone images should be read, summarized, and retrieved like any other document."
## Objective
When `BOR_IMAGES=true`, standalone image files in ANY ingest path (direct upload, uploaded archive, git/local source walk) become `Document` rows — bytes persisted to `image_dir`, `is_image`/`image_path` set, `content` = the vision description (task 03) — and an images-off sync never prunes existing image docs.
## Work
1. `app/models.py` — `Document` (L102): add, with house docstrings (the `summary` L140 / `created_at_manual` precedent):
- `is_image: Mapped[bool] = mapped_column(Boolean, default=False, server_default=text("false"), nullable=False)` — True iff the doc's content is a vision description of an image (phase 122); the image bytes live at `image_path`.
- `image_path: Mapped[str | None] = mapped_column(Text, default=None)` — absolute path of the persistent copy in `settings.image_dir`; NULL for text docs.
2. `alembic/versions/` — new revision: both columns (`is_image` NOT NULL server_default 'false'; `image_path` nullable) + downgrade.
3. `app/rag/importer.py`:
- `iter_importable_files` (L235) / the walk in `import_sources` (L279): when `llm.settings.images`, accept a file iff its extension matches `import_extension_set` OR the image frozenset (task 01) — pass the image set in (the function takes explicit extension sets; the image set is NOT merged into `import_extension_set`).
- `_index_file` (L435): image branch FIRST (before the `read_text` at L453) — if the path's extension is in the image set: `data = full_path.read_bytes()`, `digest = sha256(data)`, and on new/changed: copy `data` to `image_dir/<doc-id or uuid4>.<ext>` (dir created with `mkdir(parents=True, exist_ok=True)`), set `is_image=True` + `image_path` on the `Document` row, `content` = the description (task 03's `describe_image` — this task wires the call; the function lands in task 03, so for THIS task store `content = ""` placeholder ONLY if task 03 is not yet merged — the phases run task-ordered, so in practice task 03's function exists; wire it directly and let task 03 implement it. If implementing strictly per task: this task stores `content` via a `_describe_or_skip` hook that task 03 fills — keep the seam single and commented).
- A CHANGED image (hash differs) deletes the stale `image_path` copy before replacing it.
- The unchanged/hash path (L470+) works unmodified for images (byte digest); the phase-118 summary-backfill branch (L480) re-describes an image doc whose `summary` is NULL on the next sync (same fail-soft).
- `_prune` (L649): the prune guard (LOCKED derived decision) — when `settings.images` is FALSE, skip every `is_image` doc (invisible to the walk ≠ deleted); toggle TRUE → normal prune + delete the `image_path` copy of each pruned image doc (also on the normal prune path when the file is gone).
- `ImportSummary` (L100): new `images_failed: int = 0` counter + its slot in `format_counts`/`log` (L135–151) — task 03 increments it; add it now so the log shape is stable.
4. ASSUMPTION (A4 re-stated): only standalone image FILES are ingested — no archive-of-documents extraction, no PDF/DOCX embedded-image pulls (the archive unpacker already places image members on disk; the walk now just accepts them).
## Testing & Quality
- Unit: `tests/unit/test_image_documents.py` (task 06) — the walk accepts `.png` only when `images=True` (off → ignored, the byte-identical default), the binary branch (digest over bytes, copy made, fields set, `read_text` never called for an image), the changed-image stale-copy delete, the prune guard (off → image doc survives; on + file gone → pruned + copy deleted), `images_failed` in the log line.
- Integration: `tests/integration/test_docs_api.py` (task 06) — the `import_sources` end-to-end cases.
- Coverage: **>90%** on the touched modules.
## Completion Criteria
- [ ] `alembic upgrade head` applies; a seeded image walk with `images=True` creates the doc row + `image_dir` copy; `images=False` ignores the file entirely.
- [ ] A sync with `images=False` leaves a pre-existing image doc untouched (prune guard).
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.