phase: 122_image_documents
**Phase 122 (image documents) — final verification pass: all green. No code changes were needed; defects found: none.**
**Verified (implementation already complete in working tree, reviewed end-to-end):**
- Toggle (`BOR_IMAGES`/`BOR_IMAGE_EXTENSIONS`/`BOR_IMAGE_DIR`, off by default) + `GET /api/config` `images` flag
- Ingest: bytes digest, `image_dir` persistent copy, `content = summary = vision description` (chat-model call; only text embedded), fail-soft skip + `images_failed` counter
- Serve/display: `/api/documents/{id}/image` route (404 matrix), viewer `<img>` + description, Sources 48px lazy thumbnails, chat inline source figure (alt = summary), agent `read` marker
- Prune guard: images-off syncs never prune `is_image` docs
**Test / lint / coverage (exact commands & outcomes):**
- `uv run pytest` → exit 0 (green; note: pytest 9.1.1 `-q` omits the final count line in output — exit code authoritative)
- `uv run pytest --cov=app --cov-report=term-missing` → **2715 passed, exit 0, TOTAL 99%** (>90% gate)
- `uv run ruff check . && uv run pyright` → "All checks passed!" / "0 errors, 0 warnings, 0 informations"
- `uv run pytest tests/e2e/test_image_documents.py -v --no-cov` → **4 passed, exit 0** (isolation)
**Completion criteria:** (1) images=true → described/embedded/displayed docs: ✅ (E2E + integration) · (2) images=false byte-identical + image docs survive sync: ✅ (E2E negative app + unit/integration) · (3) viewer + chat rendering with alt text; failed description skips + logs, sync completes: ✅ · (4) test/lint/coverage gates: ✅ · (5) commit + phase move: deferred to harness per this pass's rules (working tree left uncommitted).
**Notable deviation (pre-existing, documented in code):** image route uses `require_user` (phase-79 posture, same gate as the document content endpoint) rather than the phase text's "public" parenthetical — matches the endpoint it mirrors.
**Next pending phase:** `123_chat_image_questions`.
This commit is contained in:
+8
-5
@@ -224,6 +224,7 @@ from app.rag.retriever import (
|
||||
retrieve,
|
||||
select_related,
|
||||
select_suggested,
|
||||
source_ref_with_image,
|
||||
weak_hit_titles,
|
||||
)
|
||||
from app.rag.scaffolding import ScaffoldingFilter # phase 71: the streaming filter
|
||||
@@ -984,14 +985,16 @@ async def chat(
|
||||
# since phase 119). The UI renders the row as the
|
||||
# de-emphasized related-docs row, never a citation chip;
|
||||
# old clients ignore the field.
|
||||
# Phase 122 (task 05): both tiers build their refs
|
||||
# through the shared helper — a ref for an image doc
|
||||
# carries the optional ``image_url`` (the bytes route),
|
||||
# a text doc's stays byte-identical to pre-phase (the
|
||||
# key is omitted, never null).
|
||||
cited_refs: list[SourceRef] = []
|
||||
if not plan.deflected:
|
||||
cited_refs = [
|
||||
SourceRef(source=d.source, path=d.path, title=d.title)
|
||||
for d in cited_docs
|
||||
]
|
||||
cited_refs = [source_ref_with_image(d) for d in cited_docs]
|
||||
related_refs = [
|
||||
SourceRef(source=d.source, path=d.path, title=d.title)
|
||||
source_ref_with_image(d)
|
||||
for d in plan.related_docs
|
||||
if (d.source, d.path) not in cited_seen
|
||||
]
|
||||
|
||||
+14
-8
@@ -1,6 +1,7 @@
|
||||
"""Public app metadata (display name + version) for the frontend brand
|
||||
layer, the phase-59 docs-push flag (the "Save as doc" gating), and the
|
||||
phase-62 UI customization strings (composer placeholder, footer line).
|
||||
layer, the phase-59 docs-push flag (the "Save as doc" gating), the
|
||||
phase-122 image flag (UI affordance gating), and the phase-62 UI
|
||||
customization strings (composer placeholder, footer line).
|
||||
|
||||
Phase 91 (task 01): the three UI strings are now the EFFECTIVE values —
|
||||
the ``ui_settings`` row (admin Theme tab) over the env values (B1: DB
|
||||
@@ -29,17 +30,21 @@ router = APIRouter(tags=["config"])
|
||||
@router.get("/config")
|
||||
def app_config(settings: Settings = Depends(get_settings)) -> dict[str, str | bool]: # noqa: B008
|
||||
"""Public app metadata for the frontend brand layer (phase 39) +
|
||||
the phase-59 ``docs_repo_configured`` flag + the phase-62 UI
|
||||
customization keys (``input_placeholder``, ``footer_text``) — all
|
||||
display strings, the SAME boot fetch (no new network surface) and
|
||||
the same public posture as ``app_name`` (no secrets). Phase 91:
|
||||
the phase-59 ``docs_repo_configured`` flag + the phase-122
|
||||
``images`` flag + the phase-62 UI customization keys
|
||||
(``input_placeholder``, ``footer_text``) — all display strings,
|
||||
the SAME boot fetch (no new network surface) and the same public
|
||||
posture as ``app_name`` (no secrets). Phase 91:
|
||||
``app_name`` / ``input_placeholder`` / ``footer_text`` are the
|
||||
EFFECTIVE values (the admin Theme tab's ``ui_settings`` row over
|
||||
the env values — DB-over-env, B1); the frontend brand layer treats
|
||||
an empty string as "keep the template default" (the unset =>
|
||||
byte-identical contract). Phase 91 (task 03): the retired
|
||||
CSS-file theming's ``theme`` key is gone — the five keys are the
|
||||
entire response."""
|
||||
CSS-file theming's ``theme`` key is gone. Phase 122 (task 01):
|
||||
``images`` mirrors ``settings.images`` (the ``BOR_IMAGES`` master
|
||||
switch) — consumed by the chat composer (phase 123) to show/hide
|
||||
the image-attach control, optionally by the Sources page (an
|
||||
"images off" hint). The six keys are the entire response."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
effective = theming.effective_settings(db, settings)
|
||||
@@ -49,6 +54,7 @@ def app_config(settings: Settings = Depends(get_settings)) -> dict[str, str | bo
|
||||
"app_name": effective["app_name"],
|
||||
"version": settings.app_version,
|
||||
"docs_repo_configured": settings.docs_configured,
|
||||
"images": settings.images,
|
||||
"input_placeholder": effective["input_placeholder"],
|
||||
"footer_text": effective["footer_text"],
|
||||
}
|
||||
|
||||
+158
-17
@@ -23,6 +23,18 @@ GET /api/docs/tree — the admin's full recursive KB tree in one fetch
|
||||
walks, with the file metadata the RAG view's rows and stat cards need
|
||||
(the view drills client-side; ``GET /api/docs`` is untouched).
|
||||
|
||||
GET /api/documents/{doc_id}/image — the phase-122 (task 04) image
|
||||
BYTES route: the persistent copy behind an ``is_image`` document's
|
||||
``image_path``, served with the extension's ``Content-Type`` (the
|
||||
``app.rag.summarizer.IMAGE_MIMES`` map — one map, one truth) and a
|
||||
``Cache-Control: private, max-age=3600`` header (the bytes are
|
||||
content-hashed — long enough, bustable by re-upload). User-gated like
|
||||
the content endpoint (phase 79 — the ONLY anonymous surface is the
|
||||
shared chats): a missing doc, a non-image doc, a doc whose
|
||||
``image_path`` is NULL, or a row whose copy was lost all map to 404
|
||||
``document not found`` (the router's unknown-document shape —
|
||||
traversal/UUID-guessing has no filesystem surface to hit).
|
||||
|
||||
PATCH /api/folders/summary — the admin folder-description editor
|
||||
(phase 97, task 03): update / create / clear a stored
|
||||
``folder_summaries`` row, marking every non-empty save
|
||||
@@ -34,11 +46,13 @@ contrast with the phase-57 ``is_summary`` re-embed above.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from collections.abc import Mapping, Sequence
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi.responses import FileResponse
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
@@ -52,6 +66,7 @@ from app.rag.doc_dates import normalize_doc_date
|
||||
from app.rag.folder_summaries import MIN_DOCS_PER_FOLDER, folder_of
|
||||
from app.rag.importer import match_extension
|
||||
from app.rag.llm import EmbeddingError, LLMClient
|
||||
from app.rag.summarizer import IMAGE_FALLBACK_MIME, IMAGE_MIMES
|
||||
from app.schemas import (
|
||||
DateResult,
|
||||
DateUpdate,
|
||||
@@ -171,6 +186,11 @@ def get_document_content(
|
||||
if row is None:
|
||||
raise HTTPException(status_code=404, detail="document not found")
|
||||
doc, chunks = row
|
||||
# Phase 122 (task 04): the image affordance — ``is_image`` is ALWAYS
|
||||
# present on the wire (text docs: false — the one new key); for an
|
||||
# image doc, ``image_url`` is the bytes route's path (absent for
|
||||
# text docs, and for an image row whose copy path is NULL — never
|
||||
# null, the ``DocContent`` omission rule).
|
||||
return DocContent(
|
||||
source=doc.source,
|
||||
path=doc.path,
|
||||
@@ -181,6 +201,62 @@ def get_document_content(
|
||||
content=doc.content,
|
||||
indexed_at=doc.indexed_at.isoformat(),
|
||||
chunks=chunks,
|
||||
is_image=doc.is_image,
|
||||
image_url=(
|
||||
f"/api/documents/{doc.id}/image" if doc.is_image and doc.image_path else None
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/documents/{doc_id}/image", response_class=FileResponse)
|
||||
def get_document_image(
|
||||
doc_id: str,
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
_user: None = Depends(require_user), # noqa: B008 # phase 79 posture (see docstring)
|
||||
) -> FileResponse:
|
||||
"""The phase-122 (task 04) image BYTES route — the persistent copy
|
||||
behind an ``is_image`` document's ``image_path``.
|
||||
|
||||
Same auth posture as the document content endpoint (phase 79 —
|
||||
``require_user``: admin OR live token holder; the ONLY anonymous
|
||||
surface is the shared chats — the image is part of a document's
|
||||
content, so it travels under the same gate): anonymous callers get
|
||||
401 ``authentication required`` before any row is read.
|
||||
|
||||
404 ``document not found`` (the router's unknown-document shape) in
|
||||
every non-servable case — a missing id (an unparseable string maps
|
||||
here too, not to a 422 — a guessed id is an unknown document), a
|
||||
text doc, an image doc whose ``image_path`` is NULL, or a row whose
|
||||
copy was lost on disk (defensive — the row exists, the bytes
|
||||
don't). There is no path parameter to a filesystem value: the path
|
||||
comes from the ROW (the importer's ``image_dir`` copy), so there is
|
||||
no traversal surface.
|
||||
|
||||
Servable rows stream the exact bytes with the extension's
|
||||
``Content-Type`` (the ``IMAGE_MIMES`` map — one map, one truth with
|
||||
the describe call's data-URL mime; an unexpected extension takes
|
||||
``application/octet-stream``) and ``Cache-Control: private,
|
||||
max-age=3600`` (the bytes are content-hashed — long enough to be
|
||||
useful, bustable by re-upload).
|
||||
"""
|
||||
try:
|
||||
uid = uuid.UUID(doc_id)
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=404, detail="document not found") from None
|
||||
doc = db.scalar(select(Document).where(Document.id == uid))
|
||||
if doc is None or not doc.is_image or not doc.image_path:
|
||||
raise HTTPException(status_code=404, detail="document not found")
|
||||
image_file = Path(doc.image_path)
|
||||
if not image_file.is_file():
|
||||
# Defensive: the row exists but the copy was lost (the owner
|
||||
# cleaned the image dir, the disk was wiped) — the viewer's
|
||||
# onerror fallback renders the "image unavailable" note.
|
||||
raise HTTPException(status_code=404, detail="document not found")
|
||||
media_type = IMAGE_MIMES.get(image_file.suffix.lower(), IMAGE_FALLBACK_MIME)
|
||||
return FileResponse(
|
||||
image_file,
|
||||
media_type=media_type,
|
||||
headers={"Cache-Control": "private, max-age=3600"},
|
||||
)
|
||||
|
||||
|
||||
@@ -446,6 +522,13 @@ def _folder_counts(
|
||||
return folders, counts
|
||||
|
||||
|
||||
#: One image-docs map value (phase 122, task 04):
|
||||
#: ``(doc_id, summary)`` — the id the bytes route's URL is built from
|
||||
#: and the summary the RAG view's thumbnail uses as ``alt`` (the vision
|
||||
#: description; ``None`` = the fail-soft backfill corner).
|
||||
ImageDocInfo = tuple[str, str]
|
||||
|
||||
|
||||
def _level_children(
|
||||
source: str,
|
||||
folder: str,
|
||||
@@ -453,6 +536,7 @@ def _level_children(
|
||||
counts: dict[str, int],
|
||||
rows: Sequence[TreeFileRow],
|
||||
summaries: Mapping[tuple[str, str], str],
|
||||
images: Mapping[tuple[str, str], ImageDocInfo] | None = None,
|
||||
) -> list[KbTreeFolder | KbTreeFile]:
|
||||
"""One level's children (pure): subfolders in path order, then the
|
||||
direct files in input (catalog) order.
|
||||
@@ -477,10 +561,19 @@ def _level_children(
|
||||
= the subtree's MAX document ``created_at``: the max over this
|
||||
folder's direct files' dates and its subfolder children's (already
|
||||
recursive) ``updated_at`` values, via :func:`_subtree_max`.
|
||||
|
||||
Since phase 122 (task 04), a file node whose ``(source, path)`` is
|
||||
in *images* carries the thumbnail affordance (``is_image`` +
|
||||
``image_url`` built from the mapped doc id + the mapped ``summary``
|
||||
— see :class:`app.schemas.KbTreeFile`); every other file node is
|
||||
the pre-phase shape (the omission rule keeps its wire shape
|
||||
byte-identical).
|
||||
"""
|
||||
children: list[KbTreeFolder | KbTreeFile] = []
|
||||
for sub in sorted(g for g in folders if folder_of(g) == folder):
|
||||
sub_children = _level_children(source, sub, folders, counts, rows, summaries)
|
||||
sub_children = _level_children(
|
||||
source, sub, folders, counts, rows, summaries, images
|
||||
)
|
||||
children.append(
|
||||
KbTreeFolder(
|
||||
path=sub,
|
||||
@@ -494,15 +587,34 @@ def _level_children(
|
||||
)
|
||||
for path, title, chunks, indexed_at, created_at in rows:
|
||||
if folder_of(path) == folder:
|
||||
children.append(
|
||||
KbTreeFile(
|
||||
path=path,
|
||||
title=title,
|
||||
chunks=chunks,
|
||||
created_at=created_at,
|
||||
indexed_at=indexed_at,
|
||||
image_info = images.get((source, path)) if images else None
|
||||
if image_info is not None:
|
||||
# Phase 122 (task 04): the image-docs node — the RAG
|
||||
# view's Path cell renders the 48px thumbnail from the
|
||||
# bytes route's URL with ``alt = summary``.
|
||||
doc_id, doc_summary = image_info
|
||||
children.append(
|
||||
KbTreeFile(
|
||||
path=path,
|
||||
title=title,
|
||||
chunks=chunks,
|
||||
created_at=created_at,
|
||||
indexed_at=indexed_at,
|
||||
is_image=True,
|
||||
image_url=f"/api/documents/{doc_id}/image",
|
||||
summary=doc_summary,
|
||||
)
|
||||
)
|
||||
else:
|
||||
children.append(
|
||||
KbTreeFile(
|
||||
path=path,
|
||||
title=title,
|
||||
chunks=chunks,
|
||||
created_at=created_at,
|
||||
indexed_at=indexed_at,
|
||||
)
|
||||
)
|
||||
)
|
||||
return children
|
||||
|
||||
|
||||
@@ -530,6 +642,7 @@ def build_kb_tree(
|
||||
names: Sequence[str],
|
||||
doc_rows: Sequence[TreeDocRow],
|
||||
summaries: Mapping[tuple[str, str], str],
|
||||
images: Mapping[tuple[str, str], ImageDocInfo] | None = None,
|
||||
) -> list[KbTreeSource]:
|
||||
"""The pure tree builder behind ``GET /api/docs/tree`` (phase 97,
|
||||
task 02) — module-level and DB-free so unit tests drive it
|
||||
@@ -543,7 +656,13 @@ def build_kb_tree(
|
||||
``{(source, folder_path): summary}`` over the stored
|
||||
``folder_summaries`` rows (``folder_path = ""`` = the source root;
|
||||
rows for sources the tree does not list are simply never
|
||||
referenced).
|
||||
referenced). *images* (phase 122, task 04) —
|
||||
``{(source, path): (doc_id, summary)}`` over the stored image docs
|
||||
(``is_image`` rows with a servable ``image_path`` — the endpoint
|
||||
composes the bounded select); the default ``None``/empty map keeps
|
||||
EVERY file node the pre-phase shape (byte-identical wire — a
|
||||
pre-phase KB has no image rows, so the endpoint's own map is empty
|
||||
for it).
|
||||
|
||||
Shape, per the phase-97 ``00_phase.md`` "The tree endpoint":
|
||||
|
||||
@@ -614,10 +733,10 @@ def build_kb_tree(
|
||||
if name in listed: # defensive: list_source_names dedupes
|
||||
continue
|
||||
listed.add(name)
|
||||
tree.append(_source_node(name, by_source.get(name, ()), summaries))
|
||||
tree.append(_source_node(name, by_source.get(name, ()), summaries, images))
|
||||
for source in sorted(by_source):
|
||||
if source not in listed:
|
||||
tree.append(_source_node(source, by_source[source], summaries))
|
||||
tree.append(_source_node(source, by_source[source], summaries, images))
|
||||
return tree
|
||||
|
||||
|
||||
@@ -625,6 +744,7 @@ def _source_node(
|
||||
source: str,
|
||||
rows: Sequence[TreeFileRow],
|
||||
summaries: Mapping[tuple[str, str], str],
|
||||
images: Mapping[tuple[str, str], ImageDocInfo] | None = None,
|
||||
) -> KbTreeSource:
|
||||
"""One source node (pure): whole-source count + the source-root
|
||||
summary + the root level's children (direct subfolders + direct
|
||||
@@ -646,7 +766,7 @@ def _source_node(
|
||||
0-document source (no children, no dates).
|
||||
"""
|
||||
folders, counts = _folder_counts(rows)
|
||||
children = _level_children(source, "", folders, counts, rows, summaries)
|
||||
children = _level_children(source, "", folders, counts, rows, summaries, images)
|
||||
return KbTreeSource(
|
||||
name=source,
|
||||
documents=len(rows),
|
||||
@@ -677,9 +797,12 @@ def list_kb_tree(
|
||||
excluded — the tree has no document ids) + ALL stored
|
||||
``folder_summaries`` rows (a bounded select — one row per
|
||||
existing folder at the ≥ 1-doc rule; rows for sources the tree
|
||||
does not list are never referenced by the builder) — through the
|
||||
pure :func:`build_kb_tree`. ``GET /api/docs`` itself is
|
||||
untouched.
|
||||
does not list are never referenced by the builder) + the phase-122
|
||||
(task 04) image-docs map (a second bounded select over the
|
||||
``is_image`` rows with a servable ``image_path`` — empty for every
|
||||
pre-phase KB, so the response stays byte-identical to pre-phase)
|
||||
— through the pure :func:`build_kb_tree`. ``GET /api/docs`` itself
|
||||
is untouched.
|
||||
"""
|
||||
names = list_source_names(db)
|
||||
rows = db.execute(
|
||||
@@ -712,4 +835,22 @@ def list_kb_tree(
|
||||
select(FolderSummary.source, FolderSummary.folder_path, FolderSummary.summary)
|
||||
).all()
|
||||
}
|
||||
return KbTree(sources=build_kb_tree(names, doc_rows, summaries))
|
||||
# Phase 122 (task 04): the image-docs affordance map — a bounded
|
||||
# select over the ``is_image`` rows only (a handful of rows at KB
|
||||
# scale, never the whole catalog; the catalogue query above stays
|
||||
# byte-identical). EMPTY for every pre-phase KB (no image rows), so
|
||||
# the response stays byte-identical to pre-phase — the fields are
|
||||
# row-driven, not toggle-driven (a surviving image doc keeps its
|
||||
# thumbnail through a toggle-off sync, the prune guard's UX side).
|
||||
images: dict[tuple[str, str], tuple[str, str]] = {
|
||||
(source, path): (str(doc_id), summary)
|
||||
for source, path, doc_id, summary in db.execute(
|
||||
select(
|
||||
Document.source,
|
||||
Document.path,
|
||||
Document.id,
|
||||
Document.summary,
|
||||
).where(Document.is_image.is_(True), Document.image_path.is_not(None))
|
||||
).all()
|
||||
}
|
||||
return KbTree(sources=build_kb_tree(names, doc_rows, summaries, images))
|
||||
|
||||
@@ -366,6 +366,36 @@ class Settings(BaseSettings):
|
||||
#: pattern).
|
||||
upload_max_mb: int = 512
|
||||
|
||||
# --- Image documents (phase 122: standalone images as documents) ---
|
||||
#: Master switch for image-document indexing (phase 122,
|
||||
#: ``BOR_IMAGES``; ``0``/``false`` = off — the DEFAULT, LOCKED A3).
|
||||
#: Enable only when ``llm_chat_model`` supports vision: image
|
||||
#: descriptions are generated by the chat model, and the description
|
||||
#: is the ONLY part of an image that gets indexed (the embedding
|
||||
#: model never sees pixels). While off, the import walks ignore
|
||||
#: image files and a sync never prunes existing ``is_image``
|
||||
#: documents (the phase-122 prune guard — the image is invisible to
|
||||
#: an images-off walk, not a deleted file).
|
||||
images: bool = False
|
||||
#: Comma-separated, case-insensitive file extensions (no dot) treated
|
||||
#: as standalone images when ``images`` is on (phase 122,
|
||||
#: ``BOR_IMAGE_EXTENSIONS``). Stored as a raw CSV string (the
|
||||
#: ``import_extensions`` house convention) and parsed on demand via
|
||||
#: :py:meth:`image_extension_set`. A SEPARATE set from
|
||||
#: ``import_extension_set`` — images are never user-added via
|
||||
#: ``BOR_IMPORT_EXTENSIONS`` (the ``images`` toggle is the single
|
||||
#: knob). The validator rejects an empty list and malformed tokens,
|
||||
#: exactly like ``import_extensions`` (a typo would otherwise index
|
||||
#: zero images silently).
|
||||
image_extensions: str = "png,jpg,jpeg,webp,gif,bmp"
|
||||
#: Where ingested image bytes are copied for serving (phase 122,
|
||||
#: ``BOR_IMAGE_DIR``). Raw string — ``Path.expanduser()`` is applied
|
||||
#: by the importer, not here (the ``sources_dir``/``upload_dir``
|
||||
#: convention). Deliberately separate from ``sources_dir`` (git
|
||||
#: checkouts, re-cloned) and ``upload_dir`` (replaced on every
|
||||
#: upload): the served copy must outlive the source file.
|
||||
image_dir: str = "~/bor-sources/images"
|
||||
|
||||
# --- Docs push (phase 59: save a chat answer as documentation) ---
|
||||
#: The git repo a saved chat answer is committed to (phase 59, D3):
|
||||
#: **any** remote — a URL (``https://``, ``ssh://``, ``git@``) or a
|
||||
@@ -471,6 +501,25 @@ class Settings(BaseSettings):
|
||||
)
|
||||
return v
|
||||
|
||||
@field_validator("image_extensions")
|
||||
@classmethod
|
||||
def _image_extensions_known(cls, v: str) -> str:
|
||||
"""Reject an empty list or malformed tokens loudly (the
|
||||
``import_extensions`` precedent, phase 122): a typo like
|
||||
``png,jpeb`` would otherwise index zero images silently."""
|
||||
exts = {part.strip().lstrip(".").lower() for part in v.split(",") if part.strip()}
|
||||
if not exts:
|
||||
raise ValueError("image_extensions must name at least one format")
|
||||
malformed = sorted(
|
||||
ext for ext in exts if re.fullmatch(r"[a-z0-9]{1,16}", ext) is None
|
||||
)
|
||||
if malformed:
|
||||
raise ValueError(
|
||||
f"image_extensions contains malformed token(s): {', '.join(malformed)} — "
|
||||
"each extension must be lowercase letters/digits only, 1-16 chars, no dot"
|
||||
)
|
||||
return v
|
||||
|
||||
@field_validator("agent_max_rounds")
|
||||
@classmethod
|
||||
def _agent_max_rounds_non_negative(cls, v: int) -> int:
|
||||
@@ -647,6 +696,19 @@ class Settings(BaseSettings):
|
||||
if part.strip()
|
||||
)
|
||||
|
||||
@property
|
||||
def image_extension_set(self) -> frozenset[str]:
|
||||
"""Lowercased, dotted image-extension set (``.png``) for the
|
||||
phase-122 walk filter — SEPARATE from
|
||||
:py:attr:`import_extension_set` (images are never user-added via
|
||||
``BOR_IMPORT_EXTENSIONS``; the ``images`` toggle is the single
|
||||
knob)."""
|
||||
return frozenset(
|
||||
f".{part.strip().lstrip('.').lower()}"
|
||||
for part in self.image_extensions.split(",")
|
||||
if part.strip()
|
||||
)
|
||||
|
||||
@property
|
||||
def git_source_list(self) -> list[str]:
|
||||
"""Non-empty, stripped git URLs from :py:attr:`git_sources` (phase 28).
|
||||
|
||||
@@ -139,6 +139,29 @@ class Document(Base):
|
||||
#: failed, and until the phase-118 backfill stores one on the next
|
||||
#: sync.
|
||||
summary: Mapped[str | None] = mapped_column(Text, default=None)
|
||||
#: True iff this document is a standalone image (phase 122,
|
||||
#: LOCKED A3): ``content`` (and ``summary``) is the CHAT model's
|
||||
#: vision description of the image — the ONLY embedded text (the
|
||||
#: embedding model never sees pixels), and the image bytes
|
||||
#: themselves live at :py:attr:`image_path` (served by the document
|
||||
#: image route, task 04). ``False`` for every text document,
|
||||
#: including all pre-phase-122 rows (the server default keeps them
|
||||
#: valid without a backfill). An ``is_image`` doc is INVISIBLE to
|
||||
#: an images-off walk, not a deleted file — the importer's prune
|
||||
#: guard (the phase-122 derived decision) protects it while the
|
||||
#: toggle is off.
|
||||
is_image: Mapped[bool] = mapped_column(
|
||||
Boolean, default=False, server_default=text("false"), nullable=False
|
||||
)
|
||||
#: Absolute path of the image's PERSISTENT copy in
|
||||
#: ``settings.image_dir`` (phase 122) — the importer copies each
|
||||
#: ingested image there (``<doc-id>.<ext>``) because the source
|
||||
#: file is disposable: uploads are replaced on every upload, git
|
||||
#: checkouts are re-cloned, local dirs are user-edited. The copy is
|
||||
#: written only when the doc is new or its hash changes, deleted on
|
||||
#: a content change (the stale copy) and on prune. NULL for text
|
||||
#: documents.
|
||||
image_path: Mapped[str | None] = mapped_column(Text, default=None)
|
||||
|
||||
chunks: Mapped[list[Chunk]] = relationship(
|
||||
back_populates="document", cascade="all, delete-orphan"
|
||||
|
||||
+23
-2
@@ -427,6 +427,18 @@ READ_TRUNCATION_NOTICE = (
|
||||
"document."
|
||||
)
|
||||
|
||||
#: The image-document marker (phase 122, task 05): the line prefixed to
|
||||
#: the vision DESCRIPTION a ``read`` of a standalone-image document
|
||||
#: returns — the model must reason about what it is reading (the text
|
||||
#: below is a description GENERATED from the image, not the image's
|
||||
#: own words). It sits on the result's THIRD line: the ``Document …``
|
||||
#: header and the phase-106 date line stay byte-identical (the E2E
|
||||
#: mock's ``_READ_RESULT_PREFIX`` header contract), and a NON-image
|
||||
#: doc's result carries no marker at all (byte-identical to pre-122).
|
||||
IMAGE_DOC_MARKER = (
|
||||
"Image document — the text below is a description generated from the image:"
|
||||
)
|
||||
|
||||
#: The no-source ``ls`` refusal with the teaching parenthetical
|
||||
#: appended (phase 72): used when a stripped scope has no ``/`` and
|
||||
#: matches no registered source (the incident's ``ls(path='.')``). The
|
||||
@@ -1270,6 +1282,12 @@ def _execute_tool(
|
||||
return _no_document_refusal(db, arg)
|
||||
holder.read_docs.append(doc)
|
||||
holder.tool_calls += 1
|
||||
# Phase 122 (task 05): an image document's content IS the vision
|
||||
# description — the marker line (a third line between the
|
||||
# byte-identical header/date lines and the text) tells the model
|
||||
# what it is reading. A text doc's ``marker`` is "" — the result
|
||||
# stays byte-identical to pre-122.
|
||||
marker = f"{IMAGE_DOC_MARKER}\n" if doc.is_image else ""
|
||||
cap = settings.read_max_chars
|
||||
if len(doc.content) > cap:
|
||||
# Phase 95 (owner permission 2026-09-10, ``TODO.md`` L5): the
|
||||
@@ -1293,17 +1311,20 @@ def _execute_tool(
|
||||
return (
|
||||
f"Document {doc.source}/{doc.path}:\n"
|
||||
f"date: {doc.created_at:%Y-%m-%d}\n"
|
||||
f"{marker}"
|
||||
f"{doc.content[:cap]}\n"
|
||||
f"{TRUNCATION_MARKER}\n"
|
||||
f"{READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content))}"
|
||||
)
|
||||
# At or under the cap: the pre-phase-95 result plus the
|
||||
# phase-106 D5 date line (first line byte-identical — the
|
||||
# mock's header contract; no marker, no notice, no holder
|
||||
# entry, no ToolResultPiece).
|
||||
# mock's header contract; no truncation marker, no notice, no
|
||||
# holder entry, no ToolResultPiece) — and, phase 122, the
|
||||
# image-document marker line for image docs only.
|
||||
return (
|
||||
f"Document {doc.source}/{doc.path}:\n"
|
||||
f"date: {doc.created_at:%Y-%m-%d}\n"
|
||||
f"{marker}"
|
||||
f"{doc.content}"
|
||||
)
|
||||
if call.name == "grep":
|
||||
|
||||
+374
-13
@@ -30,7 +30,12 @@ by their exact lowercased full filename (``Dockerfile`` under the
|
||||
no longer exist **or no longer match the format filter** — this is how
|
||||
previously-imported junk (e.g. dot-dir READMEs) leaves the index. Per-file
|
||||
logging uses the verbs ``added | updated | unchanged | pruned`` plus a
|
||||
summary line with per-format counts (PLAN §9).
|
||||
summary line with per-format counts (PLAN §9). Phase 122 prune guard
|
||||
(LOCKED, derived from A3/A4): while the ``images`` toggle is OFF, an
|
||||
``is_image`` doc is INVISIBLE to the walk, not a deleted file — prune
|
||||
skips it (turning the toggle off and syncing must never destroy image
|
||||
documents); a toggle-ON run prunes a deleted image file normally and
|
||||
deletes its ``image_dir`` copy with the row.
|
||||
|
||||
Document dates (phase 106, D2/D4): every import sources
|
||||
``documents.created_at`` from the file's source — the per-file git
|
||||
@@ -54,6 +59,32 @@ backfill runs BEFORE the ``created_at_manual`` early-return (the manual
|
||||
flag protects the DATE only, D1) and the strict ``is None`` check leaves
|
||||
owner-set summaries (even empty strings, phase 57) alone.
|
||||
|
||||
Standalone images (phase 122, LOCKED A3): with the ``images`` toggle
|
||||
(``BOR_IMAGES``) ON, the walk also admits the image extension set
|
||||
(``BOR_IMAGE_EXTENSIONS`` — a SEPARATE set from ``import_extension_set``;
|
||||
images are never user-added via ``BOR_IMPORT_EXTENSIONS``, the toggle is
|
||||
the single knob). Such a file takes the binary index path
|
||||
(:func:`_index_image_file`): the sha256 digest is over the raw BYTES
|
||||
(content identity — the digest rule is unchanged), the bytes are copied
|
||||
to the persistent home ``settings.image_dir/<doc-id>.<ext>`` (dir created
|
||||
on demand; the copy is written ONLY after a successful description, so a
|
||||
failure never leaves an orphan; a changed image deletes the stale copy
|
||||
first; a pruned image doc deletes its copy), the row carries
|
||||
``is_image=True`` + ``image_path``, and ``content`` is the vision
|
||||
description — the ONLY embedded text of the document (the embedding model
|
||||
never sees pixels; ``read_text`` is never called for an image). The
|
||||
description comes through the single seam :func:`_describe_or_skip`
|
||||
(task 03: :func:`app.rag.summarizer.describe_image` — ONE CHAT-model
|
||||
(vision) call with the image bytes as a base64 data URL; the ``lite``
|
||||
summary model is NOT assumed vision-capable, LOCKED A3); a failed/empty
|
||||
description SKIPS the doc entirely (no row, no copy) — counted in
|
||||
``images_failed`` + a warning, the sync continues (fail-soft). The normal
|
||||
chunk pipeline then embeds ``content`` and the phase-30 summary path runs
|
||||
on it — image-aware (task 03): for an image doc the description IS the
|
||||
summary (stored verbatim, no ``lite`` call, no pointer line), so the
|
||||
``is_summary`` position −1 chunk mirrors ``Document.summary``, which
|
||||
equals ``Document.content``.
|
||||
|
||||
``import_sources`` accepts an optional per-file ``progress`` callback
|
||||
(phase 64, task 01) reporting the file being processed right now.
|
||||
"""
|
||||
@@ -61,11 +92,12 @@ from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import uuid
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Protocol
|
||||
from typing import Any, Protocol
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
@@ -76,7 +108,12 @@ from app.models import Chunk, Document
|
||||
from app.rag.chunker import chunk_document, extract_title
|
||||
from app.rag.doc_dates import file_mtime_datetime, normalize_doc_date
|
||||
from app.rag.llm import EmbeddingError, LLMError
|
||||
from app.rag.summarizer import generate_summary
|
||||
from app.rag.summarizer import (
|
||||
IMAGE_FALLBACK_MIME,
|
||||
IMAGE_MIMES,
|
||||
describe_image,
|
||||
generate_summary,
|
||||
)
|
||||
|
||||
logger = logging.getLogger("app.importer")
|
||||
|
||||
@@ -94,9 +131,12 @@ class Embedder(Protocol):
|
||||
|
||||
async def embed(self, texts: list[str]) -> list[list[float]]: ...
|
||||
|
||||
async def chat(self, messages: list[dict[str, str]], model: str | None = None) -> str: ...
|
||||
# ^ the one-shot completion the summarizer uses for the ``lite`` model
|
||||
# (phase 30, task 01); :class:`app.rag.llm.LLMClient` satisfies it.
|
||||
async def chat(self, messages: list[dict[str, Any]], model: str | None = None) -> str: ...
|
||||
# ^ the one-shot completion the summarizer uses — the ``lite`` model
|
||||
# for text summaries (phase 30, task 01) and the CHAT (vision)
|
||||
# model for the phase-122 image description (multimodal content:
|
||||
# a string or a list of OpenAI-compatible parts); :class:`app.rag.
|
||||
# llm.LLMClient` satisfies it.
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -129,6 +169,12 @@ class ImportSummary:
|
||||
#: so no ``sources_meta`` bump, no overview/folder-summary
|
||||
#: regeneration).
|
||||
dates_updated: int = 0
|
||||
#: Image docs (phase 122, LOCKED A3) whose vision description failed
|
||||
#: or came back empty — the doc is SKIPPED entirely (no row, no
|
||||
#: ``image_dir`` copy): an undescribed image is unsearchable noise.
|
||||
#: Fail-soft: the sync continues, this counter + the warning line
|
||||
#: are the signal.
|
||||
images_failed: int = 0
|
||||
#: Files walked, keyed by lowercased extension (``md``, ``yaml``, …).
|
||||
formats: dict[str, int] = field(default_factory=dict)
|
||||
|
||||
@@ -143,7 +189,7 @@ class ImportSummary:
|
||||
logger.info(
|
||||
"import: summary files=%d added=%d updated=%d unchanged=%d pruned=%d "
|
||||
"errors=%d chunks=%d embed_batches=%d summaries=%d summary_errors=%d "
|
||||
"summary_backfilled=%d dates_updated=%d formats=%s",
|
||||
"summary_backfilled=%d dates_updated=%d images_failed=%d formats=%s",
|
||||
self.files,
|
||||
self.added,
|
||||
self.updated,
|
||||
@@ -156,6 +202,7 @@ class ImportSummary:
|
||||
self.summary_errors,
|
||||
self.summary_backfilled,
|
||||
self.dates_updated,
|
||||
self.images_failed,
|
||||
self.format_counts(),
|
||||
)
|
||||
|
||||
@@ -238,6 +285,7 @@ def iter_importable_files(
|
||||
excluded: frozenset[str] = EXCLUDED_DIRS,
|
||||
ignore: tuple[str, ...] = (),
|
||||
include_hidden: bool = False,
|
||||
image_extensions: frozenset[str] = frozenset(),
|
||||
) -> list[Path]:
|
||||
"""All importable files under *root* (sorted), per the A9 scope rules.
|
||||
|
||||
@@ -255,6 +303,13 @@ def iter_importable_files(
|
||||
source-relative POSIX path starts with any entry; the default ``()``
|
||||
keeps every existing caller byte-identical. The *ignore* tuple
|
||||
composes additively in both states.
|
||||
|
||||
*image_extensions* (phase 122) is the lowercased dotted
|
||||
image-extension set admitted IN ADDITION to *extensions* — passed by
|
||||
:func:`import_sources` only while the ``images`` toggle is on (it
|
||||
reads ``llm.settings``; the image set is never merged into
|
||||
*extensions*). The empty default admits nothing: every existing caller
|
||||
(and the toggle-off walk) stays byte-identical to pre-phase.
|
||||
"""
|
||||
if not root.is_dir():
|
||||
return []
|
||||
@@ -270,7 +325,12 @@ def iter_importable_files(
|
||||
continue
|
||||
if ignore and is_ignored(rel.as_posix(), ignore):
|
||||
continue
|
||||
if match_extension(path, extensions) is None:
|
||||
matched = match_extension(path, extensions)
|
||||
if matched is None and image_extensions:
|
||||
# Phase 122: the image set is admitted IN ADDITION to the
|
||||
# import set (toggle on only — the caller passes it in).
|
||||
matched = match_extension(path, image_extensions)
|
||||
if matched is None:
|
||||
continue
|
||||
files.append(path)
|
||||
return files
|
||||
@@ -340,10 +400,23 @@ async def import_sources(
|
||||
mtime fallback applies to every file — which IS the behavior
|
||||
change, D4: an unchanged file now refreshes its stored date from
|
||||
its source on every run (the backfill-correction case).
|
||||
|
||||
Images (phase 122): when ``llm.settings.images`` is on, BOTH walks
|
||||
(the progress pre-walk and the processing loop — same rules, so
|
||||
``total`` counts images) also admit ``llm.settings.image_extension_set``
|
||||
files, each indexed through the binary image path (see the module
|
||||
docstring). ``prune=True`` with the toggle ON prunes a deleted image
|
||||
file normally (row + ``image_dir`` copy); with the toggle OFF the
|
||||
prune skips ``is_image`` docs (the prune guard — the image is
|
||||
invisible to the walk, not a deleted file).
|
||||
"""
|
||||
if limit is not None and limit <= 0:
|
||||
raise ValueError("limit must be >= 1")
|
||||
summary = ImportSummary()
|
||||
# Phase 122: the image set is admitted by the walks ONLY while the
|
||||
# toggle is on — the empty set admits nothing, so the toggle-off run
|
||||
# (walk, counts, prune) stays byte-identical to pre-phase.
|
||||
image_exts = llm.settings.image_extension_set if llm.settings.images else frozenset()
|
||||
owns_session = session is None
|
||||
if session is None:
|
||||
session = SessionLocal()
|
||||
@@ -364,6 +437,7 @@ async def import_sources(
|
||||
include_hidden=_include_hidden_for_root(
|
||||
root, include_hidden_by_root
|
||||
),
|
||||
image_extensions=image_exts,
|
||||
)
|
||||
)
|
||||
try:
|
||||
@@ -386,6 +460,7 @@ async def import_sources(
|
||||
llm.settings.import_extension_set,
|
||||
ignore=ignore,
|
||||
include_hidden=include_hidden,
|
||||
image_extensions=image_exts,
|
||||
):
|
||||
if limit is not None and summary.files >= limit:
|
||||
break
|
||||
@@ -394,9 +469,11 @@ async def import_sources(
|
||||
summary.files += 1
|
||||
# Phase 102: the matched bare token (``dockerfile`` for an
|
||||
# extensionless ``Dockerfile``), never ``unknown`` — the
|
||||
# file is in scope, so the walk matched it.
|
||||
# file is in scope, so the walk matched it. Phase 122: an
|
||||
# image file matches the image set, not the import set.
|
||||
ext = (
|
||||
match_extension(path, llm.settings.import_extension_set)
|
||||
or match_extension(path, image_exts)
|
||||
or "unknown"
|
||||
)
|
||||
summary.formats[ext] = summary.formats.get(ext, 0) + 1
|
||||
@@ -423,7 +500,9 @@ async def import_sources(
|
||||
if limit is not None:
|
||||
logger.warning("import: --prune ignored because --limit was given")
|
||||
else:
|
||||
summary.pruned = _prune(session, source_names, seen)
|
||||
summary.pruned = _prune(
|
||||
session, source_names, seen, images=llm.settings.images
|
||||
)
|
||||
summary.embed_batches = llm.embed_batches
|
||||
summary.log()
|
||||
return summary
|
||||
@@ -448,8 +527,24 @@ async def _index_file(
|
||||
git last-commit datetime from the caller's ``doc_dates_by_root``
|
||||
map, or ``None`` (every non-git case): the file's mtime is read
|
||||
here, once, and becomes the source date (the D2 fallback).
|
||||
|
||||
Image files (phase 122, toggle on) delegate to
|
||||
:func:`_index_image_file` — the binary path (bytes digest,
|
||||
persistent copy, ``content`` = the vision description) — BEFORE
|
||||
any text read: ``read_text`` is never called for an image.
|
||||
"""
|
||||
settings = llm.settings
|
||||
# Phase 122 (task 02): the image branch FIRST. Only reachable while
|
||||
# the ``images`` toggle is on — the walk never admits image files
|
||||
# while it is off, and with it off this check is a no-op (the text
|
||||
# path below stays byte-identical to pre-phase).
|
||||
if settings.images:
|
||||
image_set = settings.image_extension_set
|
||||
if match_extension(full_path, image_set) is not None:
|
||||
return await _index_image_file(
|
||||
session, source=source, rel=rel, full_path=full_path, llm=llm,
|
||||
summary=summary, raw_date=raw_date,
|
||||
)
|
||||
content = full_path.read_text(encoding="utf-8", errors="replace").replace("\x00", "")
|
||||
digest = hashlib.sha256(content.encode("utf-8")).hexdigest()
|
||||
doc = session.scalar(select(Document).where(Document.source == source, Document.path == rel))
|
||||
@@ -609,9 +704,24 @@ async def _store_summary(
|
||||
a success counts ``summary_backfilled`` instead of ``summaries``
|
||||
(the doc content is untouched, so the import's KB-change signal must
|
||||
not move); the rest of the mechanics are identical.
|
||||
|
||||
Image docs (phase 122, task 03, LOCKED A3): for an ``is_image`` doc
|
||||
the vision description — ``content`` (which equals ``doc.content``
|
||||
on the backfill path) — IS the summary: no ``lite`` call, no
|
||||
pointer line (the summary mirrors the description verbatim, so
|
||||
``doc.summary`` == ``doc.content``). The phase-30 chunk mechanics
|
||||
(one ``is_summary`` position −1 chunk, replacement, best-effort
|
||||
rollback) are unchanged; the only remaining failure class is the
|
||||
summary chunk's embed (the ``doc`` row + content chunks survive —
|
||||
fail-soft, same as the text path).
|
||||
"""
|
||||
try:
|
||||
text = await generate_summary(llm, source=source, path=rel, content=content)
|
||||
if doc.is_image:
|
||||
# Phase 122 (task 03): the description IS the summary —
|
||||
# stored verbatim (no ``lite`` call, no pointer line).
|
||||
text = content
|
||||
else:
|
||||
text = await generate_summary(llm, source=source, path=rel, content=content)
|
||||
# Replacement: at most one summary chunk per document at a time.
|
||||
# Removing from the collection is what the ``delete-orphan``
|
||||
# cascade turns into a row delete on flush — and it keeps the
|
||||
@@ -646,14 +756,265 @@ async def _store_summary(
|
||||
logger.error("import: summary failed source=%s path=%s — %s", source, rel, e)
|
||||
|
||||
|
||||
def _prune(session: Session, source_names: set[str], seen: set[tuple[str, str]]) -> int:
|
||||
"""Delete documents of *source_names* whose file is no longer in *seen*."""
|
||||
async def _describe_or_skip(
|
||||
llm: Embedder, *, data: bytes, source: str, rel: str, full_path: Path
|
||||
) -> str | None:
|
||||
"""Phase 122 — the SINGLE seam for the image description (task 03).
|
||||
|
||||
Returns the vision description that becomes the image document's
|
||||
``content`` (and ``summary`` — the ONLY embedded text of the doc),
|
||||
or ``None`` when the description failed or came back empty — the
|
||||
caller then SKIPS the doc entirely (no row, no copy) and counts
|
||||
``summary.images_failed`` (LOCKED A3, fail-soft: the sync continues,
|
||||
the warning + counter are the signal).
|
||||
|
||||
The seam is one line by design (the importer's tests patch exactly
|
||||
this function): :func:`app.rag.summarizer.describe_image` — ONE
|
||||
CHAT-model (vision) call (LOCKED A3) with the bytes as a data URL
|
||||
whose mime comes from :data:`app.rag.summarizer.IMAGE_MIMES`
|
||||
(dotted extension; an unlisted ``BOR_IMAGE_EXTENSIONS`` token takes
|
||||
the generic fallback — a rejection there fails soft like any other
|
||||
description error). ``source``/``rel`` stay on the signature so the
|
||||
caller (and the patch) reads like the document being described;
|
||||
the failure's doc identity is logged by the caller's warning.
|
||||
"""
|
||||
mime = IMAGE_MIMES.get(full_path.suffix.lower(), IMAGE_FALLBACK_MIME)
|
||||
return await describe_image(llm, data=data, mime=mime)
|
||||
|
||||
|
||||
def _delete_image_copy(image_path: str | None) -> None:
|
||||
"""Best-effort removal of a stale image copy (phase 122).
|
||||
|
||||
A missing path is a no-op (already gone — e.g. the owner cleaned
|
||||
the image dir); an unreadable one is logged, never raised — copy
|
||||
cleanup must not break the sync (the doc row's fate is decided by
|
||||
the upsert/prune logic, not by filesystem hygiene).
|
||||
"""
|
||||
if not image_path:
|
||||
return
|
||||
try:
|
||||
Path(image_path).unlink(missing_ok=True)
|
||||
except OSError as e:
|
||||
logger.warning("import: could not delete image copy %s — %s", image_path, e)
|
||||
|
||||
|
||||
async def _index_image_file(
|
||||
session: Session,
|
||||
*,
|
||||
source: str,
|
||||
rel: str,
|
||||
full_path: Path,
|
||||
llm: Embedder,
|
||||
summary: ImportSummary,
|
||||
raw_date: datetime | None = None,
|
||||
) -> None:
|
||||
"""The phase-122 image branch of :func:`_index_file` — a standalone
|
||||
image is indexed from its BYTES, never its text:
|
||||
|
||||
* the sha256 digest is over the raw bytes (the digest rule is
|
||||
content identity — the same bytes are the same document);
|
||||
* the PERSISTENT copy lands in ``settings.image_dir`` as
|
||||
``<doc-id>.<ext>`` (the dir is created on demand; the copy is
|
||||
written only AFTER a successful description, so a failure never
|
||||
leaves an orphan; a changed image deletes the stale copy before
|
||||
replacing it);
|
||||
* ``content`` is the vision description — the ONLY embedded text of
|
||||
the document (the embedding model never sees pixels) — and the
|
||||
normal chunk pipeline then embeds it, with the phase-30 summary
|
||||
path running on it (the ``is_summary`` position −1 chunk mirrors
|
||||
``Document.summary``).
|
||||
|
||||
``raw_date`` follows the text path exactly (the phase-106 D2
|
||||
fallback: no source date in the map → the file's mtime, read before
|
||||
the unchanged early-return because the unchanged path refreshes the
|
||||
stored date from the same source; the D1 manual-date lock and the
|
||||
D4 refresh apply unmodified).
|
||||
|
||||
Fail-soft (LOCKED A3): a failed/empty description SKIPS the doc
|
||||
entirely (no row, no copy) — ``summary.images_failed`` + a warning,
|
||||
the sync continues.
|
||||
"""
|
||||
settings = llm.settings
|
||||
data = full_path.read_bytes()
|
||||
digest = hashlib.sha256(data).hexdigest()
|
||||
doc = session.scalar(select(Document).where(Document.source == source, Document.path == rel))
|
||||
if raw_date is None:
|
||||
# D2 fallback (same as the text path): no source date in the map
|
||||
# → the file's mtime (one stat).
|
||||
raw_date = file_mtime_datetime(full_path)
|
||||
|
||||
if doc is not None and doc.content_hash == digest:
|
||||
# Unchanged image (byte digest) — the text path's unchanged
|
||||
# branch, unmodified in shape.
|
||||
summary.unchanged += 1
|
||||
logger.info("import: unchanged source=%s path=%s", source, rel)
|
||||
# Phase 118 (A2) backfill, image flavour: an unchanged image doc
|
||||
# whose summary is still NULL (an earlier fail-soft summary miss
|
||||
# — for an image, the one remaining failure class: the summary
|
||||
# chunk's embed) gets the same best-effort summary pass. For an
|
||||
# image the summary IS the stored description (``doc.content``),
|
||||
# so the image-aware ``_store_summary`` (task 03) re-stores it
|
||||
# verbatim with one ``is_summary`` chunk; a failure (the
|
||||
# embed) keeps the doc as-is (no row mutation) — fail-soft,
|
||||
# same as the text path.
|
||||
if doc.summary is None:
|
||||
await _store_summary(
|
||||
session, doc=doc, source=source, rel=rel, content=doc.content,
|
||||
llm=llm, summary=summary, backfill=True,
|
||||
)
|
||||
if doc.created_at_manual:
|
||||
# D1/D4: the owner's correction survives the sync — no write
|
||||
# at all (the text path's manual-date early-return).
|
||||
return
|
||||
# D4: the date refreshes on every sync, including unchanged
|
||||
# files, and may go OLDER (no monotonic guard).
|
||||
target = normalize_doc_date(raw_date)
|
||||
if target != doc.created_at:
|
||||
doc.created_at = target
|
||||
session.commit()
|
||||
summary.dates_updated += 1
|
||||
logger.info(
|
||||
"import: date-refreshed source=%s path=%s date=%s",
|
||||
source, rel, doc.created_at.isoformat(),
|
||||
)
|
||||
return
|
||||
|
||||
verb = "updated" if doc is not None else "added"
|
||||
# The description is the doc's content (task 03: and its summary) —
|
||||
# it is generated BEFORE anything is written, so a failure skips the
|
||||
# doc with no row and no copy (the copy is only made after a
|
||||
# successful description — a failure never leaves an orphan).
|
||||
content = await _describe_or_skip(
|
||||
llm, data=data, source=source, rel=rel, full_path=full_path
|
||||
)
|
||||
if content is None:
|
||||
# LOCKED A3 fail-soft: an undescribed image is unsearchable
|
||||
# noise — skip the doc entirely (no row, no copy).
|
||||
summary.images_failed += 1
|
||||
logger.warning("import: image description failed source=%s path=%s", source, rel)
|
||||
return
|
||||
|
||||
# The persistent copy: uploads are replaced on every upload, git
|
||||
# checkouts are re-cloned, local dirs are user-edited — the served
|
||||
# bytes must outlive the source file. Named by the doc id: a new
|
||||
# doc's id is the uuid4 chosen here (row and copy agree); a changed
|
||||
# doc keeps its id (the copy path is stable).
|
||||
doc_id = doc.id if doc is not None else uuid.uuid4()
|
||||
image_dir = Path(settings.image_dir).expanduser()
|
||||
image_dir.mkdir(parents=True, exist_ok=True)
|
||||
if doc is not None:
|
||||
# A CHANGED image (hash differs): the stale copy is deleted
|
||||
# before replacement.
|
||||
_delete_image_copy(doc.image_path)
|
||||
copy_path = image_dir / f"{doc_id}{full_path.suffix.lower()}"
|
||||
copy_path.write_bytes(data)
|
||||
|
||||
# The non-markdown title rule (the image's content is prose, but the
|
||||
# doc IS the image — the file stem is the title).
|
||||
title = full_path.stem
|
||||
if doc is None:
|
||||
doc = Document(
|
||||
id=doc_id,
|
||||
source=source,
|
||||
path=rel,
|
||||
full_path=str(full_path),
|
||||
title=title,
|
||||
content=content,
|
||||
content_hash=digest,
|
||||
indexed_at=datetime.now(UTC),
|
||||
created_at=normalize_doc_date(raw_date),
|
||||
is_image=True,
|
||||
image_path=str(copy_path),
|
||||
)
|
||||
session.add(doc)
|
||||
else:
|
||||
doc.full_path = str(full_path)
|
||||
doc.title = title
|
||||
doc.content = content
|
||||
doc.content_hash = digest
|
||||
doc.indexed_at = datetime.now(UTC)
|
||||
# Phase 106 (D4): a content change is a new document version —
|
||||
# the date is re-sourced and a previous manual correction is
|
||||
# reset (it referred to the old content).
|
||||
doc.created_at = normalize_doc_date(raw_date)
|
||||
doc.created_at_manual = False
|
||||
doc.is_image = True
|
||||
doc.image_path = str(copy_path)
|
||||
|
||||
session.flush() # guarantees doc.id even for brand-new rows
|
||||
|
||||
# Phase 1+2 — the UNCHANGED pipeline on the description: chunk,
|
||||
# replace the chunk rows (embeddings NULL), embed, and commit the
|
||||
# whole file atomically (one transaction per file). The token-cap
|
||||
# retry loop is copied from the text path; a description is short,
|
||||
# so it never fires in practice.
|
||||
target = max(400, settings.chunk_target_chars)
|
||||
while True:
|
||||
chunks_text = chunk_document(content, rel, target, settings.chunk_overlap_chars)
|
||||
doc.chunks = [
|
||||
Chunk(document_id=doc.id, position=i, content=c) for i, c in enumerate(chunks_text)
|
||||
]
|
||||
session.flush() # delete-orphan cascade drops the previous rows
|
||||
if not doc.chunks:
|
||||
break
|
||||
try:
|
||||
vectors = await llm.embed([c.content for c in doc.chunks])
|
||||
for row, vec in zip(doc.chunks, vectors, strict=True):
|
||||
row.embedding = vec
|
||||
break
|
||||
except EmbeddingError as e:
|
||||
if "token cap" not in str(e) or target <= 400:
|
||||
raise
|
||||
logger.info(
|
||||
"import: re-chunking at %d chars after endpoint token cap: %s",
|
||||
target // 2,
|
||||
rel,
|
||||
)
|
||||
target //= 2
|
||||
session.commit()
|
||||
|
||||
if verb == "added":
|
||||
summary.added += 1
|
||||
else:
|
||||
summary.updated += 1
|
||||
summary.chunks += len(chunks_text)
|
||||
logger.info("import: %s source=%s path=%s chunks=%d", verb, source, rel, len(chunks_text))
|
||||
|
||||
# Phase 30 shape on the description — image-aware (task 03, LOCKED
|
||||
# A3): the description IS the summary (stored verbatim, no
|
||||
# ``lite`` call), so ``doc.summary`` == ``doc.content`` and the
|
||||
# ``is_summary`` position −1 chunk mirrors it.
|
||||
await _store_summary(
|
||||
session, doc=doc, source=source, rel=rel, content=content, llm=llm, summary=summary
|
||||
)
|
||||
|
||||
|
||||
def _prune(
|
||||
session: Session,
|
||||
source_names: set[str],
|
||||
seen: set[tuple[str, str]],
|
||||
images: bool = False,
|
||||
) -> int:
|
||||
"""Delete documents of *source_names* whose file is no longer in *seen*.
|
||||
|
||||
*images* (phase 122 prune guard, LOCKED, derived from A3/A4): while
|
||||
the image toggle is OFF (``images`` False), every ``is_image`` doc is
|
||||
SKIPPED — the image is invisible to an images-off walk, not a deleted
|
||||
file, so pruning it would silently destroy image documents on the
|
||||
first images-off sync. Toggle ON → normal semantics: a deleted image
|
||||
file prunes its doc, and the pruned image's ``image_dir`` copy is
|
||||
deleted with it.
|
||||
"""
|
||||
if not source_names:
|
||||
return 0
|
||||
pruned = 0
|
||||
docs = session.scalars(select(Document).where(Document.source.in_(source_names))).all()
|
||||
for doc in docs:
|
||||
if (doc.source, doc.path) not in seen:
|
||||
if doc.is_image and not images:
|
||||
# Prune guard: invisible to the walk, not deleted.
|
||||
continue
|
||||
_delete_image_copy(doc.image_path)
|
||||
session.delete(doc)
|
||||
pruned += 1
|
||||
logger.info("import: pruned source=%s path=%s", doc.source, doc.path)
|
||||
|
||||
@@ -92,6 +92,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import get_settings
|
||||
from app.models import Chunk, Document
|
||||
from app.schemas import SourceRef
|
||||
|
||||
#: Shared overflow marker (phase 15; imported by ``app.rag.prompts``)
|
||||
#: — used by the steering (<tuning>) section, the phase-118 NULL-summary
|
||||
@@ -939,3 +940,27 @@ def select_related(
|
||||
continue
|
||||
out.append(doc)
|
||||
return out
|
||||
|
||||
|
||||
def source_ref_with_image(doc: Document) -> SourceRef:
|
||||
"""One :class:`~app.schemas.SourceRef` wire frame for *doc* — the
|
||||
phase-122 (task 05) SHARED frame builder: the chat API's cited
|
||||
tier (the agent's read docs) and the related tier both run through
|
||||
it, so the per-doc ref shape has exactly one construction site.
|
||||
|
||||
The ref carries the chip identity (``source`` / ``path`` /
|
||||
``title``) and, ONLY for a standalone-image document (``is_image``),
|
||||
the optional ``image_url`` — the image BYTES route
|
||||
``/api/documents/<id>/image`` (the chat's sources block renders the
|
||||
compact inline image from it, the summary as alt + caption —
|
||||
"shown in the chat nicely", TODO L6). For a TEXT document the field
|
||||
stays ``None`` and is DROPPED by the model's serializer (never
|
||||
``null`` — the omission rule): a text-doc frame is byte-identical
|
||||
to pre-phase. The frame's doc id rides the path — the same way the
|
||||
document content endpoint's ``(source, path)`` lookup does (no new
|
||||
id leak beyond what the frame already carries).
|
||||
"""
|
||||
ref = SourceRef(source=doc.source, path=doc.path, title=doc.title)
|
||||
if doc.is_image:
|
||||
ref.image_url = f"/api/documents/{doc.id}/image"
|
||||
return ref
|
||||
|
||||
+131
-6
@@ -1,4 +1,5 @@
|
||||
"""Document summarizer (phase 30, task 03).
|
||||
"""Document summarizer (phase 30, task 03) + image descriptions
|
||||
(phase 122, task 03).
|
||||
|
||||
Builds the ``SUMMARY_MODE`` prompt for one document, calls the aipi
|
||||
``lite`` model through the one-shot ``LLMClient.chat`` (phase 30,
|
||||
@@ -22,18 +23,33 @@ Quality contracts enforced here:
|
||||
summarizer re-asserts defensively and never hands the importer a
|
||||
pointer-only row).
|
||||
|
||||
The ``SUMMARY_MODE`` marker follows the ``DEFLECT_MODE`` convention:
|
||||
the deterministic E2E mock LLM keys on it in the system prompt
|
||||
(``tests/e2e/mock_llm.py`` — wired in task 06).
|
||||
Image descriptions (phase 122, LOCKED A3): :func:`describe_image` is
|
||||
this module's second one-shot generation path — a SINGLE CHAT-model
|
||||
(vision) call describing one image's bytes as a base64 data URL. The
|
||||
description becomes the image document's ``content`` AND ``summary``
|
||||
(it is the ONLY embedded text of the doc — the embedding model never
|
||||
sees pixels, and the ``lite`` summary model is deliberately NOT used:
|
||||
it is not assumed vision-capable). Fail-soft by contract: any client
|
||||
error, empty reply, or non-2xx yields ``None`` — the importer skips
|
||||
the doc, counts ``images_failed``, and the sync continues.
|
||||
|
||||
The ``SUMMARY_MODE`` / ``IMAGE_DESCRIPTION_MODE`` markers follow the
|
||||
``DEFLECT_MODE`` convention: the deterministic E2E mock LLM keys on
|
||||
them (``tests/e2e/mock_llm.py`` — the image branch is wired by task
|
||||
06's story suite).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Protocol
|
||||
import base64
|
||||
import logging
|
||||
from typing import Any, Protocol
|
||||
|
||||
from app.config import Settings, get_settings
|
||||
from app.rag.llm import LLMError
|
||||
from app.rag.retriever import TRUNCATION_MARKER
|
||||
|
||||
logger = logging.getLogger("app.summarizer")
|
||||
|
||||
#: System-prompt marker for summary generation — the E2E mock LLM keys on
|
||||
#: it (same convention as ``DEFLECT_MODE``, PLAN §6).
|
||||
SUMMARY_MODE = "SUMMARY_MODE"
|
||||
@@ -51,6 +67,55 @@ SUMMARY_INSTRUCTION = (
|
||||
#: Full system prompt: marker first (the mock's key), then the instruction.
|
||||
SYSTEM_PROMPT = f"{SUMMARY_MODE}: {SUMMARY_INSTRUCTION}"
|
||||
|
||||
#: Image-description marker (phase 122, task 03) — the deterministic
|
||||
#: E2E mock LLM keys on it (same convention as ``SUMMARY_MODE`` /
|
||||
#: ``DEFLECT_MODE``, PLAN §6; the story suite wires the mock's branch
|
||||
#: in task 06). It heads the text part of the multimodal describe
|
||||
#: message, so it rides the user message, not a system prompt.
|
||||
IMAGE_DESCRIPTION_MODE = "IMAGE_DESCRIPTION_MODE"
|
||||
|
||||
#: Locked instruction for the CHAT (vision) model (phase 122, LOCKED
|
||||
#: A3): the description is the ONLY retrievable text of the image
|
||||
#: document (the embedding model never sees pixels), so it must be a
|
||||
#: faithful, retrieval-oriented account that carries the image's full
|
||||
#: meaning — what is depicted, any visible text/labels/titles,
|
||||
#: diagram/table structure, salient details.
|
||||
DESCRIBE_INSTRUCTION = (
|
||||
"Describe this image faithfully, in plain text, for a search index. "
|
||||
"State what is depicted, transcribe any visible text, labels, or "
|
||||
"titles, describe the structure of any diagram, table, or layout, and "
|
||||
"call out the most salient details. Write 2-4 sentences of substance. "
|
||||
"Do not use markdown. Do not invent anything that is not visible in "
|
||||
"the image. Your description is the ONLY text that will ever be "
|
||||
"retrieved for this image — it must carry the image's full meaning."
|
||||
)
|
||||
|
||||
#: The full describe prompt: marker first (the mock's key), then the
|
||||
#: instruction — the single text part of the multimodal user message.
|
||||
DESCRIBE_PROMPT = f"{IMAGE_DESCRIPTION_MODE}: {DESCRIBE_INSTRUCTION}"
|
||||
|
||||
#: Extension → MIME type for the image family (phase 122). Dotted,
|
||||
#: lowercase keys — the ``image_extension_set`` shape. Task 03 uses it
|
||||
#: for the describe call's data-URL mime; task 04's serve route reuses
|
||||
#: it for the image bytes' ``Content-Type`` (one map, one truth). A
|
||||
#: ``BOR_IMAGE_EXTENSIONS`` token outside this map (a custom format)
|
||||
#: takes the :data:`IMAGE_FALLBACK_MIME` data-URL mime in the describe
|
||||
#: call — the vision endpoint may reject it, and the fail-soft skip
|
||||
#: (``images_failed``) is the honest outcome.
|
||||
IMAGE_MIMES: dict[str, str] = {
|
||||
".png": "image/png",
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".webp": "image/webp",
|
||||
".gif": "image/gif",
|
||||
".bmp": "image/bmp",
|
||||
}
|
||||
|
||||
#: The data-URL mime for an image extension :data:`IMAGE_MIMES` does not
|
||||
#: name (phase 122) — the best-effort generic, never a guess at a
|
||||
#: specific type.
|
||||
IMAGE_FALLBACK_MIME = "application/octet-stream"
|
||||
|
||||
|
||||
class SummaryLLM(Protocol):
|
||||
"""The one-shot chat surface the summarizer needs.
|
||||
@@ -58,12 +123,17 @@ class SummaryLLM(Protocol):
|
||||
:class:`app.rag.llm.LLMClient` satisfies it; unit tests pass a
|
||||
duck-typed fake (``chat`` + ``settings``) instead — same pattern as
|
||||
the importer's ``Embedder`` protocol.
|
||||
|
||||
``content`` may be a string (text calls — ``generate_summary``) or
|
||||
a list of OpenAI-compatible parts (phase 122 multimodal image
|
||||
descriptions — ``{type: "text", …}`` + ``{type: "image_url", …}``);
|
||||
the client passes message dicts through untouched.
|
||||
"""
|
||||
|
||||
settings: Settings
|
||||
|
||||
async def chat(
|
||||
self, messages: list[dict[str, str]], model: str | None = None
|
||||
self, messages: list[dict[str, Any]], model: str | None = None
|
||||
) -> str: ...
|
||||
|
||||
|
||||
@@ -121,3 +191,58 @@ async def generate_summary(
|
||||
"refusing to store a silent summary"
|
||||
)
|
||||
return f"{summary}\nSource: {source}/{path}"
|
||||
|
||||
|
||||
async def describe_image(
|
||||
llm: SummaryLLM,
|
||||
*,
|
||||
data: bytes,
|
||||
mime: str,
|
||||
settings: Settings | None = None,
|
||||
) -> str | None:
|
||||
"""One-shot CHAT-model (vision) description of one image (phase 122,
|
||||
LOCKED A3) — the text that becomes the image document's ``content``
|
||||
AND ``summary`` (the ONLY embedded text of the doc; the embedding
|
||||
model never sees pixels).
|
||||
|
||||
ONE chat-model call against ``settings.llm_chat_model`` (the vision
|
||||
model — the ``lite`` summary model is NOT assumed vision-capable)
|
||||
with the multimodal user message the OpenAI-compatible API expects:
|
||||
``[{type: "text", text: DESCRIBE_PROMPT}, {type: "image_url",
|
||||
image_url: {url: <data URL from *data* + *mime*>}}]`` — no system
|
||||
prompt, no tools, no app-level retries beyond the client's own
|
||||
(SDK-level + the house one-shot empty-content policy) — a
|
||||
description failure must not stall a sync.
|
||||
|
||||
Returns the stripped reply cut exactly at
|
||||
``(settings or llm.settings).summary_max_chars`` (the phase-30 cap —
|
||||
the description IS the summary, so it keeps the same uniform
|
||||
ceiling). Returns ``None`` on any client error, empty reply, or
|
||||
non-2xx (the client raises :class:`LLMError` for all three classes)
|
||||
— the caller (the importer's ``_describe_or_skip`` seam) fails soft:
|
||||
the doc is skipped, counted in ``images_failed``, and the sync
|
||||
continues (LOCKED A3).
|
||||
"""
|
||||
data_url = f"data:{mime};base64,{base64.b64encode(data).decode('ascii')}"
|
||||
messages: list[dict[str, Any]] = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": DESCRIBE_PROMPT},
|
||||
{"type": "image_url", "image_url": {"url": data_url}},
|
||||
],
|
||||
}
|
||||
]
|
||||
model = llm.settings.llm_chat_model
|
||||
try:
|
||||
raw = await llm.chat(messages, model=model)
|
||||
except LLMError as e:
|
||||
logger.warning("image description failed (model=%s): %s", model, e)
|
||||
return None
|
||||
text = raw.strip()
|
||||
if not text:
|
||||
# The client already rejects empty content; this is the
|
||||
# defensive re-assert (the duck-typed fakes may return it).
|
||||
return None
|
||||
limit = (settings or llm.settings).summary_max_chars
|
||||
return text[:limit]
|
||||
|
||||
+101
-2
@@ -106,11 +106,36 @@ class SourceRef(BaseModel):
|
||||
rows server-side, so every server-built SSE ref fits by construction
|
||||
(A3: the SSE path is provably unaffected); the cap binds only
|
||||
client-saved refs — bounded at the boundary with a 422.
|
||||
|
||||
Phase 122 (task 05): ``image_url`` — the image BYTES route
|
||||
(``/api/documents/<id>/image``) for a ref whose document is a
|
||||
standalone image: the chat's sources block renders the compact
|
||||
inline image from it (the "shown in the chat nicely" contract, TODO
|
||||
L6). It is the ONLY new frame field (the doc id rides the path —
|
||||
the same way the document content endpoint's ``(source, path)``
|
||||
lookup does). For a TEXT document the field stays ``None`` and is
|
||||
DROPPED on serialization (never ``null`` — the :class:`DocContent`
|
||||
omission precedent), so a text-doc frame is byte-identical to
|
||||
pre-phase. Server-built refs go through the shared
|
||||
:func:`app.rag.retriever.source_ref_with_image` (one shape, both
|
||||
frame tiers); a client-saved ref without the field parses with the
|
||||
``None`` default (pre-phase saved chats restore unchanged).
|
||||
"""
|
||||
|
||||
source: str = Field(max_length=120)
|
||||
path: str = Field(max_length=1000)
|
||||
title: str = Field(max_length=500)
|
||||
#: Phase 122 (task 05) — see the class docstring. ``None`` (every
|
||||
#: text doc, and every pre-phase client-saved ref) is omitted on
|
||||
#: serialization — the key is ABSENT, never ``null``.
|
||||
image_url: str | None = None
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
data = handler(self)
|
||||
if self.image_url is None:
|
||||
data.pop("image_url", None)
|
||||
return data
|
||||
|
||||
|
||||
class ChatThinkingEvent(BaseModel):
|
||||
@@ -275,6 +300,20 @@ class KbTreeFile(BaseModel):
|
||||
``GET /api/docs`` returns) / ``created_at`` (phase 106, D8 — the
|
||||
document's creation date) / ``indexed_at`` (ISO-8601) are verbatim
|
||||
from the catalogue row the endpoint reads.
|
||||
|
||||
Image affordance (phase 122, task 04): for an ``is_image`` file node
|
||||
the three ``is_image`` / ``image_url`` / ``summary`` keys ride the
|
||||
node (the RAG view's Path cell renders the 48px thumbnail from
|
||||
``image_url`` with ``alt = summary``). For a TEXT file node all
|
||||
three are OMITTED from the wire shape (the
|
||||
:func:`_drop_image_fields` omission rule — a pre-phase KB, which has
|
||||
no image rows, serializes byte-identically to pre-phase, and the
|
||||
RAG view reads ``is_image === true`` — it never expects the keys on
|
||||
a text node). ``image_url`` is ``None``-omitted even on an image
|
||||
node (a row whose ``image_path`` was lost renders the glyph
|
||||
fallback); ``summary`` stays ``null`` on an image node (the alt
|
||||
falls back to the title client-side — the fail-soft backfill
|
||||
corner).
|
||||
"""
|
||||
|
||||
kind: Literal["file"] = "file"
|
||||
@@ -286,6 +325,23 @@ class KbTreeFile(BaseModel):
|
||||
#: the ``Created`` column (before ``Indexed``).
|
||||
created_at: str
|
||||
indexed_at: str
|
||||
#: Phase 122 (task 04) — true iff the file is an image document
|
||||
#: (LOCKED A3). Omitted from a text node's wire shape (see the class
|
||||
#: docstring); the builder sets it only for a node whose
|
||||
#: ``(source, path)`` is in the endpoint's image-docs map.
|
||||
is_image: bool = False
|
||||
#: Phase 122 (task 04) — the image bytes route
|
||||
#: (``/api/documents/<id>/image``) for the RAG view's thumbnail;
|
||||
#: ``None`` (→ absent) when the row has no servable copy.
|
||||
image_url: str | None = None
|
||||
#: Phase 122 (task 04) — the document's summary (for an image doc,
|
||||
#: the vision description — the thumbnail's ``alt``); ``None`` for a
|
||||
#: fail-soft row still awaiting the backfill.
|
||||
summary: str | None = None
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
return _drop_image_fields(self, handler)
|
||||
|
||||
|
||||
class KbTreeFolder(BaseModel):
|
||||
@@ -374,8 +430,11 @@ class KbTree(BaseModel):
|
||||
98, D3): true iff its recursive document count ≥
|
||||
``MIN_DOCS_PER_FOLDER`` (1) AND it has no stored ``folder_summaries``
|
||||
row — exactly ``missing_folder_summaries``'s candidate set (the
|
||||
marker never drifts from the gap-fill); FILE nodes carry no flag
|
||||
(the file table has no description column).
|
||||
marker never drifts from the gap-fill). FILE nodes carry no pending
|
||||
flag (the file table has no description column) — but, since phase
|
||||
122 (task 04), an image FILE node carries the thumbnail affordance
|
||||
keys (``is_image`` / ``image_url`` / ``summary`` — omitted on text
|
||||
nodes, see :class:`KbTreeFile`).
|
||||
"""
|
||||
|
||||
sources: list[KbTreeSource]
|
||||
@@ -400,6 +459,27 @@ class DocContent(BaseModel):
|
||||
content: str
|
||||
indexed_at: str
|
||||
chunks: int
|
||||
#: Phase 122 (task 04) — true iff the document is a standalone
|
||||
#: image (LOCKED A3: ``content`` is the vision description, the
|
||||
#: bytes live behind :attr:`image_url`). ALWAYS present on the wire
|
||||
#: (text docs: ``false`` — the wire-additive key, the phase-106
|
||||
#: ``created_at`` pattern); the viewer renders the ``<img>`` block
|
||||
#: only when true.
|
||||
is_image: bool = False
|
||||
#: Phase 122 (task 04) — the image bytes route
|
||||
#: (``/api/documents/<id>/image``) for the viewer's ``<img>``.
|
||||
#: ABSENT from the wire for text docs (``None`` → dropped by the
|
||||
#: serializer — never ``null``, the :func:`_drop_absent_share_url`
|
||||
#: omission precedent); also absent for an image row whose
|
||||
#: ``image_path`` is NULL (the viewer's onerror fallback covers it).
|
||||
image_url: str | None = None
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
data = handler(self)
|
||||
if self.image_url is None:
|
||||
data.pop("image_url", None)
|
||||
return data
|
||||
|
||||
|
||||
class SummaryUpdate(BaseModel):
|
||||
@@ -891,6 +971,25 @@ class SavedChatUpdate(BaseModel):
|
||||
messages: list[ChatMessage] = Field(min_length=1, max_length=200)
|
||||
|
||||
|
||||
def _drop_image_fields(model: KbTreeFile, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
"""The phase-122 (task 04) image-affordance omission rule for
|
||||
:class:`KbTreeFile` file nodes: a TEXT node (``is_image`` false) drops
|
||||
ALL three image keys — a pre-phase KB (no image rows) serializes
|
||||
byte-identically to pre-phase, and the RAG view's file row stays the
|
||||
pre-phase bare-link cell. An IMAGE node keeps ``is_image`` +
|
||||
``summary`` (a ``null`` summary is meaningful — the alt falls back
|
||||
client-side) and drops ``image_url`` only when ``None`` (the
|
||||
row-without-a-copy corner — never a ``null`` on the wire, the
|
||||
:func:`_drop_absent_share_url` precedent)."""
|
||||
data = handler(model)
|
||||
if not model.is_image:
|
||||
for key in ("is_image", "image_url", "summary"):
|
||||
data.pop(key, None)
|
||||
elif data.get("image_url") is None:
|
||||
data.pop("image_url", None)
|
||||
return data
|
||||
|
||||
|
||||
def _drop_absent_share_url(model: BaseModel, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
"""The ``share_url`` omission rule (phase 51, task 02): ``None`` →
|
||||
ABSENT from the JSON (not ``"share_url": null``) — an unshared chat
|
||||
|
||||
Reference in New Issue
Block a user