phase: 123_chat_image_questions
Build and Push Containers / build-and-push-app (push) Successful in 1m54s
Build and Push Containers / build-and-push-db (push) Failing after 13s

All gates green. Verification complete.

**Phase 123 — final verification pass (all 4 tasks already in `complete/`)**

- Verified the full implementation is in the working tree: `app/api/chat_images.py` (upload/serve pair), `ChatRequest.image`/`ChatMessage.image` (path-validated, omitted-when-None), toggle-off + stale-file hinted error frames, `build_user_content` multimodal build at both sites (chat.py deflected branch + `run_agent`), config-gated composer attach/preview/upload-then-send, restore + shared rendering, CSP `img-src 'self' data:` carve-out, mock-LLM capture buffer.
- `uv run pytest` → **2796 passed**, exit 0 (unit + integration).
- `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (29/4615 missed; phase-123 modules 99–100%).
- `uv run pytest tests/e2e/test_chat_image_questions.py -v --no-cov` → **5 passed** in isolation.
- `uv run ruff check . && uv run pyright` → clean (0 errors).

**Completion criteria:** (1) attach→send→multimodal text+image to the model, bubble/reload/shared all render it, saved chat stores the PATH with `"base64" not in json.dumps(stored)` — **verified** (E2E tests 1–4 + integration round-trip); (2) `BOR_IMAGES=false` — control hidden, exact hinted error frame, zero model calls / no query_log row — **verified** (E2E test 5 + integration); (3) text-only byte-identical (`content` stays a plain `str`) — **verified** (unit + integration); (4) all gates green — **verified**; (5) commit + phase move — left to the harness per pipeline rules (no `git add`/`commit` run).

No defects found; no live-infrastructure changes (repo + local dev DB only). **Next pending phase: none** — 123 is the last phase in `todo/`.
This commit is contained in:
2026-09-25 05:19:18 -04:00
parent a19d78d284
commit bef24e05e2
48 changed files with 3910 additions and 71 deletions
+128 -2
View File
@@ -179,6 +179,27 @@ as ``reasoning_content`` on the assistant message (the preserve-
thinking wire convention, A4). The per-turn log line records
``history_msgs=N`` after ``kb_chars=N`` (0 when the request carries no
history — the two-message request stays byte-identical).
Question images (phase 123, TODO L6; LOCKED A5/A7): the request may
attach ONE image to the CURRENT question — ``ChatRequest.image`` is the
STORED path from ``POST /api/chat-images`` (``app.api.chat_images``),
never a data URL. The turn validates it BEFORE any model call (the
toggle gate — ``settings.images`` false settles the phase-114 hinted
error frame; the stale-file gate — the stored bytes deleted out-of-
band settles the same frame shape) and, when valid, the user message's
content becomes the multimodal list ``[{type: "text", …}, {type:
"image_url", image_url: {url: <data URL>}}]`` — built by
:func:`build_user_content` at BOTH construction sites (this module's
deflected-branch ``messages`` list and the grounded branch's
``run_agent`` call, which builds its own ``[system, *history, user]``
— the flow is pinned in ``app.rag.agent.run_agent``). The data URL is
the shared :func:`app.rag.summarizer.image_data_url` (one
construction, both call sites — the phase-122 describe path).
``image=None`` keeps the plain-string content byte-identical to
pre-phase. A question image is turn-local: it is NEVER indexed as a
document, and prior turns' images are never replayed into the model's
history (``history_to_messages`` is unchanged — the text of a prior
turn that had an image stands alone, LOCKED A7).
"""
from __future__ import annotations
@@ -188,6 +209,7 @@ import logging
import time
from collections.abc import AsyncIterator, Sequence
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
from fastapi import APIRouter, Depends
@@ -229,6 +251,11 @@ from app.rag.retriever import (
)
from app.rag.scaffolding import ScaffoldingFilter # phase 71: the streaming filter
from app.rag.suggestions import derive_suggestions
from app.rag.summarizer import (
IMAGE_FALLBACK_MIME,
IMAGE_MIMES,
image_data_url,
) # phase 123: the phase-122 mime map + the shared data-URL helper
from app.schemas import (
ChatDoneEvent,
ChatErrorEvent,
@@ -281,6 +308,37 @@ def sse_event(payload: dict[str, Any]) -> str:
return f"data: {json.dumps(payload, ensure_ascii=False)}\n\n"
def build_user_content(
message: str,
image_path: str | None,
settings: Settings,
) -> str | list[dict[str, Any]]:
"""The current turn's user message content (phase 123, task 01).
*image_path* ``None`` (every text-only question) → the plain
question string — byte-identical to pre-phase-123 (the multimodal
branch is inert). Set → the OpenAI-compatible multimodal content
list: the text part + the image part, a data URL built server-side
from the stored bytes (``settings.chat_image_dir`` + the path's
filename) and the phase-122 ``IMAGE_MIMES`` map (one map, one
truth) through the shared :func:`app.rag.summarizer.image_data_url`
helper (one construction, both call sites — the phase-122 describe
path). The caller has already validated the path shape (the
``ChatRequest.image`` schema guard) and the file's existence (the
pre-stream turn gate). The image applies to the CURRENT turn only
(LOCKED A7) — prior turns' images are never replayed.
"""
if image_path is None:
return message
filename = image_path.rsplit("/", 1)[-1]
data = (Path(settings.chat_image_dir).expanduser() / filename).read_bytes()
mime = IMAGE_MIMES.get(Path(filename).suffix.lower(), IMAGE_FALLBACK_MIME)
return [
{"type": "text", "text": message},
{"type": "image_url", "image_url": {"url": image_data_url(data, mime)}},
]
@dataclass
class TurnPlan:
"""What one chat turn sends to the LLM and reports on ``done``."""
@@ -494,6 +552,59 @@ async def chat(
retries_used = 0 # phase 67: LLM requests restarted this turn (log line)
try:
settings = get_settings()
# Phase 123 (TODO L6, task 01): the question's attached
# image — validated BEFORE any model call (the embed is
# a model call): a rejected turn calls nothing and
# settles with the phase-114 error frame (the existing
# error-path convention — no ``done``, no ``query_log``
# row, no persisted record; the question is not saved).
# Two gates, in order:
# 1. the ``images`` toggle (``BOR_IMAGES``) — off with
# an image set: the HINTED frame (the client's
# banner shows the hint in place of its default
# reachability copy, phase 114);
# 2. the stored file — the schema already pinned the
# path shape, but the file may have been deleted
# out-of-band (the stale-path edge): the same frame
# shape, no hint (the banner's default copy is the
# honest fallback — there is nothing to point at).
if request.image is not None:
if not settings.images:
logger.warning(
"chat: image question rejected (images toggle "
"off) question=%r image=%r",
request.message,
request.image,
)
settled = True # terminal: the error frame settles the turn
yield sse_event(
ChatErrorEvent(
detail="Image support is turned off on this server.",
hint=(
"Enable BOR_IMAGES in the server's .env "
"(and restart) to ask with an image."
),
).model_dump()
)
return
image_file = (
Path(settings.chat_image_dir).expanduser()
/ request.image.rsplit("/", 1)[-1]
)
if not image_file.is_file():
logger.warning(
"chat: image question rejected (stored file "
"missing) question=%r image=%r",
request.message,
request.image,
)
settled = True # terminal: the error frame settles the turn
yield sse_event(
ChatErrorEvent(
detail="That image is no longer available."
).model_dump()
)
return
# Phase 74 (TODO L4): the client's prior turns, mapped ONCE
# per turn — trimmed newest-first against the settings
# budgets, assistant turns carrying their prior thinking as
@@ -640,10 +751,21 @@ async def chat(
).model_dump()
)
return
# Phase 123 (task 01): the user message's content — the
# plain question string (``image=None`` — byte-identical
# to pre-phase) or the multimodal content list (text
# part + image_url data URL; the image was validated
# above). BOTH construction sites use the same build:
# this deflected-branch list and the grounded branch's
# ``run_agent`` call below (run_agent builds its own
# ``[system, *history, user]`` — pinned there).
user_content: str | list[dict[str, Any]] = build_user_content(
request.message, request.image, settings
)
messages: list[dict[str, Any]] = [
{"role": "system", "content": plan.system_prompt},
*hist, # phase 74: the trimmed prior turns (empty by default)
{"role": "user", "content": request.message},
{"role": "user", "content": user_content},
]
# 3. Stream the answer (grounded, or an honest deflection).
@@ -689,7 +811,11 @@ async def chat(
llm,
db_factory, # SEC-14-04: session factory, not a long-lived session
system_prompt=plan.system_prompt,
user_message=request.message,
# Phase 123 (task 01): the plain question string
# (image=None) or the multimodal content list
# (validated above) — run_agent builds its own
# user message from this value (see its docstring).
user_message=user_content,
seed_docs=plan.suggested_docs, # phase 118 (A4): the suggestion tier
settings=settings,
holder=holder,
+158
View File
@@ -0,0 +1,158 @@
"""Question-image upload/serve pair (phase 123, TODO L6 — attach an
image to a question).
A user attaches ONE image to a chat question (LOCKED A5): the bytes are
stored server-side under ``chat_image_dir/<uuid4().hex>.<ext>`` — NOT
base64 in saved/shared chats. ``POST /api/chat-images`` answers
``{"path": "/api/chat-images/<uuid>.<ext>"}`` and that path is what
``ChatRequest.image`` / ``ChatMessage.image`` carry (a stored PATH,
never a data URL — the upload endpoint owns the size/mime
enforcement). ``GET /api/chat-images/{filename}`` serves the bytes back
so the user bubble, the refreshed page, and the shared chat can render
the attachment. A question image is NEVER indexed as a document (no
importer call) — it is turn-local storage, not a source.
Auth posture: the upload is user-gated exactly like the chat turn it
feeds (``require_user`` — the question itself is user-gated, and an
anonymous 10 MiB disk-fill would be a DoS); the serve route is PUBLIC
like saved-chat content (phase 55 A1 — a saved chat's id is already its
credential, and the image is part of that content; the filename is an
unguessable ``uuid4().hex`` — no enumeration value).
The extension is the source of truth (the Content-Type header is a
hint — the archive-uploader precedent): it must be in the phase-122
image set (``settings.image_extension_set`` — the same frozenset the
image-document walk uses), lowercased; total bytes are streamed with a
``chat_image_max_mb`` cap (413, fixed detail naming the cap — never
echoing the filename).
"""
from __future__ import annotations
import logging
import re
import uuid
from pathlib import Path
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile
from fastapi.responses import FileResponse
from app.config import get_settings
from app.core.auth import require_user
from app.rag.summarizer import IMAGE_FALLBACK_MIME, IMAGE_MIMES
logger = logging.getLogger(__name__)
router = APIRouter(tags=["chat-images"])
#: Stream receive chunk (the git-sources upload's 1 MiB pattern).
_STREAM_CHUNK = 1 << 20
#: Stored question-image filename guard: ``<uuid4().hex>.<ext>`` — 32
#: hex chars + one of the six image extensions (the ``ChatRequest.image``
#: path pattern's filename part, kept in lockstep with it). Anything
#: else 404s — no path traversal by construction (the route parameter
#: cannot carry a ``/`` and the regex rejects everything but the
#: upload endpoint's own naming).
_CHAT_IMAGE_FILENAME_RE = re.compile(r"^[0-9a-fA-F]{32}\.(png|jpe?g|webp|gif|bmp)$")
@router.post("/chat-images")
async def upload_chat_image(
file: UploadFile = File(...), # noqa: B008
_user: None = Depends(require_user), # noqa: B008
) -> dict[str, str]:
"""Store one question image (phase 123, task 01; LOCKED A5).
Gates, in order:
1. **extension** — the file name's extension (lowercased) must be in
the phase-122 image set (``settings.image_extension_set``); the
Content-Type header is a hint, never a source of truth (the
archive-uploader precedent). A missing/unknown extension is a
422 naming the accepted set — a fixed detail, no filename echo.
2. **size** — the bytes are streamed (1 MiB chunks) with the
``chat_image_max_mb`` cap; over-cap is a 413 naming the cap
(fixed detail — never echoing the filename), the temp file is
removed, and nothing is stored.
The file lands as ``<uuid4().hex>.<ext>`` in ``chat_image_dir``
(created on demand; a dotfile temp is renamed into place, so a
failed/partial receive never leaves a servable-looking file). The
response is the served path — ``{"path":
"/api/chat-images/<uuid>.<ext>"}`` — the value ``ChatRequest.image``
accepts (never a data URL, never the on-disk location).
"""
settings = get_settings()
filename = file.filename or ""
ext = Path(filename).suffix.lower()
if ext not in settings.image_extension_set:
accepted = ", ".join(sorted(settings.image_extension_set))
raise HTTPException(
status_code=422, detail=f"only {accepted} images are accepted"
)
root = Path(settings.chat_image_dir).expanduser()
root.mkdir(parents=True, exist_ok=True)
name = f"{uuid.uuid4().hex}{ext}"
temp = root / f".{name}.upload"
max_bytes = settings.chat_image_max_mb * 1024 * 1024
total = 0
try:
# Stream with the cap — the dotfile temp is hidden from any
# listing of the store dir (the git-sources upload pattern).
with open(temp, "wb") as out:
while chunk := await file.read(_STREAM_CHUNK):
total += len(chunk)
if total > max_bytes:
raise HTTPException(
status_code=413,
detail=(
f"the image exceeds the "
f"{settings.chat_image_max_mb} MB limit"
),
)
out.write(chunk)
temp.rename(root / name)
except BaseException:
# A failed receive (413, broken pipe, cancellation) leaves no
# file behind — the final name was never created.
temp.unlink(missing_ok=True)
raise
logger.info(
"chat-image: uploaded name=%s bytes=%d", name, total
)
return {"path": f"/api/chat-images/{name}"}
@router.get("/chat-images/{filename}", response_class=FileResponse)
def serve_chat_image(filename: str) -> FileResponse:
"""Serve one stored question image (phase 123, task 01).
PUBLIC (no auth dependency) — like saved-chat content: the saved
chat's id is already its credential (phase 55 A1), the image is part
of that content, and the filename is an unguessable ``uuid4().hex``
(no enumeration value).
Every non-servable case is a 404 with the same fixed detail — a
filename that does not match ``<uuid-hex>.<ext>`` (the regex guard:
no path traversal by construction, no 422 that would hint at
accepted shapes) and a matching name whose file is missing (the
stale-path edge — the file was deleted out-of-band). Servable files
stream the exact bytes with the phase-122 ``IMAGE_MIMES``
``Content-Type`` (one map, one truth with the describe call's
data-URL mime; the six-extension guard makes the fallback
unreachable) and ``Cache-Control: private, max-age=3600`` (the
phase-122 serve-route convention — the bytes are content-hashed
uuids, bustable by re-upload).
"""
if _CHAT_IMAGE_FILENAME_RE.fullmatch(filename) is None:
raise HTTPException(status_code=404, detail="chat image not found")
path = Path(get_settings().chat_image_dir).expanduser() / filename
if not path.is_file():
raise HTTPException(status_code=404, detail="chat image not found")
media_type = IMAGE_MIMES.get(path.suffix.lower(), IMAGE_FALLBACK_MIME)
return FileResponse(
path,
media_type=media_type,
headers={"Cache-Control": "private, max-age=3600"},
)