phase: 123_chat_image_questions
All gates green. Verification complete. **Phase 123 — final verification pass (all 4 tasks already in `complete/`)** - Verified the full implementation is in the working tree: `app/api/chat_images.py` (upload/serve pair), `ChatRequest.image`/`ChatMessage.image` (path-validated, omitted-when-None), toggle-off + stale-file hinted error frames, `build_user_content` multimodal build at both sites (chat.py deflected branch + `run_agent`), config-gated composer attach/preview/upload-then-send, restore + shared rendering, CSP `img-src 'self' data:` carve-out, mock-LLM capture buffer. - `uv run pytest` → **2796 passed**, exit 0 (unit + integration). - `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (29/4615 missed; phase-123 modules 99–100%). - `uv run pytest tests/e2e/test_chat_image_questions.py -v --no-cov` → **5 passed** in isolation. - `uv run ruff check . && uv run pyright` → clean (0 errors). **Completion criteria:** (1) attach→send→multimodal text+image to the model, bubble/reload/shared all render it, saved chat stores the PATH with `"base64" not in json.dumps(stored)` — **verified** (E2E tests 1–4 + integration round-trip); (2) `BOR_IMAGES=false` — control hidden, exact hinted error frame, zero model calls / no query_log row — **verified** (E2E test 5 + integration); (3) text-only byte-identical (`content` stays a plain `str`) — **verified** (unit + integration); (4) all gates green — **verified**; (5) commit + phase move — left to the harness per pipeline rules (no `git add`/`commit` run). No defects found; no live-infrastructure changes (repo + local dev DB only). **Next pending phase: none** — 123 is the last phase in `todo/`.
This commit is contained in:
+128
-2
@@ -179,6 +179,27 @@ as ``reasoning_content`` on the assistant message (the preserve-
|
||||
thinking wire convention, A4). The per-turn log line records
|
||||
``history_msgs=N`` after ``kb_chars=N`` (0 when the request carries no
|
||||
history — the two-message request stays byte-identical).
|
||||
|
||||
Question images (phase 123, TODO L6; LOCKED A5/A7): the request may
|
||||
attach ONE image to the CURRENT question — ``ChatRequest.image`` is the
|
||||
STORED path from ``POST /api/chat-images`` (``app.api.chat_images``),
|
||||
never a data URL. The turn validates it BEFORE any model call (the
|
||||
toggle gate — ``settings.images`` false settles the phase-114 hinted
|
||||
error frame; the stale-file gate — the stored bytes deleted out-of-
|
||||
band settles the same frame shape) and, when valid, the user message's
|
||||
content becomes the multimodal list ``[{type: "text", …}, {type:
|
||||
"image_url", image_url: {url: <data URL>}}]`` — built by
|
||||
:func:`build_user_content` at BOTH construction sites (this module's
|
||||
deflected-branch ``messages`` list and the grounded branch's
|
||||
``run_agent`` call, which builds its own ``[system, *history, user]``
|
||||
— the flow is pinned in ``app.rag.agent.run_agent``). The data URL is
|
||||
the shared :func:`app.rag.summarizer.image_data_url` (one
|
||||
construction, both call sites — the phase-122 describe path).
|
||||
``image=None`` keeps the plain-string content byte-identical to
|
||||
pre-phase. A question image is turn-local: it is NEVER indexed as a
|
||||
document, and prior turns' images are never replayed into the model's
|
||||
history (``history_to_messages`` is unchanged — the text of a prior
|
||||
turn that had an image stands alone, LOCKED A7).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -188,6 +209,7 @@ import logging
|
||||
import time
|
||||
from collections.abc import AsyncIterator, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
@@ -229,6 +251,11 @@ from app.rag.retriever import (
|
||||
)
|
||||
from app.rag.scaffolding import ScaffoldingFilter # phase 71: the streaming filter
|
||||
from app.rag.suggestions import derive_suggestions
|
||||
from app.rag.summarizer import (
|
||||
IMAGE_FALLBACK_MIME,
|
||||
IMAGE_MIMES,
|
||||
image_data_url,
|
||||
) # phase 123: the phase-122 mime map + the shared data-URL helper
|
||||
from app.schemas import (
|
||||
ChatDoneEvent,
|
||||
ChatErrorEvent,
|
||||
@@ -281,6 +308,37 @@ def sse_event(payload: dict[str, Any]) -> str:
|
||||
return f"data: {json.dumps(payload, ensure_ascii=False)}\n\n"
|
||||
|
||||
|
||||
def build_user_content(
|
||||
message: str,
|
||||
image_path: str | None,
|
||||
settings: Settings,
|
||||
) -> str | list[dict[str, Any]]:
|
||||
"""The current turn's user message content (phase 123, task 01).
|
||||
|
||||
*image_path* ``None`` (every text-only question) → the plain
|
||||
question string — byte-identical to pre-phase-123 (the multimodal
|
||||
branch is inert). Set → the OpenAI-compatible multimodal content
|
||||
list: the text part + the image part, a data URL built server-side
|
||||
from the stored bytes (``settings.chat_image_dir`` + the path's
|
||||
filename) and the phase-122 ``IMAGE_MIMES`` map (one map, one
|
||||
truth) through the shared :func:`app.rag.summarizer.image_data_url`
|
||||
helper (one construction, both call sites — the phase-122 describe
|
||||
path). The caller has already validated the path shape (the
|
||||
``ChatRequest.image`` schema guard) and the file's existence (the
|
||||
pre-stream turn gate). The image applies to the CURRENT turn only
|
||||
(LOCKED A7) — prior turns' images are never replayed.
|
||||
"""
|
||||
if image_path is None:
|
||||
return message
|
||||
filename = image_path.rsplit("/", 1)[-1]
|
||||
data = (Path(settings.chat_image_dir).expanduser() / filename).read_bytes()
|
||||
mime = IMAGE_MIMES.get(Path(filename).suffix.lower(), IMAGE_FALLBACK_MIME)
|
||||
return [
|
||||
{"type": "text", "text": message},
|
||||
{"type": "image_url", "image_url": {"url": image_data_url(data, mime)}},
|
||||
]
|
||||
|
||||
|
||||
@dataclass
|
||||
class TurnPlan:
|
||||
"""What one chat turn sends to the LLM and reports on ``done``."""
|
||||
@@ -494,6 +552,59 @@ async def chat(
|
||||
retries_used = 0 # phase 67: LLM requests restarted this turn (log line)
|
||||
try:
|
||||
settings = get_settings()
|
||||
# Phase 123 (TODO L6, task 01): the question's attached
|
||||
# image — validated BEFORE any model call (the embed is
|
||||
# a model call): a rejected turn calls nothing and
|
||||
# settles with the phase-114 error frame (the existing
|
||||
# error-path convention — no ``done``, no ``query_log``
|
||||
# row, no persisted record; the question is not saved).
|
||||
# Two gates, in order:
|
||||
# 1. the ``images`` toggle (``BOR_IMAGES``) — off with
|
||||
# an image set: the HINTED frame (the client's
|
||||
# banner shows the hint in place of its default
|
||||
# reachability copy, phase 114);
|
||||
# 2. the stored file — the schema already pinned the
|
||||
# path shape, but the file may have been deleted
|
||||
# out-of-band (the stale-path edge): the same frame
|
||||
# shape, no hint (the banner's default copy is the
|
||||
# honest fallback — there is nothing to point at).
|
||||
if request.image is not None:
|
||||
if not settings.images:
|
||||
logger.warning(
|
||||
"chat: image question rejected (images toggle "
|
||||
"off) question=%r image=%r",
|
||||
request.message,
|
||||
request.image,
|
||||
)
|
||||
settled = True # terminal: the error frame settles the turn
|
||||
yield sse_event(
|
||||
ChatErrorEvent(
|
||||
detail="Image support is turned off on this server.",
|
||||
hint=(
|
||||
"Enable BOR_IMAGES in the server's .env "
|
||||
"(and restart) to ask with an image."
|
||||
),
|
||||
).model_dump()
|
||||
)
|
||||
return
|
||||
image_file = (
|
||||
Path(settings.chat_image_dir).expanduser()
|
||||
/ request.image.rsplit("/", 1)[-1]
|
||||
)
|
||||
if not image_file.is_file():
|
||||
logger.warning(
|
||||
"chat: image question rejected (stored file "
|
||||
"missing) question=%r image=%r",
|
||||
request.message,
|
||||
request.image,
|
||||
)
|
||||
settled = True # terminal: the error frame settles the turn
|
||||
yield sse_event(
|
||||
ChatErrorEvent(
|
||||
detail="That image is no longer available."
|
||||
).model_dump()
|
||||
)
|
||||
return
|
||||
# Phase 74 (TODO L4): the client's prior turns, mapped ONCE
|
||||
# per turn — trimmed newest-first against the settings
|
||||
# budgets, assistant turns carrying their prior thinking as
|
||||
@@ -640,10 +751,21 @@ async def chat(
|
||||
).model_dump()
|
||||
)
|
||||
return
|
||||
# Phase 123 (task 01): the user message's content — the
|
||||
# plain question string (``image=None`` — byte-identical
|
||||
# to pre-phase) or the multimodal content list (text
|
||||
# part + image_url data URL; the image was validated
|
||||
# above). BOTH construction sites use the same build:
|
||||
# this deflected-branch list and the grounded branch's
|
||||
# ``run_agent`` call below (run_agent builds its own
|
||||
# ``[system, *history, user]`` — pinned there).
|
||||
user_content: str | list[dict[str, Any]] = build_user_content(
|
||||
request.message, request.image, settings
|
||||
)
|
||||
messages: list[dict[str, Any]] = [
|
||||
{"role": "system", "content": plan.system_prompt},
|
||||
*hist, # phase 74: the trimmed prior turns (empty by default)
|
||||
{"role": "user", "content": request.message},
|
||||
{"role": "user", "content": user_content},
|
||||
]
|
||||
|
||||
# 3. Stream the answer (grounded, or an honest deflection).
|
||||
@@ -689,7 +811,11 @@ async def chat(
|
||||
llm,
|
||||
db_factory, # SEC-14-04: session factory, not a long-lived session
|
||||
system_prompt=plan.system_prompt,
|
||||
user_message=request.message,
|
||||
# Phase 123 (task 01): the plain question string
|
||||
# (image=None) or the multimodal content list
|
||||
# (validated above) — run_agent builds its own
|
||||
# user message from this value (see its docstring).
|
||||
user_message=user_content,
|
||||
seed_docs=plan.suggested_docs, # phase 118 (A4): the suggestion tier
|
||||
settings=settings,
|
||||
holder=holder,
|
||||
|
||||
Reference in New Issue
Block a user