phase: 123_chat_image_questions
All gates green. Verification complete. **Phase 123 — final verification pass (all 4 tasks already in `complete/`)** - Verified the full implementation is in the working tree: `app/api/chat_images.py` (upload/serve pair), `ChatRequest.image`/`ChatMessage.image` (path-validated, omitted-when-None), toggle-off + stale-file hinted error frames, `build_user_content` multimodal build at both sites (chat.py deflected branch + `run_agent`), config-gated composer attach/preview/upload-then-send, restore + shared rendering, CSP `img-src 'self' data:` carve-out, mock-LLM capture buffer. - `uv run pytest` → **2796 passed**, exit 0 (unit + integration). - `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (29/4615 missed; phase-123 modules 99–100%). - `uv run pytest tests/e2e/test_chat_image_questions.py -v --no-cov` → **5 passed** in isolation. - `uv run ruff check . && uv run pyright` → clean (0 errors). **Completion criteria:** (1) attach→send→multimodal text+image to the model, bubble/reload/shared all render it, saved chat stores the PATH with `"base64" not in json.dumps(stored)` — **verified** (E2E tests 1–4 + integration round-trip); (2) `BOR_IMAGES=false` — control hidden, exact hinted error frame, zero model calls / no query_log row — **verified** (E2E test 5 + integration); (3) text-only byte-identical (`content` stays a plain `str`) — **verified** (unit + integration); (4) all gates green — **verified**; (5) commit + phase move — left to the harness per pipeline rules (no `git add`/`commit` run). No defects found; no live-infrastructure changes (repo + local dev DB only). **Next pending phase: none** — 123 is the last phase in `todo/`.
This commit is contained in:
+15
-1
@@ -1393,7 +1393,7 @@ async def run_agent(
|
||||
db_factory: Callable[[], Session],
|
||||
*,
|
||||
system_prompt: str,
|
||||
user_message: str,
|
||||
user_message: str | list[dict[str, Any]],
|
||||
seed_docs: Sequence[Document],
|
||||
settings: Settings,
|
||||
holder: AgentHolder,
|
||||
@@ -1424,6 +1424,20 @@ async def run_agent(
|
||||
unchanged. ``()`` (the default) keeps the pre-phase-74 two-message
|
||||
request byte-identical.
|
||||
|
||||
User message (phase 123, TODO L6): *user_message* is the current
|
||||
turn's user content — the plain question string (every text-only
|
||||
turn, byte-identical to pre-phase) OR the multimodal content list
|
||||
``[{type: "text", …}, {type: "image_url", …}]`` for a question that
|
||||
carried an image. FLOW (pinned): the API layer
|
||||
(``app.api.chat``) builds the content via its ``build_user_content``
|
||||
helper and passes it HERE — ``run_agent`` builds its OWN
|
||||
``[system, *history, user]`` list from this value (it does NOT
|
||||
receive the already-built ``messages``; the deflected branch is the
|
||||
one that consumes chat.py's list directly), so a single value
|
||||
covers both shapes and the tool rounds / recovery / retries operate
|
||||
on it untouched. Prior turns' images are never replayed (LOCKED A7
|
||||
— *history* is text-only by construction).
|
||||
|
||||
Retries (phase 67, owner-locked A2): every model request goes through
|
||||
:func:`chat_stream_retried` — a failed round is retried **before** its
|
||||
first piece (same messages, ``settings.llm_retries`` restarts, a flat
|
||||
|
||||
+7
-3
@@ -487,11 +487,15 @@ class LLMClient:
|
||||
|
||||
Messages are passed to the request body VERBATIM: string-only
|
||||
``{role, content}`` dicts are byte-identical on the wire to the
|
||||
pre-phase-74 requests, and an assistant message may additionally
|
||||
pre-phase-74 requests, an assistant message may additionally
|
||||
carry ``reasoning_content`` (the client's prior thinking, phase
|
||||
74 — the same wire field the model uses for its OWN reasoning on
|
||||
the response side; the ``openai`` SDK passes message dicts
|
||||
through untouched, so no transport change).
|
||||
the response side), and a user message's content may be the
|
||||
multimodal parts list of a question that carried an image
|
||||
(phase 123 — ``[{type: "text", …}, {type: "image_url", …}]``,
|
||||
built by ``app.api.chat.build_user_content``); the ``openai``
|
||||
SDK passes message dicts through untouched, so no transport
|
||||
change covers all three shapes.
|
||||
|
||||
``stream=True`` against the OpenAI-compatible endpoint, yielding
|
||||
typed :class:`StreamPiece` values. Wire convention (verified live
|
||||
|
||||
+13
-1
@@ -117,6 +117,18 @@ IMAGE_MIMES: dict[str, str] = {
|
||||
IMAGE_FALLBACK_MIME = "application/octet-stream"
|
||||
|
||||
|
||||
def image_data_url(data: bytes, mime: str) -> str:
|
||||
"""One image's bytes as a data URL for a multimodal message (phase
|
||||
122; factored for phase 123, task 01).
|
||||
|
||||
``data:<mime>;base64,<ascii>`` — the OpenAI-compatible ``image_url``
|
||||
payload's ``url``. ONE helper, both call sites: :func:`describe_image`
|
||||
(document-image descriptions) and the chat question-image path
|
||||
(``app.api.chat``'s multimodal user message, phase 123).
|
||||
"""
|
||||
return f"data:{mime};base64,{base64.b64encode(data).decode('ascii')}"
|
||||
|
||||
|
||||
class SummaryLLM(Protocol):
|
||||
"""The one-shot chat surface the summarizer needs.
|
||||
|
||||
@@ -223,7 +235,7 @@ async def describe_image(
|
||||
the doc is skipped, counted in ``images_failed``, and the sync
|
||||
continues (LOCKED A3).
|
||||
"""
|
||||
data_url = f"data:{mime};base64,{base64.b64encode(data).decode('ascii')}"
|
||||
data_url = image_data_url(data, mime)
|
||||
messages: list[dict[str, Any]] = [
|
||||
{
|
||||
"role": "user",
|
||||
|
||||
Reference in New Issue
Block a user