phase: 123_chat_image_questions
All gates green. Verification complete. **Phase 123 — final verification pass (all 4 tasks already in `complete/`)** - Verified the full implementation is in the working tree: `app/api/chat_images.py` (upload/serve pair), `ChatRequest.image`/`ChatMessage.image` (path-validated, omitted-when-None), toggle-off + stale-file hinted error frames, `build_user_content` multimodal build at both sites (chat.py deflected branch + `run_agent`), config-gated composer attach/preview/upload-then-send, restore + shared rendering, CSP `img-src 'self' data:` carve-out, mock-LLM capture buffer. - `uv run pytest` → **2796 passed**, exit 0 (unit + integration). - `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (29/4615 missed; phase-123 modules 99–100%). - `uv run pytest tests/e2e/test_chat_image_questions.py -v --no-cov` → **5 passed** in isolation. - `uv run ruff check . && uv run pyright` → clean (0 errors). **Completion criteria:** (1) attach→send→multimodal text+image to the model, bubble/reload/shared all render it, saved chat stores the PATH with `"base64" not in json.dumps(stored)` — **verified** (E2E tests 1–4 + integration round-trip); (2) `BOR_IMAGES=false` — control hidden, exact hinted error frame, zero model calls / no query_log row — **verified** (E2E test 5 + integration); (3) text-only byte-identical (`content` stays a plain `str`) — **verified** (unit + integration); (4) all gates green — **verified**; (5) commit + phase move — left to the harness per pipeline rules (no `git add`/`commit` run). No defects found; no live-infrastructure changes (repo + local dev DB only). **Next pending phase: none** — 123 is the last phase in `todo/`.
This commit is contained in:
+128
-2
@@ -179,6 +179,27 @@ as ``reasoning_content`` on the assistant message (the preserve-
|
||||
thinking wire convention, A4). The per-turn log line records
|
||||
``history_msgs=N`` after ``kb_chars=N`` (0 when the request carries no
|
||||
history — the two-message request stays byte-identical).
|
||||
|
||||
Question images (phase 123, TODO L6; LOCKED A5/A7): the request may
|
||||
attach ONE image to the CURRENT question — ``ChatRequest.image`` is the
|
||||
STORED path from ``POST /api/chat-images`` (``app.api.chat_images``),
|
||||
never a data URL. The turn validates it BEFORE any model call (the
|
||||
toggle gate — ``settings.images`` false settles the phase-114 hinted
|
||||
error frame; the stale-file gate — the stored bytes deleted out-of-
|
||||
band settles the same frame shape) and, when valid, the user message's
|
||||
content becomes the multimodal list ``[{type: "text", …}, {type:
|
||||
"image_url", image_url: {url: <data URL>}}]`` — built by
|
||||
:func:`build_user_content` at BOTH construction sites (this module's
|
||||
deflected-branch ``messages`` list and the grounded branch's
|
||||
``run_agent`` call, which builds its own ``[system, *history, user]``
|
||||
— the flow is pinned in ``app.rag.agent.run_agent``). The data URL is
|
||||
the shared :func:`app.rag.summarizer.image_data_url` (one
|
||||
construction, both call sites — the phase-122 describe path).
|
||||
``image=None`` keeps the plain-string content byte-identical to
|
||||
pre-phase. A question image is turn-local: it is NEVER indexed as a
|
||||
document, and prior turns' images are never replayed into the model's
|
||||
history (``history_to_messages`` is unchanged — the text of a prior
|
||||
turn that had an image stands alone, LOCKED A7).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -188,6 +209,7 @@ import logging
|
||||
import time
|
||||
from collections.abc import AsyncIterator, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
@@ -229,6 +251,11 @@ from app.rag.retriever import (
|
||||
)
|
||||
from app.rag.scaffolding import ScaffoldingFilter # phase 71: the streaming filter
|
||||
from app.rag.suggestions import derive_suggestions
|
||||
from app.rag.summarizer import (
|
||||
IMAGE_FALLBACK_MIME,
|
||||
IMAGE_MIMES,
|
||||
image_data_url,
|
||||
) # phase 123: the phase-122 mime map + the shared data-URL helper
|
||||
from app.schemas import (
|
||||
ChatDoneEvent,
|
||||
ChatErrorEvent,
|
||||
@@ -281,6 +308,37 @@ def sse_event(payload: dict[str, Any]) -> str:
|
||||
return f"data: {json.dumps(payload, ensure_ascii=False)}\n\n"
|
||||
|
||||
|
||||
def build_user_content(
|
||||
message: str,
|
||||
image_path: str | None,
|
||||
settings: Settings,
|
||||
) -> str | list[dict[str, Any]]:
|
||||
"""The current turn's user message content (phase 123, task 01).
|
||||
|
||||
*image_path* ``None`` (every text-only question) → the plain
|
||||
question string — byte-identical to pre-phase-123 (the multimodal
|
||||
branch is inert). Set → the OpenAI-compatible multimodal content
|
||||
list: the text part + the image part, a data URL built server-side
|
||||
from the stored bytes (``settings.chat_image_dir`` + the path's
|
||||
filename) and the phase-122 ``IMAGE_MIMES`` map (one map, one
|
||||
truth) through the shared :func:`app.rag.summarizer.image_data_url`
|
||||
helper (one construction, both call sites — the phase-122 describe
|
||||
path). The caller has already validated the path shape (the
|
||||
``ChatRequest.image`` schema guard) and the file's existence (the
|
||||
pre-stream turn gate). The image applies to the CURRENT turn only
|
||||
(LOCKED A7) — prior turns' images are never replayed.
|
||||
"""
|
||||
if image_path is None:
|
||||
return message
|
||||
filename = image_path.rsplit("/", 1)[-1]
|
||||
data = (Path(settings.chat_image_dir).expanduser() / filename).read_bytes()
|
||||
mime = IMAGE_MIMES.get(Path(filename).suffix.lower(), IMAGE_FALLBACK_MIME)
|
||||
return [
|
||||
{"type": "text", "text": message},
|
||||
{"type": "image_url", "image_url": {"url": image_data_url(data, mime)}},
|
||||
]
|
||||
|
||||
|
||||
@dataclass
|
||||
class TurnPlan:
|
||||
"""What one chat turn sends to the LLM and reports on ``done``."""
|
||||
@@ -494,6 +552,59 @@ async def chat(
|
||||
retries_used = 0 # phase 67: LLM requests restarted this turn (log line)
|
||||
try:
|
||||
settings = get_settings()
|
||||
# Phase 123 (TODO L6, task 01): the question's attached
|
||||
# image — validated BEFORE any model call (the embed is
|
||||
# a model call): a rejected turn calls nothing and
|
||||
# settles with the phase-114 error frame (the existing
|
||||
# error-path convention — no ``done``, no ``query_log``
|
||||
# row, no persisted record; the question is not saved).
|
||||
# Two gates, in order:
|
||||
# 1. the ``images`` toggle (``BOR_IMAGES``) — off with
|
||||
# an image set: the HINTED frame (the client's
|
||||
# banner shows the hint in place of its default
|
||||
# reachability copy, phase 114);
|
||||
# 2. the stored file — the schema already pinned the
|
||||
# path shape, but the file may have been deleted
|
||||
# out-of-band (the stale-path edge): the same frame
|
||||
# shape, no hint (the banner's default copy is the
|
||||
# honest fallback — there is nothing to point at).
|
||||
if request.image is not None:
|
||||
if not settings.images:
|
||||
logger.warning(
|
||||
"chat: image question rejected (images toggle "
|
||||
"off) question=%r image=%r",
|
||||
request.message,
|
||||
request.image,
|
||||
)
|
||||
settled = True # terminal: the error frame settles the turn
|
||||
yield sse_event(
|
||||
ChatErrorEvent(
|
||||
detail="Image support is turned off on this server.",
|
||||
hint=(
|
||||
"Enable BOR_IMAGES in the server's .env "
|
||||
"(and restart) to ask with an image."
|
||||
),
|
||||
).model_dump()
|
||||
)
|
||||
return
|
||||
image_file = (
|
||||
Path(settings.chat_image_dir).expanduser()
|
||||
/ request.image.rsplit("/", 1)[-1]
|
||||
)
|
||||
if not image_file.is_file():
|
||||
logger.warning(
|
||||
"chat: image question rejected (stored file "
|
||||
"missing) question=%r image=%r",
|
||||
request.message,
|
||||
request.image,
|
||||
)
|
||||
settled = True # terminal: the error frame settles the turn
|
||||
yield sse_event(
|
||||
ChatErrorEvent(
|
||||
detail="That image is no longer available."
|
||||
).model_dump()
|
||||
)
|
||||
return
|
||||
# Phase 74 (TODO L4): the client's prior turns, mapped ONCE
|
||||
# per turn — trimmed newest-first against the settings
|
||||
# budgets, assistant turns carrying their prior thinking as
|
||||
@@ -640,10 +751,21 @@ async def chat(
|
||||
).model_dump()
|
||||
)
|
||||
return
|
||||
# Phase 123 (task 01): the user message's content — the
|
||||
# plain question string (``image=None`` — byte-identical
|
||||
# to pre-phase) or the multimodal content list (text
|
||||
# part + image_url data URL; the image was validated
|
||||
# above). BOTH construction sites use the same build:
|
||||
# this deflected-branch list and the grounded branch's
|
||||
# ``run_agent`` call below (run_agent builds its own
|
||||
# ``[system, *history, user]`` — pinned there).
|
||||
user_content: str | list[dict[str, Any]] = build_user_content(
|
||||
request.message, request.image, settings
|
||||
)
|
||||
messages: list[dict[str, Any]] = [
|
||||
{"role": "system", "content": plan.system_prompt},
|
||||
*hist, # phase 74: the trimmed prior turns (empty by default)
|
||||
{"role": "user", "content": request.message},
|
||||
{"role": "user", "content": user_content},
|
||||
]
|
||||
|
||||
# 3. Stream the answer (grounded, or an honest deflection).
|
||||
@@ -689,7 +811,11 @@ async def chat(
|
||||
llm,
|
||||
db_factory, # SEC-14-04: session factory, not a long-lived session
|
||||
system_prompt=plan.system_prompt,
|
||||
user_message=request.message,
|
||||
# Phase 123 (task 01): the plain question string
|
||||
# (image=None) or the multimodal content list
|
||||
# (validated above) — run_agent builds its own
|
||||
# user message from this value (see its docstring).
|
||||
user_message=user_content,
|
||||
seed_docs=plan.suggested_docs, # phase 118 (A4): the suggestion tier
|
||||
settings=settings,
|
||||
holder=holder,
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
"""Question-image upload/serve pair (phase 123, TODO L6 — attach an
|
||||
image to a question).
|
||||
|
||||
A user attaches ONE image to a chat question (LOCKED A5): the bytes are
|
||||
stored server-side under ``chat_image_dir/<uuid4().hex>.<ext>`` — NOT
|
||||
base64 in saved/shared chats. ``POST /api/chat-images`` answers
|
||||
``{"path": "/api/chat-images/<uuid>.<ext>"}`` and that path is what
|
||||
``ChatRequest.image`` / ``ChatMessage.image`` carry (a stored PATH,
|
||||
never a data URL — the upload endpoint owns the size/mime
|
||||
enforcement). ``GET /api/chat-images/{filename}`` serves the bytes back
|
||||
so the user bubble, the refreshed page, and the shared chat can render
|
||||
the attachment. A question image is NEVER indexed as a document (no
|
||||
importer call) — it is turn-local storage, not a source.
|
||||
|
||||
Auth posture: the upload is user-gated exactly like the chat turn it
|
||||
feeds (``require_user`` — the question itself is user-gated, and an
|
||||
anonymous 10 MiB disk-fill would be a DoS); the serve route is PUBLIC
|
||||
like saved-chat content (phase 55 A1 — a saved chat's id is already its
|
||||
credential, and the image is part of that content; the filename is an
|
||||
unguessable ``uuid4().hex`` — no enumeration value).
|
||||
|
||||
The extension is the source of truth (the Content-Type header is a
|
||||
hint — the archive-uploader precedent): it must be in the phase-122
|
||||
image set (``settings.image_extension_set`` — the same frozenset the
|
||||
image-document walk uses), lowercased; total bytes are streamed with a
|
||||
``chat_image_max_mb`` cap (413, fixed detail naming the cap — never
|
||||
echoing the filename).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile
|
||||
from fastapi.responses import FileResponse
|
||||
|
||||
from app.config import get_settings
|
||||
from app.core.auth import require_user
|
||||
from app.rag.summarizer import IMAGE_FALLBACK_MIME, IMAGE_MIMES
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(tags=["chat-images"])
|
||||
|
||||
#: Stream receive chunk (the git-sources upload's 1 MiB pattern).
|
||||
_STREAM_CHUNK = 1 << 20
|
||||
|
||||
#: Stored question-image filename guard: ``<uuid4().hex>.<ext>`` — 32
|
||||
#: hex chars + one of the six image extensions (the ``ChatRequest.image``
|
||||
#: path pattern's filename part, kept in lockstep with it). Anything
|
||||
#: else 404s — no path traversal by construction (the route parameter
|
||||
#: cannot carry a ``/`` and the regex rejects everything but the
|
||||
#: upload endpoint's own naming).
|
||||
_CHAT_IMAGE_FILENAME_RE = re.compile(r"^[0-9a-fA-F]{32}\.(png|jpe?g|webp|gif|bmp)$")
|
||||
|
||||
|
||||
@router.post("/chat-images")
|
||||
async def upload_chat_image(
|
||||
file: UploadFile = File(...), # noqa: B008
|
||||
_user: None = Depends(require_user), # noqa: B008
|
||||
) -> dict[str, str]:
|
||||
"""Store one question image (phase 123, task 01; LOCKED A5).
|
||||
|
||||
Gates, in order:
|
||||
|
||||
1. **extension** — the file name's extension (lowercased) must be in
|
||||
the phase-122 image set (``settings.image_extension_set``); the
|
||||
Content-Type header is a hint, never a source of truth (the
|
||||
archive-uploader precedent). A missing/unknown extension is a
|
||||
422 naming the accepted set — a fixed detail, no filename echo.
|
||||
2. **size** — the bytes are streamed (1 MiB chunks) with the
|
||||
``chat_image_max_mb`` cap; over-cap is a 413 naming the cap
|
||||
(fixed detail — never echoing the filename), the temp file is
|
||||
removed, and nothing is stored.
|
||||
|
||||
The file lands as ``<uuid4().hex>.<ext>`` in ``chat_image_dir``
|
||||
(created on demand; a dotfile temp is renamed into place, so a
|
||||
failed/partial receive never leaves a servable-looking file). The
|
||||
response is the served path — ``{"path":
|
||||
"/api/chat-images/<uuid>.<ext>"}`` — the value ``ChatRequest.image``
|
||||
accepts (never a data URL, never the on-disk location).
|
||||
"""
|
||||
settings = get_settings()
|
||||
filename = file.filename or ""
|
||||
ext = Path(filename).suffix.lower()
|
||||
if ext not in settings.image_extension_set:
|
||||
accepted = ", ".join(sorted(settings.image_extension_set))
|
||||
raise HTTPException(
|
||||
status_code=422, detail=f"only {accepted} images are accepted"
|
||||
)
|
||||
|
||||
root = Path(settings.chat_image_dir).expanduser()
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
name = f"{uuid.uuid4().hex}{ext}"
|
||||
temp = root / f".{name}.upload"
|
||||
max_bytes = settings.chat_image_max_mb * 1024 * 1024
|
||||
total = 0
|
||||
try:
|
||||
# Stream with the cap — the dotfile temp is hidden from any
|
||||
# listing of the store dir (the git-sources upload pattern).
|
||||
with open(temp, "wb") as out:
|
||||
while chunk := await file.read(_STREAM_CHUNK):
|
||||
total += len(chunk)
|
||||
if total > max_bytes:
|
||||
raise HTTPException(
|
||||
status_code=413,
|
||||
detail=(
|
||||
f"the image exceeds the "
|
||||
f"{settings.chat_image_max_mb} MB limit"
|
||||
),
|
||||
)
|
||||
out.write(chunk)
|
||||
temp.rename(root / name)
|
||||
except BaseException:
|
||||
# A failed receive (413, broken pipe, cancellation) leaves no
|
||||
# file behind — the final name was never created.
|
||||
temp.unlink(missing_ok=True)
|
||||
raise
|
||||
logger.info(
|
||||
"chat-image: uploaded name=%s bytes=%d", name, total
|
||||
)
|
||||
return {"path": f"/api/chat-images/{name}"}
|
||||
|
||||
|
||||
@router.get("/chat-images/{filename}", response_class=FileResponse)
|
||||
def serve_chat_image(filename: str) -> FileResponse:
|
||||
"""Serve one stored question image (phase 123, task 01).
|
||||
|
||||
PUBLIC (no auth dependency) — like saved-chat content: the saved
|
||||
chat's id is already its credential (phase 55 A1), the image is part
|
||||
of that content, and the filename is an unguessable ``uuid4().hex``
|
||||
(no enumeration value).
|
||||
|
||||
Every non-servable case is a 404 with the same fixed detail — a
|
||||
filename that does not match ``<uuid-hex>.<ext>`` (the regex guard:
|
||||
no path traversal by construction, no 422 that would hint at
|
||||
accepted shapes) and a matching name whose file is missing (the
|
||||
stale-path edge — the file was deleted out-of-band). Servable files
|
||||
stream the exact bytes with the phase-122 ``IMAGE_MIMES``
|
||||
``Content-Type`` (one map, one truth with the describe call's
|
||||
data-URL mime; the six-extension guard makes the fallback
|
||||
unreachable) and ``Cache-Control: private, max-age=3600`` (the
|
||||
phase-122 serve-route convention — the bytes are content-hashed
|
||||
uuids, bustable by re-upload).
|
||||
"""
|
||||
if _CHAT_IMAGE_FILENAME_RE.fullmatch(filename) is None:
|
||||
raise HTTPException(status_code=404, detail="chat image not found")
|
||||
path = Path(get_settings().chat_image_dir).expanduser() / filename
|
||||
if not path.is_file():
|
||||
raise HTTPException(status_code=404, detail="chat image not found")
|
||||
media_type = IMAGE_MIMES.get(path.suffix.lower(), IMAGE_FALLBACK_MIME)
|
||||
return FileResponse(
|
||||
path,
|
||||
media_type=media_type,
|
||||
headers={"Cache-Control": "private, max-age=3600"},
|
||||
)
|
||||
@@ -396,6 +396,22 @@ class Settings(BaseSettings):
|
||||
#: upload): the served copy must outlive the source file.
|
||||
image_dir: str = "~/bor-sources/images"
|
||||
|
||||
# --- Chat image questions (phase 123: attach an image to a question) ---
|
||||
#: Where a user's question image bytes are stored (phase 123,
|
||||
#: ``BOR_CHAT_IMAGE_DIR`` — the ``image_dir`` convention: a sibling of
|
||||
#: phase 122's document-image dir, separate because question-images
|
||||
#: are per-conversation, not per-source). Raw string —
|
||||
#: ``Path.expanduser()`` is applied by the upload/serve routes, not
|
||||
#: here. Files land as ``<uuid4().hex>.<ext>`` — the uuid is the
|
||||
#: credential (no enumeration value; the saved/shared chat record
|
||||
#: carries the served path, never base64, LOCKED A5).
|
||||
chat_image_dir: str = "~/bor-sources/chat-images"
|
||||
#: Cap in MiB for one question-image upload (phase 123, LOCKED A5 —
|
||||
#: the ~10 MB cap; ``BOR_CHAT_IMAGE_MAX_MB``). ``<= 0`` would reject
|
||||
#: every upload — a typo, so the validator fails loudly at startup
|
||||
#: (the ``upload_max_mb`` pattern).
|
||||
chat_image_max_mb: int = 10
|
||||
|
||||
# --- Docs push (phase 59: save a chat answer as documentation) ---
|
||||
#: The git repo a saved chat answer is committed to (phase 59, D3):
|
||||
#: **any** remote — a URL (``https://``, ``ssh://``, ``git@``) or a
|
||||
@@ -574,6 +590,15 @@ class Settings(BaseSettings):
|
||||
raise ValueError("upload_max_mb must be > 0 (MiB)")
|
||||
return v
|
||||
|
||||
@field_validator("chat_image_max_mb")
|
||||
@classmethod
|
||||
def _chat_image_max_mb_positive(cls, v: int) -> int:
|
||||
"""``0``/negative would reject every question-image upload — fail
|
||||
loud at startup (the ``upload_max_mb`` precedent, phase 123)."""
|
||||
if v <= 0:
|
||||
raise ValueError("chat_image_max_mb must be > 0 (MiB)")
|
||||
return v
|
||||
|
||||
@field_validator("history_max_turns")
|
||||
@classmethod
|
||||
def _history_max_turns_non_negative(cls, v: int) -> int:
|
||||
|
||||
@@ -38,7 +38,8 @@ from __future__ import annotations
|
||||
from starlette.datastructures import MutableHeaders
|
||||
from starlette.types import ASGIApp, Message, Receive, Scope, Send
|
||||
|
||||
#: The exact owner-approved policy (phase 82, decision A1).
|
||||
#: The exact owner-approved policy (phase 82, decision A1), extended
|
||||
#: by phase 123's ``img-src`` carve-out (see below).
|
||||
#: ``default-src 'self'`` is inherited by every sub-policy that has no
|
||||
#: explicit entry (``script-src``, ``style-src``, ``connect-src``, …),
|
||||
#: ``base-uri 'none'`` blocks base-tag hijacking, and
|
||||
@@ -46,7 +47,26 @@ from starlette.types import ASGIApp, Message, Receive, Scope, Send
|
||||
#: (verified unnecessary — see module docstring), no ``report-uri`` /
|
||||
#: ``report-to`` (no collector in the homelab — a report would just
|
||||
#: vanish).
|
||||
CSP = "default-src 'self'; base-uri 'none'; frame-ancestors 'none'"
|
||||
#
|
||||
#: Phase 123 (chat image questions, owner-confirmed 2026-09-24) added
|
||||
#: the one scoped relaxation the design requires: ``img-src 'self'
|
||||
#: data:``. The question-image composer renders the picked file as a
|
||||
#: ``data:`` URL — the PREVIEW thumbnail (before the send-time upload
|
||||
#: there is no served path yet) and the LIVE user bubble (the data URL
|
||||
#: needs no fetch) — and ``default-src 'self'`` alone blocks ``data:``
|
||||
#: images in every real browser (the phase-123 E2E caught it: the
|
||||
#: bubble degraded to the "image unavailable" line). The carve-out is
|
||||
#: ``img-src`` ONLY: ``data:`` never becomes a source for scripts,
|
||||
#: styles, or fetches (those keep the strict ``default-src 'self'``
|
||||
#: inheritance), and the bytes are the user's OWN locally-picked file
|
||||
#: (no exfiltration vector — an ``<img>`` cannot read them back).
|
||||
#: Restored / shared bubbles render from the served path (``'self'``),
|
||||
#: so the ``data:`` allowance exists for the two pre-upload/first-paint
|
||||
#: surfaces only.
|
||||
CSP = (
|
||||
"default-src 'self'; base-uri 'none'; frame-ancestors 'none'; "
|
||||
"img-src 'self' data:"
|
||||
)
|
||||
|
||||
|
||||
class SecurityHeadersMiddleware:
|
||||
|
||||
@@ -22,6 +22,7 @@ from starlette.responses import FileResponse
|
||||
|
||||
from app.api.auth import router as auth_router
|
||||
from app.api.chat import router as chat_router
|
||||
from app.api.chat_images import router as chat_images_router
|
||||
from app.api.chats import (
|
||||
public_router as chats_public_router,
|
||||
)
|
||||
@@ -116,6 +117,10 @@ def create_app() -> FastAPI:
|
||||
app.include_router(docs_router, prefix="/api")
|
||||
app.include_router(git_sources_router, prefix="/api")
|
||||
app.include_router(chat_router, prefix="/api")
|
||||
# Phase 123: the question-image upload/serve pair (POST is
|
||||
# user-gated like the chat turn; GET is public like saved-chat
|
||||
# content — the uuid filename is the credential).
|
||||
app.include_router(chat_images_router, prefix="/api")
|
||||
app.include_router(steering_router, prefix="/api")
|
||||
app.include_router(sync_router, prefix="/api")
|
||||
app.include_router(chats_router, prefix="/api")
|
||||
|
||||
+15
-1
@@ -1393,7 +1393,7 @@ async def run_agent(
|
||||
db_factory: Callable[[], Session],
|
||||
*,
|
||||
system_prompt: str,
|
||||
user_message: str,
|
||||
user_message: str | list[dict[str, Any]],
|
||||
seed_docs: Sequence[Document],
|
||||
settings: Settings,
|
||||
holder: AgentHolder,
|
||||
@@ -1424,6 +1424,20 @@ async def run_agent(
|
||||
unchanged. ``()`` (the default) keeps the pre-phase-74 two-message
|
||||
request byte-identical.
|
||||
|
||||
User message (phase 123, TODO L6): *user_message* is the current
|
||||
turn's user content — the plain question string (every text-only
|
||||
turn, byte-identical to pre-phase) OR the multimodal content list
|
||||
``[{type: "text", …}, {type: "image_url", …}]`` for a question that
|
||||
carried an image. FLOW (pinned): the API layer
|
||||
(``app.api.chat``) builds the content via its ``build_user_content``
|
||||
helper and passes it HERE — ``run_agent`` builds its OWN
|
||||
``[system, *history, user]`` list from this value (it does NOT
|
||||
receive the already-built ``messages``; the deflected branch is the
|
||||
one that consumes chat.py's list directly), so a single value
|
||||
covers both shapes and the tool rounds / recovery / retries operate
|
||||
on it untouched. Prior turns' images are never replayed (LOCKED A7
|
||||
— *history* is text-only by construction).
|
||||
|
||||
Retries (phase 67, owner-locked A2): every model request goes through
|
||||
:func:`chat_stream_retried` — a failed round is retried **before** its
|
||||
first piece (same messages, ``settings.llm_retries`` restarts, a flat
|
||||
|
||||
+7
-3
@@ -487,11 +487,15 @@ class LLMClient:
|
||||
|
||||
Messages are passed to the request body VERBATIM: string-only
|
||||
``{role, content}`` dicts are byte-identical on the wire to the
|
||||
pre-phase-74 requests, and an assistant message may additionally
|
||||
pre-phase-74 requests, an assistant message may additionally
|
||||
carry ``reasoning_content`` (the client's prior thinking, phase
|
||||
74 — the same wire field the model uses for its OWN reasoning on
|
||||
the response side; the ``openai`` SDK passes message dicts
|
||||
through untouched, so no transport change).
|
||||
the response side), and a user message's content may be the
|
||||
multimodal parts list of a question that carried an image
|
||||
(phase 123 — ``[{type: "text", …}, {type: "image_url", …}]``,
|
||||
built by ``app.api.chat.build_user_content``); the ``openai``
|
||||
SDK passes message dicts through untouched, so no transport
|
||||
change covers all three shapes.
|
||||
|
||||
``stream=True`` against the OpenAI-compatible endpoint, yielding
|
||||
typed :class:`StreamPiece` values. Wire convention (verified live
|
||||
|
||||
+13
-1
@@ -117,6 +117,18 @@ IMAGE_MIMES: dict[str, str] = {
|
||||
IMAGE_FALLBACK_MIME = "application/octet-stream"
|
||||
|
||||
|
||||
def image_data_url(data: bytes, mime: str) -> str:
|
||||
"""One image's bytes as a data URL for a multimodal message (phase
|
||||
122; factored for phase 123, task 01).
|
||||
|
||||
``data:<mime>;base64,<ascii>`` — the OpenAI-compatible ``image_url``
|
||||
payload's ``url``. ONE helper, both call sites: :func:`describe_image`
|
||||
(document-image descriptions) and the chat question-image path
|
||||
(``app.api.chat``'s multimodal user message, phase 123).
|
||||
"""
|
||||
return f"data:{mime};base64,{base64.b64encode(data).decode('ascii')}"
|
||||
|
||||
|
||||
class SummaryLLM(Protocol):
|
||||
"""The one-shot chat surface the summarizer needs.
|
||||
|
||||
@@ -223,7 +235,7 @@ async def describe_image(
|
||||
the doc is skipped, counted in ``images_failed``, and the sync
|
||||
continues (LOCKED A3).
|
||||
"""
|
||||
data_url = f"data:{mime};base64,{base64.b64encode(data).decode('ascii')}"
|
||||
data_url = image_data_url(data, mime)
|
||||
messages: list[dict[str, Any]] = [
|
||||
{
|
||||
"role": "user",
|
||||
|
||||
+57
-1
@@ -1,6 +1,7 @@
|
||||
"""Pydantic request/response schemas (API contract)."""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Annotated, Any, Literal
|
||||
@@ -60,7 +61,8 @@ class HistoryTurn(BaseModel):
|
||||
class ChatRequest(BaseModel):
|
||||
"""``POST /api/chat`` body: the current question plus the optional
|
||||
prior turns (phase 74 — the client-provided history, stateless per
|
||||
A10).
|
||||
A10) and, since phase 123, the question's attached image (the
|
||||
stored path — :attr:`image`).
|
||||
|
||||
``history`` is the client's earlier turns, oldest first (the
|
||||
``bor.chat.v1`` record minus the current question); the mapper
|
||||
@@ -74,6 +76,30 @@ class ChatRequest(BaseModel):
|
||||
|
||||
message: str = Field(min_length=1, max_length=4000)
|
||||
history: list[HistoryTurn] = Field(default_factory=list, max_length=100)
|
||||
#: Phase 123 (TODO L6, LOCKED A5): the STORED PATH of the question's
|
||||
#: attached image — ``/api/chat-images/<uuid4-hex>.<ext>`` exactly as
|
||||
#: ``POST /api/chat-images`` returns it. NEVER a raw data URL: the
|
||||
#: upload endpoint already did the size/mime enforcement, and
|
||||
#: re-validating a 10 MB base64 string at the schema would be the
|
||||
#: anti-pattern. ``None`` (the default, every text-only question)
|
||||
#: keeps the turn byte-identical to pre-phase-123.
|
||||
image: str | None = Field(default=None, max_length=500)
|
||||
|
||||
@field_validator("image")
|
||||
@classmethod
|
||||
def _image_is_stored_path(cls, v: str | None) -> str | None:
|
||||
"""A set ``image`` must be the upload endpoint's stored-path
|
||||
shape — ``/api/chat-images/<uuid4().hex>.<ext>`` (32 hex chars,
|
||||
the six image extensions; the pattern is pinned in the
|
||||
phase-123 design and mirrored by the serve route's filename
|
||||
guard, ``app.api.chat_images``). A data URL, a bare filename, a
|
||||
traversal, a wrong extension, or a mistyped uuid all 422 with
|
||||
ONE fixed detail — no echo of the input."""
|
||||
if v is None:
|
||||
return v
|
||||
if re.fullmatch(r"/api/chat-images/[0-9a-fA-F]{32}\.(png|jpe?g|webp|gif|bmp)", v) is None:
|
||||
raise ValueError("image must be an uploaded chat image path")
|
||||
return v
|
||||
|
||||
class LoginRequest(BaseModel):
|
||||
"""``POST /api/login`` body (phase 16): the single admin's password.
|
||||
@@ -928,6 +954,36 @@ class ChatMessage(BaseModel):
|
||||
# persisted error detail (the phase-48 ``stopped`` precedent).
|
||||
failed: bool | None = None
|
||||
error: str | None = Field(default=None, max_length=500)
|
||||
# Phase 123 (task 01, LOCKED A5): the question's attached image —
|
||||
# the STORED PATH (``/api/chat-images/<uuid>.<ext>``, from
|
||||
# ``POST /api/chat-images``), on the USER record only: the
|
||||
# attachment belongs to the question, so a brain record never
|
||||
# carries it (the answer may cite the image doc's sources, but the
|
||||
# attachment itself is the user's). The saved/shared shape gains
|
||||
# this one optional key — omitted when ``None`` (the
|
||||
# :meth:`_drop_image_when_absent` serializer below), so a text-only
|
||||
# chat round-trips byte-identically to pre-phase-123 (the
|
||||
# phase-50 contract; the phase-122 ``SourceRef.image_url``
|
||||
# omission precedent). The path is ≤ 500 chars — no phase-83
|
||||
# cap pressure (it is never a data URL, LOCKED A5).
|
||||
image: str | None = Field(default=None, max_length=500)
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _drop_image_when_absent(self, handler: SerializerFunctionWrapHandler) -> Any:
|
||||
"""The phase-123 image omission rule: ``image: None`` (every
|
||||
text-only record, and every pre-phase-123 record) serializes
|
||||
WITHOUT the key — ABSENT, never ``null`` — so the stored
|
||||
``bor.chat.v1`` JSONB and the saved/shared wire shape stay
|
||||
byte-identical to pre-phase for text-only chats (the phase-50
|
||||
round-trip contract). A record WITH an image keeps the path —
|
||||
the user bubble, the refreshed page, and the shared chat all
|
||||
render it from the served route (the image is part of the
|
||||
chat's content, so it rides the same public/credential-
|
||||
is-the-id trust model)."""
|
||||
data = handler(self)
|
||||
if self.image is None:
|
||||
data.pop("image", None)
|
||||
return data
|
||||
|
||||
|
||||
class SavedChatCreate(BaseModel):
|
||||
|
||||
Reference in New Issue
Block a user