phase: 123_chat_image_questions
All gates green. Verification complete. **Phase 123 — final verification pass (all 4 tasks already in `complete/`)** - Verified the full implementation is in the working tree: `app/api/chat_images.py` (upload/serve pair), `ChatRequest.image`/`ChatMessage.image` (path-validated, omitted-when-None), toggle-off + stale-file hinted error frames, `build_user_content` multimodal build at both sites (chat.py deflected branch + `run_agent`), config-gated composer attach/preview/upload-then-send, restore + shared rendering, CSP `img-src 'self' data:` carve-out, mock-LLM capture buffer. - `uv run pytest` → **2796 passed**, exit 0 (unit + integration). - `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (29/4615 missed; phase-123 modules 99–100%). - `uv run pytest tests/e2e/test_chat_image_questions.py -v --no-cov` → **5 passed** in isolation. - `uv run ruff check . && uv run pyright` → clean (0 errors). **Completion criteria:** (1) attach→send→multimodal text+image to the model, bubble/reload/shared all render it, saved chat stores the PATH with `"base64" not in json.dumps(stored)` — **verified** (E2E tests 1–4 + integration round-trip); (2) `BOR_IMAGES=false` — control hidden, exact hinted error frame, zero model calls / no query_log row — **verified** (E2E test 5 + integration); (3) text-only byte-identical (`content` stays a plain `str`) — **verified** (unit + integration); (4) all gates green — **verified**; (5) commit + phase move — left to the harness per pipeline rules (no `git add`/`commit` run). No defects found; no live-infrastructure changes (repo + local dev DB only). **Next pending phase: none** — 123 is the last phase in `todo/`.
This commit is contained in:
@@ -0,0 +1,50 @@
|
||||
# Phase 123 — Chat image questions: attach an image to a question
|
||||
|
||||
**Source:** `TODO.md` L6 — "…users should be able to submit images as part of their question in brain of reese."
|
||||
**Story:** n/a (feature request; completes the phase-122 image capability on the question side).
|
||||
**Context:** Phase 122 (todo, this pipeline) — `BOR_IMAGES` toggle + `GET /api/config` `images` flag (task 01), the ext→mime map, `image_dir` storage convention. `app/schemas.py:60` — `ChatRequest` (`message` min 1/max 4000, `history` ≤100 — `HistoryTurn` is text-only), `ChatMessage` (L742, `extra="forbid"`, phase-83 value bounds). `app/api/chat.py` — the turn pipeline: the user message is built at L645 (`{"role": "user", "content": request.message}`; a grounded turn runs `run_agent`, a deflected turn a direct `chat_stream` on the same `messages`), and `app/rag/agent.py:1370/1448` — `run_agent(..., user_message: str)` builds its own `[system, user]` (verify the data flow — if `run_agent` receives the already-built `messages`, the single edit site is chat.py). The phase-114 SSE error-frame-with-hint pattern (the "question too long" frame — `ChatErrorEvent.detail` + optional `hint`, consumed by the banner at app.js L2210). `frontend/index.html` — the composer (label L287, `#message-input` L292, `#send-btn` L325). `frontend/assets/app.js` — `handleSend` (L2306), `runTurn` (the turn driver + the user append/save-point-1 at send), `addMessage("user", …)` (user bubble), `rememberBrainTurn` (L2108, the brain save point), `renderStoredMessage` (L1642, user branch). `frontend/assets/shared.js` — the shared page's message render (text-only today). `app/api/config.py:30` — the public flags dict (phase 122 task 01 added `images`).
|
||||
|
||||
## Objective
|
||||
The user attaches one image to a question: a masked-by-server upload stores the bytes, the vision model (the chat model) receives a multimodal message, the user's bubble renders the image, the record persists the image path (not base64) so refresh and shared chats render it, and `BOR_IMAGES=false` rejects the request with a helpful hint.
|
||||
|
||||
## Dependencies
|
||||
- `122_image_documents` (todo) — CODE dependency: the `BOR_IMAGES`/`images` config flag (the toggle gates this feature), the ext→mime map, and the `image_dir` storage convention (this phase's `chat_image_dir` follows it).
|
||||
- Code dependencies (all complete): phase 14/50/55 conversation persistence, phase 74 history mapping, phase 114 SSE error-hint frames, phase 51 shared chats.
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
- **Storage (task 01, LOCKED A5):** user question-images are server-stored, NOT base64-in-saved-chats: `Settings.chat_image_dir: str = "~/bor-sources/chat-images"` (`BOR_CHAT_IMAGE_DIR`, the `image_dir` convention — a sibling of phase 122's `image_dir`, separate because question-images are per-conversation, not per-source) + `Settings.chat_image_max_mb: int = 10` (`BOR_CHAT_IMAGE_MAX_MB`, the ~10 MB cap of A5; `upload_max_mb`'s fail-loud validator precedent for `<= 0`). `POST /api/chat-images` (multipart, in `app/api/chat.py` or a small new `app/api/chat_images.py` router — the executor's call, following the repo's one-concern-per-module style): accepts an image file, validates the mime/ext against the SAME six-extension set as phase 122 (reuse the frozenset; the Content-Type header is a hint — the EXTENSION is the source of truth, the archive-uploader precedent), rejects oversize with a 413 (the fixed-detail style), stores `chat_image_dir/<uuid4().hex>.<ext>`, returns `{ "path": "/api/chat-images/<uuid-hex>.<ext>" }`. `GET /api/chat-images/{filename}` serves the bytes (404 on missing/unknown — the filename is a uuid, no enumeration value) with the phase-122 mime map; PUBLIC like saved-chat content (a saved chat's id is already its credential — phase 55 A1 — the image is part of that content).
|
||||
- **Request (task 01):** `ChatRequest.image: str | None = Field(default=None, max_length=500)` — a STORED PATH, pattern-validated (`^/api/chat-images/[0-9a-fA-F]{32}\.(png|jpe?g|webp|gif|bmp)$` — the stored filename is `uuid4().hex.<ext>`) — never a raw data URL (the upload endpoint already did the size/mime enforcement; re-validating a 10 MB base64 string in the schema would be the anti-pattern). Toggle OFF (`settings.images` false) with `image` set → the turn settles with the phase-114 SSE error frame: `detail` "Image support is turned off on this server." + `hint` "Enable BOR_IMAGES in the server's .env (and restart) to ask with an image." (the question itself is NOT persisted — a rejected turn saves nothing, the existing error-path convention). `image` set but file missing → the same frame shape with a "that image is no longer available" detail (a stale-path edge: the stored file was deleted out-of-band).
|
||||
- **Multimodal (task 01):** the user message becomes `{"role": "user", "content": [{"type": "text", "text": request.message}, {"type": "image_url", "image_url": {"url": <data URL from the stored file>}}]}` at BOTH construction sites (chat.py:645 and agent.py:1448 if it builds independently — verify the flow; when `request.image` is None the content stays the plain string, byte-identical to today). The data URL is built server-side from the stored bytes + mime map (the phase-122 `describe_image` data-URL construction — reuse it). `HistoryTurn`/`history_to_messages` are UNCHANGED (LOCKED A7): prior turns' images are never replayed into the model's history — the history budget is text, and a 10 MB image per past turn would blow every budget; the model simply sees the text of a prior turn that had an image.
|
||||
- **Persistence (tasks 01+02):** `ChatMessage.image: str | None = Field(default=None, max_length=500)` — the stored path, on the USER record (the image belongs to the question). The user record is saved at save-point-1 (send), BEFORE the turn resolves — so the client uploads FIRST (`POST /api/chat-images`) and stores the returned path in the user record, then POSTs `/api/chat` with `image=<path>`. Saved chats, shared chats, and the localStorage shape all carry the path (≤500 chars — no phase-83 cap pressure). A brain record never carries `image` (the answer may reference the image's sources, but the attachment is the user's).
|
||||
- **Composer (task 02):** the attach control appears ONLY when `GET /api/config` says `images: true` (phase 122's flag; fetched once at boot like the other config — the composer reads the existing cached config if present). A paperclip button (SVG, the icon style of the other composer glyphs, `aria-label="Attach an image"`) before the input → hidden `<input type="file" accept="image/png,image/jpeg,image/webp,image/gif,image/bmp">` → on select: a preview strip above the input (thumbnail ≤48px, the filename, a remove ✕) + the file's data URL kept client-side until send; on send with an attachment: `POST /api/chat-images` (the file) → the returned path goes into the user record + the `/api/chat` body → the preview clears. Upload failure (oversize, non-image, server down) → the phase-114-style out-of-turn banner ("Couldn't attach the image — …") and the send is BLOCKED (no question without the image the user attached — ASSUMPTION A8, locked below). The user bubble renders the image (from the data URL live, from the stored path after restore) with `alt = filename`, capped height, above/beside the text (the theme's bubble treatment; the image is part of the question, visible in both the live bubble and the restore).
|
||||
- **Restore + shared (task 03):** `renderStoredMessage`'s user branch: `m.image` present → the user bubble includes `<img src="{m.image}" alt="…">` (a load failure collapses to a small "image unavailable" line — never a broken icon). The shared page (`shared.js`) renders the user image the same way (the image route is public — the shared view is faithful; no new shared-shape field beyond `ChatMessage.image`, which the public `messages` shape already carries).
|
||||
- **NOT touched:** the history budget/trimming, the honesty gate, the suggestion chips, phase-122's document-image pipeline (a QUESTION image is a separate concern — it is NOT indexed as a document), and the stop/failed-turn paths (they persist whatever records exist, including the new `image` key, unmodified).
|
||||
|
||||
## Tasks
|
||||
1. `01_vision_request.md` — `POST/GET /api/chat-images`, `ChatRequest.image` + `ChatMessage.image`, the toggle-off/stale error frames, the multimodal user message at both construction sites.
|
||||
2. `02_composer_attach.md` — the config-gated attach control, preview, upload-then-send, the user bubble's image.
|
||||
3. `03_restore_and_shared.md` — `ChatMessage.image` on restore (chat page) and on the shared page.
|
||||
4. `04_chat_image_tests.md` — unit + integration + isolated E2E `test_chat_image_questions.py`.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_chat_image_questions.py` (new, task 04) — the path pattern validator (accepts well-formed, rejects data URLs / wrong ext / traversal), the multimodal message build (both sites; `image=None` → byte-identical plain string), the toggle-off + stale-file error frames (detail + hint shapes), the upload endpoint's mime/size/ext rules (tmp-dir settings), the serve route (200/404), `ChatMessage.image` bounds + omission.
|
||||
- Integration: `tests/integration/test_chat_api.py` (extend, task 04) — upload → `POST /api/chat` with `image=<path>` → the mock client RECEIVES the multimodal content list (text part + image_url data URL); `image` with `images=false` → the SSE error frame with the hint and NO model call, no persisted record; `image=None` requests are byte-identical to pre-phase; a saved chat round-trips a user record with `image`; a shared chat serves it.
|
||||
- E2E: `tests/e2e/test_chat_image_questions.py` (new, task 04) — isolated run per AGENTS.md §4, `BOR_IMAGES=true`: attach a fixture PNG in the composer → preview + remove works → send → the user bubble shows the image → the (mock) answer streams → reload → the user bubble restores WITH its image → open the shared link → the shared page shows the image. Plus the default-off negative: with `BOR_IMAGES` unset, the attach control is ABSENT from the DOM.
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] With `BOR_IMAGES=true`: attach → send → the vision model gets text+image; the user bubble, the refreshed page, and the shared chat all show the image; the saved chat stores the PATH (assert no base64 in the stored payload).
|
||||
- [ ] With `BOR_IMAGES=false`: the attach control is absent, an API request with `image` gets the hinted error frame, and no model call / record happens.
|
||||
- [ ] Text-only questions behave byte-identically to pre-phase (the multimodal branch is inert).
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agents/phases/complete/` by the pipeline gate.
|
||||
|
||||
## Locked decisions
|
||||
- **A5 — one image per question; the ~10 MB cap (`BOR_CHAT_IMAGE_MAX_MB`); server-stored bytes under `chat_image_dir`; the saved/shared record carries the path, never base64 (owner-confirmed 2026-09-24, roadmap confirmation).**
|
||||
- **A7 — a question's image applies to the CURRENT turn only; prior turns' images are never replayed into the model's history (the text of a prior turn stands alone) (owner-confirmed: same confirmation — the proposed design).**
|
||||
- **A8 — if the image upload fails, the send is blocked with a banner (the question is never sent without the image the user attached) (owner-confirmed: same confirmation).**
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add app/ frontend/ tests/ .env.example .agents/phases/ && git commit --no-gpg-sign -m "feat(chat): attach an image to a question — vision input, in-bubble render, persisted and shared"
|
||||
```
|
||||
Reference in New Issue
Block a user