Phase 49 (owner request, chat 2026-08-28: "The git sources page should remove local directory and should instead accept a tarball or zipfile upload which it will unpack and scan … reuploading the same tarball should not create a new folder, but should unpack and overwrite the previously unpacked content" — design confirmed in the same conversation): * POST /api/git-sources/upload (admin-only, require_admin): accepts .tar/.tar.gz/.tgz/.zip, streams it with the BOR_UPLOAD_MAX_MB cap (bounds BOTH the compressed upload and the total extracted bytes — zip-bomb guard), safely unpacks (absolute/traversal/symlink/hardlink escape and device/FIFO members rejected), and atomically swaps the content in over BOR_UPLOAD_DIR/<name>/ (name = filename minus the archive suffix — no missing window, a failed upload never touches the existing folder/row/KB). The git_sources row is upserted by path (kind='local', no duplicates, added_at preserved), the models are checked fail-fast (503 sanitized when down — the folder/row stay committed and the next sync/re-upload retries idempotently), and the source is scanned synchronously in the request (single-source import_sources prune=True + change-gated KB overview), answering 200 with the sync-style counts. One upload at a time (409); the request session is released before the scan so a concurrent TRUNCATE cannot deadlock against it. * app/rag/archive_upload.py: ArchiveUploadError, ARCHIVE_SUFFIXES, archive_source_name (safe-name derivation), unpack_archive (guarded zip/tar extraction with the extracted-byte cap, no partial state), swap_in (atomic replace with restore-on-failure) — fully unit-tested. * app/config.py + .env.example: BOR_UPLOAD_DIR (default ~/bor-sources/uploads, deliberately separate from the git checkouts) and BOR_UPLOAD_MAX_MB (default 512; a validator fails loud at startup on <= 0). * python-multipart added to the dependencies — FastAPI's required multipart parser (an A2 implementation detail, phase locked decision). * The Sources page: the phase-38 "Add a local directory" form is removed; #archive-upload-form takes its place (labeled file input, "Upload & scan" button, the §7.4 never-stale lifecycle, inline role=alert error, role=status count line); hint + table caption updated. The POST /api/git-sources kind=local API contract is UNCHANGED — a plain directory is still registrable via the API, and existing Local rows list/remove/sync exactly as before. * The phase-38 story E2E (test_local_directory_sources.py) is rewritten API-driven — the form it drove is gone; its acceptance stands. * The story E2E (test_archive_upload_sources.py): the swap, upload→scan→list (the deterministic "Uploading…" in-flight state, the Local row, /api/docs + the RAG catalog), same-filename re-upload (in-place replace, prune, no duplicate row, v2-only folder), the 422 inline error + recovery (the form is not wedged), and the anonymous gate + 403. * README: the archive-upload section (formats, naming rule, in-place replace, both new settings), the local-directory form removal noted, config reference rows for BOR_UPLOAD_DIR / BOR_UPLOAD_MAX_MB. Gates: unit+integration green, app/ coverage 99%, the story E2E green in isolation, the regression suites (git sources admin, local directory sources, sync button, import documents, nav rename, smoke, shared header) green in isolation, ruff + pyright clean. Note: per this phase's file-level staging, frontend/assets/styles.css also carries the small same-day in-flight owner rework already in the working tree (the .sign-in-mobile companion rule for the phase-48 mobile sign-in copy); the phase-49 change is the upload form's block.
285 lines
8.7 KiB
Python
285 lines
8.7 KiB
Python
"""Pydantic request/response schemas (API contract)."""
|
|
from __future__ import annotations
|
|
|
|
import uuid
|
|
from datetime import datetime
|
|
from typing import Literal
|
|
|
|
from pydantic import BaseModel, Field, field_validator
|
|
|
|
|
|
class HealthResponse(BaseModel):
|
|
status: str
|
|
db: str
|
|
version: str
|
|
environment: str
|
|
|
|
|
|
class SuggestionList(BaseModel):
|
|
suggestions: list[str]
|
|
|
|
|
|
class ChatRequest(BaseModel):
|
|
message: str = Field(min_length=1, max_length=4000)
|
|
|
|
|
|
class LoginRequest(BaseModel):
|
|
"""``POST /api/login`` body (phase 16): the single admin's password.
|
|
|
|
An empty or wrong password is a 401 with one generic detail — never a
|
|
422 that would hint at input-shape differences.
|
|
"""
|
|
|
|
password: str = ""
|
|
|
|
|
|
class WhoamiResponse(BaseModel):
|
|
"""``GET /api/whoami`` (phase 16) — drives all UI gating."""
|
|
|
|
authenticated: bool
|
|
role: str # "admin" | "anonymous"
|
|
|
|
|
|
class SourceRef(BaseModel):
|
|
source: str
|
|
path: str
|
|
title: str
|
|
|
|
|
|
class ChatThinkingEvent(BaseModel):
|
|
"""SSE thinking event: one chunk of the model's reasoning (phase 17).
|
|
|
|
PLAN §4 extension (A15, owner permission 2026-08-23): frames of the
|
|
shape ``{type: "thinking", text: str}`` stream ahead of the
|
|
``delta`` frames in practice (the model reasons before it answers). The
|
|
client renders them in a collapsible "Thinking" block; the ``done``
|
|
event shape is unchanged and thinking text never travels on it.
|
|
Sibling of :class:`ChatErrorEvent`.
|
|
"""
|
|
|
|
type: str = "thinking"
|
|
text: str
|
|
|
|
|
|
class ChatToolEvent(BaseModel):
|
|
"""SSE frame for one agent tool call (phase 37, PLAN §4 extension).
|
|
|
|
A15 extension (owner permission 2026-08-26): a grounded turn may call
|
|
the server-side document tools (``list_documents`` / ``read_document``,
|
|
see :mod:`app.rag.agent`); each model-requested call streams as
|
|
``{type: "tool", name: str, argument: str | null}`` ahead of the
|
|
answer's ``delta`` frames. ``argument`` is the read document's
|
|
``"source/path"`` for ``read_document`` and null otherwise. The client
|
|
renders each frame as a "calling tool" line/state (phase 37 task 05);
|
|
the ``delta`` / ``done`` shapes are unchanged — the read document is
|
|
reflected in ``done.sources`` instead.
|
|
"""
|
|
|
|
type: Literal["tool"] = "tool"
|
|
name: str # "list_documents" | "read_document"
|
|
argument: str | None = None # "source/path" for read_document
|
|
|
|
|
|
class ChatDoneEvent(BaseModel):
|
|
"""Final SSE event of a chat turn: metadata for the finished answer."""
|
|
|
|
type: str = "done"
|
|
deflected: bool
|
|
sources: list[SourceRef]
|
|
suggestions: list[str] = []
|
|
|
|
|
|
class ChatErrorEvent(BaseModel):
|
|
"""SSE error event: a turn that cannot complete (PLAN §4).
|
|
|
|
The client's loading-feedback state machine (phase 06) keys off this
|
|
exact shape — ``{type: "error", detail: str}`` — to flip to the error
|
|
state and re-enable the send button.
|
|
"""
|
|
|
|
type: str = "error"
|
|
detail: str
|
|
|
|
|
|
class DocSummary(BaseModel):
|
|
"""One indexed document as shown on the Sources page / API."""
|
|
|
|
id: str
|
|
source: str
|
|
path: str
|
|
title: str
|
|
chunks: int
|
|
indexed_at: str
|
|
|
|
|
|
class DocList(BaseModel):
|
|
"""Response of ``GET /api/docs`` (empty list → designed empty state)."""
|
|
|
|
documents: list[DocSummary]
|
|
|
|
|
|
class DocContent(BaseModel):
|
|
"""One indexed document's full content (feeds the viewer page, phase 10)."""
|
|
|
|
source: str
|
|
path: str
|
|
title: str
|
|
format: str
|
|
#: Lite-model summary (phase 30) — non-markdown A9 docs only; None for
|
|
#: markdown documents, pre-phase-30 rows, and the fail-soft path where
|
|
#: summary generation failed but the document was still indexed.
|
|
summary: str | None = None
|
|
content: str
|
|
indexed_at: str
|
|
chunks: int
|
|
|
|
|
|
class SteeringNoteIn(BaseModel):
|
|
"""``POST /api/steering`` body: one tuning instruction (phase 15).
|
|
|
|
The note is trimmed *before* the length constraints run, so a
|
|
whitespace-only body is a 422 and a 2000-char note with surrounding
|
|
spaces still passes.
|
|
"""
|
|
|
|
note: str = Field(min_length=1, max_length=2000)
|
|
|
|
@field_validator("note", mode="before")
|
|
@classmethod
|
|
def _trim_note(cls, v: object) -> object:
|
|
return v.strip() if isinstance(v, str) else v
|
|
|
|
|
|
class SteeringNoteUpdate(BaseModel):
|
|
"""``PUT /api/steering/{id}`` body: a new tuning instruction (phase 27).
|
|
|
|
Mirrors :class:`SteeringNoteIn` — the note is trimmed *before* the
|
|
length constraints run, so an empty/whitespace body is a 422 and a
|
|
full replacement that is ≤2000 chars after the trim still passes.
|
|
"""
|
|
|
|
note: str = Field(min_length=1, max_length=2000)
|
|
|
|
@field_validator("note", mode="before")
|
|
@classmethod
|
|
def _trim_note(cls, v: object) -> object:
|
|
return v.strip() if isinstance(v, str) else v
|
|
|
|
|
|
class SteeringNote(BaseModel):
|
|
"""One stored steering note (API shape — ISO-8601 ``created_at``)."""
|
|
|
|
id: uuid.UUID
|
|
note: str
|
|
created_at: datetime
|
|
|
|
|
|
class SteeringNoteList(BaseModel):
|
|
"""``GET /api/steering`` response: all notes, newest first."""
|
|
|
|
notes: list[SteeringNote]
|
|
|
|
|
|
class GitSourceIn(BaseModel):
|
|
"""``POST /api/git-sources`` body (phase 35, task 02; ``kind``, phase 38).
|
|
|
|
``kind`` selects the source kind and which field carries its location:
|
|
|
|
* ``"git"`` (default) — ``url`` is the repo URL. Mirrors the
|
|
phase-35 contract: trimmed *before* the length constraints run, so a
|
|
whitespace-only body is a 422 and a URL with surrounding spaces is
|
|
stored clean. Shape validation (``https://``, ``ssh://``, ``git@``)
|
|
and the kind-field rules (url present, no path) happen in the API
|
|
layer so the 422/409 details stay fixed strings that never echo the
|
|
input (credential safety).
|
|
* ``"local"`` — ``path`` is an existing directory on the server.
|
|
Trimmed here; the API layer then ``expanduser()``s it and requires an
|
|
absolute existing directory (else 422 naming the path — the path is
|
|
not a secret, unlike a git URL) and no ``url``.
|
|
"""
|
|
|
|
kind: Literal["git", "local"] = "git"
|
|
url: str | None = Field(default=None, min_length=1, max_length=500)
|
|
path: str | None = Field(default=None, min_length=1, max_length=2000)
|
|
|
|
@field_validator("url", mode="before")
|
|
@classmethod
|
|
def _trim_url(cls, v: object) -> object:
|
|
return v.strip() if isinstance(v, str) else v
|
|
|
|
@field_validator("path", mode="before")
|
|
@classmethod
|
|
def _trim_path(cls, v: object) -> object:
|
|
return v.strip() if isinstance(v, str) else v
|
|
|
|
|
|
class GitSourceOut(BaseModel):
|
|
"""One created git source as returned by ``POST`` (phase 35, task 02).
|
|
|
|
``id`` / ``added_at`` are non-null for a stored row. ``url`` is the
|
|
row's location column: the repo URL for ``kind=git`` rows and, for
|
|
``kind=local`` rows, the stored (expanded) directory path — the
|
|
phase-35 response shape is unchanged by phase 38, so a local 201
|
|
reports its path in ``url`` and the full row (``kind`` + ``path``)
|
|
via ``GET``.
|
|
"""
|
|
|
|
id: uuid.UUID | None
|
|
url: str
|
|
added_at: datetime | None
|
|
|
|
|
|
class GitSourceRow(BaseModel):
|
|
"""One row of ``GET /api/git-sources`` (phase 35; ``kind``/``path``,
|
|
phase 38, task 02).
|
|
|
|
``kind`` discriminates the row: git rows (and the git-only
|
|
``BOR_GIT_SOURCES`` env-fallback rows) carry ``url`` and
|
|
``path: null``; local rows carry ``path`` (the absolute directory,
|
|
expanded) and the same string in ``url`` (the table's NOT-NULL
|
|
location column). ``id`` / ``added_at`` are nullable: env-fallback
|
|
rows (table empty) carry neither.
|
|
"""
|
|
|
|
id: uuid.UUID | None
|
|
kind: Literal["git", "local"]
|
|
url: str
|
|
path: str | None
|
|
added_at: datetime | None
|
|
|
|
|
|
class GitSourceList(BaseModel):
|
|
"""``GET /api/git-sources`` response (phase 35, task 02).
|
|
|
|
``from_env`` is True only when the ``git_sources`` table is empty and
|
|
the list comes from ``BOR_GIT_SOURCES`` (the phase's locked fallback
|
|
— env rows are git-only and report ``kind: "git"``, ``path: null``);
|
|
once the table has rows the env var is ignored and ``from_env`` is
|
|
False — the UI is the source of truth.
|
|
"""
|
|
|
|
sources: list[GitSourceRow]
|
|
from_env: bool
|
|
|
|
|
|
class UploadOut(BaseModel):
|
|
"""``POST /api/git-sources/upload`` response (phase 49, task 02).
|
|
|
|
The uploaded source's name (filename minus the archive suffix) plus
|
|
the SAME count keys as the admin sync's success ``detail``
|
|
(``files``, ``added``, ``updated``, ``unchanged``, ``pruned``,
|
|
``errors``, ``chunks`` — ``app.api.sync._run_sync``) and the
|
|
``overview`` flag: the Sources page renders the same
|
|
"N added · N pruned" result line for both.
|
|
"""
|
|
|
|
source: str
|
|
files: int
|
|
added: int
|
|
updated: int
|
|
unchanged: int
|
|
pruned: int
|
|
errors: int
|
|
chunks: int
|
|
overview: bool
|