"""SQLAlchemy models (PostgreSQL 17 + pgvector). Data model — see ``.agents/PLAN.md`` §Data Model: * ``documents`` — one row per imported A9 file (full content, path, sha256 hash). * ``chunks`` — retrieval units; each chunk points at its parent document via ``document_id``. This is how an embedding maps back to a document path (the "feed the whole document" requirement). * ``query_log`` — observability: every question, its retrieval score, the deflection decision, and latency. * ``steering_notes`` — owner tuning notes injected into the system prompt of every chat turn (phase 15, ```` section). * ``kb_overview`` — single-row lite-generated outline of the KB's basic categories, injected as the ```` section of every chat turn (phase 31). * ``git_sources`` — admin-managed source registry (git URLs + local directories) the Sync button and import_docs import (phase 35; ``kind`` discriminator added in phase 38). * ``saved_chats`` — owner-saved chat conversations: one row per explicitly Saved conversation (auto-``title`` + the ``bor.chat.v1`` message list as JSONB, phase 14 shape) — phase 50; ``/api/chat`` stays stateless; ``share_token`` (NULL = private, ``uuid4`` = publicly readable at ``/shared/``) — phase 51; ``sources_version`` (the KB generation the conversation was saved against; 0 = the pre-counter KB, stale on the first bump) — phase 53. * ``sources_meta`` — single-row sources-version counter: which generation of the knowledge base is current, bumped exactly once per KB-changing sync so saved chats can be marked stale (phase 53). * ``doc_drafts`` — server-side drafts of chat answers saved as documentation: one row per "Save as doc" action (the long answer body lives here, never in a URL), keyed by an unguessable ``uuid4`` ``token`` (the edit screen's URL credential — the share-token trust model, phase 51); ``status`` moves ``draft`` → ``pushed`` (``branch`` + ``commit_sha`` recorded) when the push endpoint commits + pushes the file to the ``BOR_DOCS_REPO`` branch (phase 59). * ``api_tokens`` — admin-issued access tokens: one row per generated token, so a person handed a token can sign in to use the app (chat, suggestion chips, cited documents) — the ONLY content that stays anonymous is the shared chats (phase 79). ``token_hash`` is the SHA-256 hex digest of the full ``bor_…`` token string (the stored credential — the plaintext exists only in the 201 create response, returned exactly once); ``revoked_at`` set = dead (live-checked on the holder's next request), ``last_used_at`` bumped on ``POST /api/token-auth`` (task 03 — the only request that presents the token; the in-app gate re-sends the cached token on every page load). """ from __future__ import annotations import uuid from datetime import datetime from pgvector.sqlalchemy import Vector from sqlalchemy import ( Boolean, DateTime, Float, ForeignKey, Integer, String, Text, UniqueConstraint, func, ) from sqlalchemy.dialects.postgresql import JSONB, UUID from sqlalchemy.orm import Mapped, mapped_column, relationship from app.config import get_settings from app.db import Base # Single source of truth for the vector column size (see .agents/PLAN.md A6). EMBEDDING_DIM: int = get_settings().embedding_dim class Document(Base): __tablename__ = "documents" __table_args__ = (UniqueConstraint("source", "path", name="uq_documents_source_path"),) id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) source: Mapped[str] = mapped_column(String(120), index=True) # e.g. "Homelab" path: Mapped[str] = mapped_column(String(1000), index=True) # relative to source dir full_path: Mapped[str] = mapped_column(String(2000)) # absolute path at import time title: Mapped[str] = mapped_column(String(500)) content: Mapped[str] = mapped_column(Text) # full markdown — the RAG context content_hash: Mapped[str] = mapped_column(String(64), index=True) # sha256 for change detection indexed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) #: Lite-model summary, phase 30. Natural-language summary of the #: document (non-markdown A9 docs only, generated at import time by the #: aipi ``lite`` model). NULL for markdown docs, pre-phase-30 rows, and #: the fail-soft path where summary generation failed but the document #: was still indexed. summary: Mapped[str | None] = mapped_column(Text, default=None) chunks: Mapped[list[Chunk]] = relationship( back_populates="document", cascade="all, delete-orphan" ) class Chunk(Base): __tablename__ = "chunks" id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) document_id: Mapped[uuid.UUID] = mapped_column( UUID(as_uuid=True), ForeignKey("documents.id", ondelete="CASCADE"), index=True ) position: Mapped[int] = mapped_column(Integer) content: Mapped[str] = mapped_column(Text) embedding: Mapped[list[float] | None] = mapped_column(Vector(EMBEDDING_DIM)) #: Summary chunk, position −1, phase 30. Marks the single extra embedded #: chunk mirroring ``Document.summary``; default False keeps every #: pre-phase-30 row (and ordinary content chunks) valid. is_summary: Mapped[bool] = mapped_column(Boolean, default=False) document: Mapped[Document] = relationship(back_populates="chunks") class QueryLog(Base): __tablename__ = "query_log" id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) question: Mapped[str] = mapped_column(Text) top_score: Mapped[float] = mapped_column(Float, default=0.0) # best cosine similarity #: Lexical (FTS) candidates matched — the OR-tsquery hit count (A8). NULL #: for pre-hybrid rows (migration 0002). fts_hits: Mapped[int | None] = mapped_column(Integer) chunk_hits: Mapped[int] = mapped_column(Integer, default=0) deflected: Mapped[bool] = mapped_column(Boolean, default=False) # True = honest "no idea" sources: Mapped[str] = mapped_column(Text, default="") # comma-joined source paths latency_ms: Mapped[int] = mapped_column(Integer, default=0) created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) class SteeringNote(Base): """One owner tuning instruction (phase 15). Notes are read into the system prompt of **every** chat turn as the ```` section (oldest first, char-budgeted — see :func:`app.rag.prompts.build_steering_section`). """ __tablename__ = "steering_notes" id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) note: Mapped[str] = mapped_column(Text) # trimmed, 1–2000 chars (API-enforced) created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) class KbOverview(Base): """Single-row, lite-generated outline of the knowledge base (phase 31). Exactly one row (``id = 1``, enforced by the migration 0005 server defaults) holds a plain-text outline of the KB's basic categories, generated by the aipi ``lite`` model whenever an import changes the KB. Chat turns only read this row (one indexed PK lookup) and inject it into the system prompt of every turn as the ```` section — an empty row means the section is absent and the prompt stays byte-identical to the pre-phase text (phase 15 convention). """ __tablename__ = "kb_overview" id: Mapped[int] = mapped_column(Integer, primary_key=True, server_default="1") content: Mapped[str] = mapped_column(Text, server_default="") # the outline text updated_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now() ) class SourcesMeta(Base): """Single-row sources-version counter (phase 53). Exactly one row (``id = 1``, seeded by migration 0010 — the ``kb_overview`` id=1 precedent) holds the current **generation** of the knowledge base. ``version`` is bumped exactly once per sync that actually changed the KB (phase 53, task 02 — change-gated on ``added + updated + pruned > 0``), so it doubles as the invalidation marker for saved chats: a ``saved_chats`` row stamped with an older generation is *stale* — its answers predate the current index and may be Regenerated against it (phase 53, tasks 03/05). The row is seeded by the migration (not lazily on first bump), so :func:`app.rag.sources_meta.current_sources_version` is a plain PK read. """ __tablename__ = "sources_meta" id: Mapped[int] = mapped_column(Integer, primary_key=True, server_default="1") #: The KB generation. 0 = the pre-counter KB (everything indexed #: before phase 53); incremented by one per KB-changing sync. version: Mapped[int] = mapped_column(Integer, server_default="0") updated_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), onupdate=func.now() ) class GitSource(Base): """One admin-managed source (phase 35; kind discriminator, phase 38). The UI-maintained list the Sync button (phase 32) and import_docs (phase 28) import from. ``kind`` discriminates: ``git`` rows carry a repo ``url`` (cloned/pulled), ``local`` rows carry an existing directory ``path`` (walked directly). DB rows win over the BOR_GIT_SOURCES env var (git-only fallback), which is a fallback while this table is empty (see app.rag.git_sources.effective_git_sources). """ __tablename__ = "git_sources" id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) url: Mapped[str] = mapped_column(Text, unique=True, nullable=False) #: Source-kind discriminator (phase 38): "git" (default) or "local" #: — enforced by the ``ck_git_sources_kind`` CHECK constraint. kind: Mapped[str] = mapped_column(Text, default="git", server_default="'git'") #: Absolute directory of a ``local`` source; NULL for git rows. #: Unique — Postgres treats NULLs as distinct under a unique index. path: Mapped[str | None] = mapped_column(Text, unique=True) added_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) class DocDraft(Base): """One server-side draft of a chat answer saved as documentation (phase 59, task 01). A long answer body must live on the **server**, never in a URL: the "Save as doc" action POSTs the answer's raw markdown to ``POST /api/doc-drafts`` (task 02), which stores it here and hands back an unguessable 128-bit ``uuid4`` ``token`` — the edit screen's URL credential (``/doc-edit.html?draft=``, the share-token trust model, phase 51). ``status`` stays ``draft`` until the push endpoint (task 04) commits + pushes the file to the ``BOR_DOCS_REPO`` branch — then it is ``pushed``, with ``branch`` and ``commit_sha`` recorded (the UI's branch + sha feedback; D3: no PR tooling — the owner opens the PR themselves). """ __tablename__ = "doc_drafts" id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) #: The URL credential (``/doc-edit.html?draft=``): an #: unguessable 128-bit ``uuid4`` — never the row id, never #: sequential/guessable. Unique NOT NULL: unlike the NULLable #: ``saved_chats.share_token`` there is no "un-drafted" state, so #: NULLs never occur (always set on create). token: Mapped[uuid.UUID] = mapped_column( UUID(as_uuid=True), unique=True, nullable=False, default=uuid.uuid4 ) #: The document's title. Defaults client-side to the last user #: question (whitespace-collapsed, ≤120 chars — the chat auto-title #: convention, phase 50); the edit screen changes anything. title: Mapped[str] = mapped_column(Text) #: The in-repo file path (default ``docs/.md``). Guard-railled #: by the API layer (task 02 — repo-relative, no ``..``); the #: column itself is plain TEXT (the ``documents.path`` precedent). path: Mapped[str] = mapped_column(Text) #: The markdown body — the answer's raw text (never HTML — the #: ``bor.chat.v1`` record's ``text``), edited on the edit screen. body: Mapped[str] = mapped_column(Text) #: "draft" until the push endpoint commits + pushes the file, then #: "pushed" — the domain is enforced by the API layer (the #: ``git_sources.kind`` phase-38 precedent: plain TEXT + server #: default, no CHECK constraint). status: Mapped[str] = mapped_column(Text, default="draft", server_default="'draft'") #: Set on push (task 04): the branch the commit landed on (the #: ``BOR_DOCS_BRANCH`` name); NULL while still a draft. branch: Mapped[str | None] = mapped_column(Text) #: ... and the pushed branch's new HEAD sha (must equal #: ``git rev-parse `` in the repo); NULL while still a #: draft. commit_sha: Mapped[str | None] = mapped_column(Text) created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) updated_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), onupdate=func.now() ) class SavedChat(Base): """One owner-saved chat conversation (phase 50). Only conversations the owner explicitly **Saves** are stored — ``/api/chat`` itself stays stateless (A10, owner-locked extension 2026-08-29) and nothing is stored about a conversation that was not saved. ``messages`` holds the exact ``bor.chat.v1`` localStorage record shape (phase 14 — raw text, never HTML), so a saved chat restores pixel-identical through the existing ``renderStoredMessage`` path. """ __tablename__ = "saved_chats" id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) #: Auto-title (the first question, API-side). Plain column on purpose: #: a future rename needs no migration. title: Mapped[str] = mapped_column(String(500)) #: ``list[dict]`` in the ``bor.chat.v1`` record shape #: (``{who, text, sources?, deflected?, suggestions?, thinking?, #: tools?, stopped?}``); the API always supplies a list, so no #: default is needed. messages: Mapped[list] = mapped_column(JSONB) #: Anonymous share link (phase 51): a 128-bit ``uuid4`` token; when #: set, the chat is publicly readable at ``/shared/`` without #: any admin session, and unsharing (token → NULL) revokes it. #: Unique — Postgres treats NULLs as distinct under a unique index #: (the ``git_sources.path`` precedent, phase 38), so any number of #: unshared chats coexist while two identical tokens can never exist. share_token: Mapped[uuid.UUID | None] = mapped_column( UUID(as_uuid=True), unique=True, nullable=True ) #: The sources version (KB generation) the conversation was saved #: against (phase 53): stamped by the API at save time. Existing #: rows (saved before the counter existed) stamp ``0`` — "the #: pre-counter KB" — and become stale on the first KB-changing sync #: (stale = ``sources_version < current``, computed server-side by #: the chats API, phase 53 task 03). sources_version: Mapped[int] = mapped_column(Integer, server_default="0") created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) updated_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), onupdate=func.now() ) class ApiToken(Base): """One admin-issued access token (phase 79, task 01). The admin generates named tokens and hands them out so people can sign in to the app and use it (chat, suggestion chips, cited documents) — the ONLY content that stays anonymous is the shared chats (extending the phase-16 single-admin auth; the ``require_user`` live-check and the admin token API land in tasks 02/03). Trust model — the plaintext token (``bor_`` + 32 hex chars) exists only in the 201 response of the create call, returned **exactly once**; the row never carries it. The stored credential is the SHA-256 hex digest of the **full** token string (``token_hash``): hashing the full string, not the suffix, so a stripped prefix can never collide. The ``saved_chats.share_token`` / ``doc_drafts.token`` lineage — but HASHED: unlike those unguessable ``uuid4`` link tokens these are long-lived hand-out credentials, and a leaked database must not hand anyone working tokens. """ __tablename__ = "api_tokens" id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4) #: The hand-out name (e.g. "alice") — display-only: no index, not #: unique (two tokens may share a label). label: Mapped[str] = mapped_column(String(120), nullable=False) #: The stored credential: the SHA-256 hex digest of the full #: ``bor_…`` token string (the ``documents.content_hash`` #: String(64) precedent). Unique — the lookup is a unique-index hit #: (``ix_api_tokens_token_hash`` — the explicit unique-index shape #: of ``ix_saved_chats_share_token``, phase 51). token_hash: Mapped[str] = mapped_column(String(64), unique=True, nullable=False) created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) #: Bumped to now() on ``POST /api/token-auth`` (task 03 calls the #: service's ``mark_used`` and commits); NULL until the token is #: first used. last_used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) #: Set on revocation (task 02) — the row is dead from that moment #: (enforced immediately on the holder's next request); NULL while #: active. revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)