phase: 191_source_name_labels_tags
All gates re-verified against a stable tree. Report: **Verified (all 6 tasks in `complete/`; this was the final pass)** - `SourceLabel`+alembic `0030` (dev DB `alembic current` = `0030 (head)`, table present, 0 rows); admin-only `GET/PUT/DELETE /api/source-labels` (`require_admin` router-wide, `resolve_source_name`, partial-update caps/404s); Sources-page "Names, labels and tags" card (read-only derived-name `<code>`, per-row `saveInFlight`, 404 note that survives the settle, chips reuse `.git-source-hidden-count`); `question_name_hits` D2 rule; ONE `all_labels` read per turn in `chat.py` feeding both the `source_name_hit` flag and the prompt surfaces; `source_name_hit_bonus=0.005` in `_selection_order`; ` name=`/` label=` in `build_high_prompt` + `build_overview_prompt(names=…)`. - **Completion criteria:** E2E `tests/e2e/test_source_name_labels.py` ✅ 7 passed (arms a–e: editor all kinds, control/named question, rename-never-breaks-identity, 404 note, reset) · full suite ✅ · coverage ✅ · README/AGENTS ✅ · phase dir in `complete/` ✅ · commit left to the harness. - **Gates:** `uv run pytest --cov=app --cov-report=term-missing` → **exit 0, TOTAL 7160 stmts / 16 missed = 99%** (>90%) · `uv run pytest tests/e2e/test_source_name_labels.py -v --no-cov` → **7 passed, exit 0** · regression E2E isolated, exit 0: `test_document_viewer`, `test_admin_auth`, `test_api_tokens`, `test_git_sources_admin`, `test_kb_tree`, `test_local_directory_sources` · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors. - **Battery (task 04 criterion):** `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` at default vs `BOR_SOURCE_NAME_HIT_BONUS=0` → **`diff` empty (line-for-line identical)**, matching the recorded arms (dev KB is empty, so every question deflects; the observable boost is unit/E2E-pinned). - **Documentation:** README — fixed the `/git-sources.html` rows (nav label is **Sources**) and cross-linked the phase-191 card from that section; full audit otherwise clean: all 86 `BOR_*` settings documented with defaults matching `app/config.py`, 0 broken file/dir references and 0 broken anchors in README+AGENTS, AGENTS rule 17 re-checked line-by-line against the code. - **Notable / deviations:** (1) a **concurrent pipeline process** (`phased-execution/progress.mjs` for `191_…_00_phase`) was mutating this same working tree mid-pass — it rewrote `tests/e2e/test_source_name_labels.py` (11→7 tests), `frontend/assets/git-sources.js`, `tests/unit/test_source_labels_editor_ui.py` and `AGENTS.md` at 05:27; every gate above was (re-)run after those edits with the tree no source file newer than the green full run. (2) The task-06 report/phase text claims regression suites `test_chat_token_gate.py`, `test_git_sources_page.py`, `test_rag_browser.py` are green — **those files have never existed in git history** (those runs exit 4); I ran the real equivalents listed above instead (phase files are not mine to edit). (3) **No live changes**: the VM DB already has `source_labels` (0 rows) but the deployed app predates this uncommitted code, so deploying `/api/source-labels` to the live service is the owner's post-commit step, not part of this phase's criteria. - **Next pending phase:** none — `191_source_name_labels_tags` is the only entry in `.agents/phases/todo/`.
This commit is contained in:
File diff suppressed because it is too large
Load Diff
@@ -236,7 +236,17 @@ def test_invalid_index_is_dropped_and_rebuilt(db: Session, alembic: Config) -> N
|
||||
data, and timing a real interruption is nondeterministic; the dev/
|
||||
test DB role is a superuser, and a role without catalog UPDATE
|
||||
rights skips cleanly instead of leaving the DB inconsistent (the
|
||||
stamp that follows the forced state, never before it)."""
|
||||
stamp that follows the forced state, never before it).
|
||||
|
||||
Phase 191 fix: the stamp moves the version table ALL the way below
|
||||
0029, so every LATER migration's objects must be genuinely dropped
|
||||
first (``downgrade("0029")``) — a stamped-below head with a later
|
||||
migration's table still physically present would make every
|
||||
following ``upgrade(head)`` re-run that migration and die on
|
||||
``relation already exists`` (observed with 0030's
|
||||
``source_labels`` once it landed). The only lie that stays between
|
||||
the stamp and the re-run is the corrupted index — exactly the
|
||||
state this test is about."""
|
||||
command.upgrade(alembic, "head")
|
||||
assert _index_valid(db) is True, "precondition: a valid index to corrupt"
|
||||
|
||||
@@ -253,8 +263,12 @@ def test_invalid_index_is_dropped_and_rebuilt(db: Session, alembic: Config) -> N
|
||||
db.rollback()
|
||||
pytest.skip(f"forcing INVALID needs catalog UPDATE rights: {exc}")
|
||||
|
||||
# The stamp moves the version table WITHOUT touching the index, so
|
||||
# the next upgrade() re-runs 0029 against the corrupted state.
|
||||
# Drop every post-0029 migration's objects for real BEFORE the
|
||||
# stamp (the phase-191 note above), then move the version table
|
||||
# WITHOUT touching the index, so the next upgrade() re-runs 0029
|
||||
# against the corrupted state and head is re-derivable cleanly.
|
||||
db.rollback() # release the session before the DDL (house pattern)
|
||||
command.downgrade(alembic, "0029")
|
||||
command.stamp(alembic, "0028")
|
||||
assert _version(db) == "0028"
|
||||
assert _index_valid(db) is False, "the forced INVALID state must be visible"
|
||||
|
||||
@@ -0,0 +1,311 @@
|
||||
"""Integration: migration 0030 (source_labels) schema contract
|
||||
(phase 191, task 01).
|
||||
|
||||
Drives the **real Alembic engine** against the live dev database
|
||||
(``podman compose up -d db``), mirroring the house pattern of
|
||||
``test_migration_0028.py`` (information_schema assertions on the state
|
||||
the migration must leave). The tests target the 0029 → 0030 step
|
||||
explicitly so later migrations cannot break them:
|
||||
|
||||
* upgrade 0029 → 0030 → the ``source_labels`` table exists with the
|
||||
full column contract (D1) — ``source`` VARCHAR(120) NOT NULL and the
|
||||
sole primary key (one label row per source; the join key is the
|
||||
DERIVED ``documents.source`` name — an alias key, never a rename),
|
||||
``display_name`` / ``label`` TEXT nullable (the ≤120 / ≤200 caps are
|
||||
API-boundary concerns, the phase-121 plain-column pattern), ``tags``
|
||||
JSONB NOT NULL with the ``'[]'`` server default (the phase-89
|
||||
``ignore_paths`` shape — a source with no tags reads ``[]``, never
|
||||
NULL), ``updated_at`` TIMESTAMPTZ NOT NULL with the now() server
|
||||
default (house style) — while the 0029 schema
|
||||
(``ix_chunks_embedding_hnsw``) survives;
|
||||
* the NO-BACKFILL contract: right after the upgrade the table is EMPTY
|
||||
(a source without a row behaves exactly as pre-phase);
|
||||
* inserted rows round-trip (name/label/tags verbatim, ``updated_at``
|
||||
stamped by the server default);
|
||||
* a duplicate ``source`` raises ``IntegrityError`` (the PK pin) and a
|
||||
NULL ``tags`` is rejected (NOT NULL — the reset is ``[]``, never
|
||||
NULL);
|
||||
* downgrade to 0029 → the table is GONE (A13 — reversible), the rest of
|
||||
the schema (the HNSW index, ``app_settings``) survives;
|
||||
* upgrade back to 0030 → the table is back (round-trip).
|
||||
|
||||
The ``alembic`` fixture guarantees the DB ends at head even if a test
|
||||
fails or the process is interrupted.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from alembic.config import Config
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from alembic import command
|
||||
from app.db import db_available
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def alembic(db: Session) -> Iterator[Config]:
|
||||
"""Real Alembic config bound to the dev DB (URL from app settings).
|
||||
|
||||
Starts at head (repairs an interrupted earlier run); teardown
|
||||
upgrades to head no matter what happened, so the dev DB is never
|
||||
left below head.
|
||||
"""
|
||||
if not db_available():
|
||||
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
||||
cfg = Config() # no alembic.ini file — env.py gets the URL from app config
|
||||
cfg.set_main_option("script_location", "alembic")
|
||||
command.upgrade(cfg, "head")
|
||||
try:
|
||||
yield cfg
|
||||
finally:
|
||||
# Release the test session's open transaction BEFORE the repair
|
||||
# DDL (the 0028 fixture's lesson): psycopg3 keeps its snapshot
|
||||
# alive while idle-in-transaction, and a later 0029
|
||||
# CREATE INDEX CONCURRENTLY would wait on it forever.
|
||||
db.rollback()
|
||||
command.upgrade(cfg, "head")
|
||||
|
||||
|
||||
def _version(db: Session) -> str | None:
|
||||
return db.execute(text("SELECT version_num FROM alembic_version")).scalar()
|
||||
|
||||
|
||||
def _table_exists(db: Session, table: str) -> bool:
|
||||
return (
|
||||
db.execute(
|
||||
text("SELECT 1 FROM information_schema.tables WHERE table_name = :t"),
|
||||
{"t": table},
|
||||
).scalar()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
def _column(db: Session, table: str, column: str) -> tuple[Any, ...] | None:
|
||||
"""(data_type, is_nullable, column_default, character_maximum_length)
|
||||
for one table column."""
|
||||
row = db.execute(
|
||||
text(
|
||||
"SELECT data_type, is_nullable, column_default, character_maximum_length"
|
||||
" FROM information_schema.columns"
|
||||
" WHERE table_name = :t AND column_name = :c"
|
||||
),
|
||||
{"t": table, "c": column},
|
||||
).fetchone()
|
||||
return tuple(row) if row is not None else None
|
||||
|
||||
|
||||
def _pk_columns(db: Session, table: str) -> list[str]:
|
||||
"""The table's PRIMARY KEY columns in ordinal position."""
|
||||
rows = db.execute(
|
||||
text(
|
||||
"SELECT kcu.column_name"
|
||||
" FROM information_schema.table_constraints tc"
|
||||
" JOIN information_schema.key_column_usage kcu"
|
||||
" ON kcu.constraint_name = tc.constraint_name"
|
||||
" AND kcu.table_name = tc.table_name"
|
||||
" WHERE tc.table_name = :t"
|
||||
" AND tc.constraint_type = 'PRIMARY KEY'"
|
||||
" ORDER BY kcu.ordinal_position"
|
||||
),
|
||||
{"t": table},
|
||||
).fetchall()
|
||||
return [r[0] for r in rows]
|
||||
|
||||
|
||||
def _hnsw_index_exists(db: Session) -> bool:
|
||||
return (
|
||||
db.execute(
|
||||
text(
|
||||
"SELECT 1 FROM pg_indexes"
|
||||
" WHERE indexname = 'ix_chunks_embedding_hnsw'"
|
||||
)
|
||||
).scalar()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
def _clear_rows(db: Session) -> None:
|
||||
db.execute(text("DELETE FROM source_labels"))
|
||||
db.commit()
|
||||
|
||||
|
||||
def test_upgrade_to_0030_creates_source_labels(db: Session, alembic: Config) -> None:
|
||||
"""Upgrade 0029 → 0030: the table exists with the full D1 column
|
||||
contract, the table is ABSENT at 0029, and the 0029 HNSW index
|
||||
survives the additive upgrade."""
|
||||
command.downgrade(alembic, "0029") # start from the pre-0030 state
|
||||
assert _version(db) == "0029"
|
||||
assert not _table_exists(db, "source_labels"), (
|
||||
"source_labels must be absent at 0029"
|
||||
)
|
||||
|
||||
command.upgrade(alembic, "0030")
|
||||
assert _version(db) == "0030", "alembic_version must be at 0030"
|
||||
assert _table_exists(db, "source_labels"), "the table must exist at 0030"
|
||||
|
||||
source = _column(db, "source_labels", "source")
|
||||
assert source is not None, "source_labels.source is missing"
|
||||
assert source[0] == "character varying", "source must be VARCHAR"
|
||||
assert source[1] == "NO", "source must be NOT NULL (PK)"
|
||||
assert source[2] is None, "source must have no server default"
|
||||
assert source[3] == 120, "source must be String(120) (mirrors documents.source)"
|
||||
|
||||
display = _column(db, "source_labels", "display_name")
|
||||
assert display is not None, "source_labels.display_name is missing"
|
||||
assert display[0] == "text", "display_name must be plain TEXT (caps are API-side)"
|
||||
assert display[1] == "YES", "display_name must be nullable (NULL = no name)"
|
||||
|
||||
label = _column(db, "source_labels", "label")
|
||||
assert label is not None, "source_labels.label is missing"
|
||||
assert label[0] == "text", "label must be plain TEXT (caps are API-side)"
|
||||
assert label[1] == "YES", "label must be nullable (NULL = no label)"
|
||||
|
||||
tags = _column(db, "source_labels", "tags")
|
||||
assert tags is not None, "source_labels.tags is missing"
|
||||
assert tags[0] == "jsonb", "tags must be JSONB (the ignore_paths shape)"
|
||||
assert tags[1] == "NO", "tags must be NOT NULL"
|
||||
assert tags[2] is not None and "[]" in str(tags[2]), (
|
||||
"tags must carry the '[]' server default (no tags reads [], never NULL)"
|
||||
)
|
||||
|
||||
updated = _column(db, "source_labels", "updated_at")
|
||||
assert updated is not None, "source_labels.updated_at is missing"
|
||||
assert updated[0] == "timestamp with time zone", "updated_at must be TIMESTAMPTZ"
|
||||
assert updated[1] == "NO", "updated_at must be NOT NULL"
|
||||
assert updated[2] is not None and "now" in str(updated[2]), (
|
||||
"updated_at must carry the now() server default (house style)"
|
||||
)
|
||||
|
||||
assert _pk_columns(db, "source_labels") == ["source"], (
|
||||
"source must be the sole primary key (one label row per source)"
|
||||
)
|
||||
|
||||
# The NO-BACKFILL contract (D1): the migration creates an EMPTY
|
||||
# table — every source keeps its pre-phase "no row" behavior.
|
||||
count = db.execute(text("SELECT count(*) FROM source_labels")).scalar()
|
||||
assert count == 0, "0030 must not backfill any label rows"
|
||||
|
||||
# The 0029 schema survives the additive upgrade.
|
||||
assert _hnsw_index_exists(db), "ix_chunks_embedding_hnsw (0029) must survive"
|
||||
|
||||
|
||||
def test_inserted_rows_round_trip(db: Session, alembic: Config) -> None:
|
||||
"""At head, the D1 values round-trip verbatim (name / label / tags)
|
||||
and ``updated_at`` is stamped by the server default."""
|
||||
command.upgrade(alembic, "head")
|
||||
try:
|
||||
db.execute(
|
||||
text(
|
||||
"INSERT INTO source_labels (source, display_name, label, tags)"
|
||||
" VALUES ('gitea-forge', 'the git server',"
|
||||
" 'self-hosted forge + Gitea', '[\"gitea\", \"forge\"]'::jsonb)"
|
||||
)
|
||||
)
|
||||
db.execute(
|
||||
text(
|
||||
"INSERT INTO source_labels (source)"
|
||||
" VALUES ('bare-row')"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
rows = db.execute(
|
||||
text(
|
||||
"SELECT source, display_name, label, tags, updated_at"
|
||||
" FROM source_labels ORDER BY source"
|
||||
)
|
||||
).fetchall()
|
||||
assert len(rows) == 2, "both rows must be stored"
|
||||
assert rows[0][0] == "bare-row"
|
||||
assert rows[0][1] is None and rows[0][2] is None
|
||||
assert rows[0][3] == [], "the server default must read as the empty list"
|
||||
assert rows[0][4] is not None, "updated_at must be stamped (server default)"
|
||||
assert rows[1][0] == "gitea-forge", "source must round-trip verbatim"
|
||||
assert rows[1][1] == "the git server"
|
||||
assert rows[1][2] == "self-hosted forge + Gitea"
|
||||
assert rows[1][3] == ["gitea", "forge"], "the tags array must round-trip"
|
||||
finally:
|
||||
_clear_rows(db)
|
||||
|
||||
|
||||
def test_duplicate_source_is_rejected(db: Session, alembic: Config) -> None:
|
||||
"""The PK is the uniqueness guard: two label rows for one source
|
||||
would make "which name wins" undefined, so the second insert must
|
||||
raise ``IntegrityError``."""
|
||||
command.upgrade(alembic, "head")
|
||||
try:
|
||||
db.execute(
|
||||
text("INSERT INTO source_labels (source) VALUES ('homelab')")
|
||||
)
|
||||
db.commit()
|
||||
with pytest.raises(IntegrityError):
|
||||
db.execute(
|
||||
text(
|
||||
"INSERT INTO source_labels (source, display_name)"
|
||||
" VALUES ('homelab', 'second name')"
|
||||
)
|
||||
)
|
||||
db.rollback() # the IntegrityError aborts the open transaction
|
||||
finally:
|
||||
_clear_rows(db)
|
||||
|
||||
|
||||
def test_null_tags_is_rejected(db: Session, alembic: Config) -> None:
|
||||
"""``tags`` is NOT NULL: "no tags" is ``[]`` (the server default),
|
||||
never NULL — readers never branch on the None case."""
|
||||
command.upgrade(alembic, "head")
|
||||
try:
|
||||
with pytest.raises(IntegrityError):
|
||||
db.execute(
|
||||
text(
|
||||
"INSERT INTO source_labels (source, tags)"
|
||||
" VALUES ('nulltags', NULL)"
|
||||
)
|
||||
)
|
||||
db.rollback() # the IntegrityError aborts the open transaction
|
||||
stored = db.execute(text("SELECT count(*) FROM source_labels")).scalar()
|
||||
assert stored == 0, "the rejected row must not be stored"
|
||||
finally:
|
||||
_clear_rows(db)
|
||||
|
||||
|
||||
def test_downgrade_to_0029_drops_the_table(db: Session, alembic: Config) -> None:
|
||||
"""Downgrade 0030 → 0029: the table is gone (A13 — fully
|
||||
reversible) while the rest of the schema survives (the 0029 HNSW
|
||||
index, ``app_settings``). The names/labels/tags are re-enterable
|
||||
from the Sources page — nothing outside this table was mutated."""
|
||||
command.downgrade(alembic, "0029")
|
||||
assert _version(db) == "0029"
|
||||
assert not _table_exists(db, "source_labels"), "source_labels must be dropped"
|
||||
|
||||
assert _hnsw_index_exists(db), "ix_chunks_embedding_hnsw (0029) must survive"
|
||||
strip = _column(db, "documents", "page_strip_mode")
|
||||
assert strip is not None and strip[3] == 16, (
|
||||
"documents.page_strip_mode must survive the downgrade"
|
||||
)
|
||||
key = _column(db, "app_settings", "key")
|
||||
assert key is not None and key[3] == 100, "app_settings.key must survive"
|
||||
|
||||
|
||||
def test_upgrade_round_trip_restores_the_table(db: Session, alembic: Config) -> None:
|
||||
"""Downgrade to 0029, then upgrade back to 0030: the table is back
|
||||
with the column contract and PK intact (A13 round-trip)."""
|
||||
command.downgrade(alembic, "0029")
|
||||
command.upgrade(alembic, "0030")
|
||||
assert _version(db) == "0030", "round-trip upgrade must land at 0030"
|
||||
assert _table_exists(db, "source_labels"), "the table must be back"
|
||||
|
||||
source = _column(db, "source_labels", "source")
|
||||
assert source is not None, "source_labels.source must be back"
|
||||
assert source[0] == "character varying", "source must be VARCHAR after the round-trip"
|
||||
assert source[1] == "NO", "source must be NOT NULL after the round-trip"
|
||||
assert source[3] == 120, "source must be String(120) after the round-trip"
|
||||
tags = _column(db, "source_labels", "tags")
|
||||
assert tags is not None, "source_labels.tags must be back"
|
||||
assert tags[0] == "jsonb", "tags must be JSONB after the round-trip"
|
||||
assert tags[1] == "NO", "tags must be NOT NULL after the round-trip"
|
||||
assert _pk_columns(db, "source_labels") == ["source"]
|
||||
@@ -0,0 +1,753 @@
|
||||
"""Integration: the admin ``/api/source-labels`` surface (phase 191, task 02).
|
||||
|
||||
The wire half of "Sources need a name, a label, tag(s)" (``TODO.md`` L4)
|
||||
— the catalogue the Sources page loads, the upsert its Save drives and
|
||||
the reset its Reset button drives. Driven through the real app with the
|
||||
house ``TestClient`` cookie jar (the ``test_ui_settings_api.py`` phase-91
|
||||
fixture pattern: the admin login and the token-user matrix), against the
|
||||
real Postgres (``podman compose up -d db``); no LLM involved.
|
||||
|
||||
Contract under test:
|
||||
|
||||
* **the gate** — an anonymous caller AND a signed-in token USER (role
|
||||
``"user"``) get 403 ``{"detail": "admin only"}`` on all three routes
|
||||
(the router-wide ``require_admin``, the phase-91/168 admin-surface
|
||||
shape, one fixed detail with nothing to enumerate), and nothing lands;
|
||||
* **the catalogue GET** — the union of the ``git_sources`` rows of BOTH
|
||||
kinds and the indexed document sources no row derives (the uploads),
|
||||
de-duplicated and sorted by ``source``, each entry exactly
|
||||
``{source, display_name, label, tags}`` with its label row or the
|
||||
``null``/``[]`` defaults; an env-only deployment (empty table) lists
|
||||
the ``BOR_GIT_SOURCES`` names; three tables, three SELECTs — the
|
||||
assembly never queries per source;
|
||||
* **the PUT** — the round trip (set all three → 200 → the GET reads them
|
||||
back), the partial-update shape (an omitted field is untouched, an
|
||||
explicit ``null`` clears, ``[]`` clears the tags, an all-omitted body
|
||||
is a 200 no-op), the trim-then-cap ordering, the case-insensitive tag
|
||||
de-dupe preserving the first case, the framework's standard 422 for a
|
||||
121-char name / 201-char label / 11 tags / 33-char tag / whitespace-only
|
||||
tag / a key outside the shape — each leaving a pre-seeded row
|
||||
BYTE-IDENTICAL — the 404 for a source that is neither a row name nor an
|
||||
indexed document (nothing created), the 422 for a > 120-char path, and
|
||||
the ``normalize_tags`` funnel (its ``ValueError`` is a 422, never a
|
||||
500);
|
||||
* **the DELETE** — 200 with the entry's post-reset state, the GET shows
|
||||
the nulls after it, a second DELETE is 404;
|
||||
* **the alias contract (D1)** — labelling a source touches
|
||||
``source_labels`` and NOTHING else: ``documents.source``, the row's
|
||||
derived name and the catalogue's identity are byte-identical after a
|
||||
rename, which is what keeps a saved chat's ``source/path`` locator, a
|
||||
shared page and a verbatim caption valid across a rename.
|
||||
|
||||
``source_labels``, ``git_sources`` and the catalogue are global state:
|
||||
truncated around every test.
|
||||
|
||||
Requires: podman compose up -d db
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from collections.abc import Callable, Iterator
|
||||
from contextlib import contextmanager
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
from sqlalchemy import event, text
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api import source_labels as source_labels_api
|
||||
from app.config import Settings
|
||||
from app.db import engine
|
||||
from app.main import app as fastapi_app
|
||||
from app.models import Document, GitSource
|
||||
from tests.conftest import ADMIN_PASSWORD
|
||||
|
||||
#: The catalogue entry's exact wire shape (``extra="forbid"`` makes the
|
||||
#: omission structural — the row's ``updated_at`` never rides along).
|
||||
_ENTRY_KEYS = {"source", "display_name", "label", "tags"}
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_state(db: Session) -> Iterator[None]:
|
||||
"""Every table this module reads or writes is global state (the
|
||||
catalogue is a FULL scan of the registry and the documents, so
|
||||
another suite's rows would move the pins)."""
|
||||
_wipe(db)
|
||||
yield
|
||||
_wipe(db)
|
||||
|
||||
|
||||
def _wipe(db: Session) -> None:
|
||||
db.execute(text("DELETE FROM source_labels"))
|
||||
db.execute(text("TRUNCATE git_sources"))
|
||||
db.execute(text("TRUNCATE chunks, documents"))
|
||||
db.execute(text("TRUNCATE api_tokens"))
|
||||
db.commit()
|
||||
|
||||
|
||||
def _login(client: TestClient) -> TestClient:
|
||||
client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||
return client
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def env_sources(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> Callable[[str], None]:
|
||||
"""Pin the ``BOR_GIT_SOURCES`` fallback the way
|
||||
``test_git_sources_api.py`` does — with a fresh ``Settings`` that
|
||||
ignores the operator's ``.env`` (a dev ``.env`` with a repo in it
|
||||
would otherwise BE the catalogue while the table is empty, the
|
||||
phase-35 fallback). Returns the setter so one test can name the list
|
||||
it wants; the empty string is the "nothing configured" state."""
|
||||
|
||||
def _set(git_sources: str = "") -> None:
|
||||
monkeypatch.setattr(
|
||||
"app.rag.git_sources.get_settings",
|
||||
lambda: Settings(
|
||||
_env_file=None, # pyright: ignore[reportCallIssue]
|
||||
git_sources=git_sources,
|
||||
),
|
||||
)
|
||||
|
||||
return _set
|
||||
|
||||
|
||||
def _doc(db: Session, source: str, path: str) -> Document:
|
||||
"""One indexed document — the catalogue's (b) arm reads
|
||||
``documents.source`` and NOTHING else."""
|
||||
doc = Document(
|
||||
id=uuid.uuid4(),
|
||||
source=source,
|
||||
path=path,
|
||||
full_path=f"/srv/{source}/{path}",
|
||||
title=path.rsplit("/", 1)[-1],
|
||||
content=f"body of {path}",
|
||||
content_hash="0" * 64,
|
||||
)
|
||||
db.add(doc)
|
||||
return doc
|
||||
|
||||
|
||||
def _seed_three_kinds(db: Session) -> None:
|
||||
"""One git row, one local row and one upload-shaped document.
|
||||
|
||||
The git row is a ``file://`` URL (the phase-135 E2E-fixture form) and
|
||||
carries the ``.git`` suffix, so the pin also proves the name is the
|
||||
importer's DERIVED one (``repo_name``) rather than the raw URL; a
|
||||
document exists under that derived name too, which is what the
|
||||
de-duplication has to collapse.
|
||||
"""
|
||||
db.add_all(
|
||||
[
|
||||
GitSource(url="file:///srv/repos/gitea-forge.git", kind="git"),
|
||||
GitSource(
|
||||
url="/srv/docs/media-vault",
|
||||
kind="local",
|
||||
path="/srv/docs/media-vault",
|
||||
),
|
||||
]
|
||||
)
|
||||
db.add(_doc(db, "gitea-forge", "docs/app.md"))
|
||||
db.add(_doc(db, "corpus-upload", "notes/a.md"))
|
||||
db.commit()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _captured_statements() -> Iterator[list[str]]:
|
||||
"""Every SQL statement the ENGINE runs inside the block.
|
||||
|
||||
The house statement-capture idiom (``test_import_session_retention``
|
||||
/ ``test_prune_cascade_statements``): a local listener on the shared
|
||||
engine, removed in ``finally`` — the request runs on another thread
|
||||
under ``TestClient``, so the engine (not the session) is the place
|
||||
both sides see.
|
||||
"""
|
||||
captured: list[str] = []
|
||||
|
||||
def _listen(
|
||||
conn, cursor, statement, parameters, context, executemany
|
||||
) -> None: # the engine's own signature — no types to restate
|
||||
captured.append(statement)
|
||||
|
||||
event.listen(engine, "before_cursor_execute", _listen)
|
||||
try:
|
||||
yield captured
|
||||
finally:
|
||||
event.remove(engine, "before_cursor_execute", _listen)
|
||||
|
||||
|
||||
def _selects(statements: list[str]) -> list[str]:
|
||||
"""The SELECTs only, whitespace-collapsed for comparability, with
|
||||
the pool's ``SELECT 1`` health check dropped — it is a checkout
|
||||
artifact, not a read this endpoint asked for, and pinning it would
|
||||
make the count depend on how warm the pool happens to be."""
|
||||
return [
|
||||
collapsed
|
||||
for statement in statements
|
||||
if (collapsed := " ".join(statement.split())).upper().startswith("SELECT")
|
||||
and collapsed.upper() != "SELECT 1"
|
||||
]
|
||||
|
||||
|
||||
def _row(db: Session, source: str) -> dict[str, Any]:
|
||||
"""One ``source_labels`` row read RAW — the pin never trusts the
|
||||
shape under test to report its own storage."""
|
||||
db.expire_all()
|
||||
found = db.execute(
|
||||
text(
|
||||
"SELECT source, display_name, label, tags FROM source_labels"
|
||||
" WHERE source = :s"
|
||||
),
|
||||
{"s": source},
|
||||
).first()
|
||||
if found is None:
|
||||
return {}
|
||||
return {
|
||||
"source": found[0],
|
||||
"display_name": found[1],
|
||||
"label": found[2],
|
||||
"tags": found[3],
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The admin gate (the phase-91 shape)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_anonymous_is_403_on_every_route(client: TestClient, db: Session) -> None:
|
||||
"""Router-wide ``require_admin``: one fixed detail on all three
|
||||
routes, and nothing landed (the ``test_ui_settings_api`` contract)."""
|
||||
r = client.get("/api/source-labels")
|
||||
assert r.status_code == 403
|
||||
assert r.json() == {"detail": "admin only"}
|
||||
|
||||
r = client.put("/api/source-labels/gitea-forge", json={"display_name": "nope"})
|
||||
assert r.status_code == 403
|
||||
assert r.json() == {"detail": "admin only"}
|
||||
|
||||
r = client.delete("/api/source-labels/gitea-forge")
|
||||
assert r.status_code == 403
|
||||
assert r.json() == {"detail": "admin only"}
|
||||
|
||||
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
|
||||
|
||||
|
||||
def test_token_user_is_403_on_every_route(client: TestClient) -> None:
|
||||
"""A signed-in token USER is not the admin: the owner's naming surface
|
||||
is closed to them on all three methods (the phase-79 token matrix —
|
||||
the same leg the phase-91/168 admin surfaces pin)."""
|
||||
admin = _login(TestClient(fastapi_app))
|
||||
r = admin.post("/api/tokens", json={"label": "alice"})
|
||||
assert r.status_code == 201, r.text
|
||||
token = r.json()["token"]
|
||||
assert client.post("/api/token-auth", json={"token": token}).status_code == 204
|
||||
assert client.get("/api/whoami").json() == {"authenticated": True, "role": "user"}
|
||||
|
||||
assert client.get("/api/source-labels").status_code == 403
|
||||
r = client.put("/api/source-labels/gitea-forge", json={"display_name": "nope"})
|
||||
assert r.status_code == 403
|
||||
assert r.json() == {"detail": "admin only"}
|
||||
assert client.delete("/api/source-labels/gitea-forge").status_code == 403
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET — the catalogue
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_get_empty_deployment_is_an_empty_catalogue(
|
||||
admin_client: TestClient, env_sources: Callable[[str], None]
|
||||
) -> None:
|
||||
"""No rows and no env list: the response is the empty list, not a 404
|
||||
and not a missing key — the page always has a shape to render."""
|
||||
env_sources("")
|
||||
r = admin_client.get("/api/source-labels")
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json() == {"labels": []}
|
||||
|
||||
|
||||
def test_get_lists_git_local_and_upload_sources(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""The D5 union, sorted, de-duplicated, with the exact entry shape.
|
||||
|
||||
``gitea-forge`` (a git row, derived from the URL basename with the
|
||||
``.git`` stripped) also has a document under that name — the entry
|
||||
still appears exactly ONCE. ``corpus-upload`` has documents and no
|
||||
row, so only arm (b) can list it (and the E2E needs it to).
|
||||
"""
|
||||
_seed_three_kinds(db)
|
||||
|
||||
r = admin_client.get("/api/source-labels")
|
||||
assert r.status_code == 200, r.text
|
||||
labels = r.json()["labels"]
|
||||
assert [entry["source"] for entry in labels] == [
|
||||
"corpus-upload",
|
||||
"gitea-forge",
|
||||
"media-vault",
|
||||
]
|
||||
for entry in labels:
|
||||
assert set(entry) == _ENTRY_KEYS
|
||||
assert entry["display_name"] is None
|
||||
assert entry["label"] is None
|
||||
assert entry["tags"] == []
|
||||
|
||||
|
||||
def test_get_carries_the_stored_label_on_one_entry_only(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""One labelled row, two unlabelled ones: the label rides its own
|
||||
entry and the others keep the null/``[]`` defaults (a label row is
|
||||
data beside the name, never a rename — D1)."""
|
||||
_seed_three_kinds(db)
|
||||
assert (
|
||||
admin_client.put(
|
||||
"/api/source-labels/corpus-upload",
|
||||
json={"display_name": "the uploads", "tags": ["upload"]},
|
||||
).status_code
|
||||
== 200
|
||||
)
|
||||
|
||||
labels = admin_client.get("/api/source-labels").json()["labels"]
|
||||
by_source = {entry["source"]: entry for entry in labels}
|
||||
assert by_source["corpus-upload"] == {
|
||||
"source": "corpus-upload",
|
||||
"display_name": "the uploads",
|
||||
"label": None,
|
||||
"tags": ["upload"],
|
||||
}
|
||||
assert by_source["gitea-forge"]["display_name"] is None
|
||||
assert by_source["media-vault"]["tags"] == []
|
||||
|
||||
|
||||
def test_get_lists_env_source_names_while_the_table_is_empty(
|
||||
admin_client: TestClient, env_sources: Callable[[str], None]
|
||||
) -> None:
|
||||
"""An env-only deployment (no ``git_sources`` rows): the
|
||||
``BOR_GIT_SOURCES`` names are the catalogue, derived exactly as the
|
||||
importer would name what it clones — and one of them is labelable."""
|
||||
env_sources("https://a.example.com/one.git, git@b.example.com:two.git ,")
|
||||
|
||||
labels = admin_client.get("/api/source-labels").json()["labels"]
|
||||
assert [entry["source"] for entry in labels] == ["one", "two"]
|
||||
|
||||
r = admin_client.put("/api/source-labels/one", json={"label": "the first repo"})
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json()["label"] == "the first repo"
|
||||
|
||||
|
||||
def test_get_reads_three_tables_and_never_one_per_source(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""The one-query-per-table discipline: with four sources and one
|
||||
label row the GET costs exactly three SELECTs — the registry, the
|
||||
distinct document sources, the label rows. A per-source label lookup
|
||||
(or a third existence query) fails this pin."""
|
||||
_seed_three_kinds(db)
|
||||
db.add(_doc(db, "third-source", "b.md"))
|
||||
db.commit()
|
||||
admin_client.put("/api/source-labels/media-vault", json={"display_name": "vault"})
|
||||
|
||||
with _captured_statements() as statements:
|
||||
body = admin_client.get("/api/source-labels").json()
|
||||
selects = _selects(statements)
|
||||
|
||||
assert [entry["source"] for entry in body["labels"]] == [
|
||||
"corpus-upload",
|
||||
"gitea-forge",
|
||||
"media-vault",
|
||||
"third-source",
|
||||
]
|
||||
assert len(selects) == 3, selects
|
||||
assert sum(1 for s in selects if "FROM git_sources" in s) == 1
|
||||
assert sum(1 for s in selects if "DISTINCT documents.source" in s) == 1
|
||||
assert sum(1 for s in selects if "FROM source_labels" in s) == 1
|
||||
# No whole-entity document read — the catalogue never hydrates content.
|
||||
assert not any("documents.content" in s for s in selects), selects
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# PUT — the upsert
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_put_round_trips_all_three_fields(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""Set name + label + tags on a git source: 200 carries the stored
|
||||
shape, the row exists, and the GET reads it back."""
|
||||
_seed_three_kinds(db)
|
||||
|
||||
r = admin_client.put(
|
||||
"/api/source-labels/gitea-forge",
|
||||
json={
|
||||
"display_name": "the git server",
|
||||
"label": "self-hosted forge + Gitea",
|
||||
"tags": ["gitea", "forge"],
|
||||
},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json() == {
|
||||
"source": "gitea-forge",
|
||||
"display_name": "the git server",
|
||||
"label": "self-hosted forge + Gitea",
|
||||
"tags": ["gitea", "forge"],
|
||||
}
|
||||
assert _row(db, "gitea-forge") == {
|
||||
"source": "gitea-forge",
|
||||
"display_name": "the git server",
|
||||
"label": "self-hosted forge + Gitea",
|
||||
"tags": ["gitea", "forge"],
|
||||
}
|
||||
entry = admin_client.get("/api/source-labels").json()["labels"]
|
||||
assert entry[1]["display_name"] == "the git server"
|
||||
|
||||
|
||||
def test_put_partial_update_shape(admin_client: TestClient, db: Session) -> None:
|
||||
"""D5 three ways: a value SETS, an OMITTED field is untouched, an
|
||||
explicit ``null`` CLEARS (and ``tags: []`` clears the tags while the
|
||||
strings stay)."""
|
||||
_seed_three_kinds(db)
|
||||
admin_client.put(
|
||||
"/api/source-labels/media-vault",
|
||||
json={"display_name": "the media box", "label": "jellyfin + arrs", "tags": ["jellyfin"]},
|
||||
)
|
||||
|
||||
# Only the label moves — the name and the tags keep their values.
|
||||
r = admin_client.put("/api/source-labels/media-vault", json={"label": "arrs only"})
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json()["display_name"] == "the media box"
|
||||
assert r.json()["label"] == "arrs only"
|
||||
assert r.json()["tags"] == ["jellyfin"]
|
||||
|
||||
# An explicit null clears JUST that field.
|
||||
r = admin_client.put("/api/source-labels/media-vault", json={"display_name": None})
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json() == {
|
||||
"source": "media-vault",
|
||||
"display_name": None,
|
||||
"label": "arrs only",
|
||||
"tags": ["jellyfin"],
|
||||
}
|
||||
|
||||
# An empty tag list clears the tags; the label stays.
|
||||
r = admin_client.put("/api/source-labels/media-vault", json={"tags": []})
|
||||
assert r.json() == {
|
||||
"source": "media-vault",
|
||||
"display_name": None,
|
||||
"label": "arrs only",
|
||||
"tags": [],
|
||||
}
|
||||
assert _row(db, "media-vault")["tags"] == []
|
||||
|
||||
|
||||
def test_put_all_omitted_body_is_a_200_no_op(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""The ``GitSourcePatchIn`` "all absent → 200 no-op" precedent: an
|
||||
empty body changes nothing on a labelled row (and creates a defaults
|
||||
row on an unlabelled one)."""
|
||||
_seed_three_kinds(db)
|
||||
admin_client.put(
|
||||
"/api/source-labels/gitea-forge",
|
||||
json={"display_name": "the git server", "tags": ["gitea"]},
|
||||
)
|
||||
|
||||
r = admin_client.put("/api/source-labels/gitea-forge", json={})
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json() == {
|
||||
"source": "gitea-forge",
|
||||
"display_name": "the git server",
|
||||
"label": None,
|
||||
"tags": ["gitea"],
|
||||
}
|
||||
assert _row(db, "gitea-forge")["display_name"] == "the git server"
|
||||
|
||||
|
||||
def test_put_trims_values_and_dedupes_tags(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""Trim before the caps (``GitSourceIn._trim_url``), and the ONE tag
|
||||
rule: case-insensitive de-dupe with the FIRST case kept, order
|
||||
otherwise preserved."""
|
||||
_seed_three_kinds(db)
|
||||
|
||||
r = admin_client.put(
|
||||
"/api/source-labels/gitea-forge",
|
||||
json={
|
||||
"display_name": " the git server ",
|
||||
"label": " self-hosted forge ",
|
||||
"tags": [" Gitea ", "gitea", "FORGE", "forge"],
|
||||
},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json()["display_name"] == "the git server"
|
||||
assert r.json()["label"] == "self-hosted forge"
|
||||
assert r.json()["tags"] == ["Gitea", "FORGE"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("field", "value", "needle"),
|
||||
[
|
||||
("display_name", "x" * 121, "string_too_long"),
|
||||
("label", "x" * 201, "string_too_long"),
|
||||
("tags", ["tag"] * 11, "too_long"),
|
||||
("tags", ["x" * 33], "string_too_long"),
|
||||
("tags", [" "], "string_too_short"),
|
||||
("tags", [""], "string_too_short"),
|
||||
],
|
||||
)
|
||||
def test_put_validation_422s_leave_the_row_unchanged(
|
||||
admin_client: TestClient,
|
||||
db: Session,
|
||||
field: str,
|
||||
value: Any,
|
||||
needle: str,
|
||||
) -> None:
|
||||
"""Every D5 cap is the framework's standard 422 (the pydantic body,
|
||||
not a fixed ``detail``), names the offending field, and writes
|
||||
NOTHING: the pre-seeded row is byte-identical afterwards.
|
||||
|
||||
The trim runs BEFORE the cap, so a 121-char value that trims to 120
|
||||
is legal — the 120-char leg of each boundary is asserted too, or the
|
||||
cap could silently be one short."""
|
||||
_seed_three_kinds(db)
|
||||
admin_client.put(
|
||||
"/api/source-labels/gitea-forge",
|
||||
json={"display_name": "the git server", "label": "forge", "tags": ["gitea"]},
|
||||
)
|
||||
before = _row(db, "gitea-forge")
|
||||
|
||||
if field == "display_name":
|
||||
legal = "y" * 120
|
||||
elif field == "label":
|
||||
legal = "y" * 200
|
||||
else:
|
||||
legal = ["tag"] * 10 if len(value) == 11 else ["y" * 32]
|
||||
r = admin_client.put("/api/source-labels/gitea-forge", json={field: value})
|
||||
assert r.status_code == 422, r.text
|
||||
detail = r.json()["detail"]
|
||||
assert isinstance(detail, list), r.text
|
||||
assert detail[0]["type"] == needle, detail
|
||||
assert field in detail[0]["loc"], detail
|
||||
assert _row(db, "gitea-forge") == before
|
||||
|
||||
assert admin_client.put(
|
||||
"/api/source-labels/gitea-forge", json={field: legal}
|
||||
).status_code == 200
|
||||
|
||||
|
||||
def test_put_rejects_a_key_outside_the_shape(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""``extra="forbid"`` (the phase-91 voice): an undeclared key is a
|
||||
422 and no row is created."""
|
||||
_seed_three_kinds(db)
|
||||
|
||||
r = admin_client.put(
|
||||
"/api/source-labels/media-vault",
|
||||
json={"display_name": "vault", "source": "gitea-forge"},
|
||||
)
|
||||
assert r.status_code == 422, r.text
|
||||
assert r.json()["detail"][0]["type"] == "extra_forbidden"
|
||||
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
|
||||
|
||||
|
||||
def test_put_rejects_a_malformed_tags_value(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""A ``tags`` value that is not a list of strings is pydantic's own
|
||||
422, in its own voice (``list_type`` / ``string_type``): the trimmer
|
||||
passes anything it does not understand straight through rather than
|
||||
guessing, so the framework names the type and NOTHING is written."""
|
||||
_seed_three_kinds(db)
|
||||
|
||||
r = admin_client.put("/api/source-labels/media-vault", json={"tags": "gitea"})
|
||||
assert r.status_code == 422, r.text
|
||||
assert r.json()["detail"][0]["type"] == "list_type"
|
||||
|
||||
r = admin_client.put(
|
||||
"/api/source-labels/media-vault", json={"tags": ["gitea", 7]}
|
||||
)
|
||||
assert r.status_code == 422, r.text
|
||||
assert r.json()["detail"][0]["type"] == "string_type"
|
||||
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
|
||||
|
||||
|
||||
def test_put_folds_empty_strings_to_null(admin_client: TestClient, db: Session) -> None:
|
||||
"""An empty / whitespace-only ``display_name`` or ``label`` is the
|
||||
CLEAR leg, not a stored empty string: "no name" and "an empty name"
|
||||
must not be two different things for the prompt builders (task 05)."""
|
||||
_seed_three_kinds(db)
|
||||
admin_client.put(
|
||||
"/api/source-labels/corpus-upload",
|
||||
json={"display_name": "the uploads", "label": "the upload folder"},
|
||||
)
|
||||
|
||||
r = admin_client.put(
|
||||
"/api/source-labels/corpus-upload",
|
||||
json={"display_name": " ", "label": ""},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json()["display_name"] is None
|
||||
assert r.json()["label"] is None
|
||||
assert _row(db, "corpus-upload")["display_name"] is None
|
||||
|
||||
|
||||
def test_put_404s_a_source_that_exists_nowhere(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""D5's no-orphan-row arm: neither a registry name nor an indexed
|
||||
document → 404, and NOTHING is written (a dead label would boost
|
||||
nothing and never appear in the UI to be cleared)."""
|
||||
_seed_three_kinds(db)
|
||||
|
||||
r = admin_client.put("/api/source-labels/no-such-source", json={"label": "x"})
|
||||
assert r.status_code == 404, r.text
|
||||
assert r.json() == {"detail": "source not found"}
|
||||
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
|
||||
|
||||
# A near-miss on the derived name is still unknown (no fuzzy leg).
|
||||
r = admin_client.put("/api/source-labels/gitea-forg", json={"label": "x"})
|
||||
assert r.status_code == 404, r.text
|
||||
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
|
||||
|
||||
|
||||
def test_put_422s_an_overlong_source_path(admin_client: TestClient) -> None:
|
||||
"""The path param carries the ``documents.source`` 120 cap: a longer
|
||||
name is the framework's standard 422 on the ``path`` location (it
|
||||
never reaches the query)."""
|
||||
r = admin_client.put(f"/api/source-labels/{'x' * 121}", json={"label": "x"})
|
||||
assert r.status_code == 422, r.text
|
||||
detail = r.json()["detail"]
|
||||
assert detail[0]["type"] == "string_too_long"
|
||||
assert detail[0]["loc"] == ["path", "source"]
|
||||
|
||||
|
||||
def test_put_maps_the_tag_normalizer_error_to_422(
|
||||
admin_client: TestClient,
|
||||
db: Session,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The belt: ``normalize_tags`` is the ONE tag rule and its
|
||||
``ValueError`` is the framework's 422 — never a 500. The request
|
||||
shape caps the same numbers today, so the belt is pinned by
|
||||
substituting the raising normalizer (the row stays byte-identical,
|
||||
which is the part that must not regress)."""
|
||||
_seed_three_kinds(db)
|
||||
admin_client.put(
|
||||
"/api/source-labels/gitea-forge", json={"display_name": "the git server"}
|
||||
)
|
||||
before = _row(db, "gitea-forge")
|
||||
|
||||
def _boom(_raw: list[str]) -> list[str]:
|
||||
raise ValueError("at most 10 tags are allowed")
|
||||
|
||||
monkeypatch.setattr(source_labels_api, "normalize_tags", _boom)
|
||||
|
||||
r = admin_client.put("/api/source-labels/gitea-forge", json={"tags": ["gitea"]})
|
||||
assert r.status_code == 422, r.text
|
||||
assert r.json() == {"detail": "at most 10 tags are allowed"}
|
||||
assert _row(db, "gitea-forge") == before
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DELETE — the reset
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_delete_resets_then_404s(admin_client: TestClient, db: Session) -> None:
|
||||
"""200 with the entry's post-reset state (the phase-168
|
||||
``DELETE /api/admin/settings/{key}`` convention — the row re-renders
|
||||
from the response it already has), the GET shows the nulls, and a
|
||||
second DELETE is a 404 with one fixed detail."""
|
||||
_seed_three_kinds(db)
|
||||
admin_client.put(
|
||||
"/api/source-labels/gitea-forge",
|
||||
json={"display_name": "the git server", "label": "forge", "tags": ["gitea"]},
|
||||
)
|
||||
|
||||
r = admin_client.delete("/api/source-labels/gitea-forge")
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json() == {
|
||||
"source": "gitea-forge",
|
||||
"display_name": None,
|
||||
"label": None,
|
||||
"tags": [],
|
||||
}
|
||||
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
|
||||
|
||||
entry = {e["source"]: e for e in admin_client.get("/api/source-labels").json()["labels"]}
|
||||
assert entry["gitea-forge"] == {
|
||||
"source": "gitea-forge",
|
||||
"display_name": None,
|
||||
"label": None,
|
||||
"tags": [],
|
||||
}
|
||||
|
||||
r = admin_client.delete("/api/source-labels/gitea-forge")
|
||||
assert r.status_code == 404, r.text
|
||||
assert r.json() == {"detail": "source label not found"}
|
||||
|
||||
|
||||
def test_delete_works_for_a_source_that_has_vanished(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""The reset never consults the registry: a label whose source has
|
||||
since been removed (phase 69's total removal) stays CLEARABLE, or a
|
||||
label could never be taken back off a gone source."""
|
||||
_seed_three_kinds(db)
|
||||
admin_client.put("/api/source-labels/corpus-upload", json={"label": "the upload"})
|
||||
db.execute(text("DELETE FROM documents WHERE source = 'corpus-upload'"))
|
||||
db.commit()
|
||||
|
||||
r = admin_client.delete("/api/source-labels/corpus-upload")
|
||||
assert r.status_code == 200, r.text
|
||||
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The alias contract (D1)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_labelling_a_source_touches_nothing_but_the_label_row(
|
||||
admin_client: TestClient, db: Session
|
||||
) -> None:
|
||||
"""The owner-confirmed alias contract, at the API: a rename writes
|
||||
``source_labels`` and leaves ``documents.source`` (and with it every
|
||||
``source/path`` locator, citation and KB-tree node) byte-identical —
|
||||
and the catalogue's identity column stays the DERIVED name."""
|
||||
_seed_three_kinds(db)
|
||||
before = db.execute(
|
||||
text("SELECT source, path FROM documents ORDER BY source, path")
|
||||
).fetchall()
|
||||
|
||||
r = admin_client.put(
|
||||
"/api/source-labels/gitea-forge", json={"display_name": "the git server"}
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
db.expire_all()
|
||||
after = db.execute(
|
||||
text("SELECT source, path FROM documents ORDER BY source, path")
|
||||
).fetchall()
|
||||
assert after == before
|
||||
assert [row[0] for row in after] == ["corpus-upload", "gitea-forge"]
|
||||
|
||||
catalogue = admin_client.get("/api/source-labels").json()["labels"]
|
||||
assert [entry["source"] for entry in catalogue] == [
|
||||
"corpus-upload",
|
||||
"gitea-forge",
|
||||
"media-vault",
|
||||
]
|
||||
assert catalogue[1]["display_name"] == "the git server"
|
||||
|
||||
# And the PUT still keys on the DERIVED name after the rename (the
|
||||
# display name is never an address).
|
||||
assert admin_client.put(
|
||||
"/api/source-labels/the git server", json={"label": "nope"}
|
||||
).status_code == 404
|
||||
@@ -13,6 +13,7 @@ import json
|
||||
import uuid
|
||||
from collections.abc import Iterator, Sequence
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import pytest
|
||||
@@ -27,6 +28,7 @@ from app.rag.agent import AGENT_TOOLS, SHOW_TEXT_TOOL
|
||||
from app.rag.llm import StreamPiece, ToolCallPiece
|
||||
from app.rag.prompts import SUGGEST_INTRO, build_deflect_prompt
|
||||
from app.rag.retriever import RetrievedChunk, weak_hit_titles
|
||||
from app.rag.source_labels import SourceLabelView
|
||||
from app.rag.suggestions import MAX_SUGGESTIONS, derive_suggestions
|
||||
from tests.conftest import ADMIN_PASSWORD
|
||||
|
||||
@@ -1817,3 +1819,114 @@ def test_deflected_turn_resolves_nothing_and_logs_zero(
|
||||
done = [f for f in frames if f.get("type") == "done"]
|
||||
assert done and done[0]["deflected"] is True
|
||||
assert "answer_text" not in done[0]
|
||||
|
||||
|
||||
# ---------- phase 191 (D4, task 05): the turn's prompt learns the name ----------
|
||||
|
||||
|
||||
def _label_view(
|
||||
source: str = "Homelab",
|
||||
display_name: str | None = None,
|
||||
label: str | None = None,
|
||||
) -> SourceLabelView:
|
||||
return SourceLabelView(source=source, display_name=display_name, label=label, tags=())
|
||||
|
||||
|
||||
def test_plan_turn_high_prompt_carries_the_source_name() -> None:
|
||||
"""The HIGH branch hands the label views to ``build_high_prompt``: the
|
||||
suggested block gains `` name="…"`` / `` label="…"`` right after
|
||||
``source="…"`` — the model sees the name→docs mapping."""
|
||||
doc = _doc("Gitea Upgrade", "GITEA_DOC_CONTENT", summary="Upgrading Gitea.")
|
||||
names = {"Homelab": _label_view(display_name="the git server", label="self-hosted forge")}
|
||||
plan = chat_api.plan_turn(
|
||||
[_chunk(doc, 0.90, cosine=0.90)],
|
||||
_settings(threshold=0.30),
|
||||
names=names,
|
||||
)
|
||||
assert plan.deflected is False
|
||||
assert 'source="Homelab" name="the git server" label="self-hosted forge" path=' in (
|
||||
plan.system_prompt
|
||||
)
|
||||
assert "Upgrading Gitea." in plan.system_prompt # the body is untouched
|
||||
|
||||
|
||||
def test_plan_turn_without_names_is_byte_identical() -> None:
|
||||
"""THE kill pin on the call-through: ``names`` omitted, ``None`` and
|
||||
``{}`` each build the exact pre-phase prompt (and every other TurnPlan
|
||||
field is identical)."""
|
||||
doc = _doc("Gitea Upgrade", "GITEA_DOC_CONTENT", summary="Upgrading Gitea.")
|
||||
chunks = [_chunk(doc, 0.90, cosine=0.90)]
|
||||
settings = _settings(threshold=0.30)
|
||||
pre = chat_api.plan_turn(chunks, settings)
|
||||
for names in (None, {}):
|
||||
plan = chat_api.plan_turn(chunks, settings, names=names)
|
||||
assert plan.system_prompt == pre.system_prompt
|
||||
assert plan.deflected == pre.deflected
|
||||
assert [d.id for d in plan.suggested_docs] == [d.id for d in pre.suggested_docs]
|
||||
assert ' name="' not in pre.system_prompt
|
||||
|
||||
|
||||
def test_plan_turn_low_prompt_ignores_names() -> None:
|
||||
"""A deflected turn has no document blocks: the LOW prompt is
|
||||
byte-identical whether or not the source is named (D4 touches the
|
||||
suggested tier only)."""
|
||||
doc = _doc("Mongolia Trivia", "MONGOLIA_CONTENT")
|
||||
settings = _settings(threshold=0.62)
|
||||
chunks = [_chunk(doc, 0.05, cosine=0.10, fts_hit=True)]
|
||||
pre = chat_api.plan_turn(chunks, settings)
|
||||
plan = chat_api.plan_turn(chunks, settings, names={"Homelab": _label_view("Homelab", "x")})
|
||||
assert plan.deflected is True
|
||||
assert plan.system_prompt == pre.system_prompt
|
||||
# The pre-phase deflected build, unchanged, is what the LOW branch owes.
|
||||
assert plan.system_prompt == build_deflect_prompt(
|
||||
weak_hit_titles(chunks), notes=[], kb_overview=""
|
||||
)
|
||||
|
||||
|
||||
def test_chat_module_issues_exactly_one_label_read_per_turn() -> None:
|
||||
"""The marker pin for the "one query per turn" contract: the endpoint
|
||||
reads the label rows ONCE (``all_labels``) and derives both the D2 hit
|
||||
set and the prompt map from that read — no second label query, and task
|
||||
04's convenience wrapper (which would re-query) is gone from the
|
||||
module."""
|
||||
src = Path(chat_api.__file__).read_text(encoding="utf-8")
|
||||
assert src.count("all_labels(") == 1
|
||||
assert "question_name_hit_sources(" not in src
|
||||
assert src.count("names=labels") == 1, "the single read is what the prompt receives"
|
||||
assert "question_name_hits(" in src, "the pure D2 rule stays applied per view"
|
||||
|
||||
|
||||
def test_endpoint_single_label_read_feeds_flags_and_prompt(
|
||||
client: TestClient,
|
||||
gate_env: tuple[_FakeSession, _CannedLLM],
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The one-read contract on the wire (task 04 + task 05): the ONE
|
||||
``all_labels`` result of a turn sets the D2 boost flags AND the
|
||||
``<document>`` name/label attributes — one query, two surfaces."""
|
||||
_session, llm = gate_env
|
||||
doc = _doc("Gitea Upgrade", "GITEA_DOC_CONTENT", summary="Upgrading Gitea.")
|
||||
chunk = _chunk(doc, 0.90, cosine=0.90)
|
||||
assert chunk.source_name_hit is False # the pre-flag state
|
||||
monkeypatch.setattr(chat_api, "retrieve", _fake_retriever([chunk]))
|
||||
views = {
|
||||
"Homelab": SourceLabelView(
|
||||
source="Homelab",
|
||||
display_name="the git server",
|
||||
label="self-hosted forge",
|
||||
tags=("gitea",),
|
||||
)
|
||||
}
|
||||
reads: list[int] = []
|
||||
|
||||
def _counting_all_labels(_db: Any) -> dict[str, SourceLabelView]:
|
||||
reads.append(1)
|
||||
return views
|
||||
|
||||
monkeypatch.setattr(chat_api, "all_labels", _counting_all_labels)
|
||||
frames = _ask(client, "how do I restart the git server?")
|
||||
assert [f for f in frames if f.get("type") == "done"], "the turn completed"
|
||||
assert len(reads) == 1, "ONE label read for the whole turn"
|
||||
assert chunk.source_name_hit is True, "the same read sets the task-04 boost flag"
|
||||
system = llm.seen[0][0]["content"]
|
||||
assert 'source="Homelab" name="the git server" label="self-hosted forge"' in system
|
||||
|
||||
@@ -45,6 +45,9 @@ def test_defaults_match_locked_decisions(monkeypatch: pytest.MonkeyPatch) -> Non
|
||||
# selection-time document score is ON by default (0.005 — the
|
||||
# owner-tunable starting point; ``0`` is the kill switch).
|
||||
assert s.name_hit_bonus == 0.005
|
||||
# Phase 191, D3: the SECOND bounded selection bonus (the source-name
|
||||
# hit) ships ON at the same phase-119 starting point.
|
||||
assert s.source_name_hit_bonus == 0.005
|
||||
# Owner instruction 2026-08-22: answers may run up to 32 768 tokens.
|
||||
assert s.max_output_tokens == 32_768
|
||||
# Phase 17: the model's thinking streams by default (kill-switch off).
|
||||
@@ -553,6 +556,36 @@ def test_name_hit_bonus_rejects_negative(monkeypatch: pytest.MonkeyPatch) -> Non
|
||||
_settings()
|
||||
|
||||
|
||||
def test_source_name_hit_bonus_default_and_env_override(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Phase 191, D3 (LOCKED A3): the bounded SOURCE-name bonus (the
|
||||
second, independent selection knob over the ``source_labels`` row)
|
||||
is ON by default at the phase-119 starting point (0.005) and
|
||||
env-tunable so the owner re-tunes live (the ``BOR_NAME_HIT_BONUS``
|
||||
pattern). ``0`` is legal — the byte-identical kill switch."""
|
||||
monkeypatch.delenv("BOR_SOURCE_NAME_HIT_BONUS", raising=False)
|
||||
s = _settings()
|
||||
assert s.source_name_hit_bonus == 0.005
|
||||
monkeypatch.setenv("BOR_SOURCE_NAME_HIT_BONUS", "0")
|
||||
assert _settings().source_name_hit_bonus == 0.0
|
||||
monkeypatch.setenv("BOR_SOURCE_NAME_HIT_BONUS", "0.01")
|
||||
assert _settings().source_name_hit_bonus == 0.01
|
||||
|
||||
|
||||
def test_source_name_hit_bonus_rejects_negative(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""``0`` is the byte-identical kill switch (the pre-phase-191
|
||||
selection order) — a NEGATIVE bonus would demote source-named
|
||||
documents (the exact opposite of D2/D3), so the validator fails
|
||||
loudly at startup naming the field (the ``name_hit_bonus``
|
||||
pattern, phase 191)."""
|
||||
monkeypatch.setenv("BOR_SOURCE_NAME_HIT_BONUS", "-0.1")
|
||||
with pytest.raises(ValidationError, match="source_name_hit_bonus"):
|
||||
_settings()
|
||||
|
||||
|
||||
def test_retrieval_ef_search_default_and_env_override(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
|
||||
@@ -373,7 +373,13 @@ def test_branch_field_css_rules_are_house_styled() -> None:
|
||||
assert "var(--ink-soft)" in hint_rule and "font-size: 0.8rem" in hint_rule, (
|
||||
"the .field-hint values (the token field's hint styling)"
|
||||
)
|
||||
tag = css.find(".git-source-hidden-count,\n.git-source-branch-count {")
|
||||
# Phase 191 (task 03): the name/label/tags row's two tags ("named" and
|
||||
# one tag chip) joined this SAME rule — the reuse that keeps the token
|
||||
# input from shipping its own tag CSS.
|
||||
tag = css.find(
|
||||
".git-source-hidden-count,\n.git-source-branch-count,\n"
|
||||
".source-label-state,\n.source-label-tag {"
|
||||
)
|
||||
assert tag != -1, "the row tag joins the 'hidden on' tag rule"
|
||||
tag_rule = css[tag : css.find("}", tag)]
|
||||
assert "background: var(--bg)" in tag_rule and "border" in tag_rule, (
|
||||
|
||||
@@ -398,8 +398,14 @@ def test_hidden_checkbox_css_rules_exist_with_accent_color() -> None:
|
||||
)
|
||||
# Phase 135 (task 04): the "branch: <name>" state tag joined this
|
||||
# rule — the "hidden on" tag keeps the same idiom (text + a
|
||||
# distinct background, never color alone).
|
||||
tag = _css_rule(css, ".git-source-hidden-count,\n.git-source-branch-count")
|
||||
# distinct background, never color alone). Phase 191 (task 03): the
|
||||
# name/label/tags row's "named" tag and its tag CHIPS joined the same
|
||||
# rule (the token input reuses this tag, it ships none of its own).
|
||||
tag = _css_rule(
|
||||
css,
|
||||
".git-source-hidden-count,\n.git-source-branch-count,\n"
|
||||
".source-label-state,\n.source-label-tag",
|
||||
)
|
||||
assert "var(--ink)" in tag and "var(--bg)" in tag, (
|
||||
"text + a distinct background (never color alone)"
|
||||
)
|
||||
|
||||
@@ -10,13 +10,22 @@ pins the deterministic ``Sources: …`` inventory tail (LOCKED A2): the
|
||||
format/order/counts, the 1-doc survival, the happy-path stored content,
|
||||
the byte-identical zero-doc / ``LLMError`` paths, and the byte-identical
|
||||
model input.
|
||||
|
||||
And the phase-191 display names (task 05, D4): the INPUT lines lead with
|
||||
the owner's ``display_name`` when the source has a label row (the raw
|
||||
source name is replaced in the ``lite`` model's input ONLY — the stored
|
||||
row and its ``Sources: …`` tail keep the derived name), byte-identical
|
||||
for every unnamed source, and fed by exactly ONE ``all_labels`` read per
|
||||
run (the marker pins at the end).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import inspect
|
||||
import logging
|
||||
import uuid
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import select, text
|
||||
@@ -24,6 +33,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import Settings, get_settings
|
||||
from app.models import Document, KbOverview
|
||||
from app.rag import overview as overview_module
|
||||
from app.rag.llm import LLMError
|
||||
from app.rag.overview import (
|
||||
KB_OVERVIEW_MODE,
|
||||
@@ -35,6 +45,8 @@ from app.rag.overview import (
|
||||
regenerate_overview,
|
||||
)
|
||||
from app.rag.retriever import TRUNCATION_MARKER
|
||||
from app.rag.source_labels import all_labels as real_all_labels
|
||||
from app.rag.source_labels import set_label
|
||||
|
||||
REPLY = "- Homelab\n - Kubernetes (k3s)\n- Deployments\n - Borg backups"
|
||||
|
||||
@@ -650,3 +662,208 @@ def test_regenerate_llm_error_preserves_an_existing_inventory_tail(
|
||||
assert row is not None and row.content == previous
|
||||
finally:
|
||||
_truncate_documents(db)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 191 (D4, task 05): the input lines carry the owner's display name
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: The owner's alias for the ``gitea-infra`` source (phase 191 D1 — an
|
||||
#: alias, never the identity).
|
||||
GIT_SERVER = "the git server"
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def clean_source_labels(db: Session):
|
||||
"""Empty ``source_labels`` before and after (migration-0030 state)."""
|
||||
db.execute(text("DELETE FROM source_labels"))
|
||||
db.commit()
|
||||
yield
|
||||
db.execute(text("DELETE FROM source_labels"))
|
||||
db.commit()
|
||||
|
||||
|
||||
# ---------- build_overview_prompt: the names map (pure) ----------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"names",
|
||||
[None, {}, {"Other": "another name"}],
|
||||
ids=["names=None", "names={}", "no-entry-for-the-source"],
|
||||
)
|
||||
def test_input_lines_without_a_names_entry_are_byte_identical_to_pre_phase(
|
||||
names: dict[str, str] | None,
|
||||
) -> None:
|
||||
"""THE kill pin (phase 191 D4): no map, an empty map and a map that
|
||||
misses the row's source each build the pre-phase input byte-for-byte —
|
||||
the same cap, the same separator, the same order."""
|
||||
rows = [
|
||||
("Homelab", "k8s.md", "Kubernetes", "A cluster.\nSource: Homelab/k8s.md"),
|
||||
("Homelab", "borg.md", "Borg", None),
|
||||
]
|
||||
pre = build_overview_prompt(rows, max_chars=10_000)
|
||||
assert build_overview_prompt(rows, max_chars=10_000, names=names) == pre
|
||||
|
||||
|
||||
def test_input_line_leads_with_the_display_name_when_named() -> None:
|
||||
"""A named source: the display name REPLACES the raw source name in the
|
||||
lite model's input — the path / title / summary halves untouched."""
|
||||
rows = [("gitea-infra", "docs/install.md", "Install", "The install notes.")]
|
||||
_system, user = build_overview_prompt(rows, names={"gitea-infra": GIT_SERVER})
|
||||
assert user == f"{GIT_SERVER} — docs/install.md — Install — The install notes."
|
||||
assert "gitea-infra" not in user
|
||||
|
||||
|
||||
def test_input_line_without_an_entry_keeps_the_raw_source_name() -> None:
|
||||
"""Mixed catalogue: only the named source's lines change — every other
|
||||
line stays exactly as the pre-phase builder wrote it."""
|
||||
rows = [
|
||||
("gitea-infra", "a.md", "A", "Sum A."),
|
||||
("Homelab", "b.md", "B", "Sum B."),
|
||||
]
|
||||
_system, user = build_overview_prompt(rows, names={"gitea-infra": GIT_SERVER})
|
||||
assert user.splitlines() == [
|
||||
f"{GIT_SERVER} — a.md — A — Sum A.",
|
||||
"Homelab — b.md — B — Sum B.",
|
||||
]
|
||||
|
||||
|
||||
def test_named_summary_less_line_has_no_dangling_dash() -> None:
|
||||
"""The pre-phase "no summary → no trailing field" rule survives the
|
||||
name swap."""
|
||||
_system, user = build_overview_prompt(
|
||||
[("gitea-infra", "a.md", "A", None)], names={"gitea-infra": GIT_SERVER}
|
||||
)
|
||||
assert user == f"{GIT_SERVER} — a.md — A"
|
||||
|
||||
|
||||
def test_named_entry_with_an_empty_display_name_falls_back_to_the_source() -> None:
|
||||
"""Defensive: an empty name in the map is "no name" (the API trims
|
||||
empty submissions to NULL, so this never reaches the wire — the input
|
||||
line must never lead with a blank)."""
|
||||
_system, user = build_overview_prompt(
|
||||
[("gitea-infra", "a.md", "A", None)], names={"gitea-infra": ""}
|
||||
)
|
||||
assert user == "gitea-infra — a.md — A"
|
||||
|
||||
|
||||
def test_names_take_part_in_the_input_cap() -> None:
|
||||
"""The cap counts the whole line (the name included) — the truncation
|
||||
contract is unchanged: cut at the cap, marker on its own line."""
|
||||
rows = [("gitea-infra", "a.md", "A", None)]
|
||||
_system, plain = build_overview_prompt(rows, max_chars=10)
|
||||
_system, named = build_overview_prompt(rows, max_chars=10, names={"gitea-infra": GIT_SERVER})
|
||||
assert named != plain
|
||||
assert named.endswith(TRUNCATION_MARKER)
|
||||
assert named.startswith(GIT_SERVER[:10])
|
||||
|
||||
|
||||
# ---------- regenerate_overview: the ONE all_labels read (DB) ----------
|
||||
|
||||
|
||||
def test_regenerate_model_input_carries_the_display_name(
|
||||
db: Session, clean_overview, clean_source_labels
|
||||
) -> None:
|
||||
"""The wire half of D4: the single read feeds the input lines, and the
|
||||
unnamed source's line is untouched."""
|
||||
_truncate_documents(db)
|
||||
set_label(db, "gitea-infra", display_name=GIT_SERVER)
|
||||
db.commit()
|
||||
try:
|
||||
base = datetime(2024, 6, 15, 12, 0, 0, tzinfo=UTC)
|
||||
_add_doc(db, "gitea-infra", "a.md", "A", "Sum A.", created_at=base + timedelta(days=1))
|
||||
_add_doc(db, "Homelab", "b.md", "B", "Sum B.", created_at=base)
|
||||
llm = _FakeLLM()
|
||||
assert asyncio.run(regenerate_overview(llm, db)) is True
|
||||
user = llm.messages[1]["content"]
|
||||
assert user.splitlines() == [
|
||||
f"{GIT_SERVER} — a.md — A — Sum A.",
|
||||
"Homelab — b.md — B — Sum B.",
|
||||
]
|
||||
finally:
|
||||
_truncate_documents(db)
|
||||
|
||||
|
||||
def test_regenerate_stored_content_keeps_the_derived_names(
|
||||
db: Session, clean_overview, clean_source_labels
|
||||
) -> None:
|
||||
"""The ALIAS contract (D1) on the stored row: the deterministic
|
||||
``Sources: …`` inventory tail names sources by their DERIVED name —
|
||||
a rename never changes the identity the rest of the app keys on."""
|
||||
_truncate_documents(db)
|
||||
set_label(db, "gitea-infra", display_name=GIT_SERVER, label="self-hosted forge")
|
||||
db.commit()
|
||||
try:
|
||||
_add_doc(db, "gitea-infra", "a.md", "A")
|
||||
assert asyncio.run(regenerate_overview(_FakeLLM(), db)) is True
|
||||
row = db.get(KbOverview, 1)
|
||||
assert row is not None
|
||||
assert row.content == REPLY + "\nSources: gitea-infra (1 doc)"
|
||||
assert GIT_SERVER not in row.content
|
||||
finally:
|
||||
_truncate_documents(db)
|
||||
|
||||
|
||||
def test_regenerate_without_label_rows_is_byte_identical_to_pre_phase(
|
||||
db: Session, clean_overview, clean_source_labels
|
||||
) -> None:
|
||||
"""No label rows (the whole-KB kill pin): the model input and the
|
||||
stored content are exactly the pre-phase bytes."""
|
||||
_truncate_documents(db)
|
||||
try:
|
||||
_add_doc(db, "Homelab", "a.md", "A", "Sum A.")
|
||||
llm = _FakeLLM()
|
||||
assert asyncio.run(regenerate_overview(llm, db)) is True
|
||||
assert llm.messages[1]["content"] == "Homelab — a.md — A — Sum A."
|
||||
row = db.get(KbOverview, 1)
|
||||
assert row is not None and row.content == REPLY + "\nSources: Homelab (1 doc)"
|
||||
finally:
|
||||
_truncate_documents(db)
|
||||
|
||||
|
||||
def test_regenerate_reads_the_label_rows_once(
|
||||
db: Session, clean_overview, clean_source_labels, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""ONE ``all_labels`` read per run (the task's single-query contract):
|
||||
the regenerator calls it exactly once and derives the name map from
|
||||
those views — no second query."""
|
||||
_truncate_documents(db)
|
||||
set_label(db, "gitea-infra", display_name=GIT_SERVER)
|
||||
db.commit()
|
||||
calls: list[int] = []
|
||||
|
||||
def _counting_all_labels(session: Session) -> dict:
|
||||
calls.append(1)
|
||||
return real_all_labels(session)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"app.rag.overview.all_labels",
|
||||
_counting_all_labels,
|
||||
)
|
||||
try:
|
||||
_add_doc(db, "gitea-infra", "a.md", "A")
|
||||
_add_doc(db, "gitea-infra", "b.md", "B")
|
||||
assert asyncio.run(regenerate_overview(_FakeLLM(), db)) is True
|
||||
assert len(calls) == 1
|
||||
finally:
|
||||
_truncate_documents(db)
|
||||
|
||||
|
||||
# ---------- the marker pins: one read, in the right function ----------
|
||||
|
||||
OVERVIEW_PY = Path(overview_module.__file__).resolve().read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def test_overview_module_issues_exactly_one_all_labels_read() -> None:
|
||||
"""The marker pin: the ONE ``all_labels(` call in the module — the
|
||||
overview builder never queries labels on its own."""
|
||||
assert OVERVIEW_PY.count("all_labels(") == 1
|
||||
|
||||
|
||||
def test_the_single_read_lives_in_regenerate_overview() -> None:
|
||||
"""The read sits in the caller (which already holds the session), not
|
||||
in the pure builder — so the builder stays a pure function of its
|
||||
arguments and one read covers the whole run."""
|
||||
src = inspect.getsource(regenerate_overview)
|
||||
assert src.count("all_labels(") == 1
|
||||
assert "all_labels(" not in inspect.getsource(build_overview_prompt)
|
||||
|
||||
@@ -39,6 +39,12 @@ phase-124 ``show_image`` clause through one mechanism, and every build
|
||||
the flags do not select stays byte-identical (the constant's own sha pin
|
||||
lives in :mod:`tests.unit.test_prompt_lock`).
|
||||
|
||||
And the phase-191 source names (task 05, D4): a ``<document>`` block can
|
||||
carry the owner's label row as the ADDITIVE ``name`` / ``label``
|
||||
attributes (order ``source`` → ``name?`` → ``label?`` → ``path`` →
|
||||
``title`` → ``date``, XML-escaped values), with no map / no entry for the
|
||||
source byte-identical to pre-phase (the kill pin).
|
||||
|
||||
And the phase-119 cite discipline (task 04, LOCKED A5): the intro's
|
||||
final sentence ("cite the document(s) you used, by path", phase 118)
|
||||
is REPLACED — the HIGH prompt carries the discipline sentence exactly
|
||||
@@ -65,6 +71,7 @@ from app.rag.prompts import (
|
||||
SHOW_TEXT_CLAUSE,
|
||||
SUGGEST_INTRO,
|
||||
TOOLS_SECTION,
|
||||
_attr,
|
||||
_base,
|
||||
_document_body,
|
||||
_extract_context_line,
|
||||
@@ -74,6 +81,7 @@ from app.rag.prompts import (
|
||||
build_steering_section,
|
||||
)
|
||||
from app.rag.retriever import TRUNCATION_MARKER
|
||||
from app.rag.source_labels import SourceLabelView
|
||||
|
||||
#: A small multi-line outline standing in for the lite-generated one.
|
||||
OVERVIEW = "- Homelab\n - Kubernetes (k3s)\n- Deployments\n - Borg backups"
|
||||
@@ -1944,3 +1952,204 @@ def test_high_prompt_pdf_image_doc_block_exact_shape() -> None:
|
||||
assert prompt.index(line) < prompt.index("A diagram of the cluster.")
|
||||
assert block_start < prompt.index(line) < prompt.index("</document>")
|
||||
assert "The second page of the guide" not in prompt
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 191 (D4, task 05): the ``<document>`` block learns the owner's name
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: The owner's display name for the fixture source (phase 191 D1 — an
|
||||
#: ALIAS beside the derived name, never a replacement for it).
|
||||
GIT_SERVER = "the git server"
|
||||
|
||||
#: The owner's one-line description of the same source.
|
||||
FORGE_LABEL = "self-hosted forge + Gitea"
|
||||
|
||||
|
||||
def _label_view(
|
||||
source: str = "Homelab",
|
||||
display_name: str | None = None,
|
||||
label: str | None = None,
|
||||
tags: tuple[str, ...] = (),
|
||||
) -> SourceLabelView:
|
||||
"""One label-row view (task 01's frozen dataclass) for the pins."""
|
||||
return SourceLabelView(
|
||||
source=source,
|
||||
display_name=display_name,
|
||||
label=label,
|
||||
tags=tags,
|
||||
)
|
||||
|
||||
|
||||
def _block_head(prompt: str) -> str:
|
||||
"""The opening tag of the FIRST ``<document>`` block (attribute
|
||||
order/escaping pins read it whole, so nothing is asserted loosely)."""
|
||||
start = prompt.index("<document ")
|
||||
return prompt[start : prompt.index(">", start) + 1]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"names",
|
||||
[
|
||||
None,
|
||||
{},
|
||||
{"Other": _label_view("Other", display_name=GIT_SERVER, label=FORGE_LABEL)},
|
||||
],
|
||||
ids=["names=None", "names={}", "no-entry-for-the-source"],
|
||||
)
|
||||
def test_high_prompt_without_a_label_entry_is_byte_identical_to_pre_phase(
|
||||
names: dict[str, SourceLabelView] | None,
|
||||
) -> None:
|
||||
"""THE kill pin (phase 191 D4): no map, an empty map and a map with no
|
||||
entry for the document's source each build a prompt byte-identical to
|
||||
the pre-phase build — no ``name``, no ``label``, no stray space."""
|
||||
doc = _doc(
|
||||
"kubernetes.md",
|
||||
"Talos Linux on three nodes.",
|
||||
"Kubernetes Homelab Cluster",
|
||||
summary="A Talos Linux cluster on three nodes.",
|
||||
)
|
||||
pre = build_high_prompt([doc])
|
||||
assert build_high_prompt([doc], names=names) == pre
|
||||
assert ' name="' not in pre and ' label="' not in pre
|
||||
|
||||
|
||||
def test_no_label_block_keeps_the_exact_pre_phase_head() -> None:
|
||||
"""The block head for a label-less source, byte for byte (the shape the
|
||||
phase-106/118/125 pins rely on: ``source`` → ``path`` → ``title`` →
|
||||
``date``, single-spaced)."""
|
||||
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes Homelab Cluster", summary="S.")
|
||||
assert _block_head(build_high_prompt([doc], names={})) == (
|
||||
'<document source="Homelab" path="kubernetes.md" '
|
||||
'title="Kubernetes Homelab Cluster" date="2024-06-15">'
|
||||
)
|
||||
|
||||
|
||||
def test_document_block_name_attribute_sits_immediately_after_source() -> None:
|
||||
"""A ``display_name``-only row: exactly `` name="…"`` added, directly
|
||||
after ``source="…"`` and before ``path`` (the attribute-order pin)."""
|
||||
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
|
||||
prompt = build_high_prompt([doc], names={"Homelab": _label_view(display_name=GIT_SERVER)})
|
||||
assert _block_head(prompt) == (
|
||||
'<document source="Homelab" name="the git server" path="kubernetes.md" '
|
||||
'title="Kubernetes" date="2024-06-15">'
|
||||
)
|
||||
assert ' label="' not in prompt
|
||||
|
||||
|
||||
def test_document_block_label_only_view_adds_exactly_the_label_attribute() -> None:
|
||||
"""A ``label``-only row gains ONE attribute — an unset field adds
|
||||
NOTHING (never an empty ``name=""``)."""
|
||||
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
|
||||
prompt = build_high_prompt([doc], names={"Homelab": _label_view(label=FORGE_LABEL)})
|
||||
assert _block_head(prompt) == (
|
||||
'<document source="Homelab" label="self-hosted forge + Gitea" '
|
||||
'path="kubernetes.md" title="Kubernetes" date="2024-06-15">'
|
||||
)
|
||||
assert ' name="' not in prompt
|
||||
|
||||
|
||||
def test_document_block_row_with_neither_field_is_byte_identical() -> None:
|
||||
"""A row that exists but carries neither name nor label (the owner
|
||||
saved only tags) leaves the block untouched — tags never render in the
|
||||
block (they are a retrieval input, D2/D3)."""
|
||||
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
|
||||
pre = build_high_prompt([doc])
|
||||
names = {"Homelab": _label_view(tags=("gitea", "forge"))}
|
||||
built = build_high_prompt([doc], names=names)
|
||||
assert built == pre
|
||||
assert "gitea" not in built
|
||||
|
||||
|
||||
def test_document_block_carries_both_attributes_in_the_locked_order() -> None:
|
||||
"""Both fields set → both attributes, in the fixed order
|
||||
``source`` → ``name`` → ``label`` → ``path`` → ``title`` → ``date``."""
|
||||
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
|
||||
names = {"Homelab": _label_view(display_name=GIT_SERVER, label=FORGE_LABEL)}
|
||||
head = _block_head(build_high_prompt([doc], names=names))
|
||||
order = ['source="', 'name="', 'label="', 'path="', 'title="', 'date="']
|
||||
positions = [head.index(attr) for attr in order]
|
||||
assert positions == sorted(positions)
|
||||
assert head == (
|
||||
'<document source="Homelab" name="the git server" '
|
||||
'label="self-hosted forge + Gitea" path="kubernetes.md" '
|
||||
'title="Kubernetes" date="2024-06-15">'
|
||||
)
|
||||
|
||||
|
||||
def test_document_block_name_and_label_are_xml_escaped() -> None:
|
||||
"""The escaping pin (task 05 work item 1): a ``"`` in owner text renders
|
||||
as ``"`` so the attribute cannot break; ``&``/``<`` fold too."""
|
||||
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
|
||||
names = {
|
||||
"Homelab": _label_view(
|
||||
display_name='Reese "forge" & <co>',
|
||||
label='say "hi"',
|
||||
)
|
||||
}
|
||||
prompt = build_high_prompt([doc], names=names)
|
||||
assert ' name="Reese "forge" & <co>"' in prompt
|
||||
assert ' label="say "hi""' in prompt
|
||||
# The unescaped form never reaches the wire: the attribute stays closed.
|
||||
assert ' name="Reese "forge"' not in prompt
|
||||
head = _block_head(prompt)
|
||||
assert head.count('"') % 2 == 0, "an unescaped quote would break the attribute"
|
||||
|
||||
|
||||
def test_attribute_escaping_helper_covers_the_xml_five() -> None:
|
||||
"""``_attr`` is the ONE escaper for the new attribute values: the XML
|
||||
specials go, the single quote stays (the delimiter is the double
|
||||
quote, and ``Reese's`` must stay readable to the model)."""
|
||||
assert _attr('a & b < c > d " e') == "a & b < c > d " e"
|
||||
assert _attr("Reese's forge") == "Reese's forge"
|
||||
|
||||
|
||||
def test_names_apply_per_document_source_and_never_touch_the_body() -> None:
|
||||
"""Only the blocks whose source has an entry gain the attribute; the
|
||||
summary body (and the other source's block) are untouched."""
|
||||
named = _doc("a.md", "CONTENT_A", "Title A", summary="SUMMARY A.")
|
||||
other = _doc("b.md", "CONTENT_B", "Title B", summary="SUMMARY B.")
|
||||
other.source = "Other"
|
||||
prompt = build_high_prompt(
|
||||
[named, other],
|
||||
names={"Homelab": _label_view(display_name=GIT_SERVER)},
|
||||
)
|
||||
assert 'source="Homelab" name="the git server" path="a.md"' in prompt
|
||||
assert 'source="Other" path="b.md"' in prompt
|
||||
assert prompt.count(GIT_SERVER) == 1, "the name rides the attribute, nothing else"
|
||||
assert "SUMMARY A." in prompt and "SUMMARY B." in prompt
|
||||
|
||||
|
||||
def test_names_on_an_empty_documents_build_change_nothing() -> None:
|
||||
"""No blocks → nothing to annotate: the honest "(no documents
|
||||
matched…)" build is byte-identical with a label row in hand."""
|
||||
names = {"Homelab": _label_view(display_name=GIT_SERVER, label=FORGE_LABEL)}
|
||||
assert build_high_prompt([], names=names) == build_high_prompt([])
|
||||
|
||||
|
||||
def test_names_compose_with_folder_lines_and_display_clauses() -> None:
|
||||
"""The additive attributes ride alongside the phase-119 folder lines and
|
||||
the phase-124/182 display clauses — the section order and the intro are
|
||||
untouched (the folder lines stay plain, no new markup)."""
|
||||
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
|
||||
names = {"Homelab": _label_view(display_name=GIT_SERVER)}
|
||||
prompt = build_high_prompt(
|
||||
[doc],
|
||||
folder_lines=["docs/gitea.md — Gitea install"],
|
||||
show_image=True,
|
||||
show_text=True,
|
||||
names=names,
|
||||
)
|
||||
assert 'source="Homelab" name="the git server" path="kubernetes.md"' in prompt
|
||||
assert prompt.index(SUGGEST_INTRO) < prompt.index("docs/gitea.md — Gitea install")
|
||||
assert prompt.index("docs/gitea.md — Gitea install") < prompt.index("<document ")
|
||||
assert prompt.endswith(TOOLS_SECTION[: -len("</tools>")]
|
||||
+ SHOW_IMAGE_CLAUSE + "\n\n" + SHOW_TEXT_CLAUSE + "\n</tools>")
|
||||
|
||||
|
||||
def test_deflect_prompt_has_no_name_or_label_surface() -> None:
|
||||
"""D4 touches the suggested-tier block only: the deflected prompt is
|
||||
byte-identical (it carries no document blocks to annotate)."""
|
||||
prompt = build_deflect_prompt(["Kubernetes Homelab Cluster"])
|
||||
assert ' name="' not in prompt and ' label="' not in prompt
|
||||
assert GIT_SERVER not in prompt
|
||||
|
||||
@@ -1260,9 +1260,12 @@ def _bonus_rc(
|
||||
cosine: float,
|
||||
position: int = 0,
|
||||
name_hit: bool = False,
|
||||
source_name_hit: bool = False,
|
||||
) -> RetrievedChunk:
|
||||
"""One fused-list candidate (name-hit rows follow the D1 lexical
|
||||
convention: ``cosine=0.0``, ``fts_hit=True``)."""
|
||||
convention: ``cosine=0.0``, ``fts_hit=True``). The
|
||||
``source_name_hit`` flag (phase 191, D2) is set at the retrieval
|
||||
boundary in chat.py — here directly, on the fused row."""
|
||||
return RetrievedChunk(
|
||||
chunk_id=uuid.uuid4(),
|
||||
position=position,
|
||||
@@ -1272,6 +1275,7 @@ def _bonus_rc(
|
||||
cosine=cosine,
|
||||
fts_hit=cosine == 0.0,
|
||||
name_hit=name_hit,
|
||||
source_name_hit=source_name_hit,
|
||||
)
|
||||
|
||||
|
||||
@@ -1479,24 +1483,185 @@ def test_weak_hit_titles_bonus_adjusted_order() -> None:
|
||||
|
||||
|
||||
def test_bonus_lives_in_the_selection_layer_only() -> None:
|
||||
"""LOCKED A3: the bonus never touches the chunk objects —
|
||||
"""LOCKED A3 (mirrored by phase 191 for the SECOND bonus): the
|
||||
bonuses never touch the chunk objects —
|
||||
``score``/``cosine``/``fts_hit`` (the A8 gate's inputs —
|
||||
``query_log.top_score`` is the best fused chunk score, the same
|
||||
values) are unchanged after every selection walk, even with the
|
||||
bonus lifting a document."""
|
||||
values) and the ``name_hit``/``source_name_hit`` flags themselves
|
||||
are unchanged after every selection walk, even with the bonuses
|
||||
lifting a document."""
|
||||
chunks = _bonus_chunks()
|
||||
chunks[-1].source_name_hit = True
|
||||
before = {
|
||||
rc.chunk_id: (rc.score, rc.cosine, rc.fts_hit, rc.name_hit) for rc in chunks
|
||||
rc.chunk_id: (
|
||||
rc.score,
|
||||
rc.cosine,
|
||||
rc.fts_hit,
|
||||
rc.name_hit,
|
||||
rc.source_name_hit,
|
||||
)
|
||||
for rc in chunks
|
||||
}
|
||||
select_suggested(chunks, n=5, bonus=0.005)
|
||||
select_related(chunks, set(), cap=5, bonus=0.005)
|
||||
weak_hit_titles(chunks, bonus=0.005)
|
||||
select_suggested(chunks, n=5, bonus=0.005, source_bonus=0.005)
|
||||
select_related(chunks, set(), cap=5, bonus=0.005, source_bonus=0.005)
|
||||
weak_hit_titles(chunks, bonus=0.005, source_bonus=0.005)
|
||||
after = {
|
||||
rc.chunk_id: (rc.score, rc.cosine, rc.fts_hit, rc.name_hit) for rc in chunks
|
||||
rc.chunk_id: (
|
||||
rc.score,
|
||||
rc.cosine,
|
||||
rc.fts_hit,
|
||||
rc.name_hit,
|
||||
rc.source_name_hit,
|
||||
)
|
||||
for rc in chunks
|
||||
}
|
||||
assert before == after
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 191, D3 — the bounded SOURCE-name bonus (the second, independent
|
||||
# knob; the phase-119 mirror: kill switch, bounded per-document apply,
|
||||
# the fuse OR-merge, the settings defaults)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_selection_order_source_bonus_zero_is_byte_identical_to_pre_phase() -> None:
|
||||
"""The kill switch (LOCKED A3, phase-119 mirror): with
|
||||
*source_bonus* ``0`` the walk is byte-identical to the
|
||||
pre-phase-191 walk even when EVERY chunk carries the flag — same
|
||||
order, same effective scores (the phase-119 name bonus alone). With
|
||||
BOTH knobs ``0`` the pre-phase golden walk returns byte-identical."""
|
||||
chunks = _bonus_chunks()
|
||||
for rc in chunks:
|
||||
rc.source_name_hit = True
|
||||
out = _selection_order(chunks, 0.005, 0.0)
|
||||
assert [d.path for d, _eff, _cos, _idx in out] == [
|
||||
"gitea/README.md", "a.md", "b.md", "c.md", "d.md"
|
||||
]
|
||||
# No +0.005 rides the effective scores — the name bonus alone.
|
||||
assert out[0][1] == pytest.approx(0.016 + 0.005)
|
||||
off = _selection_order(chunks, 0.0, 0.0)
|
||||
assert [d.path for d, *_ in off] == GOLDEN_PRE_PHASE_ORDER
|
||||
assert [eff for _d, eff, _cos, _idx in off] == [
|
||||
0.0200, 0.0160, 0.0150, 0.0150, 0.0100
|
||||
]
|
||||
|
||||
|
||||
def test_selection_order_source_bonus_inert_without_source_hits() -> None:
|
||||
"""No ``source_name_hit`` chunk present → the second bonus cannot
|
||||
fire: with both knobs on the walk equals the phase-119 name-bonus
|
||||
walk exactly (the flagged-signal presence, not just the knob, gates
|
||||
the re-sort)."""
|
||||
out = _selection_order(_bonus_chunks(), 0.005, 0.005)
|
||||
assert [d.path for d, _eff, _cos, _idx in out] == [
|
||||
"gitea/README.md", "a.md", "b.md", "c.md", "d.md"
|
||||
]
|
||||
assert out[0][1] == pytest.approx(0.016 + 0.005)
|
||||
|
||||
|
||||
def test_selection_order_source_bonus_lifts_named_doc_over_equal_fused() -> None:
|
||||
"""The boost (D3): a document whose source is a D2 hit
|
||||
(``source_name_hit=True``) ranks ABOVE an equal-fused-score
|
||||
non-hit — the fused tie normally resolves on cosine (a's 0.9 beats
|
||||
the hit's 0.0), the 0.005 head start decides instead."""
|
||||
gitserver = _doc("gitserver.md", "G" * 50)
|
||||
a = _doc("a.md", "A" * 50)
|
||||
chunks = [
|
||||
_bonus_rc(a, 0.0200, 0.9),
|
||||
_bonus_rc(gitserver, 0.0200, 0.0, source_name_hit=True),
|
||||
]
|
||||
assert [d.path for d, *_ in _selection_order(chunks, 0.0, 0.0)] == [
|
||||
"a.md", "gitserver.md"
|
||||
]
|
||||
out = _selection_order(chunks, 0.0, 0.005)
|
||||
assert [d.path for d, *_ in out] == ["gitserver.md", "a.md"]
|
||||
assert out[0][1] == pytest.approx(0.0200 + 0.005)
|
||||
|
||||
|
||||
def test_selection_order_two_bonuses_bounded_at_two_per_document() -> None:
|
||||
"""A document carrying BOTH flags gets EXACTLY +0.010 (the bounded
|
||||
two-bonus pin — no amplification): every one of the doc's THREE
|
||||
chunks is flagged for both signals, yet each bonus applies ONCE per
|
||||
document — best score + bonus + source_bonus, never more."""
|
||||
gitserver = _doc("gitserver.md", "G" * 50)
|
||||
a = _doc("a.md", "A" * 50)
|
||||
chunks = [
|
||||
_bonus_rc(a, 0.0300, 0.9),
|
||||
_bonus_rc(gitserver, 0.0260, 0.0, 0, name_hit=True, source_name_hit=True),
|
||||
_bonus_rc(gitserver, 0.0200, 0.0, 1, name_hit=True, source_name_hit=True),
|
||||
_bonus_rc(gitserver, 0.0100, 0.0, 2, name_hit=True, source_name_hit=True),
|
||||
]
|
||||
out = _selection_order(chunks, 0.005, 0.005)
|
||||
assert [d.path for d, *_ in out] == ["gitserver.md", "a.md"]
|
||||
assert out[0][1] == pytest.approx(0.0260 + 0.010)
|
||||
|
||||
|
||||
def test_selection_helpers_pull_the_source_bonus_from_settings(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""All three selection helpers default *source_bonus* to the LIVE
|
||||
``BOR_SOURCE_NAME_HIT_BONUS`` setting (the phase-119 ``bonus``
|
||||
pattern): the settings default 0.005 lifts the flagged doc, a
|
||||
settings kill switch (``0``) and an explicit ``source_bonus=0``
|
||||
both reproduce the pre-phase-191 walk."""
|
||||
chunks = _bonus_chunks()
|
||||
chunks[-1].score = 0.0199 # d.md just UNDER a.md's 0.0200
|
||||
chunks[-1].source_name_hit = True
|
||||
on = Settings(_env_file=None) # pyright: ignore[reportCallIssue]
|
||||
assert on.source_name_hit_bonus == 0.005 # the production default
|
||||
monkeypatch.setattr(retriever, "get_settings", lambda: on)
|
||||
lifted = ["d.md", "gitea/README.md", "a.md", "b.md", "c.md"]
|
||||
assert [d.path for d in select_suggested(chunks, n=5)] == lifted
|
||||
assert [d.path for d in select_related(chunks, set(), cap=5)] == lifted
|
||||
assert weak_hit_titles(chunks) == lifted
|
||||
# Explicit kill switch — the pre-phase-191 (name-bonus-only) walk.
|
||||
assert [d.path for d in select_suggested(chunks, n=5, source_bonus=0.0)] == [
|
||||
"gitea/README.md", "a.md", "d.md", "b.md", "c.md"
|
||||
]
|
||||
# Settings kill switch (BOR_SOURCE_NAME_HIT_BONUS=0) — the same walk.
|
||||
off = Settings(_env_file=None, source_name_hit_bonus=0.0) # pyright: ignore[reportCallIssue]
|
||||
monkeypatch.setattr(retriever, "get_settings", lambda: off)
|
||||
pre = ["gitea/README.md", "a.md", "d.md", "b.md", "c.md"]
|
||||
assert [d.path for d in select_suggested(chunks, n=5)] == pre
|
||||
assert [d.path for d in select_related(chunks, set(), cap=5)] == pre
|
||||
assert weak_hit_titles(chunks) == pre
|
||||
|
||||
|
||||
def test_fuse_or_s_source_name_hit_on_double_hit() -> None:
|
||||
"""A vector row merged with a flagged source-name row keeps
|
||||
``source_name_hit=True`` — the merge ORs the flag in (the
|
||||
``name_hit`` mirror), so the selection tier still sees the hit on
|
||||
the fused list."""
|
||||
v = _rc("gitserver.md", cosine=0.9)
|
||||
l1 = _rc("gitserver.md", cosine=0.1) # the lexical copy of the same chunk
|
||||
l1.chunk_id = v.chunk_id
|
||||
l1.source_name_hit = True
|
||||
out = fuse([v], [l1], k=60)
|
||||
assert len(out) == 1
|
||||
assert out[0].source_name_hit is True
|
||||
assert out[0].fts_hit is True
|
||||
assert out[0].score == pytest.approx(2 / 61)
|
||||
|
||||
|
||||
def test_fuse_keeps_source_name_hit_on_lexical_only_hit() -> None:
|
||||
"""A source-name-hit row with no vector rank keeps
|
||||
``source_name_hit=True`` through the fusion (the ``replace()``
|
||||
copy carries the field — the ``name_hit`` mirror)."""
|
||||
row = _rc("gitserver.md")
|
||||
row.source_name_hit = True
|
||||
out = fuse([], [row], k=60)
|
||||
assert len(out) == 1
|
||||
assert out[0].source_name_hit is True
|
||||
|
||||
|
||||
def test_fuse_default_source_name_hit_stays_false_for_ordinary_rows() -> None:
|
||||
"""Neither list flagged ⇒ fusion never invents a source-name hit —
|
||||
ordinary vector and FTS rows are ``source_name_hit=False``."""
|
||||
out = fuse([_rc("a.md", cosine=0.8)], [_rc("b.md", fts_hit=True)], k=60)
|
||||
assert len(out) == 2
|
||||
assert all(rc.source_name_hit is False for rc in out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 162, D1 — the accent-folded name normalization (LOCKED A1): the
|
||||
# 2026-10-07 live gap, the ASCII filename vs the accented content layer
|
||||
|
||||
@@ -253,6 +253,29 @@ def test_sync_session_bound_field_types() -> None:
|
||||
assert Settings.model_fields["sync_prune_batch"].default == 500
|
||||
|
||||
|
||||
def test_source_name_hit_bonus_row_is_a_plain_live_overridable_row() -> None:
|
||||
"""Phase 191 (task 04, D3): the second selection-bonus row sits in
|
||||
``Retrieval & honesty`` (right beside ``name_hit_bonus`` — the two
|
||||
independently kill-able knobs) with the ``Source name hit bonus``
|
||||
label and mirrors the ``name_hit_bonus`` entry's flags exactly:
|
||||
plain — not ``secret`` (a float), not ``restart_required`` (the
|
||||
selection walk reads ``get_settings()`` per turn, so a saved
|
||||
override applies on the next question), overridable (the overlay is
|
||||
generic over ``float`` fields). ``0`` is the byte-identical kill
|
||||
switch; a negative value fails startup, not the panel."""
|
||||
meta = CATALOG["source_name_hit_bonus"]
|
||||
assert meta.group == "Retrieval & honesty"
|
||||
assert meta.label == "Source name hit bonus"
|
||||
assert meta.secret is False
|
||||
assert meta.restart_required is False
|
||||
assert meta.overridable is True
|
||||
assert "source_name_hit_bonus" not in SECRETS
|
||||
assert "source_name_hit_bonus" not in RESTART_REQUIRED
|
||||
assert "source_name_hit_bonus" not in NON_OVERRIDABLE
|
||||
assert Settings.model_fields["source_name_hit_bonus"].annotation is float
|
||||
assert Settings.model_fields["source_name_hit_bonus"].default == 0.005
|
||||
|
||||
|
||||
def test_labels_nonempty_and_unique() -> None:
|
||||
labels = [meta.label for meta in CATALOG.values()]
|
||||
assert all(label.strip() for label in labels), "empty label in the catalogue"
|
||||
|
||||
@@ -0,0 +1,370 @@
|
||||
"""Unit: source name/label/tags helpers + the D2 matching rule (phase 191, task 01).
|
||||
|
||||
The pure half (``normalize_tags``, ``question_name_hits``) needs no DB;
|
||||
the session helpers run against the local compose Postgres
|
||||
(``podman compose up -d db``) — the house DB-test pattern of
|
||||
``test_sources_meta.py``: the helpers are thin session wrappers whose
|
||||
contract (upsert, partial-update shape, flush-not-commit, absent-row
|
||||
``None``/``False``) only holds against a real database. Skips with clear
|
||||
instructions when the stack is not up.
|
||||
|
||||
The D2 rule is the heart of the phase — every branch is pinned: the
|
||||
name/label/tag hit classes, the case-insensitive ``_normalize_name``
|
||||
form (the phase-162 accent fold included), the 4-char minimum on BOTH
|
||||
sides, the NO digit-class rule ("gitea" hits digitless), ``None`` →
|
||||
``False``, and the no-overlap baseline.
|
||||
|
||||
A fixture truncates ``source_labels`` before and after every test so
|
||||
the suite leaves the dev DB exactly as migration 0030 left it (empty).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.db import SessionLocal
|
||||
from app.rag.source_labels import (
|
||||
TAGS_MAX_COUNT,
|
||||
SourceLabelView,
|
||||
all_labels,
|
||||
delete_label,
|
||||
get_label,
|
||||
label_for,
|
||||
normalize_tags,
|
||||
question_name_hit_sources,
|
||||
question_name_hits,
|
||||
set_label,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_source_labels(db: Session):
|
||||
"""Empty ``source_labels`` before and after every test (the
|
||||
migration-0030 state)."""
|
||||
db.execute(text("DELETE FROM source_labels"))
|
||||
db.commit()
|
||||
yield
|
||||
db.execute(text("DELETE FROM source_labels"))
|
||||
db.commit()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The model round-trip + the upsert / partial-update shape (D1)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_set_label_creates_row_with_stamp(db: Session) -> None:
|
||||
"""All three fields set: the row is created verbatim and
|
||||
``updated_at`` carries the server stamp (the house style)."""
|
||||
row = set_label(
|
||||
db,
|
||||
"gitea-forge",
|
||||
display_name="the git server",
|
||||
label="self-hosted forge + Gitea",
|
||||
tags=["gitea", "forge"],
|
||||
)
|
||||
db.commit()
|
||||
assert row.source == "gitea-forge"
|
||||
stored = get_label(db, "gitea-forge")
|
||||
assert stored is not None
|
||||
assert stored.display_name == "the git server"
|
||||
assert stored.label == "self-hosted forge + Gitea"
|
||||
assert stored.tags == ["gitea", "forge"] # stored exactly as normalized
|
||||
assert stored.updated_at is not None, "updated_at must be server-stamped"
|
||||
|
||||
|
||||
def test_set_label_defaults_omitted_fields(db: Session) -> None:
|
||||
"""A brand-new row created with only ``display_name`` carries the
|
||||
D1 defaults: ``label`` NULL and ``tags`` ``[]`` (NOT NULL column)."""
|
||||
set_label(db, "uploads", display_name="the uploads")
|
||||
db.commit()
|
||||
stored = get_label(db, "uploads")
|
||||
assert stored is not None
|
||||
assert stored.display_name == "the uploads"
|
||||
assert stored.label is None
|
||||
assert stored.tags == []
|
||||
|
||||
|
||||
def test_partial_update_omitted_field_untouched(db: Session) -> None:
|
||||
"""The partial-update shape: an OMITTED field leaves the stored
|
||||
value untouched (the ``_UNSET`` sentinel owns the difference)."""
|
||||
set_label(
|
||||
db, "homelab", display_name="the lab", label="first", tags=["homelab"]
|
||||
)
|
||||
db.commit()
|
||||
set_label(db, "homelab", label="second")
|
||||
db.commit()
|
||||
stored = get_label(db, "homelab")
|
||||
assert stored is not None
|
||||
assert stored.label == "second", "the passed field overwrites"
|
||||
assert stored.display_name == "the lab", "the omitted field is untouched"
|
||||
assert stored.tags == ["homelab"], "the omitted tags are untouched"
|
||||
|
||||
|
||||
def test_partial_update_explicit_none_clears(db: Session) -> None:
|
||||
"""An explicit ``None`` CLEARS the field (the D5 shape); ``tags``
|
||||
clears to ``[]`` — the column stays NOT NULL."""
|
||||
set_label(
|
||||
db, "homelab", display_name="the lab", label="first", tags=["homelab"]
|
||||
)
|
||||
db.commit()
|
||||
set_label(db, "homelab", display_name=None, label=None, tags=None)
|
||||
db.commit()
|
||||
stored = get_label(db, "homelab")
|
||||
assert stored is not None
|
||||
assert stored.display_name is None
|
||||
assert stored.label is None
|
||||
assert stored.tags == []
|
||||
|
||||
|
||||
def test_set_label_flushes_without_committing(db: Session) -> None:
|
||||
"""Flush-only (the phase-53 session convention): a rolled-back
|
||||
session rolls the upsert back with it."""
|
||||
set_label(db, "homelab", display_name="the lab")
|
||||
db.rollback()
|
||||
other = SessionLocal()
|
||||
try:
|
||||
assert get_label(other, "homelab") is None, "the uncommitted row must roll back"
|
||||
finally:
|
||||
other.close()
|
||||
|
||||
|
||||
def test_source_is_the_sole_pk(db: Session) -> None:
|
||||
"""One label row per source: the raw duplicate insert raises."""
|
||||
db.execute(text("INSERT INTO source_labels (source) VALUES ('dup')"))
|
||||
db.commit()
|
||||
with pytest.raises(IntegrityError):
|
||||
db.execute(text("INSERT INTO source_labels (source) VALUES ('dup')"))
|
||||
db.rollback()
|
||||
|
||||
|
||||
def test_tags_server_default_is_empty_list(db: Session) -> None:
|
||||
"""A row inserted without ``tags`` reads ``[]`` (the phase-89
|
||||
``ignore_paths`` server-default shape — no NULL ever surfaces)."""
|
||||
db.execute(text("INSERT INTO source_labels (source) VALUES ('nodefault')"))
|
||||
db.commit()
|
||||
stored = get_label(db, "nodefault")
|
||||
assert stored is not None
|
||||
assert stored.tags == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The read side: label_for / all_labels / delete_label
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_label_for_returns_frozen_view(db: Session) -> None:
|
||||
"""``label_for`` maps the row to the frozen view; ``tags`` arrives
|
||||
as a tuple."""
|
||||
set_label(db, "homelab", display_name="the lab", tags=["a", "b"])
|
||||
db.commit()
|
||||
view = label_for(db, "homelab")
|
||||
assert view is not None
|
||||
assert view == SourceLabelView(
|
||||
source="homelab", display_name="the lab", label=None, tags=("a", "b")
|
||||
)
|
||||
assert isinstance(view.tags, tuple)
|
||||
|
||||
|
||||
def test_label_for_absent_returns_none(db: Session) -> None:
|
||||
"""No row → ``None`` — the "no label row = pre-phase" contract."""
|
||||
assert label_for(db, "never-named") is None
|
||||
|
||||
|
||||
def test_all_labels_keyed_by_source(db: Session) -> None:
|
||||
"""One query, every row, keyed by ``source`` (the views the boost
|
||||
and the API iterate)."""
|
||||
set_label(db, "alpha", display_name="the alpha")
|
||||
set_label(db, "beta", label="the beta row")
|
||||
db.commit()
|
||||
labels = all_labels(db)
|
||||
assert set(labels) == {"alpha", "beta"}
|
||||
assert labels["alpha"].display_name == "the alpha"
|
||||
assert labels["beta"].label == "the beta row"
|
||||
assert labels["alpha"].tags == ()
|
||||
|
||||
|
||||
def test_all_labels_empty_table(db: Session) -> None:
|
||||
"""The pre-phase state: an empty table reads as an empty mapping."""
|
||||
assert all_labels(db) == {}
|
||||
|
||||
|
||||
def test_delete_label_removes_row(db: Session) -> None:
|
||||
"""The reset path: the row is gone, the helper reports ``True``."""
|
||||
set_label(db, "homelab", display_name="the lab")
|
||||
db.commit()
|
||||
assert delete_label(db, "homelab") is True
|
||||
db.commit()
|
||||
assert get_label(db, "homelab") is None
|
||||
|
||||
|
||||
def test_delete_label_absent_returns_false(db: Session) -> None:
|
||||
"""An absent row is ``False`` — the API maps that to 404 (D5)."""
|
||||
assert delete_label(db, "never-named") is False
|
||||
|
||||
|
||||
def test_delete_label_flushes_without_committing(db: Session) -> None:
|
||||
"""Flush-only both ways: a rolled-back delete keeps the row."""
|
||||
set_label(db, "homelab", display_name="the lab")
|
||||
db.commit()
|
||||
assert delete_label(db, "homelab") is True
|
||||
db.rollback()
|
||||
other = SessionLocal()
|
||||
try:
|
||||
assert get_label(other, "homelab") is not None, "the rollback must restore"
|
||||
finally:
|
||||
other.close()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# normalize_tags (the API's 422 source, D5)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_normalize_tags_trims_and_drops_empties() -> None:
|
||||
"""Entries are trimmed; entries that are empty afterwards dropped;
|
||||
``[]`` is legal (it clears)."""
|
||||
assert normalize_tags([" gitea ", "", " ", "forge"]) == ["gitea", "forge"]
|
||||
assert normalize_tags([]) == []
|
||||
assert normalize_tags(["", " "]) == []
|
||||
|
||||
|
||||
def test_normalize_tags_dedupes_case_insensitively_first_case_wins() -> None:
|
||||
"""Case-insensitive de-dupe preserving the FIRST case seen, order
|
||||
otherwise kept."""
|
||||
assert normalize_tags(["Gitea", "gitea", "GITEA", "Forge"]) == ["Gitea", "Forge"]
|
||||
|
||||
|
||||
def test_normalize_tags_rejects_too_many_tags() -> None:
|
||||
"""> 10 entries raise (counted on the trimmed non-empty list)."""
|
||||
with pytest.raises(ValueError):
|
||||
normalize_tags([f"tag{i}" for i in range(TAGS_MAX_COUNT + 1)])
|
||||
# exactly 10 is legal
|
||||
assert len(normalize_tags([f"tag{i}" for i in range(TAGS_MAX_COUNT)])) == 10
|
||||
|
||||
|
||||
def test_normalize_tags_rejects_overlong_tag() -> None:
|
||||
"""A > 32-char entry raises (the cap is checked after the trim)."""
|
||||
with pytest.raises(ValueError):
|
||||
normalize_tags(["gitea", "x" * 33])
|
||||
assert normalize_tags(["x" * 32]) == ["x" * 32] # exactly 32 is legal
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# question_name_hits — the D2 rule (every branch pinned)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_VIEW = SourceLabelView(
|
||||
source="gitea-forge",
|
||||
display_name="the git server",
|
||||
label="self-hosted forge + Gitea",
|
||||
tags=("gitea", "forgejo"),
|
||||
)
|
||||
|
||||
|
||||
def test_hit_on_display_name_token() -> None:
|
||||
"""A question token equal to a display-name token hits — and the
|
||||
phase-119 digit-class rule does NOT apply ("server" is digitless)."""
|
||||
assert question_name_hits(_VIEW, "restart the SERVER properly") is True
|
||||
|
||||
|
||||
def test_hit_on_label_token() -> None:
|
||||
"""A token of the ``label`` field hits (``+Gitea`` normalizes to
|
||||
``gitea``)."""
|
||||
assert question_name_hits(_VIEW, "upgrade gitea today") is True
|
||||
|
||||
|
||||
def test_hit_on_tag_entry() -> None:
|
||||
"""A tag entry hits even when neither name nor label mentions it."""
|
||||
view = SourceLabelView(
|
||||
source="s", display_name=None, label=None, tags=("forgejo",)
|
||||
)
|
||||
assert question_name_hits(view, "mirror forgejo repos") is True
|
||||
|
||||
|
||||
def test_hit_is_accent_folded_both_sides() -> None:
|
||||
"""The phase-162 fold rides ``_normalize_name``: the accented source
|
||||
name answers to the ASCII question token (and vice versa)."""
|
||||
view = SourceLabelView(
|
||||
source="besta", display_name="BESTÅ", label=None, tags=()
|
||||
)
|
||||
assert question_name_hits(view, "how tall is the besta shelf") is True
|
||||
assert question_name_hits(view, "BESTÅ shelf height") is True
|
||||
|
||||
|
||||
def test_digitless_name_hits_without_digit_rule() -> None:
|
||||
"""The 2026-09-16 finding, name side: "gitea" must hit even though
|
||||
it carries no digit (the path digit rule is NOT reused here)."""
|
||||
view = SourceLabelView(
|
||||
source="gitea-forge", display_name="Gitea", label=None, tags=()
|
||||
)
|
||||
assert question_name_hits(view, "How do I deploy gitea?") is True
|
||||
|
||||
|
||||
def test_short_source_side_token_never_matches() -> None:
|
||||
"""The 4-char minimum applies to the SOURCE side: a 3-char tag can
|
||||
never match, even against an identical question token."""
|
||||
view = SourceLabelView(
|
||||
source="ops", display_name="gke", label=None, tags=("gke",)
|
||||
)
|
||||
assert question_name_hits(view, "set up gke today") is False
|
||||
|
||||
|
||||
def test_short_question_token_never_matches() -> None:
|
||||
"""The phase-119 ``NAME_TOKEN_MIN_LEN`` side: a 3-char question
|
||||
token is no candidate against a matching source token."""
|
||||
view = SourceLabelView(
|
||||
source="ops", display_name="Gke Cluster", label=None, tags=()
|
||||
)
|
||||
assert question_name_hits(view, "gke deploy") is False
|
||||
|
||||
|
||||
def test_none_view_never_hits() -> None:
|
||||
"""No label row → ``False`` — the byte-identical pre-phase contract."""
|
||||
assert question_name_hits(None, "restart the git server") is False
|
||||
|
||||
|
||||
def test_question_without_candidates_never_hits() -> None:
|
||||
"""A question with NO qualifying candidate (every token under the
|
||||
4-char floor) short-circuits to ``False`` without touching the
|
||||
source side."""
|
||||
assert question_name_hits(_VIEW, "ai ox up") is False
|
||||
|
||||
|
||||
def test_no_overlap_question_never_hits() -> None:
|
||||
"""The baseline: a question sharing no normalized token with any
|
||||
field is not a hit."""
|
||||
assert question_name_hits(_VIEW, "bake a birthday cake") is False
|
||||
|
||||
|
||||
def test_dotted_and_versioned_candidates_still_match() -> None:
|
||||
"""The reused phase-119 candidate list: a dotted token (kept whole,
|
||||
``llama.cpp`` → ``llamacpp``) matches the equal source token."""
|
||||
view = SourceLabelView(
|
||||
source="infer", display_name="llama.cpp", label=None, tags=()
|
||||
)
|
||||
assert question_name_hits(view, "tune llama.cpp flags") is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# question_name_hit_sources — the per-turn entry point (one read)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_hit_sources_matches_per_row(db: Session) -> None:
|
||||
"""The DB half: every labelled row is tested against the ONE
|
||||
``all_labels`` read; only the named source comes back."""
|
||||
set_label(db, "gitea-forge", display_name="the git server", tags=["gitea"])
|
||||
set_label(db, "media", display_name="the media box", tags=["jellyfin"])
|
||||
set_label(db, "plain") # a row with nothing set can never hit
|
||||
db.commit()
|
||||
assert question_name_hit_sources(db, "restart jellyfin tonight") == {"media"}
|
||||
assert question_name_hit_sources(db, "upgrade gitea") == {"gitea-forge"}
|
||||
assert question_name_hit_sources(db, "nothing relevant here") == set()
|
||||
|
||||
|
||||
def test_hit_sources_empty_table(db: Session) -> None:
|
||||
"""Pre-phase state: no rows → the empty set (the boost never fires)."""
|
||||
assert question_name_hit_sources(db, "restart the git server") == set()
|
||||
@@ -0,0 +1,746 @@
|
||||
"""Unit: the per-source name/label/tags editor on the Sources page
|
||||
(phase 191, task 03).
|
||||
|
||||
``TODO.md`` L4: "Sources need a name, a label, tag(s) so you can focus
|
||||
the model when prompting if you have a project name or a standard name
|
||||
you reference things by." Tasks 01/02 stored that (``source_labels``,
|
||||
migration 0030) and exposed it (``/api/source-labels``); this task is the
|
||||
surface the owner types into — a SECOND list on ``/git-sources.html``
|
||||
whose row list is the catalogue ``GET /api/source-labels`` (the union of
|
||||
every registry row's DERIVED name and every indexed document source, so
|
||||
git, local AND upload sources all get an editor without a second API):
|
||||
|
||||
* the markup — the ``#source-labels-section`` card inside the admin-gated
|
||||
``#git-sources-content`` (the page's gate is this list's gate: the
|
||||
anonymous path returns at the gate and never fetches it), its own
|
||||
``.table-wrap`` (the phase-156 bounded panel + sticky header — the
|
||||
class alone, no divergent card rule), its ``role="alert"`` load-error
|
||||
line, its empty state, and ONE shared
|
||||
``<template id="source-label-row-template">`` cloned per row: the
|
||||
read-only mono identity cell, the three fields (Name ``maxlength=120``,
|
||||
Label ``maxlength=200``, the Tags token input ``maxlength=32``), the
|
||||
``type="button"`` Save / Reset pair and the ``role="status"`` save-state
|
||||
line;
|
||||
* THE ALIAS CONTRACT (D1/A1) — the identity cell is a ``<code>`` and the
|
||||
template carries NO input there: a label row never renames
|
||||
``Document.source``, a locator, a citation or a saved chat;
|
||||
* the JS lifecycle — ``loadLabels`` → ``renderLabelRows`` → ``makeLabelRow``
|
||||
(``template.content`` + ``cloneNode``, then the per-clone id rewrite so
|
||||
every ``<label for>`` stays unique), the token input (Enter / comma adds
|
||||
a chip, Backspace drops the last, the ``normalize_tags`` caps mirrored),
|
||||
the dirty paint (``.is-dirty`` + "unsaved changes", per ROW), and Save /
|
||||
Reset on the row's OWN ``saveInFlight`` flag — the ``PUT`` / ``DELETE``
|
||||
response is the new truth (NO second GET), the 404 arm names itself
|
||||
("That source no longer exists") and the ``finally`` settles every
|
||||
outcome so "Saving…" can never stick (§7.4);
|
||||
* styles.css — the chips and the state tag JOIN the phase-89 tag rule
|
||||
(``.git-source-hidden-count`` / ``.git-source-branch-count`` /
|
||||
``.source-label-state`` / ``.source-label-tag`` are ONE rule — no new
|
||||
tag-input CSS), the fields take the ``.ignore-editor-token`` box
|
||||
language, the buttons keep the ≥44px floor, the empty state is shared
|
||||
with ``.git-sources-empty``, and the new block stays on palette
|
||||
variables (no raw hex, no new hue).
|
||||
|
||||
The browser behaviour is E2E-gated by the phase-191 story suite
|
||||
(``tests/e2e/test_source_name_labels.py``, task 06); like
|
||||
``test_source_ignore_paths.py`` / ``test_git_source_branch_ui.py`` (the
|
||||
house pattern for this page), this module pins the source-level contract a
|
||||
silent regression would break.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
|
||||
SHELL_HTML = FRONTEND / "index.html"
|
||||
JS = FRONTEND / "assets" / "git-sources.js"
|
||||
CSS = FRONTEND / "assets" / "styles.css"
|
||||
|
||||
#: The static ids the section ships (the no-collision guard keys on these
|
||||
#: — the ids inside the template are the FIRST row's and makeLabelRow
|
||||
#: rewrites the clones').
|
||||
SECTION_IDS = (
|
||||
"source-labels-section",
|
||||
"source-labels-title",
|
||||
"source-labels-error",
|
||||
"source-labels-wrap",
|
||||
"source-labels-table",
|
||||
"source-labels-tbody",
|
||||
"source-labels-empty",
|
||||
"source-label-row-template",
|
||||
)
|
||||
|
||||
#: The template's field ids (rewritten per clone to ``<base>-<n>``).
|
||||
TEMPLATE_FIELD_IDS = (
|
||||
"source-label-name",
|
||||
"source-label-label",
|
||||
"source-label-tags",
|
||||
"source-label-editor",
|
||||
)
|
||||
|
||||
#: The save-state line's copy, pinned verbatim (§7.4 + the 404 arm).
|
||||
STATUS_SAVED = "saved"
|
||||
STATUS_DIRTY = "unsaved changes"
|
||||
STATUS_SAVING = "Saving…"
|
||||
STATUS_RESETTING = "Resetting…"
|
||||
NOTE_GONE = "That source no longer exists — reload the page."
|
||||
|
||||
#: The caps of ``app/rag/source_labels`` mirrored client-side.
|
||||
TAG_MAX_CHARS = 32
|
||||
TAGS_MAX_COUNT = 10
|
||||
|
||||
#: The functions this module inspects (all live inside ``mount(root)``).
|
||||
LABEL_FNS = (
|
||||
"storedFromEntry",
|
||||
"hasStoredLabel",
|
||||
"labelSummaryText",
|
||||
"paintLabelTags",
|
||||
"addLabelTags",
|
||||
"readLabelForm",
|
||||
"fillLabelForm",
|
||||
"labelFormDirty",
|
||||
"clearLabelError",
|
||||
"setLabelStatus",
|
||||
"paintLabelRowState",
|
||||
"updateLabelRowUI",
|
||||
"setLabelEditorOpen",
|
||||
"labelFailureNote",
|
||||
"saveLabelRow",
|
||||
"resetLabelRow",
|
||||
"makeLabelRow",
|
||||
"loadLabels",
|
||||
"renderLabelRows",
|
||||
"showLabelsError",
|
||||
"hideLabelsError",
|
||||
)
|
||||
|
||||
|
||||
def _read(path: Path) -> str:
|
||||
return path.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def _html() -> str:
|
||||
return _read(SHELL_HTML)
|
||||
|
||||
|
||||
def _js() -> str:
|
||||
return _read(JS)
|
||||
|
||||
|
||||
def _css() -> str:
|
||||
return _read(CSS)
|
||||
|
||||
|
||||
def _fn(js: str, name: str) -> str:
|
||||
"""The source of a (possibly async) function via balanced-brace
|
||||
counting (the ``test_source_ignore_paths.py`` helper)."""
|
||||
for prefix in ("async function ", "function "):
|
||||
start = js.find(f"{prefix}{name}(")
|
||||
if start != -1:
|
||||
depth = 0
|
||||
for i in range(js.find("{", start), len(js)):
|
||||
if js[i] == "{":
|
||||
depth += 1
|
||||
elif js[i] == "}":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return js[start : i + 1]
|
||||
raise AssertionError(f"unbalanced braces in {name}()")
|
||||
raise AssertionError(f"{name}() must exist in git-sources.js")
|
||||
|
||||
|
||||
def _input_tag(html: str, class_attr: str) -> str:
|
||||
m = re.search(rf"<input[^>]*{class_attr}[^>]*>", html, re.S)
|
||||
assert m, f"missing <input class*=…{class_attr}> in the shell"
|
||||
return re.sub(r"\s+", " ", m.group(0))
|
||||
|
||||
|
||||
def _element_by_id(html: str, id_attr: str) -> str:
|
||||
m = re.search(rf'<[a-z]+ [^>]*id="{id_attr}"[^>]*>', html, re.S)
|
||||
assert m, f"missing the #{id_attr} element in the shell"
|
||||
return re.sub(r"\s+", " ", m.group(0))
|
||||
|
||||
|
||||
def _div_span(html: str, open_tag: str) -> str:
|
||||
"""The substring from an opening ``<div …>`` to its matching
|
||||
``</div>`` (balanced div-tag counting — the phase-174 helper)."""
|
||||
start = html.index(open_tag)
|
||||
depth = 0
|
||||
for m in re.finditer(r"<div\b|</div>", html[start:]):
|
||||
if m.group(0) == "</div>":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return html[start : start + m.end()]
|
||||
else:
|
||||
depth += 1
|
||||
raise AssertionError(f"unbalanced divs from {open_tag!r}")
|
||||
|
||||
|
||||
def _label_for(html: str, field_id: str) -> str:
|
||||
m = re.search(rf'<label[^>]*for="{field_id}"[^>]*>(.*?)</label>', html, re.S)
|
||||
assert m, f"missing <label for={field_id}> (WCAG — never aria-label-only)"
|
||||
return re.sub(r"\s+", " ", m.group(0))
|
||||
|
||||
|
||||
def _css_rule(css: str, selector: str) -> str:
|
||||
"""The BODY of one CSS rule (these rules carry no nested braces)."""
|
||||
m = re.search(re.escape(selector) + r"\s*\{([^}]*)\}", css)
|
||||
assert m is not None, f"rule for {selector!r} not found in styles.css"
|
||||
return " ".join(m.group(1).split())
|
||||
|
||||
|
||||
def _css_selector_list(css: str, needle: str) -> str:
|
||||
"""The SELECTOR LIST of the rule that styles ``needle`` (the last
|
||||
selector before the ``{``), so a shared rule is detectable."""
|
||||
i = css.find(needle)
|
||||
assert i != -1, f"{needle!r} is not in styles.css"
|
||||
open_brace = css.find("{", i)
|
||||
head = css.rfind("}", 0, i)
|
||||
start = head + 1 if head != -1 else 0
|
||||
return " ".join(css[start:open_brace].split())
|
||||
|
||||
|
||||
def _template_block(html: str) -> str:
|
||||
start = html.index('<template id="source-label-row-template">')
|
||||
end = html.index("</template>", start)
|
||||
return html[start:end]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The section scaffold (index.html)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_section_ships_inside_the_admin_gated_content() -> None:
|
||||
"""The card exists, is labelled by its own heading, and sits INSIDE
|
||||
``#git-sources-content`` — the reveal git-sources.js performs only in
|
||||
the ADMIN branch, which is what keeps the anonymous visitor from ever
|
||||
fetching ``/api/source-labels`` (the page's gate is this list's gate)."""
|
||||
html = _html()
|
||||
section = _element_by_id(html, "source-labels-section")
|
||||
assert 'class="source-labels"' in section
|
||||
assert 'aria-labelledby="source-labels-title"' in section
|
||||
content = _div_span(html, '<div id="git-sources-content" hidden>')
|
||||
assert 'id="source-labels-section"' in content, (
|
||||
"the section must live inside #git-sources-content (anonymous ships hidden)"
|
||||
)
|
||||
assert 'id="source-labels-title"' in content
|
||||
# The heading is real copy about the alias, not a placeholder.
|
||||
title = re.search(
|
||||
r'<h2 class="source-labels-title" id="source-labels-title">(.*?)</h2>', html, re.S
|
||||
)
|
||||
assert title and "Names, labels and tags" in title.group(1)
|
||||
|
||||
|
||||
def test_section_carries_its_own_error_line_and_empty_state() -> None:
|
||||
"""``#source-labels-error`` is the list's own ``role="alert"`` line
|
||||
(hidden until the GET fails) and ``#source-labels-empty`` the empty
|
||||
state (hidden by default) — the two states a failed or empty catalogue
|
||||
may reach, so the page is never stuck (the ``#git-sources-load-error``
|
||||
discipline applied to this page's second list)."""
|
||||
html = _html()
|
||||
err = _element_by_id(html, "source-labels-error")
|
||||
assert 'role="alert"' in err, "a failed catalogue is an alert, not silence"
|
||||
assert " hidden" in err, "the error line ships hidden"
|
||||
empty = _element_by_id(html, "source-labels-empty")
|
||||
assert " hidden" in empty, "the empty state ships hidden"
|
||||
assert "No sources to name yet." in html
|
||||
|
||||
|
||||
def test_table_is_a_bounded_sticky_panel_with_scoped_columns() -> None:
|
||||
"""The wrapper carries the shared ``.table-wrap`` class (the ONE
|
||||
bounded-panel + sticky-header card rule, phase 156 — an unbounded
|
||||
bespoke card would be the regression) and the table has two scoped
|
||||
column headers, the second one naming all three fields."""
|
||||
html = _html()
|
||||
wrap = _element_by_id(html, "source-labels-wrap")
|
||||
assert "table-wrap" in wrap, "the shared bounded/sticky card class"
|
||||
assert 'role="region"' in wrap and 'tabindex="0"' in wrap, (
|
||||
"a scroll region is keyboard-reachable and named (WCAG 2.1.1/1.3.1)"
|
||||
)
|
||||
assert 'id="source-labels-tbody"' in html
|
||||
table_i = html.index('id="source-labels-table"')
|
||||
tail = html[table_i : html.index("</table>", table_i)]
|
||||
assert '<th scope="col">Source</th>' in tail
|
||||
assert '<th scope="col">Name, label and tags</th>' in tail
|
||||
assert '<caption class="visually-hidden">' in tail
|
||||
|
||||
|
||||
def test_new_ids_appear_exactly_once() -> None:
|
||||
"""The no-collision guard (the phase-46/76 contract): every static id
|
||||
this task adds appears EXACTLY ONCE in the shell."""
|
||||
html = _html()
|
||||
for id_attr in SECTION_IDS + TEMPLATE_FIELD_IDS:
|
||||
n = len(re.findall(rf'id="{re.escape(id_attr)}"', html))
|
||||
assert n == 1, f'id="{id_attr}" appears {n} times in index.html (must be 1)'
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The shared row template + the alias contract
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_template_ships_one_row_with_the_three_capped_fields() -> None:
|
||||
"""ONE shared template, cloned per row: the Name (≤120), Label (≤200)
|
||||
and Tags (≤32) inputs — ``type="text"`` + ``autocomplete="off"`` +
|
||||
the ``maxlength`` mirroring the API caps (so an over-long submission
|
||||
never round-trips as a 422) — each with a real visible
|
||||
``<label for>`` (WCAG, never aria-label-only)."""
|
||||
tpl = _template_block(_html())
|
||||
assert "<tr" in tpl, "the template holds one row (a <tr> parses in a template)"
|
||||
for cls, cap, field_id in (
|
||||
("source-label-name-input", 120, "source-label-name"),
|
||||
("source-label-label-input", 200, "source-label-label"),
|
||||
("source-label-tags-input", 32, "source-label-tags"),
|
||||
):
|
||||
tag = _input_tag(tpl, cls)
|
||||
assert 'type="text"' in tag, f"{cls} is a plain text field"
|
||||
assert f'maxlength="{cap}"' in tag, f"{cls} mirrors the {cap}-char API cap"
|
||||
assert 'autocomplete="off"' in tag, f"{cls} offers no browser autofill"
|
||||
assert f'id="{field_id}"' in tag, f"{cls} carries the template id JS rewrites"
|
||||
assert _label_for(tpl, field_id), f"{cls} has a visible <label for>"
|
||||
|
||||
|
||||
def test_identity_cell_is_read_only_text_never_an_input() -> None:
|
||||
"""D1/A1 pinned in markup: the identity cell is a ``<code>`` — the
|
||||
derived source name — and the ONLY element the row's identity area
|
||||
contains; the editor is a sibling cell, so a label can never become a
|
||||
rename of ``Document.source``, a citation or a ``source/path`` locator."""
|
||||
tpl = _template_block(_html())
|
||||
m = re.search(
|
||||
r'<td class="source-label-id-cell">(.*?)</td>', tpl, re.S
|
||||
)
|
||||
assert m, "the identity cell exists"
|
||||
cell = m.group(1)
|
||||
assert '<code class="source-label-source">' in cell
|
||||
assert "<input" not in cell, "the identity cell has NO input (an alias, not a rename)"
|
||||
assert 'class="source-label-state"' in cell, "the stored-label state tag (text, 1.4.1)"
|
||||
id_i = tpl.index('class="source-label-id-cell"')
|
||||
edit_i = tpl.index('class="source-label-edit-cell"')
|
||||
assert id_i < edit_i, "the identity cell precedes the editor (it is not replaced)"
|
||||
|
||||
|
||||
def test_editor_ships_closed_with_the_a11y_pair_and_state_line() -> None:
|
||||
"""The expandable block: the toggle is a real ``type="button"`` with
|
||||
``aria-expanded="false"``, the editor div ships ``hidden`` (JS sets
|
||||
``aria-controls`` to the editor's per-clone id), Save ships
|
||||
``disabled`` (nothing is dirty on paint) and Reset ships ``hidden``
|
||||
(a row with no stored label has nothing to reset — the phase-168
|
||||
per-row reset shape). The save-state line is a live region."""
|
||||
tpl = _template_block(_html())
|
||||
toggle = re.search(r'<button[^>]*class="source-label-toggle"[^>]*>', tpl, re.S)
|
||||
assert toggle and 'type="button"' in toggle.group(0)
|
||||
assert 'aria-expanded="false"' in toggle.group(0), "collapsed by default"
|
||||
editor = re.search(r'<div class="source-label-editor"[^>]*>', tpl, re.S)
|
||||
assert editor and " hidden" in editor.group(0), "the editor ships closed"
|
||||
save = re.search(r'<button[^>]*class="source-label-save"[^>]*>', tpl, re.S)
|
||||
assert save and 'type="button"' in save.group(0) and "disabled" in save.group(0)
|
||||
reset = re.search(r'<button[^>]*class="source-label-reset"[^>]*>', tpl, re.S)
|
||||
assert reset and 'type="button"' in reset.group(0) and " hidden" in reset.group(0)
|
||||
status = re.search(r'<span class="source-label-status"[^>]*>', tpl, re.S)
|
||||
assert status and 'role="status"' in status.group(0)
|
||||
assert 'aria-live="polite"' in status.group(0), "the save state announces"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The load path
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_load_labels_fetches_the_catalogue_once_and_heals() -> None:
|
||||
"""``loadLabels`` is ONE ``GET /api/source-labels`` (no method key =
|
||||
GET), renders from ``data.labels``, and heals its own error line on
|
||||
the success path BEFORE rendering (an error followed by a good
|
||||
refresh clears it — the phase-77 rule for this page's lists). A
|
||||
network blip and a non-2xx both land on the role=alert line."""
|
||||
js = _js()
|
||||
body = _fn(js, "loadLabels")
|
||||
assert 'const LABELS_ENDPOINT = "/api/source-labels";' in js
|
||||
assert "fetch(LABELS_ENDPOINT)" in body
|
||||
assert 'method: "' not in body, "the catalogue read is a GET"
|
||||
assert body.count("fetch(") == 1, "exactly one fetch per load"
|
||||
assert "Array.isArray(data.labels)" in body, "renders from the labels array"
|
||||
heal_i = body.find("hideLabelsError();")
|
||||
render_i = body.find("renderLabelRows(")
|
||||
assert -1 < heal_i < render_i, "the success path heals the error before rendering"
|
||||
assert body.find("showLabelsError(") < body.find("hideLabelsError();")
|
||||
for fallback in (
|
||||
"Could not reach the server — the source names are not shown.",
|
||||
"The server could not list the source names",
|
||||
"The server sent an unreadable label list — try again.",
|
||||
):
|
||||
assert fallback in body, f"missing the failure line {fallback!r}"
|
||||
|
||||
|
||||
def test_render_replaces_rows_and_syncs_the_list_states() -> None:
|
||||
"""A re-render REPLACES the rows (never appends — a re-show must not
|
||||
duplicate), and the table card / empty state flip off the row count."""
|
||||
js = _js()
|
||||
body = _fn(js, "renderLabelRows")
|
||||
clear_i = body.find("labelsTbody.replaceChildren()")
|
||||
append_i = body.find("labelsTbody.appendChild(")
|
||||
assert -1 < clear_i < append_i, "the clear precedes the fill"
|
||||
assert 'labelsWrap.hidden = !hasRows' in body
|
||||
assert 'labelsEmptyEl.hidden = hasRows' in body
|
||||
|
||||
|
||||
def test_error_state_hides_the_table_and_the_empty_state() -> None:
|
||||
"""While the error is up it is the ONLY claim about the list (the
|
||||
``showLoadError`` discipline): both the table and the empty state go
|
||||
hidden, and ``hideLabelsError`` clears the text on the way out."""
|
||||
js = _js()
|
||||
show = _fn(js, "showLabelsError")
|
||||
assert "labelsErrorEl.textContent = message" in show
|
||||
assert "labelsErrorEl.hidden = false" in show
|
||||
assert "labelsWrap.hidden = true" in show
|
||||
assert "labelsEmptyEl.hidden = true" in show
|
||||
hide = _fn(js, "hideLabelsError")
|
||||
assert 'labelsErrorEl.textContent = ""' in hide
|
||||
assert "labelsErrorEl.hidden = true" in hide
|
||||
|
||||
|
||||
def test_labels_fetch_is_admin_only_and_refreshes_the_list_flows() -> None:
|
||||
"""Every ``loadLabels()`` call site sits AFTER the ``fetchIsAdmin``
|
||||
gate (anonymous never asks), the boot loads it alongside the sources,
|
||||
and each flow that CHANGES the source list (add / remove / a finished
|
||||
upload) refreshes it — so a new source is nameable without a reload."""
|
||||
js = _js()
|
||||
gate = js.find("const admin = await fetchIsAdmin();")
|
||||
assert gate != -1
|
||||
anon = js[gate : js.find("return;", gate)]
|
||||
assert "loadLabels" not in anon, "the anonymous branch never fetches labels"
|
||||
# The gate sits at the END of mount (the boot), so the pin is the
|
||||
# branch itself + the boot's own call after it: nothing before the
|
||||
# gate runs for an anonymous visitor, and the mount tail is where the
|
||||
# first catalogue read happens.
|
||||
boot = js[gate:]
|
||||
assert "await loadLabels();" in boot, "the admin boot loads the catalogue"
|
||||
assert boot.index("await loadLabels();") < boot.index("await loadSources();"), (
|
||||
"the two lists load side by side at boot (neither waits on the other)"
|
||||
)
|
||||
# add / remove / upload settle paths
|
||||
add = _fn(js, "wireAddForm")
|
||||
assert "await loadSources(); // the new row lands in the table" in add
|
||||
assert js.count("await loadLabels();") >= 3, "add + remove + boot refresh labels"
|
||||
assert "loadLabels(); // phase 191: the uploaded source joins the name list" in js
|
||||
# the re-show refresh keeps the phase-77 listener verbatim + a second
|
||||
# listener for the second list (so an unrelated table reload never
|
||||
# wipes a half-typed name draft)
|
||||
refresh_listeners = re.findall(
|
||||
r'addEventListener\("bor:view-refresh", \(\) => (\w+)\(\)\)', js
|
||||
)
|
||||
assert refresh_listeners == ["loadSources", "loadLabels"], refresh_listeners
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Save / Reset — the §7.4 contract, per row
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_save_puts_the_whole_form_and_uses_the_response_as_truth() -> None:
|
||||
"""Save: ``PUT /api/source-labels/<source>`` (``encodeURIComponent`` —
|
||||
a source name is a path segment), the three keys always ride (an empty
|
||||
Name is the deliberate CLEAR leg), and on 200 the ROW RE-READS ITS
|
||||
STORED LABEL FROM THE RESPONSE — no second GET anywhere in the save
|
||||
path (task 03: "the response is the new truth")."""
|
||||
js = _js()
|
||||
body = _fn(js, "saveLabelRow")
|
||||
fetch_i = body.find(
|
||||
"fetch(`${LABELS_ENDPOINT}/${encodeURIComponent(view.source)}`"
|
||||
)
|
||||
assert fetch_i != -1, "the PUT targets the per-source path"
|
||||
assert 'method: "PUT"' in body[fetch_i : fetch_i + 220]
|
||||
assert "body: JSON.stringify(form)" in body, "the whole form is sent"
|
||||
form_i = body.find("const form = readLabelForm(view);")
|
||||
assert -1 < form_i < fetch_i, "the form is read before the request"
|
||||
ok_i = body.find("if (r.ok)")
|
||||
stored_i = body.find("view.stored = storedFromEntry(", ok_i)
|
||||
json_i = body.find("await r.json()", ok_i)
|
||||
assert -1 < ok_i < json_i < stored_i, "the response body IS the new truth"
|
||||
assert "loadLabels()" not in body, "no second GET after a save"
|
||||
assert "loadSources()" not in body, "a save never reloads the sources table"
|
||||
assert "fillLabelForm(view)" in body[stored_i:], "the fields re-paint from the response"
|
||||
assert "setLabelStatus(view, LABEL_STATUS_SAVED, null)" in body
|
||||
|
||||
|
||||
def test_save_and_reset_guard_on_the_rows_own_in_flight_flag() -> None:
|
||||
"""The §7.4 guard scoped PER ROW (the phase-168 ``saveInFlight``
|
||||
pattern): the early return precedes everything, the flag is set BEFORE
|
||||
the fetch, and the ``finally`` drops it on every outcome — a second
|
||||
click while a request is out is ignored and no row is ever left
|
||||
claiming "Saving…" after a terminal frame."""
|
||||
js = _js()
|
||||
for name, label in (
|
||||
("saveLabelRow", "LABEL_STATUS_SAVING"),
|
||||
("resetLabelRow", "LABEL_STATUS_RESETTING"),
|
||||
):
|
||||
body = _fn(js, name)
|
||||
guard_i = body.find("if (view.saveInFlight) return;")
|
||||
set_i = body.find("view.saveInFlight = true;")
|
||||
status_i = body.find(f"setLabelStatus(view, {label}, null);")
|
||||
fetch_i = body.find("fetch(`")
|
||||
finally_i = body.find("} finally {")
|
||||
clear_i = body.find("view.saveInFlight = false;", finally_i)
|
||||
assert -1 < guard_i < set_i < status_i < fetch_i, (
|
||||
f"{name}: guard → flag → in-flight label → fetch"
|
||||
)
|
||||
assert finally_i != -1 and clear_i != -1, f"{name}: the finally must drop the flag"
|
||||
assert body.index("view.saveBtn.disabled = true") < fetch_i
|
||||
assert body.index("view.resetBtn.disabled = true") < fetch_i
|
||||
assert "updateLabelRowUI(view);" in body[clear_i:], (
|
||||
f"{name}: the settle re-decides Save from the dirty state"
|
||||
)
|
||||
|
||||
|
||||
def test_reset_deletes_and_clears_the_row() -> None:
|
||||
"""Reset: ``DELETE /api/source-labels/<source>`` → the stored label is
|
||||
nulled from the response and the fields clear with it, settling on the
|
||||
same "saved" line (the row now matches the server)."""
|
||||
body = _fn(_js(), "resetLabelRow")
|
||||
fetch_i = body.find(
|
||||
"fetch(`${LABELS_ENDPOINT}/${encodeURIComponent(view.source)}`"
|
||||
)
|
||||
assert fetch_i != -1
|
||||
assert 'method: "DELETE"' in body[fetch_i : fetch_i + 200]
|
||||
ok_i = body.find("if (r.ok)")
|
||||
stored_i = body.find("view.stored = storedFromEntry(cleared || {})", ok_i)
|
||||
assert stored_i != -1, "the stored label is nulled from the response"
|
||||
assert body.find("fillLabelForm(view)", stored_i) != -1, "the fields clear"
|
||||
assert "setLabelStatus(view, LABEL_STATUS_SAVED, null)" in body
|
||||
assert "loadLabels()" not in body, "no second GET after a reset"
|
||||
|
||||
|
||||
def test_the_404_note_names_the_arm() -> None:
|
||||
"""A 404 means the source left the catalogue — a retry cannot help —
|
||||
so the one-line note names it (task 03), while every other non-2xx
|
||||
carries the server's own detail (``apiDetail``, 422 shape-aware)."""
|
||||
js = _js()
|
||||
assert f'const LABEL_NOTE_GONE = "{NOTE_GONE}";' in js
|
||||
body = _fn(js, "labelFailureNote")
|
||||
gone_i = body.find("if (r.status === 404) return LABEL_NOTE_GONE;")
|
||||
detail_i = body.find("return await apiDetail(r, fallback);")
|
||||
assert -1 < gone_i < detail_i, "the 404 arm first, then the server detail"
|
||||
for name in ("saveLabelRow", "resetLabelRow"):
|
||||
assert "labelFailureNote(r," in _fn(js, name), f"{name} uses the shared note"
|
||||
|
||||
|
||||
def test_a_failed_saves_note_survives_the_dirty_paint() -> None:
|
||||
"""A failed save leaves the row DIRTY, and the request's ``finally``
|
||||
calls ``updateLabelRowUI`` — so the dirty hint used to overwrite the
|
||||
note the same request had just written, and the one error whose advice
|
||||
is a reload rather than a retype ("That source no longer exists —
|
||||
reload the page.") disappeared in the tick that produced it (the
|
||||
defect the phase-191 story suite walked into). Pinned: the error line
|
||||
returns from the paint untouched, Save keeps its dirty-driven
|
||||
``disabled`` decision (a retry is legitimate for every arm but the
|
||||
404), and the ONLY thing that clears a note is the owner editing
|
||||
again — the settle path never does."""
|
||||
js = _js()
|
||||
paint = _fn(js, "updateLabelRowUI")
|
||||
disable_i = paint.find("view.saveBtn.disabled = !dirty;")
|
||||
guard_i = paint.find('if (view.statusEl.classList.contains("is-error")) return;')
|
||||
hint_i = paint.find('setLabelStatus(view, LABEL_STATUS_DIRTY, "hint")')
|
||||
assert -1 < disable_i < guard_i < hint_i, (
|
||||
"the error guard sits after the Save paint, before the hint"
|
||||
)
|
||||
assert "setLabelStatus" not in paint[guard_i:hint_i], "the note is untouched"
|
||||
clear = _fn(js, "clearLabelError")
|
||||
assert 'if (view.statusEl.classList.contains("is-error"))' in clear
|
||||
assert 'setLabelStatus(view, "", null);' in clear
|
||||
row = _fn(js, "makeLabelRow")
|
||||
for opener in (
|
||||
'input.addEventListener("input", () => {',
|
||||
'tagsInput.addEventListener("keydown", (e) => {',
|
||||
):
|
||||
i = row.find(opener)
|
||||
assert i != -1, f"missing {opener}"
|
||||
assert "clearLabelError(view);" in row[i : i + 220], f"{opener} clears the stale note"
|
||||
# A collapse clears it the same way (it refills from the stored label).
|
||||
assert 'setLabelStatus(view, "", null);' in _fn(js, "setLabelEditorOpen")
|
||||
|
||||
|
||||
def test_status_line_copy_is_pinned_verbatim() -> None:
|
||||
"""The save-state line's whole vocabulary (§7.4 + the dirty hint) —
|
||||
pinned so a silent copy change cannot break the E2E and the
|
||||
screen-reader contract with it."""
|
||||
js = _js()
|
||||
for literal in (
|
||||
f'const LABEL_STATUS_SAVED = "{STATUS_SAVED}";',
|
||||
f'const LABEL_STATUS_DIRTY = "{STATUS_DIRTY}";',
|
||||
f'const LABEL_STATUS_SAVING = "{STATUS_SAVING}";',
|
||||
f'const LABEL_STATUS_RESETTING = "{STATUS_RESETTING}";',
|
||||
):
|
||||
assert literal in js, f"missing {literal!r}"
|
||||
dirty_paint = _fn(js, "updateLabelRowUI")
|
||||
assert 'setLabelStatus(view, LABEL_STATUS_DIRTY, "hint")' in dirty_paint
|
||||
assert 'view.tr.classList.toggle("is-dirty", dirty)' in dirty_paint
|
||||
assert "if (view.saveInFlight) return;" in dirty_paint, (
|
||||
"a keystroke mid-request never overwrites the in-flight label"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The token input
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_token_input_adds_on_enter_or_comma_and_drops_the_last_on_backspace() -> None:
|
||||
"""The token input's widget rules (task 03): Enter or a comma commits
|
||||
the token (``preventDefault`` — a comma never types, Enter never
|
||||
submits), Backspace on an empty input drops the LAST chip, and leaving
|
||||
the field commits a half-typed token."""
|
||||
js = _js()
|
||||
body = _fn(js, "makeLabelRow")
|
||||
enter_i = body.find('if (e.key === "Enter" || e.key === ",") {')
|
||||
assert enter_i != -1
|
||||
assert "e.preventDefault();" in body[enter_i : enter_i + 200]
|
||||
add_i = body.find("addLabelTags(view, tagsInput.value);", enter_i)
|
||||
back_i = body.find(
|
||||
'if (e.key === "Backspace" && tagsInput.value === "" && view.tags.length)'
|
||||
)
|
||||
assert -1 < enter_i < add_i < back_i, "Enter/comma precedes the Backspace rule"
|
||||
pop_i = body.find("view.tags.pop();", back_i)
|
||||
assert -1 < back_i < pop_i < body.find("paintLabelTags(view);", back_i)
|
||||
blur_i = body.find('tagsInput.addEventListener("blur"')
|
||||
assert blur_i > back_i, "blur commits a pending token"
|
||||
|
||||
|
||||
def test_tag_caps_are_the_servers_and_the_first_case_wins() -> None:
|
||||
"""The caps are ``normalize_tags``' numbers mirrored client-side
|
||||
(32 chars, 10 chips) and the de-dupe keeps the FIRST case — the chip
|
||||
list the owner sees is the list the API stores."""
|
||||
js = _js()
|
||||
assert f"const LABEL_TAG_MAX_CHARS = {TAG_MAX_CHARS};" in js
|
||||
assert f"const LABEL_TAGS_MAX_COUNT = {TAGS_MAX_COUNT};" in js
|
||||
body = _fn(js, "addLabelTags")
|
||||
assert ".slice(0, LABEL_TAG_MAX_CHARS)" in body
|
||||
assert "if (!tag) continue;" in body, "an empty token is dropped, never stored"
|
||||
assert "t.toLowerCase() === tag.toLowerCase()" in body, "case-insensitive de-dupe"
|
||||
assert "view.tags.length >= LABEL_TAGS_MAX_COUNT" in body, "the 10-chip cap"
|
||||
assert "LABEL_TAGS_MAX_COUNT" in _fn(js, "paintLabelTags")
|
||||
|
||||
|
||||
def test_chips_render_as_text_in_the_shared_tag_class() -> None:
|
||||
"""Each chip is an ``<li class="source-label-tag">`` written with
|
||||
``textContent`` (the XSS contract — owner text is never HTML) and the
|
||||
class is the one the shared phase-89 tag rule styles, so the widget
|
||||
adds NO tag-input CSS (task 03 step 1)."""
|
||||
js = _js()
|
||||
body = _fn(js, "paintLabelTags")
|
||||
assert 'li.className = "source-label-tag";' in body
|
||||
assert "li.textContent = tag;" in body
|
||||
assert "innerHTML" not in body
|
||||
selector = _css_selector_list(_css(), ".source-label-tag {")
|
||||
assert ".git-source-hidden-count" in selector, (
|
||||
"the chip joins the phase-89 tag rule (no new tag-input CSS)"
|
||||
)
|
||||
assert ".source-label-state" in selector, "the state tag shares it too"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# makeLabelRow — the clone
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_make_label_row_clones_the_template_and_rewrites_the_ids() -> None:
|
||||
"""One shared template, cloned per row: ``template.content`` +
|
||||
``cloneNode(true)``, then the per-clone id rewrite (input id + the
|
||||
matching ``<label for>``) and the ``aria-controls`` link — without it
|
||||
every clone would duplicate the template's ids and every label would
|
||||
aim at the first match (WCAG 1.3.1)."""
|
||||
js = _js()
|
||||
body = _fn(js, "makeLabelRow")
|
||||
assert "labelRowTemplate.content" in body
|
||||
assert "proto.cloneNode(true)" in body
|
||||
assert 'input.id = `${base}-${seq}`;' in body
|
||||
assert 'lbl.setAttribute("for", input.id);' in body
|
||||
assert 'editor.id = `source-label-editor-${seq}`;' in body
|
||||
assert 'toggleBtn.setAttribute("aria-controls", editor.id);' in body
|
||||
assert 'codeEl.textContent = source;' in body, "the identity is textContent"
|
||||
for fn in LABEL_FNS:
|
||||
assert "innerHTML" not in _fn(js, fn), f"{fn} must never touch innerHTML"
|
||||
|
||||
|
||||
def test_row_state_paint_names_the_source_on_every_control() -> None:
|
||||
"""The state paint: the "named" tag + summary appear only when a label
|
||||
is stored, Reset is visible only then (the phase-168 per-row reset),
|
||||
and every control's ``aria-label`` names the row's source (the
|
||||
phase-89/105 idiom — the clipped label stays reachable)."""
|
||||
js = _js()
|
||||
body = _fn(js, "paintLabelRowState")
|
||||
assert "view.stateTag.hidden = !labelled" in body
|
||||
assert "view.resetBtn.hidden = !labelled" in body
|
||||
assert 'labelled ? "Edit name" : "Name this source"' in body
|
||||
for label in (
|
||||
"${labelled ? \"Edit\" : \"Set\"} the name, label and tags for source: ${view.source}",
|
||||
"Save the name, label and tags for source: ${view.source}",
|
||||
"Reset the name, label and tags of source: ${view.source}",
|
||||
):
|
||||
assert label in body, f"missing the aria-label template {label!r}"
|
||||
|
||||
|
||||
def test_collapse_discards_the_draft_instead_of_staling_it() -> None:
|
||||
"""Closing the editor refills it from the STORED label and clears the
|
||||
status line (no request) — a closed row never claims an edit the
|
||||
server never got; a request in flight freezes the row."""
|
||||
js = _js()
|
||||
body = _fn(js, "setLabelEditorOpen")
|
||||
assert "if (view.saveInFlight) return;" in body
|
||||
assert 'view.toggleBtn.setAttribute("aria-expanded", open ? "true" : "false");' in body
|
||||
open_i = body.find("view.editor.hidden = !open;")
|
||||
fill_i = body.find("fillLabelForm(view);")
|
||||
clear_i = body.find('setLabelStatus(view, "", null);')
|
||||
assert -1 < open_i < fill_i, "a collapse refills from the stored label"
|
||||
assert fill_i < clear_i < body.find("updateLabelRowUI(view);")
|
||||
|
||||
|
||||
def test_stored_shape_folds_nulls_to_the_empty_form() -> None:
|
||||
"""``storedFromEntry`` folds the API's nulls / missing keys to ``\"\"``
|
||||
and ``[]`` (a missing row and a cleared label are ONE state — D1), and
|
||||
``hasStoredLabel`` is what drives the "named" tag and Reset."""
|
||||
js = _js()
|
||||
body = _fn(js, "storedFromEntry")
|
||||
assert 'typeof e.display_name === "string"' in body
|
||||
assert "Array.isArray(e.tags)" in body
|
||||
stored = _fn(js, "hasStoredLabel")
|
||||
assert "stored.display_name || stored.label || stored.tags.length" in stored
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# styles.css — the reuse contract
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_new_block_reuses_the_page_language() -> None:
|
||||
"""The new rules stay on the page's own language: the field box is the
|
||||
``.ignore-editor-token`` box (mono, the line border, the 44px floor),
|
||||
the row's three buttons keep the ≥44px touch target, the empty state is
|
||||
SHARED with ``.git-sources-empty``, and the dirty mark is the phase-168
|
||||
``.is-dirty`` stripe."""
|
||||
css = _css()
|
||||
field = _css_rule(css, ".source-label-input")
|
||||
assert "min-height: 44px" in field
|
||||
assert "font-family: var(--mono)" in field
|
||||
assert "border: 1px solid var(--line)" in field
|
||||
for selector in (".source-label-toggle", ".source-label-save", ".source-label-reset"):
|
||||
assert "min-height: 44px" in _css_rule(css, selector), f"{selector} lost the floor"
|
||||
selector = _css_selector_list(css, ".source-labels-empty {")
|
||||
assert ".git-sources-empty" in selector, "the empty-state rule is shared, not copied"
|
||||
dirty = _css_rule(css, ".source-label-row.is-dirty > td:first-child")
|
||||
assert "box-shadow: inset 3px 0 0 var(--brand)" in dirty, "the phase-168 stripe"
|
||||
|
||||
|
||||
def test_new_css_uses_palette_variables_only() -> None:
|
||||
"""No new hue, no CDN: every colour in the block is a palette
|
||||
variable (the phase-08/92 monochrome invariant — a raw hex would not
|
||||
follow a saved theme)."""
|
||||
css = _css()
|
||||
start = css.index("/* ---------- Source names, labels and tags (phase 191, task 03)")
|
||||
end = css.index("/* ---------- History refresh button", start)
|
||||
block = css[start:end]
|
||||
assert block, "the labelled block exists"
|
||||
offenders = re.findall(r"#[0-9a-fA-F]{3,8}\b", block)
|
||||
assert not offenders, f"raw hex colours in the phase-191 block: {offenders}"
|
||||
assert "url(" not in block, "no external asset (the no-CDN rule)"
|
||||
assert "blur(" not in block, "the phase-08 no-blur anchor"
|
||||
Reference in New Issue
Block a user