Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 191_source_name_labels_tags
Build and Push Containers / build-and-push-app (push) Successful in 7m24s
Details
Build and Push Containers / build-and-push-db (push) Successful in 36s
Details

Browse Source 
All gates re-verified against a stable tree. Report:

**Verified (all 6 tasks in `complete/`; this was the final pass)**
- `SourceLabel`+alembic `0030` (dev DB `alembic current` = `0030 (head)`, table present, 0 rows); admin-only `GET/PUT/DELETE /api/source-labels` (`require_admin` router-wide, `resolve_source_name`, partial-update caps/404s); Sources-page "Names, labels and tags" card (read-only derived-name `<code>`, per-row `saveInFlight`, 404 note that survives the settle, chips reuse `.git-source-hidden-count`); `question_name_hits` D2 rule; ONE `all_labels` read per turn in `chat.py` feeding both the `source_name_hit` flag and the prompt surfaces; `source_name_hit_bonus=0.005` in `_selection_order`; ` name=`/` label=` in `build_high_prompt` + `build_overview_prompt(names=…)`.
- **Completion criteria:** E2E `tests/e2e/test_source_name_labels.py` ✅ 7 passed (arms a–e: editor all kinds, control/named question, rename-never-breaks-identity, 404 note, reset) · full suite ✅ · coverage ✅ · README/AGENTS ✅ · phase dir in `complete/` ✅ · commit left to the harness.
- **Gates:** `uv run pytest --cov=app --cov-report=term-missing` → **exit 0, TOTAL 7160 stmts / 16 missed = 99%** (>90%) · `uv run pytest tests/e2e/test_source_name_labels.py -v --no-cov` → **7 passed, exit 0** · regression E2E isolated, exit 0: `test_document_viewer`, `test_admin_auth`, `test_api_tokens`, `test_git_sources_admin`, `test_kb_tree`, `test_local_directory_sources` · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors.
- **Battery (task 04 criterion):** `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` at default vs `BOR_SOURCE_NAME_HIT_BONUS=0` → **`diff` empty (line-for-line identical)**, matching the recorded arms (dev KB is empty, so every question deflects; the observable boost is unit/E2E-pinned).
- **Documentation:** README — fixed the `/git-sources.html` rows (nav label is **Sources**) and cross-linked the phase-191 card from that section; full audit otherwise clean: all 86 `BOR_*` settings documented with defaults matching `app/config.py`, 0 broken file/dir references and 0 broken anchors in README+AGENTS, AGENTS rule 17 re-checked line-by-line against the code.
- **Notable / deviations:** (1) a **concurrent pipeline process** (`phased-execution/progress.mjs` for `191_…_00_phase`) was mutating this same working tree mid-pass — it rewrote `tests/e2e/test_source_name_labels.py` (11→7 tests), `frontend/assets/git-sources.js`, `tests/unit/test_source_labels_editor_ui.py` and `AGENTS.md` at 05:27; every gate above was (re-)run after those edits with the tree no source file newer than the green full run. (2) The task-06 report/phase text claims regression suites `test_chat_token_gate.py`, `test_git_sources_page.py`, `test_rag_browser.py` are green — **those files have never existed in git history** (those runs exit 4); I ran the real equivalents listed above instead (phase files are not mine to edit). (3) **No live changes**: the VM DB already has `source_labels` (0 rows) but the deployed app predates this uncommitted code, so deploying `/api/source-labels` to the live service is the owner's post-commit step, not part of this phase's criteria.
- **Next pending phase:** none — `191_source_name_labels_tags` is the only entry in `.agents/phases/todo/`.
This commit is contained in:
ducoterra
2026-10-19 06:13:32 -04:00
parent c6ffa050a6
commit 4f20c256d7
70 changed files with 7378 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/todo/191_source_name_labels_tags/00_phase.md → .agents/phases/complete/191_source_name_labels_tags/00_phase.md
View File
.agents/phases/todo/191_source_name_labels_tags/01_source_labels_table.md → .agents/phases/complete/191_source_name_labels_tags/01_source_labels_table.md
View File
.agents/phases/todo/191_source_name_labels_tags/02_source_labels_api.md → .agents/phases/complete/191_source_name_labels_tags/02_source_labels_api.md
View File
.agents/phases/todo/191_source_name_labels_tags/03_source_labels_ui.md → .agents/phases/complete/191_source_name_labels_tags/03_source_labels_ui.md
View File
.agents/phases/todo/191_source_name_labels_tags/04_source_name_boost.md → .agents/phases/complete/191_source_name_labels_tags/04_source_name_boost.md
View File
.agents/phases/todo/191_source_name_labels_tags/05_prompt_surfacing.md → .agents/phases/complete/191_source_name_labels_tags/05_prompt_surfacing.md
View File
.agents/phases/todo/191_source_name_labels_tags/06_e2e.md → .agents/phases/complete/191_source_name_labels_tags/06_e2e.md
View File
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__00_phase.a1.err
View File
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__00_phase.a1.md
+10
View File
@@ -0,0 +1,10 @@
All gates re-verified against a stable tree. Report:
**Verified (all 6 tasks in `complete/`; this was the final pass)**
- `SourceLabel`+alembic `0030` (dev DB `alembic current` = `0030 (head)`, table present, 0 rows); admin-only `GET/PUT/DELETE /api/source-labels` (`require_admin` router-wide, `resolve_source_name`, partial-update caps/404s); Sources-page "Names, labels and tags" card (read-only derived-name `<code>`, per-row `saveInFlight`, 404 note that survives the settle, chips reuse `.git-source-hidden-count`); `question_name_hits` D2 rule; ONE `all_labels` read per turn in `chat.py` feeding both the `source_name_hit` flag and the prompt surfaces; `source_name_hit_bonus=0.005` in `_selection_order`; ` name=`/` label=` in `build_high_prompt` + `build_overview_prompt(names=…)`.
- **Completion criteria:** E2E `tests/e2e/test_source_name_labels.py` ✅ 7 passed (arms a–e: editor all kinds, control/named question, rename-never-breaks-identity, 404 note, reset) · full suite ✅ · coverage ✅ · README/AGENTS ✅ · phase dir in `complete/` ✅ · commit left to the harness.
- **Gates:** `uv run pytest --cov=app --cov-report=term-missing` → **exit 0, TOTAL 7160 stmts / 16 missed = 99%** (>90%) · `uv run pytest tests/e2e/test_source_name_labels.py -v --no-cov` → **7 passed, exit 0** · regression E2E isolated, exit 0: `test_document_viewer`, `test_admin_auth`, `test_api_tokens`, `test_git_sources_admin`, `test_kb_tree`, `test_local_directory_sources` · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors.
- **Battery (task 04 criterion):** `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` at default vs `BOR_SOURCE_NAME_HIT_BONUS=0` → **`diff` empty (line-for-line identical)**, matching the recorded arms (dev KB is empty, so every question deflects; the observable boost is unit/E2E-pinned).
- **Documentation:** README — fixed the `/git-sources.html` rows (nav label is **Sources**) and cross-linked the phase-191 card from that section; full audit otherwise clean: all 86 `BOR_*` settings documented with defaults matching `app/config.py`, 0 broken file/dir references and 0 broken anchors in README+AGENTS, AGENTS rule 17 re-checked line-by-line against the code.
- **Notable / deviations:** (1) a **concurrent pipeline process** (`phased-execution/progress.mjs` for `191_…_00_phase`) was mutating this same working tree mid-pass — it rewrote `tests/e2e/test_source_name_labels.py` (11→7 tests), `frontend/assets/git-sources.js`, `tests/unit/test_source_labels_editor_ui.py` and `AGENTS.md` at 05:27; every gate above was (re-)run after those edits with the tree no source file newer than the green full run. (2) The task-06 report/phase text claims regression suites `test_chat_token_gate.py`, `test_git_sources_page.py`, `test_rag_browser.py` are green — **those files have never existed in git history** (those runs exit 4); I ran the real equivalents listed above instead (phase files are not mine to edit). (3) **No live changes**: the VM DB already has `source_labels` (0 rows) but the deployed app predates this uncommitted code, so deploying `/api/source-labels` to the live service is the owner's post-commit step, not part of this phase's criteria.
- **Next pending phase:** none — `191_source_name_labels_tags` is the only entry in `.agents/phases/todo/`.
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__00_phase.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__00_phase.a1.validate
+150
View File
@@ -0,0 +1,150 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 43%]
........................................................................ [ 45%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 71%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
........................................................................ [ 99%]
........... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 298 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 231 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 332 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 494 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 865 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 82 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 178 0 100%
app/rag/retriever.py 270 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 576 0 100%
--------------------------------------------------
TOTAL 7160 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__01_source_labels_table.a1.err
View File
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__01_source_labels_table.a1.md
+18
View File
@@ -0,0 +1,18 @@
Task 01 complete. Report:
**Implemented**
- `app/models.py`: new `SourceLabel` model (D1 spec: `source` String(120) PK, `display_name`/`label` plain-Text nullable, `tags` JSONB NOT NULL default `'[]'`, `updated_at` now()/onupdate) + module docstring table entry; alias-only contract documented.
- `alembic/versions/0030_source_labels.py` (rev `0030` ← `0029`): additive CREATE TABLE, no backfill, `DROP TABLE` downgrade; verified up/down/up on the dev DB.
- `app/rag/source_labels.py`: `SourceLabelView`, `get_label`/`label_for`/`all_labels`, `set_label` (upsert, `_UNSET`-sentinel partial-update — omitted untouched, explicit `None` clears; flush-only), `delete_label`, `normalize_tags` (trim/drop-empties, ≤10 / ≤32 ValueErrors, first-case case-insensitive de-dupe), `question_name_hits` (D2: reused `name_hit_tokens` + `_normalize_name`, 4-char floor on both sides, no digit rule), `question_name_hit_sources` (one query).
- **Defect fix (pre-existing latent)**: `test_chunks_hnsw_index.py::test_invalid_index_is_dropped_and_rebuilt` stamped below head without dropping later migrations' tables — once 0030 landed, every following `upgrade(head)` died on `source_labels already exists` (221 cascade errors). Now genuinely `downgrade("0029")` before the stamp; consistent with AGENTS rule 16.
**Gates** (all green)
- `uv run pytest --cov=app --cov-report=term-missing` → **4667 passed**, TOTAL **99%**, `app/rag/source_labels.py` **100%**; dev DB left at `0030 (head)`.
- `tests/unit/test_source_labels.py` 31 pins (D2 every branch, partial-update shape, tag guards) + `tests/integration/test_migration_0030.py` 6 pins (column contract, no-backfill, PK/NOT-NULL, A13 round-trip) pass.
- `uv run ruff check .` clean; `uv run pyright` clean; `tests/e2e/test_smoke.py` 3 passed; `alembic upgrade head`/`downgrade 0029`/re-upgrade clean.
**Docs**: no doc impact (pure data layer — README/AGENTS updates are task 06's per the phase plan; AGENTS rule 16's schema-move convention remains accurate).
**Decisions**: `normalize_tags` count-guard runs before the de-dupe (11 typed = rejected, documented); whitespace-tag strictness is left to the task-02 API boundary per the task text.
**Next pending task**: `191_source_name_labels_tags/02_source_labels_api.md`.
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__01_source_labels_table.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__01_source_labels_table.a1.validate
+147
View File
@@ -0,0 +1,147 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 20%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 64%]
........................................................................ [ 66%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 98%]
........................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 292 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 231 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 325 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 105 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 494 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 865 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 162 0 100%
app/rag/retriever.py 259 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 549 0 100%
--------------------------------------------------
TOTAL 7035 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__02_source_labels_api.a1.err
View File
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__02_source_labels_api.a1.md
+19
View File
@@ -0,0 +1,19 @@
Everything is green. Final report:
**Implemented (phase 191 task 02)**
- `app/schemas.py` — `SourceLabelOut` / `SourceLabelList` / `SourceLabelUpdate` (`extra="forbid"`, trim-*before*-cap validators, `_Tag` item cap = the house `_Chip` item-cap convention).
- `app/api/source_labels.py` (NEW) — admin-gated router: catalogue `GET` (`effective_sources` names via `resolve_source_name` — git + local + the env fallback — ∪ `SELECT DISTINCT documents.source`, de-duped, sorted, ONE `all_labels` read), `PUT` (404 arm first → `normalize_tags` `ValueError`→422 → `set_label` from `model_fields_set`: omitted keeps / `null` clears / value sets, router commits), `DELETE` (404 when no row; 200 body = the entry's cleared state).
- `app/main.py` — registered in the admin-router block; `scripts/api_payload_audit.py` — the new endpoint in the byte audit.
- `tests/integration/test_source_labels_api.py` (NEW, 26 tests): 403 gate for anonymous AND token users, the git+local+upload union (sorted, no dupes, env-only arm, exactly 3 SELECTs and no `documents.content` read), round trip, partial-update shape, every cap as the framework's 422 with the row byte-identical, both 404 arms, the `ValueError`→422 funnel, and the D1 "a rename writes nothing but the label row" arm.
**Gates** — `uv run pytest` → **4693 passed**; `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%**, `app/api/source_labels.py` **100%**, `app/schemas.py` **100%**; `uv run ruff check .` → clean; `uv run pyright` → **0 errors**; E2E sanity in isolation: `tests/e2e/test_smoke.py`, `tests/e2e/test_git_sources_admin.py` pass.
**Docs** — README gains "Source names, labels and tags (phase 191)" (route table + the alias-only contract); AGENTS.md gains rule 17 (alias-never-identity, reuse `resolve_source_name`, 404/no-partial-write, the pin file).
**Decisions / deviations**
- Router uses the house prefix split (`prefix="/source-labels"` + include `prefix="/api"`) — wire paths are exactly D5's.
- `DELETE` answers 200 with the cleared `SourceLabelOut` (the task's "200"; body mirrors phase-168 `DELETE /api/admin/settings/{key}` returning the reset item).
- A whitespace-only/empty tag is the framework's 422 (trim runs before `min_length`), as this task's pins require; `normalize_tags` keeps its drop-empties contract for helper callers, still owns the case-insensitive de-dupe, and its `ValueError`→422 belt is pinned by substituting a raising normalizer.
- The GET lists *sources* (registry ∪ indexed documents), not orphan label rows; `DELETE` never consults the registry, so an orphan row stays clearable (documented in the route docstring).
**Next pending task:** `191_source_name_labels_tags/03_source_labels_ui.md`.
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__02_source_labels_api.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__02_source_labels_api.a1.validate
+149
View File
@@ -0,0 +1,149 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 73%]
........................................................................ [ 75%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
........................................................................ [ 99%]
............. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 292 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 231 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 325 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 494 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 865 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 162 0 100%
app/rag/retriever.py 259 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 576 0 100%
--------------------------------------------------
TOTAL 7116 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__03_source_labels_ui.a1.err
View File
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__03_source_labels_ui.a1.md
+15
View File
@@ -0,0 +1,15 @@
**Implemented (phase 191 task 03 — the per-source name/label/tags editor, D6)**
- `frontend/index.html`: new **Names, labels and tags** card inside the admin-gated `#git-sources-content` — bounded `.table-wrap` table (sticky header), `role="alert"` load-error line, empty state, and ONE shared `<template id="source-label-row-template">`: read-only mono identity `<code>` + Name ≤120 / Label ≤200 / Tags ≤32 fields (real `<label for>`) + Save/Reset + `role="status"` save-state line.
- `frontend/assets/git-sources.js`: `loadLabels` → `renderLabelRows` → `makeLabelRow` (template clone + per-clone id/`aria-controls` rewrite); row list comes from `GET /api/source-labels` (so git, local **and** upload/document-only sources all get a row); token input (Enter/comma adds a chip, Backspace drops the last, ≤10×≤32, case-insensitive de-dupe); per-row dirty paint (`.is-dirty` + "unsaved changes"); per-row `saveInFlight` guard; Save `PUT` / Reset `DELETE` re-read the row **from the response** (no second GET), 404 → "That source no longer exists…", `finally` never leaves "Saving…"; collapsing discards the draft; refreshed at boot / view-refresh / add / remove / finished upload; anonymous never fetches it. Existing sync/upload/git-row flows untouched.
- `frontend/assets/styles.css`: new block, palette variables only; `.source-label-state` + `.source-label-tag` **joined** the phase-89 tag rule (no bespoke tag-input CSS); `.source-labels-empty` shares `.git-sources-empty`; 44px floors.
- New `tests/unit/test_source_labels_editor_ui.py` (25 marker pins: markup, alias contract, load/save/reset/404 note, token rules, clone, CSS reuse, no-hex).
- Fixed two stale selector-list pins (phase-105/135) broken by the joined tag rule; docs: README sources section now documents the editor; AGENTS.md rule 17 gains the UI clause.
**Gates** — `uv run pytest --cov=app --cov-report=term-missing` → **4718 passed**, `TOTAL 99%`; `uv run ruff check .` → **All checks passed**; `uv run pyright` → **0 errors**. E2E in isolation (`--no-cov`): smoke 3, git_sources_admin 6, git_sources_card_rhythm 4, source_ignore_paths 6, hidden_folders_toggle 6, archive_upload_sources 5, git_source_branch 4, git_source_tokens 4 — **all pass**. Behavior verified in a real browser with a temporary 5-scenario Playwright scratch suite (3 source kinds render, chip typing + save→DB + reset→DB, reload persistence, no horizontal overflow at 375px, anonymous fetches nothing) — scratch file deleted; dev DB left clean.
**Notable decisions / deviations**
- Row list = the `GET /api/source-labels` catalogue (per the task), rendered as its own table card rather than a column of `#git-sources-table`, so a source with documents but no registry row is still nameable.
- The task's "phase-89 token input" precedent does not exist as a chip widget (phase 89 ships a one-path-per-line textarea), so the tags widget's *behavior* is new while its *styling* reuses the house tag rule + the `.ignore-editor-token` field box; no new hue, no CDN.
- Kept the phase-77 `() => loadSources()` refresh listener verbatim (pinned) and added a second `bor:view-refresh` listener for the label list — an unrelated table reload (ignore-paths/hidden saves) therefore never wipes a half-typed name draft.
**Next pending task:** `191_source_name_labels_tags/04_source_name_boost.md`.
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__03_source_labels_ui.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__03_source_labels_ui.a1.validate
+149
View File
@@ -0,0 +1,149 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 71%]
........................................................................ [ 73%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 97%]
........................................................................ [ 99%]
...................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 292 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 231 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 325 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 494 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 865 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 162 0 100%
app/rag/retriever.py 259 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 576 0 100%
--------------------------------------------------
TOTAL 7116 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__04_source_name_boost.a1.err
View File
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__04_source_name_boost.a1.md
+19
View File
@@ -0,0 +1,19 @@
**Task 04 complete — all gates green.**
**Implemented (D2/D3 source-name selection boost):**
- `app/config.py` — `source_name_hit_bonus: float = 0.005` knob + `>= 0` validator (`"source_name_hit_bonus must be >= 0 (0 = off)"`); env `BOR_SOURCE_NAME_HIT_BONUS`.
- `app/core/settings_catalog.py` — "Source name hit bonus" row in *Retrieval & honesty*, flags mirrored from `name_hit_bonus` (plain, live, overridable).
- `app/rag/retriever.py` — `RetrievedChunk.source_name_hit: bool = False`; `fuse` OR-merges it on the double-hit; `_selection_order(chunks, bonus, source_bonus=0.0)` gains `any_source_hit`, the second `+ source_bonus` term, and the extended re-sort condition (neither bonus active → no re-sort, byte-identical); `weak_hit_titles` / `select_suggested` / `select_related` gain `source_bonus: float | None = None` pulled from settings.
- `app/api/chat.py` — ONE `question_name_hit_sources(step_db, question)` read per turn after `retrieve`, before the first selection call; flag loop over the retrieved chunks (no per-chunk query); per-turn log line untouched.
- Battery record written to `.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__04_source_name_boost.battery.md` (task/phase files are harness-owned, so the three-run record lives in the reports dir): baseline vs `bonus=0` vs default-0.005 (no label rows) → **all three outputs line-for-line identical** (`diff` exit 0).
**Tests/lint/coverage:**
- 8 new `test_retriever.py` pins (kill-switch byte-identity, inert-without-hits, equal-fused lift, bounded +0.010 two-bonus, three helpers' settings defaults + both kill arms, 3 `fuse` pins); phase-119 untouched-fields pin extended; 2 `test_config.py` pins; 1 catalogue-completeness row pin.
- `uv run pytest --cov=app --cov-report=term-missing` → **4729 passed**, `app/` coverage **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
**Docs:** README — phase-191 section gains the "name focuses retrieval" paragraph + `BOR_SOURCE_NAME_HIT_BONUS` settings-table row; `.env.example` gains the knob; `AGENTS.md` rule 17 gains the task-04 boost clause (per the phase's docs note).
**Notes:** battery app is the empty dev KB (0 docs → all deflects) — the byte-identity claim holds there; the observable-boost evidence is the unit pins + the task-06 E2E arm, per the task's testing mandate. No separate integration module (task explicitly defers the flag-loop arm to task 06; chat.py stays at 99%).
**Next pending task:** `191_source_name_labels_tags/05_prompt_surfacing.md`.
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__04_source_name_boost.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__04_source_name_boost.a1.validate
+149
View File
@@ -0,0 +1,149 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 35%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 39%]
........................................................................ [ 41%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 54%]
........................................................................ [ 56%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 71%]
........................................................................ [ 73%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 98%]
................................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 297 2 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 231 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 332 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 494 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 865 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 78 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 162 0 100%
app/rag/retriever.py 270 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 576 0 100%
--------------------------------------------------
TOTAL 7139 17 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__04_source_name_boost.battery.md
+17
View File
@@ -0,0 +1,17 @@
# Phase 191 task 04 — retrieval battery record (the phase-119 convention)
Command (all three arms): `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt`
Run against the battery app = the dev Postgres (`brain-of-reese_db_1`) with **0 documents / 0 chunks / 0 `source_labels` rows** — every question deflects (`best_cosine=0.0000 fts_hits=0 -> LOW`). That is the state the task's no-row baseline names: with no label rows the D2 rule can produce no hit, so all three arms must — and do — agree line-for-line.
| Arm | Setting | Output | Result |
| --- | --- | --- | --- |
| (a) baseline (pre-change) | shipped defaults | `/tmp/battery191/baseline.txt` (29 lines) | reference |
| (b) post-change kill switch | `BOR_SOURCE_NAME_HIT_BONUS=0` | `/tmp/battery191/post_bonus0.txt` | `diff (a) (b)` = **line-for-line identical** |
| (c) post-change default | `source_name_hit_bonus = 0.005` (no label rows exist) | `/tmp/battery191/post_default.txt` | `diff (a) (c)` = **line-for-line identical** |
The observable-boost evidence therefore lives in the unit pins (the
kill-switch byte-identity, the bounded two-bonus application, the fuse
OR-merge and the three settings-default helpers in
`tests/unit/test_retriever.py`), and the named-source surfacing arm lands
with the task-06 E2E suite (`tests/e2e/test_source_name_labels.py`),
which runs its own seeded app fixture.
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__05_prompt_surfacing.a1.err
View File
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__05_prompt_surfacing.a1.md
+20
View File
@@ -0,0 +1,20 @@
**Task 05 (`191_source_name_labels_tags/05_prompt_surfacing.md`) — done**
**Implemented**
- `app/rag/prompts.py`: `build_high_prompt(..., names: Mapping[str, SourceLabelView] | None = None)`; the `<document>` head gains additive ` name="…"` (right after `source="…"`) then ` label="…"`, locked order `source→name?→label?→path→title→date`; new `_attr` escaper; unset field adds NOTHING.
- `app/rag/overview.py`: `build_overview_prompt(..., names: Mapping[str, str] | None = None)` swaps the display name into the lite model's INPUT line only; `regenerate_overview` performs ONE `all_labels` read (after the zero-doc gate) and derives the name map from those views — stored row + `Sources:` tail stay derived-name.
- `app/api/chat.py`: task-04 call site refactored to the shared `labels = all_labels(step_db)` (pure `question_name_hits` inlined, flag loop unchanged) and `plan_turn(names=labels)` → `build_high_prompt(names=…)`; `question_name_hit_sources` no longer runs in a turn. `retriever.py`/`source_labels.py` docstrings aligned.
- Tests: 12 pins in `tests/unit/test_prompts.py` (kill pins None/{}/no-entry, attribute-order, label-only, escaping, per-source scoping, LOW-prompt clean), 12 in `tests/unit/test_overview.py` (kill pins, line swap, cap, `regenerate_*` display-name/inventory/single-read + marker pins), 5 in `tests/unit/test_chat_gate.py` (plan_turn names/kill/LOW + chat.py one-read marker pin + an endpoint pin that the ONE read sets both the boost flag and the prompt attributes).
**Gates**
- `uv run pytest --cov=app --cov-report=term-missing` → **4762 passed**, TOTAL 99% (`prompts.py`/`overview.py`/`source_labels.py` 100%, `chat.py` 99% — only the pre-existing 503-cap line).
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors.
- Sanity E2E (unaffected-by-design check): `uv run pytest tests/e2e/test_chat_rag.py tests/e2e/test_kb_overview.py tests/e2e/test_name_signal_read_chips.py --no-cov` → 11 passed.
**Docs**: README — new "The name reaches the model, too." paragraph in the phase-191 section; AGENTS.md rule 17 — task-04 clause corrected to the ONE `all_labels` read and extended with the task-05 surfacing contract.
**Decisions / deviations**
- Phase-file D4 sketched `display_name (label) — path` prose; task 05 (the executable spec) specifies the additive `name=`/`label=` attributes — implemented per task 05.
- Verified L934: the block writer never escaped `source`/`path`/`title`, so there was no shared helper. Added `_attr` (`&`,`<`,`>`,`"`) applied ONLY to the two new owner-supplied values — escaping the identity attributes would have broken the "label-less block byte-identical" criterion. Single quotes deliberately kept (`Reese's` stays readable).
**Next pending task**: `191_source_name_labels_tags/06_e2e_and_gates.md`.
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__05_prompt_surfacing.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__05_prompt_surfacing.a1.validate
+150
View File
@@ -0,0 +1,150 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 43%]
........................................................................ [ 45%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 71%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
........................................................................ [ 99%]
.......... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 298 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 231 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 332 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 494 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 865 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 82 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 178 0 100%
app/rag/retriever.py 270 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 576 0 100%
--------------------------------------------------
TOTAL 7160 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__06_e2e.a1.err
View File
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__06_e2e.a1.md
+17
View File
@@ -0,0 +1,17 @@
**Task 06 (`191_source_name_labels_tags/06_e2e.md`) — done**
**Added** `tests/e2e/test_source_name_labels.py` (7 tests, own module app on port 8208). Fixture registers + indexes **all three source kinds** through the real APIs — a real `git init` repo cloned by the sync over `file://` (`forgecore`), a local dir (`homelab`), a tar.gz through `/api/git-sources/upload` (`notesdrop`) — with a token-controlled corpus whose target (`forgecore/notes/b6.md`) is pinned by the app's own retrieval at setup as rank 6 (out of the top-5 seed) and rank 1 once its source is named.
- (a) editor saves Name/Label/Tags on a git, local **and** upload row → per-row "saved", reload re-populates, identity cell stays the read-only derived `<code>`;
- (b) control turn: target in the "Nearby docs" row, no `<document>` block for it; after saving "the git server"/"self-hosted forge"/tags the **same** question seeds it with the wire-exact `<document source="forgecore" name="the git server" label="self-hosted forge" path="notes/b6.md"` (mock `/v1/e2e/captured`), and a name-free question with the same row saved holds the pre-phase tier;
- (c) alias vs identity: content API + viewer + KB tree stay DERIVED, `source=the git server` → 404, nearby-doc href derived;
- (d) vanished source → the named 404 note, gone from the catalogue on reload, orphan PUT 404; (e) Reset clears fields and the tier reverts.
**Defect found and fixed (task 03's UI):** `frontend/assets/git-sources.js` — the save's `finally` called `updateLabelRowUI`, whose dirty hint repainted the failure note in the same tick, so "That source no longer exists — reload the page." was never readable. The paint now returns before the hint on an `.is-error` line; a new `clearLabelError` drops the note on the next edit only. Pinned by `test_a_failed_saves_note_survives_the_dirty_paint` + the E2E.
**Gates**: `pytest --cov=app` → **4763 passed, TOTAL 99%**; `tests/e2e/test_source_name_labels.py -v --no-cov` → **7 passed** in isolation; regressions green unedited (name_signal_read_chips, admin_settings_panel, local_directory_sources, kb_tree, git_sources_admin, source_ignore_paths, archive_upload_sources → 51 tests); `ruff check .` clean, `pyright` 0 errors.
**Docs**: README — the failure note now documented as staying until an edit/collapse; AGENTS.md rule 17 — the error-note-survival contract and the new E2E suite named as a pinner of the alias contract.
**Deviations**: no commit and no `todo/`→`complete/` move (harness-managed per my operating instructions; working tree left staged-ready). The 404 arm uses a never-synced throwaway git row it deletes itself, so no other source's documents are touched.
**Next pending task**: none in `191_source_name_labels_tags/` — the phase is complete pending the harness commit.
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__06_e2e.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/191_source_name_labels_tags/191_source_name_labels_tags__06_e2e.a1.validate
+150
View File
@@ -0,0 +1,150 @@
........................................................................ [ 1%]
........................................................................ [ 3%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 7%]
........................................................................ [ 9%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 18%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 28%]
........................................................................ [ 30%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 39%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 43%]
........................................................................ [ 45%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 52%]
........................................................................ [ 54%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 58%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 71%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
........................................................................ [ 99%]
........... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 298 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 231 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 332 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 494 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 865 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 82 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 178 0 100%
app/rag/retriever.py 270 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 576 0 100%
--------------------------------------------------
TOTAL 7160 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/screenshots/doc_modal_scrollbar_1280.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 42 KiB

After

Width:  |  Height:  |  Size: 42 KiB

.env.example
+1
View File
@@ -102,6 +102,7 @@ BOR_RRF_K=60 # Reciprocal Rank Fusion damping constant
BOR_RECENCY_BOOST=0.0007 # max additive fused-score bonus a zero-age doc gets (phase 106); 0 = off (pre-phase ranking byte-identical) BOR_RECENCY_BOOST=0.0007 # max additive fused-score bonus a zero-age doc gets (phase 106); 0 = off (pre-phase ranking byte-identical)
BOR_RECENCY_HALF_LIFE_DAYS=365 # recency decay timescale, days (phase 106); the boost decays by e^-1 (~0.37x) per timescale of age BOR_RECENCY_HALF_LIFE_DAYS=365 # recency decay timescale, days (phase 106); the boost decays by e^-1 (~0.37x) per timescale of age
BOR_NAME_HIT_BONUS=0.005 # selection-time bonus a name-hit doc's best fused score gets (phase 119); 0 = off (pre-phase order byte-identical) BOR_NAME_HIT_BONUS=0.005 # selection-time bonus a name-hit doc's best fused score gets (phase 119); 0 = off (pre-phase order byte-identical)
BOR_SOURCE_NAME_HIT_BONUS=0.005 # selection-time bonus a source-name-hit doc gets — the question names the source label row's name/label/tag (phase 191); independent knob, 0 = off
# --- Agent document tools (grounded turns may extend context: ls / read / grep) --- # --- Agent document tools (grounded turns may extend context: ls / read / grep) ---
# BOR_AGENT_MAX_ROUNDS=10 # hard cap on agent tool rounds per turn (0 = no tools); every tool result carries the model's remaining count (phase 160) # BOR_AGENT_MAX_ROUNDS=10 # hard cap on agent tool rounds per turn (0 = no tools); every tool result carries the model's remaining count (phase 160)
AGENTS.md
+93
View File
@@ -312,6 +312,99 @@
quoted cosine/lexical margins, so editing a note there moves the numbers in quoted cosine/lexical margins, so editing a note there moves the numbers in
the module docstring and the pins together. the module docstring and the pins together.
17. **A source label is an alias, never an identity (phase 191):** the
`source_labels` row (alembic `0030`, PK = the DERIVED source name
exactly as `Document.source` carries it) holds the owner's
`display_name` / `label` / `tags`, and only
`app/api/source_labels.py` writes it — `GET /api/source-labels` (the
catalogue: the `effective_sources` rows of BOTH kinds — or the
`BOR_GIT_SOURCES` names while the table is empty — plus every indexed
document source no row derives, de-duplicated and sorted, ONE
`all_labels` read for the whole list), `PUT` (partial update: omitted
keeps, explicit `null` clears, `tags: []` clears) and `DELETE` (the
reset); `require_admin` router-wide, and every name resolution goes
through `app.rag.source_removal.resolve_source_name` — never
re-derive the basename here or the row keys nothing. A `PUT` for a
name that is neither a registry row's derived name nor an indexed
document source is 404 (no orphan label rows) and a validation
failure writes NOTHING. NEVER rename, resolve or join on the display
name: `Document.source`, the `source/path` read/grep/`show_text`
locators, the citations, the KB tree and the verbatim caption stay on
the derived name, so a rename cannot invalidate a saved chat, a
shared page or a locator — and the name reaches the model only
through the prompt/retrieval layers (tasks 04/05), never by replacing
the identity in a chip, a table or a caption. Pinned by
`tests/integration/test_source_labels_api.py` (the 403 gate for
anonymous AND token users, the git+local+upload union sorted and
de-duplicated at three SELECTs with no `documents.content` read, the
partial-update shape, every cap as the framework's 422 with the row
byte-identical, both 404 arms, the `ValueError` → 422 funnel, and the
"a rename writes nothing but the label row" arm).
The Sources-page editor (task 03) is the owner-visible half of the SAME
contract and must stay that way: its row list comes from
`GET /api/source-labels` (the catalogue union — never a re-derived
basename, never a join on the display name), the row's identity cell is a
read-only `<code>` of the DERIVED name (an input there WOULD be the
alias-contract bug), Save/Reset `PUT`/`DELETE` the per-source path and
re-read the ROW from that response (no second GET), the save state is
scoped PER ROW (`saveInFlight` — one request per row, the `finally` always
settles it, the 404 note names the arm, a collapse discards the draft) —
and a FAILED save's note SURVIVES that settle: `updateLabelRowUI` returns
before the dirty hint whenever the line carries `.is-error`, so
"That source no longer exists — reload the page." cannot be painted over
by "unsaved changes" in the tick that wrote it (the defect the task-06
story suite walked into); only the next edit (`clearLabelError`) or a
collapse clears a note, never the settle path),
and the Tags token input's chips reuse the phase-89 tag rule
(`.source-label-tag` sits in the `.git-source-hidden-count` rule group — no
bespoke tag-input CSS). Pinned by
`tests/unit/test_source_labels_editor_ui.py` and, through the real UI over
one git + one local + one uploaded source, by
`tests/e2e/test_source_name_labels.py` (task 06: the editor saves and
re-populates all three kinds, a question that USES the name moves that
source's document into the seeded `<documents>` tier with
`name`/`label` on the wire while the same row with a name-free question
moves nothing, the vanished-source 404 note is visible, Reset reverts the
tier, and the alias never appears in a locator, a nearby-doc link, the KB
tree or the viewer); the chat source chips, the RAG
source tree and the verbatim caption keep the derived name untouched.
The selection boost (task 04) is the retrieval half of the SAME
contract: ONE `all_labels` read per turn at the
retrieval boundary (`app/api/chat.py`, after `retrieve`, before the
first selection call) flags `RetrievedChunk.source_name_hit` on the
chunks whose `document.source` is a D2 hit (the pure
`question_name_hits` rule applied per view — a loop over the few
dozen chunks, never a per-chunk query; that ONE result is also what
task 05's prompt surfaces consume, so a turn issues one label query
and the standalone `question_name_hit_sources` wrapper never runs in
one), and `_selection_order` adds
`source_name_hit_bonus` (default 0.005; `0` = the byte-identical
kill switch; a negative value refuses startup) as a SECOND,
independent bonus — once per DOCUMENT, at most two bonuses stacked
with the phase-119 `name_hit_bonus` (bounded 0.010 on the fused
scale), SELECTION layer only (`score`/`cosine`/`fts_hit`/
`query_log.top_score` and the per-turn log line untouched). Pinned
by `tests/unit/test_retriever.py` (the kill-switch byte-identity,
the bounded two-bonus application, the `fuse` OR-merge, the three
selection helpers pulling the knob from settings).
The prompt surfacing (task 05) is the model-facing half of the SAME
read: `build_high_prompt(names=…)` gives the suggested `<document>`
block the ADDITIVE ` name="…"` (immediately after `source="…"`) and
` label="…"` attributes — the order is locked
(`source` → `name?` → `label?` → `path` → `title` → `date`), the two
values escape through the ONE `_attr` helper, and the identity
attributes keep their pre-phase unescaped bytes; `regenerate_overview`
derives the KB-overview INPUT lines' name map from its own ONE
`all_labels` read (`build_overview_prompt(names=…)` swaps the raw
source name for the display name in the lite model's input ONLY — the
stored outline and its deterministic `Sources: …` tail stay derived
name). An unset field adds NOTHING (never `name=""`), tags never
render, the LOW prompt never carries a name, and no map / no entry is
byte-identical to pre-phase on BOTH builders (the kill pins in
`tests/unit/test_prompts.py` and `tests/unit/test_overview.py`, which
also marker-pin the one-read-per-surface contract, as
`tests/unit/test_chat_gate.py` does on the chat call site).
## Quick reference ## Quick reference
```bash ```bash
podman compose up -d db # start Postgres 17 + pgvector podman compose up -d db # start Postgres 17 + pgvector
README.md
+80 -7
View File
@@ -64,7 +64,7 @@ client-side router. Direct URLs deep-link to the matching view.
|------|-----|----------------| |------|-----|----------------|
| **Chat** | `/` | Everyone (token gate for non-shared) | | **Chat** | `/` | Everyone (token gate for non-shared) |
| **RAG / Knowledge base** | `/sources.html` | Admin only | | **RAG / Knowledge base** | `/sources.html` | Admin only |
| **Git sources** | `/git-sources.html` | Admin only | | **Git sources** (nav label **Sources**) | `/git-sources.html` | Admin only |
| **Tuning** | `/tuning.html` | Admin only | | **Tuning** | `/tuning.html` | Admin only |
| **Saved chats** | `/history.html` | Admin only | | **Saved chats** | `/history.html` | Admin only |
| **Access tokens** | `/tokens.html` | Admin only | | **Access tokens** | `/tokens.html` | Admin only |
@@ -269,12 +269,16 @@ just looked at costs a bodiless 304 (see *Caching & deploys*).
### Git sources (`/git-sources.html`) ### Git sources (`/git-sources.html`)
The admin-managed source registry: add or remove git repositories, upload The admin-managed source registry (labelled **Sources** in the nav): add or
source archives (`.tar`, `.tar.gz`, `.tgz`, `.zip`), and register local remove git repositories, upload source archives (`.tar`, `.tar.gz`, `.tgz`,
directories — all in one table with a `kind` discriminator. No `.env` `.zip`), and register local directories — all in one table with a `kind`
editing, no restart. The **Sync sources** button — which lives on the discriminator. No `.env` editing, no restart. Below the table sits the
**RAG** page (`/sources.html`), not here — clones/pulls/walks every **Names, labels and tags** card, where each source gets your own name,
source and re-imports in one click. label and focus tags (phase 191 — see
[Source names, labels and tags](#source-names-labels-and-tags-phase-191)).
The **Sync sources** button — which lives on the **RAG** page
(`/sources.html`), not here — clones/pulls/walks every source and re-imports
in one click.
### Tuning (`/tuning.html`) ### Tuning (`/tuning.html`)
@@ -635,6 +639,74 @@ discriminator: `git` | `local`), **one admin page**, and **one Sync button**.
- The directory is re-verified to exist at sync time; a missing directory - The directory is re-verified to exist at sync time; a missing directory
fails the run loudly. fails the run loudly.
### Source names, labels and tags (phase 191)
Every source — git, local or uploaded — can carry **your** words for it: a
name ("the git server"), a one-line label ("self-hosted forge + Gitea") and
up to 10 focus tags. The storage is one row per source (`source_labels`,
migration 0030, created empty — an unnamed source behaves exactly as before).
**Edit them on the Sources page** (`/git-sources.html`, admin only), in the
**Names, labels and tags** card below the sources table. Its row list is the
catalogue `GET /api/source-labels` — the union of every registered source and
every indexed document source — so git repos, local directories **and**
uploads all get an editor without a second API. Each row leads with its
read-only mono source name (the identity), expands into the three fields
(Name ≤ 120, Label ≤ 200, Tags: type + **Enter** or a **comma** adds a chip,
**Backspace** drops the last, ten chips of ≤ 32 chars) plus **Save** /
**Reset** and its own status line — "unsaved changes" → "Saving…" → "saved",
or a one-line note on failure (a 404 says *that source no longer exists*, so
you know to reload rather than retry). A failure note **stays** on the row —
the row never repaints it back to "unsaved changes" over the reason; only
editing the fields again or collapsing the editor clears it. One save per row
at a time; the save response is what the row re-reads, so nothing on the page
claims more than the server has. A row with no stored label shows no **Reset**, and collapsing
an editor discards the un-saved draft instead of silently half-saving.
The API behind it:
| Route | What it does |
| --- | --- |
| `GET /api/source-labels` | The full catalogue: every registered source (git **and** local rows, or the `BOR_GIT_SOURCES` names while the table is empty) plus every indexed source no row derives (the uploads), de-duplicated and sorted by name, each with its label or `null`/`[]`. |
| `PUT /api/source-labels/{source}` | The upsert, partial-update style: an omitted field keeps its value, an explicit `null` clears it, `tags: []` clears the tags. Values are trimmed, the caps are 120 / 200 / ten tags of 32, tags de-duplicate case-insensitively (first case kept). A source that matches no row and no document is **404** — no label can be created for a source that does not exist; an over-cap or malformed field is the framework's **422** and writes nothing. |
| `DELETE /api/source-labels/{source}` | The reset — the row goes, and the **200** body is the entry's cleared state. No row to remove is **404**. |
Admin-only (all three, like the Theme and Settings surfaces): anonymous
callers and token users get `403 admin only`.
**A name is an alias, never a rename.** The identity of a source stays the
derived name the importer stores in `Document.source` — the clone directory,
the local directory, the upload file stem. A label row never rewrites a
document, a `source/path` locator in a saved or shared chat, a citation, a
verbatim panel caption or the RAG tree, so renaming a source cannot break
anything that was already written. What the name is *for* — focusing
retrieval and the prompt on the words you use — is the rest of the phase.
**The name focuses retrieval.** When a question token equals (case- and
accent-insensitively, 4-char minimum on both sides) a token of a source's
name, label or any tag, that source is a *source-name hit* for the turn:
every retrieved document from it gets a small, bounded head start in the
selection walk that picks the suggested tier — `source_name_hit_bonus`
(default `0.005`, env `BOR_SOURCE_NAME_HIT_BONUS`, `0` = off, a negative
value refuses startup). The bonus applies **once per document**, at most
two bonuses stacked (the phase-119 path-name bonus plus this one = 0.010
on the fused scale), and it lives in the selection layer only — chunk
scores, cosines and the honesty-gate inputs are untouched, so a deflected
answer stays deflected. With no label rows anywhere the selection walk is
byte-identical to the pre-phase one (the kill switch), which is also what
the retrieval battery records at `0`.
**The name reaches the model, too.** One label read per chat turn feeds both
the boost above and the prompt: each suggested document's `<document>` block
carries ` name="the git server" label="self-hosted forge"` beside its
`source` / `path`, so the model knows which documents your words refer to —
and the KB outline is generated from a document list that names your sources
the way you do (the display name replaces the raw source name in that one
model input; the stored outline and its deterministic `Sources: …` line keep
the derived name). Nothing here changes what the model can cite: it still
reads, greps and cites `source/path`, and a source without a label row is
byte-for-byte the pre-phase prompt.
### Sync from the UI ### Sync from the UI
The **Sync sources** button on the **RAG** page (`/sources.html`, admin The **Sync sources** button on the **RAG** page (`/sources.html`, admin
@@ -1806,6 +1878,7 @@ startup (the DB rows shadow the env, `.env` is never rewritten).
| `BOR_RECENCY_BOOST` | `0.0007` | Max additive fused-score bonus a zero-age doc gets (phase 106); `0` = off | | `BOR_RECENCY_BOOST` | `0.0007` | Max additive fused-score bonus a zero-age doc gets (phase 106); `0` = off |
| `BOR_RECENCY_HALF_LIFE_DAYS` | `365` | Recency decay timescale, days (phase 106) — the boost decays by e^-1 (~0.37x) per timescale of age | | `BOR_RECENCY_HALF_LIFE_DAYS` | `365` | Recency decay timescale, days (phase 106) — the boost decays by e^-1 (~0.37x) per timescale of age |
| `BOR_NAME_HIT_BONUS` | `0.005` | Selection-time bonus a name-hit doc's best fused score gets (phase 119); `0` = off | | `BOR_NAME_HIT_BONUS` | `0.005` | Selection-time bonus a name-hit doc's best fused score gets (phase 119); `0` = off |
| `BOR_SOURCE_NAME_HIT_BONUS` | `0.005` | Selection-time bonus a **source-name**-hit doc's best fused score gets — the question names the source's label-row name/label/tag (phase 191); independent knob, `0` = off |
### Agent tools ### Agent tools
alembic/versions/0030_source_labels.py
+85
View File
@@ -0,0 +1,85 @@
"""source_labels: per-source name, label and tags (phase 191, task 01)
Revision ID: 0030
Revises: 0029
Create Date: 2026-10-18
``TODO.md`` L4: "Sources need a name, a label, tag(s) so you can focus
the model when prompting if you have a project name or a standard name
you reference things by." This revision carries the storage half
(phase 191 D1) — the matching rule (D2), the boost (D3), the API (D5)
and the UI (D6) are the later tasks.
* ``source_labels`` — one row per SOURCE the owner has named:
* ``source`` — VARCHAR(120) NOT NULL, PRIMARY KEY. The source name
EXACTLY as ``documents.source`` carries it (the derived basename —
the clone dir / local dir / upload file stem): the join key to the
documents and to the ``git_sources`` row. The PK is the uniqueness
guard (one label row per source — the ``app_settings`` 0028
reasoning). **An ALIAS key only (D1):** a label row never renames a
document, a ``source/path`` locator, a citation or the KB tree.
* ``display_name`` — TEXT nullable (the owner's name for the source,
"the git server"). Plain TEXT: the ≤ 120-char trimmed limit is
enforced at the API boundary (the phase-121 ``git_sources.branch``
pattern — plain column, boundary validation). NULL = no name.
* ``label`` — TEXT nullable (the short one-line description /
category, ≤ 200 trimmed, same boundary rule). NULL = no label.
* ``tags`` — JSONB NOT NULL with server default ``'[]'`` (the
phase-89 ``git_sources.ignore_paths`` shape): the focus tokens,
each ≤ 32 chars, ≤ 10 entries, guarded at the API boundary
(``app.rag.source_labels.normalize_tags``).
* ``updated_at`` — TIMESTAMPTZ NOT NULL, ``now()`` server default
(the ``app_settings`` / ``folder_summaries`` house style).
NO backfill (D1): the table is created EMPTY — a source without a row
has no name/label/tags and behaves exactly as pre-phase, so "no label
row → every surface byte-identical" holds without a data step. No index
beyond the PK: every read is a PK lookup or a whole-table walk of a
handful of rows (the ``app_settings`` "never a query surface" note).
One additive, fully reversible migration (A13).
"""
from __future__ import annotations
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
from alembic import op
revision = "0030"
down_revision = "0029"
branch_labels = None
depends_on = None
def upgrade() -> None:
# No index beyond the PK (every read is a PK lookup or the whole
# handful-row table), no backfill (an empty table = every source
# keeps its pre-phase "no row" behavior).
op.create_table(
"source_labels",
sa.Column("source", sa.String(length=120), primary_key=True),
sa.Column("display_name", sa.Text(), nullable=True),
sa.Column("label", sa.Text(), nullable=True),
sa.Column(
"tags",
postgresql.JSONB(astext_type=sa.Text()),
server_default=sa.text("'[]'"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.func.now(),
nullable=False,
),
)
def downgrade() -> None:
# The table is the only 0030 artefact — dropping it leaves 0029's
# schema byte-identical (A13, fully reversible). No owner data is
# trapped: the names/labels/tags are re-enterable from the Sources
# page (task 03), and nothing else was mutated by 0030.
op.drop_table("source_labels")
app/api/chat.py
+51 -1
View File
@@ -258,7 +258,7 @@ import asyncio
import json import json
import logging import logging
import time import time
from collections.abc import AsyncIterator, Sequence from collections.abc import AsyncIterator, Mapping, Sequence
from dataclasses import asdict, dataclass, field from dataclasses import asdict, dataclass, field
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
@@ -301,6 +301,11 @@ from app.rag.retriever import (
weak_hit_titles, weak_hit_titles,
) )
from app.rag.scaffolding import ScaffoldingFilter # phase 71: the streaming filter from app.rag.scaffolding import ScaffoldingFilter # phase 71: the streaming filter
from app.rag.source_labels import (
SourceLabelView,
all_labels,
question_name_hits,
)
from app.rag.suggestions import derive_suggestions from app.rag.suggestions import derive_suggestions
from app.rag.summarizer import ( from app.rag.summarizer import (
IMAGE_FALLBACK_MIME, IMAGE_FALLBACK_MIME,
@@ -461,6 +466,7 @@ def plan_turn(
notes: Sequence[str] | None = None, notes: Sequence[str] | None = None,
kb_overview: str | None = None, kb_overview: str | None = None,
folder_lines: Sequence[str] = (), folder_lines: Sequence[str] = (),
names: Mapping[str, SourceLabelView] | None = None,
) -> TurnPlan: ) -> TurnPlan:
"""Apply the honesty gate (A8, revised) and assemble prompt + context. """Apply the honesty gate (A8, revised) and assemble prompt + context.
@@ -502,6 +508,15 @@ def plan_turn(
phase-118 shape. The :class:`TurnPlan`, the durable record, and the phase-118 shape. The :class:`TurnPlan`, the durable record, and the
per-turn log line are untouched. per-turn log line are untouched.
*names* (phase 191, D4): the source-name → ``SourceLabelView`` map of
the owner's label rows — the SAME single ``all_labels`` read the
endpoint performs for the task-04 selection boost (ONE query per
turn). The HIGH branch hands it to ``build_high_prompt`` (the
additive ``name`` / ``label`` attributes of the ``<document>``
blocks); the LOW branch IGNORES it — the deflected prompt stays
byte-identical. ``None`` ⇒ every block is byte-identical to
pre-phase.
``summary_hits`` (phase 30; phase 118 redefinition) counts the hit ``summary_hits`` (phase 30; phase 118 redefinition) counts the hit
chunks with ``is_summary`` whose parent document is in the chunks with ``is_summary`` whose parent document is in the
SUGGESTED set — both the HIGH and the LOW branch record it. SUGGESTED set — both the HIGH and the LOW branch record it.
@@ -561,6 +576,10 @@ def plan_turn(
# byte-identical to pre-phase (no clause, and agent_tools # byte-identical to pre-phase (no clause, and agent_tools
# offers four tools, not five). # offers four tools, not five).
show_text=settings.answer_text, show_text=settings.answer_text,
# Phase 191 (D4, task 05): the owner's source names ride the
# same map the boost flags came from — one all_labels read
# per turn, no second label query here.
names=names,
), ),
suggested, suggested,
related_docs, related_docs,
@@ -860,6 +879,33 @@ async def chat(
kb_overview = load_kb_overview(step_db) kb_overview = load_kb_overview(step_db)
with SessionLocal() as step_db: with SessionLocal() as step_db:
chunks = retrieve(step_db, request.message, question_vec) chunks = retrieve(step_db, request.message, question_vec)
# Phase 191 (D3 task 04 + D4 task 05): ONE
# all_labels read per turn feeds BOTH surfaces —
# the source-name hit flags here AND the prompt
# surfaces (the map rides into plan_turn →
# build_high_prompt's name/label attributes). The
# hit set is the pure D2 rule per view, then a
# loop over the few dozen retrieved chunks against
# the handful of hit sources (never a per-chunk
# query — task 01's question_name_hit_sources is
# this same pair, inlined here so the read is
# shared). The flags ride the chunk objects into
# EVERY selection call below (plan_turn's walk
# included); the source_name_hit_bonus itself is
# pulled from settings inside the selection
# helpers. No label rows → empty map → no flags
# and no name attributes → every surface is the
# pre-phase one; the per-turn log line is
# untouched (the boost is invisible to it).
labels = all_labels(step_db)
source_name_hits = {
view.source
for view in labels.values()
if question_name_hits(view, request.message)
}
for rc in chunks:
if rc.document.source in source_name_hits:
rc.source_name_hit = True
# Phase 119 (D3, LOCKED A4): the suggested-folder # Phase 119 (D3, LOCKED A4): the suggested-folder
# context — the SAME deterministic suggested walk # context — the SAME deterministic suggested walk
# plan_turn performs internally (one extra cheap # plan_turn performs internally (one extra cheap
@@ -880,6 +926,10 @@ async def chat(
notes=steering_notes, notes=steering_notes,
kb_overview=kb_overview, kb_overview=kb_overview,
folder_lines=folder_lines, folder_lines=folder_lines,
# Phase 191 (D4, task 05): the SAME single read —
# the HIGH prompt learns the name→docs mapping
# from these views (no second label query).
names=labels,
) )
except Exception: # noqa: BLE001 — DB failure mid-turn except Exception: # noqa: BLE001 — DB failure mid-turn
logger.exception( logger.exception(
app/api/source_labels.py
+249
View File
@@ -0,0 +1,249 @@
"""Source name/label/tags admin API (phase 191, task 02).
``TODO.md`` L4: "Sources need a name, a label, tag(s) so you can focus
the model when prompting if you have a project name or a standard name
you reference things by." Task 01 stored that (the ``source_labels``
table + :mod:`app.rag.source_labels`); this module is its whole HTTP
surface — the API the Sources page's per-source editor (task 03)
drives, admin-gated like the phase-91 Theme tab and the phase-168
Settings panel.
Routes (under ``/api`` via the ``main`` registration — the house prefix
convention, the :mod:`app.api.ui_settings` /
:mod:`app.api.admin_settings` style, so the wire paths are exactly the
D5 ones):
* ``GET /api/source-labels`` — the FULL source catalogue: (a) every
``git_sources`` row's DERIVED source name (git AND local rows —
:func:`app.rag.source_removal.resolve_source_name`, the one helper
that mirrors the import pipeline's naming for both kinds — and, while
the table is empty, the ``BOR_GIT_SOURCES`` env names, which
:func:`app.rag.git_sources.effective_sources` already surfaces as
synthetic git rows), plus (b) every indexed document source no row
derives (the uploads — ONE ``SELECT DISTINCT source``, no ``content``,
the phase-126 load discipline). De-duplicated (the phase-69 sibling
case: two rows, one name) and sorted by ``source``, so the page and
the phase's E2E always see the same order. Each entry carries its
label row or the ``null``/``[]`` defaults — one
:func:`app.rag.source_labels.all_labels` read for the whole list,
never a query per source. Reads only; creates nothing.
* ``PUT /api/source-labels/{source}`` — the upsert. The path param is
capped at the ``documents.source`` 120 (the framework's 422 above
it); a ``source`` matching NO registry row AND NO indexed document is
a 404 BEFORE anything is written (D5: no orphan label rows — a label
for a source that exists nowhere would be a dead boost input, and
nothing in the UI could ever list it). The body is the partial-update
shape (the phase-168 precedent): an omitted field keeps its stored
value, an explicit ``null`` clears it, a value sets it. The tags go
through :func:`app.rag.source_labels.normalize_tags` — the ONE tag
rule (trim, case-insensitive de-dupe, first case wins) — and its
``ValueError`` becomes the framework's 422, NEVER a 500. The router
owns the session here (the phase-91 commit convention): ``set_label``
flushes, this route commits, and a rejected request writes NOTHING.
200 → the stored :class:`SourceLabelOut`.
* ``DELETE /api/source-labels/{source}`` — the reset: the row goes, 404
when there is no row to remove. Like the phase-168
``DELETE /api/admin/settings/{key}`` the 200 body is the entry's
post-reset state — the same shape with the fields cleared — so the
row re-renders from the response it already has.
One invariant the routes exist to protect (D1, owner-confirmed): a
label row is an ALIAS. ``Document.source``, the ``source/path`` document
identity, the read/grep/``show_text`` locators, the citations and the KB
tree are ALL untouched by anything here — calling "gitea-forge" "the git
server" changes only what the lite model and the chat model are TOLD
(tasks 04/05), never what a saved chat, a shared page or a verbatim
panel points at. That is why ``source`` is a path param that is LOOKED
UP and never rewritten, and why the catalogue's identity column stays
the derived name.
Logging (AGENTS rule 10): one INFO line per WRITE, carrying the action
and the source, never the values (the ``admin_settings`` discipline —
the owner's text belongs in the row, not in ``server.log``). GETs log
nothing here: :class:`app.core.response_budget.ResponseBudgetMiddleware`
already writes the per-response ``api method=… bytes=…`` line for every
``/api/*`` route.
"""
from __future__ import annotations
import logging
from typing import Annotated, Any
from fastapi import APIRouter, Depends, HTTPException, Path
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.core.auth import require_admin
from app.db import get_db
from app.models import Document
from app.rag.git_sources import effective_sources
from app.rag.source_labels import (
SourceLabelView,
all_labels,
delete_label,
normalize_tags,
set_label,
)
from app.rag.source_removal import resolve_source_name
from app.schemas import SourceLabelList, SourceLabelOut, SourceLabelUpdate
logger = logging.getLogger("app.api.source_labels")
router = APIRouter(
prefix="/source-labels",
tags=["source-labels"],
dependencies=[Depends(require_admin)], # phase 191: the owner's naming surface
)
def _catalogue_names(db: Session) -> list[str]:
"""Every source the owner can label, de-duplicated and sorted.
Exactly the two reads the D5 catalogue is made of — no third:
(a) the registry: :func:`app.rag.git_sources.effective_sources` —
the ``git_sources`` rows of BOTH kinds in ``(added_at, id)``
order, or the synthetic env rows while the table is empty —
named by :func:`app.rag.source_removal.resolve_source_name`, the
single helper that spells a row's name the way the importer
stores it in ``Document.source`` (``repo_name`` of the URL for
git rows, the expanded path's basename for local rows). Reused,
never re-derived: an expression copied here would drift from the
importer and the label rows would key nothing.
(b) the indexed documents: ``SELECT DISTINCT source`` — one query
over the indexed corpus (the column only, never the entity: no
``content``, phase-126/185 discipline), so an upload — or any
document whose registry row is gone — still lists and stays
labelable.
Sorted because the page renders this order and the phase's E2E pins
it; the set collapses the phase-69 sibling case (two rows resolving
to one name) to a single entry.
"""
rows, _origin = effective_sources(db)
names: set[str] = {resolve_source_name(row) for row in rows}
names.update(db.scalars(select(Document.source).distinct()).all())
return sorted(names)
def _shape(source: str, view: SourceLabelView | None) -> SourceLabelOut:
"""One entry of the catalogue: the label row's values, or the
``null``/``[]`` defaults when the source has no row (the pre-phase
state — indistinguishable from a cleared label, and that is the
point: D1 says a label is data beside the name, never a rename)."""
return SourceLabelOut(
source=source,
display_name=view.display_name if view else None,
label=view.label if view else None,
tags=list(view.tags) if view else [],
)
@router.get("", response_model=SourceLabelList)
def list_source_labels(
db: Session = Depends(get_db), # noqa: B008
) -> SourceLabelList:
"""The catalogue: every source of every kind, each with its label or
the null defaults, sorted by ``source``.
Two reads for the names (:func:`_catalogue_names`) plus ONE for all
the label rows — the table carries a handful of rows, so the list is
assembled in Python instead of issuing a lookup per source (the
phase-119 one-query convention).
The catalogue is the SOURCE list, not the label list: a label row
whose source has since lost both its registry row and its documents
does not appear here (it can boost nothing — the D2 match runs
against retrieved chunks' ``document.source``), and it stays
clearable through ``DELETE`` below, which never consults the
registry.
"""
labels = all_labels(db)
return SourceLabelList(
labels=[_shape(name, labels.get(name)) for name in _catalogue_names(db)]
)
@router.put("/{source}", response_model=SourceLabelOut)
def put_source_label(
source: Annotated[str, Path(max_length=120)],
payload: SourceLabelUpdate,
db: Session = Depends(get_db), # noqa: B008
) -> SourceLabelOut:
"""Set or clear one source's name/label/tags (200 → the stored row).
The order is the D5 contract:
1. **404 first** — *source* matches no registry row name and no
indexed document (the same two reads the GET makes): a label for
a source that exists nowhere is refused and nothing is written.
A typo'd name would otherwise sit in the table with no row to
join, no documents to boost and nothing in the UI to list it.
2. **Then the tags** — :func:`~app.rag.source_labels.normalize_tags`
(trim, drop the entries that are empty afterwards, de-duplicate
case-insensitively with the first case kept); its ``ValueError``
becomes a 422 carrying its own fixed, input-free reason. The
request shape caps the same numbers, so today the raise is the
belt — the pin is that the belt can never fire as a 500.
3. **Then the write** — only the fields the request actually carried
(``model_fields_set`` — the phase-168 partial-update shape:
omitted keeps, ``null`` clears, a value sets) reach
:func:`~app.rag.source_labels.set_label`, and the commit happens
here (the router owns the session, the phase-91 convention). A
rejection at step 1 or 2 leaves any existing row byte-identical.
An all-omitted body is therefore a legitimate 200 no-op (the
``GitSourcePatchIn`` "all absent → 200 no-op" precedent): the row is
touched and reported, nothing changes.
"""
if source not in _catalogue_names(db):
raise HTTPException(status_code=404, detail="source not found")
write: dict[str, Any] = {}
carried = payload.model_fields_set
if "display_name" in carried:
write["display_name"] = payload.display_name
if "label" in carried:
write["label"] = payload.label
if "tags" in carried:
try:
write["tags"] = normalize_tags(payload.tags or [])
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
row = set_label(db, source, **write)
db.commit()
# The action and the key only — never the owner's text (AGENTS rule
# 10, the ``admin_settings`` write-line discipline).
logger.info(
"source labels action=set source=%s fields=%s",
source,
",".join(sorted(write)) or "none",
)
return SourceLabelOut(
source=row.source,
display_name=row.display_name,
label=row.label,
tags=list(row.tags or []),
)
@router.delete("/{source}", response_model=SourceLabelOut)
def delete_source_label(
source: Annotated[str, Path(max_length=120)],
db: Session = Depends(get_db), # noqa: B008
) -> SourceLabelOut:
"""Reset one source — the row goes and the next turn sees the
pre-phase state.
404 (one fixed detail, ``source label not found``) when there is no
label row for it. Unlike the PUT this does NOT consult the registry
or the documents: the reset has to work for a row whose source has
since been pruned or whose registry row was deleted (phase 69's
total removal), or the label would be un-clearable by construction.
"""
if not delete_label(db, source):
raise HTTPException(status_code=404, detail="source label not found")
db.commit()
logger.info("source labels action=clear source=%s", source)
return SourceLabelOut(source=source, display_name=None, label=None, tags=[])
app/config.py
+28
View File
@@ -430,6 +430,23 @@ class Settings(BaseSettings):
#: records the realized margins). #: records the realized margins).
name_hit_bonus: float = 0.005 name_hit_bonus: float = 0.005
#: (phase 191, D3, LOCKED A3): the SECOND, independent selection-time
#: bonus — a document whose SOURCE is named by the question (the D2
#: rule over the ``source_labels`` row: a question token equals, in
#: normalized form, a token of the source's display_name / label /
#: tags) gets this added to its best fused chunk score in the SAME
#: shared selection walk as the phase-119 ``name_hit_bonus`` (a
#: separate knob: the two signals stay independently kill-able).
#: Default 0.005 = the phase-119 starting point; ``0`` = off (the
#: walk returns byte-identical to pre-phase — the kill switch); a
#: negative value fails startup loudly. Bounded per DOCUMENT, applied
#: ONCE, at most TWO bonuses on the fused scale (name_hit + source
#: name_hit = 0.010 maximum head start — no amplification); the
#: bonus lives in the SELECTION layer only — ``score`` / ``cosine`` /
#: ``fts_hit`` / ``query_log.top_score`` are untouched (the
#: phase-119 LOCKED A3 clause, mirrored).
source_name_hit_bonus: float = 0.005
# --- Admin & sign-in (phase 16; A10 revised 2026-08-22) --- # --- Admin & sign-in (phase 16; A10 revised 2026-08-22) ---
# Single-admin auth via a signed session cookie (Starlette # Single-admin auth via a signed session cookie (Starlette
# SessionMiddleware — no new services, no DB tables). Both secrets are # SessionMiddleware — no new services, no DB tables). Both secrets are
@@ -972,6 +989,17 @@ class Settings(BaseSettings):
raise ValueError("name_hit_bonus must be >= 0 (0 = off)") raise ValueError("name_hit_bonus must be >= 0 (0 = off)")
return v return v
@field_validator("source_name_hit_bonus")
@classmethod
def _source_name_hit_bonus_non_negative(cls, v: float) -> float:
"""``0`` is the byte-identical kill switch (the pre-phase-191
selection order) — a NEGATIVE bonus would demote source-named
documents, the exact opposite of D2/D3 (the ``name_hit_bonus``
pattern, phase 191)."""
if v < 0:
raise ValueError("source_name_hit_bonus must be >= 0 (0 = off)")
return v
@field_validator("db_pool_size") @field_validator("db_pool_size")
@classmethod @classmethod
def _db_pool_size_positive(cls, v: int) -> int: def _db_pool_size_positive(cls, v: int) -> int:
app/core/settings_catalog.py
+7
View File
@@ -225,6 +225,13 @@ CATALOG: dict[str, SettingMeta] = {
"name_hit_bonus": SettingMeta( "name_hit_bonus": SettingMeta(
"name_hit_bonus", "Retrieval & honesty", "Name hit bonus" "name_hit_bonus", "Retrieval & honesty", "Name hit bonus"
), ),
# Phase 191 (D3): the second, independent selection bonus — the SAME
# live shape as name_hit_bonus (the selection walk reads
# get_settings() per turn, so a saved override applies on the next
# question).
"source_name_hit_bonus": SettingMeta(
"source_name_hit_bonus", "Retrieval & honesty", "Source name hit bonus"
),
# --- Chunking & embedding --- # --- Chunking & embedding ---
"chunk_target_chars": SettingMeta( "chunk_target_chars": SettingMeta(
"chunk_target_chars", "Chunking & embedding", "Chunk target chars" "chunk_target_chars", "Chunking & embedding", "Chunk target chars"
app/main.py
+6
View File
@@ -39,6 +39,7 @@ from app.api.doc_drafts import router as doc_drafts_router
from app.api.docs import router as docs_router from app.api.docs import router as docs_router
from app.api.git_sources import router as git_sources_router from app.api.git_sources import router as git_sources_router
from app.api.health import router as health_router from app.api.health import router as health_router
from app.api.source_labels import router as source_labels_router
from app.api.steering import router as steering_router from app.api.steering import router as steering_router
from app.api.suggestions import router as suggestions_router from app.api.suggestions import router as suggestions_router
from app.api.sync import router as sync_router from app.api.sync import router as sync_router
@@ -234,6 +235,11 @@ def create_app() -> FastAPI:
# the app_settings overrides — DB-over-env, live on Save) — admin-only # the app_settings overrides — DB-over-env, live on Save) — admin-only
# (router-wide require_admin; a token user stays 403). # (router-wide require_admin; a token user stays 403).
app.include_router(admin_settings_router, prefix="/api") app.include_router(admin_settings_router, prefix="/api")
# Phase 191 (task 02): the per-source name/label/tags surface (the
# catalogue GET + the upsert + the reset) — admin-only (router-wide
# require_admin; a token user stays 403). The display name is an
# ALIAS beside the derived source name, never a rename (D1).
app.include_router(source_labels_router, prefix="/api")
# Phase 51: the anonymous shared-chat read — NO admin dependency. # Phase 51: the anonymous shared-chat read — NO admin dependency.
# /api/shared/<token> is the JSON snapshot; /shared/<token> (the # /api/shared/<token> is the JSON snapshot; /shared/<token> (the
# page route below, registered without a prefix) is the page. # page route below, registered without a prefix) is the page.
app/models.py
+62
View File
@@ -77,6 +77,14 @@ Data model — see ``.agents/PLAN.md`` §Data Model:
name, the value in its env-native string form; name, the value in its env-native string form;
the row WINS over the environment (resolved the row WINS over the environment (resolved
inside ``app.config.get_settings``). inside ``app.config.get_settings``).
* ``source_labels`` — one row per source the owner has named (phase
191, TODO.md L4): the display name ("the git
server"), the short label/category, and the tag
list used to focus the model when prompting.
``source`` mirrors ``documents.source`` (the
DERIVED name) and is an ALIAS key only — a label
row never renames a document, a locator, or a
citation (phase 191 D1).
""" """
from __future__ import annotations from __future__ import annotations
@@ -717,3 +725,57 @@ class AppSetting(Base):
updated_at: Mapped[datetime] = mapped_column( updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False DateTime(timezone=True), server_default=func.now(), nullable=False
) )
class SourceLabel(Base):
"""One owner-given name / label / tag set per source (phase 191, task 01).
``TODO.md`` L4: "Sources need a name, a label, tag(s) so you can
focus the model when prompting if you have a project name or a
standard name you reference things by." One row per SOURCE (not per
document), keyed by the source name exactly as ``Document.source``
carries it (the derived basename — the clone dir / local dir /
upload file stem), so the row joins the documents and the
``git_sources`` row without a second identity.
**The display name is an ALIAS ONLY (D1, owner-confirmed):**
``Document.source``, the ``source/path`` document identity, the
read/grep/``show_text`` locators, the citations and the KB tree are
ALL untouched by a label row — renaming a source never invalidates
a saved chat, a shared page or a verbatim locator. The derived name
stays the wire identity; the display name rides beside it and is
consumed by the retrieval boost (task 04) and the prompt surfacing
(task 05) only.
No row = no name/label/tags = the pre-phase behavior exactly (the
migration creates an EMPTY table — no backfill, A13 reversible).
"""
__tablename__ = "source_labels"
#: Mirrors ``documents.source`` (String(120)) — the DERIVED source
#: name, the join key and the sole PK (one label row per source;
#: the ``folder_summaries`` PK-part precedent).
source: Mapped[str] = mapped_column(String(120), primary_key=True)
#: The owner's name for the source ("the git server"). Plain TEXT —
#: the ≤ 120-char trimmed limit is enforced at the API boundary
#: (the phase-121 ``git_sources.branch`` pattern: plain column,
#: boundary validation). NULL = no display name (pre-phase).
display_name: Mapped[str | None] = mapped_column(Text, default=None)
#: The short one-line description / category ("self-hosted forge +
#: Gitea"). Plain TEXT, ≤ 200 trimmed enforced at the API. NULL =
#: no label.
label: Mapped[str | None] = mapped_column(Text, default=None)
#: The focus tokens (the owner's "tag(s)"): ≤ 10 entries, each ≤ 32
#: chars, non-empty, de-duplicated case-insensitively — normalized
#: and guarded at the API boundary (``app.rag.source_labels.normalize_tags``).
#: JSONB NOT NULL with server default ``'[]'`` (the phase-89
#: ``ignore_paths`` shape): a source with no tags reads ``[]``.
tags: Mapped[list] = mapped_column(
JSONB, default=list, server_default=text("'[]'"), nullable=False
)
#: Fresh stamp on every write (the ``sources_meta.updated_at`` house
#: style: server-side now(), onupdate now()).
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), onupdate=func.now()
)
app/rag/overview.py
+43 -4
View File
@@ -43,6 +43,14 @@ tail (the model input stays byte-identical). The longest-prefix cap in
one-line tail (~15 chars/source) always fit; a smaller owner-set cap one-line tail (~15 chars/source) always fit; a smaller owner-set cap
cuts the tail first — accepted, the cap is owner-tunable. cuts the tail first — accepted, the cap is owner-tunable.
Source names (phase 191, D4): when a source has a label row with a
``display_name``, the INPUT line of its documents leads with that name
instead of the derived source name (``build_overview_prompt``'s *names*
map, derived from the ONE ``all_labels`` read :func:`regenerate_overview`
performs) — the owner's vocabulary reaches the outline. The stored text,
the catalogue rows and the deterministic ``Sources: …`` inventory tail
keep the DERIVED name (D1 — a label is an alias, never an identity).
The ``KB_OVERVIEW_MODE`` marker follows the ``SUMMARY_MODE`` / The ``KB_OVERVIEW_MODE`` marker follows the ``SUMMARY_MODE`` /
``DEFLECT_MODE`` convention: the deterministic E2E mock LLM keys on it ``DEFLECT_MODE`` convention: the deterministic E2E mock LLM keys on it
in the system prompt (wired in phase 31, task 05). in the system prompt (wired in phase 31, task 05).
@@ -50,7 +58,7 @@ in the system prompt (wired in phase 31, task 05).
from __future__ import annotations from __future__ import annotations
import logging import logging
from collections.abc import Sequence from collections.abc import Mapping, Sequence
from datetime import UTC, datetime from datetime import UTC, datetime
from typing import Protocol from typing import Protocol
@@ -62,6 +70,7 @@ from app.db import SessionLocal
from app.models import Document, KbOverview from app.models import Document, KbOverview
from app.rag.llm import LLMError from app.rag.llm import LLMError
from app.rag.retriever import TRUNCATION_MARKER from app.rag.retriever import TRUNCATION_MARKER
from app.rag.source_labels import all_labels
logger = logging.getLogger("app.rag.overview") logger = logging.getLogger("app.rag.overview")
@@ -146,7 +155,9 @@ def _source_inventory(rows: Sequence[tuple[str, str, str, str | None]]) -> str:
def build_overview_prompt( def build_overview_prompt(
rows: Sequence[tuple[str, str, str, str | None]], max_chars: int | None = None rows: Sequence[tuple[str, str, str, str | None]],
max_chars: int | None = None,
names: Mapping[str, str] | None = None,
) -> tuple[str, str]: ) -> tuple[str, str]:
"""The ``(system, user)`` message pair for one overview call. """The ``(system, user)`` message pair for one overview call.
@@ -166,11 +177,25 @@ def build_overview_prompt(
cap and the shared ``[…truncated…]`` marker is appended on its own cap and the shared ``[…truncated…]`` marker is appended on its own
line, so the model never sees more than the cap and the cut is line, so the model never sees more than the cap and the cut is
visible (summarizer convention, phase 30). visible (summarizer convention, phase 30).
*names* (phase 191, D4) maps a DERIVED source name to the owner's
display name for it: the input line leads with the display name
instead of the raw source name, so the ``lite`` outline is written in
the owner's vocabulary ("the git server" rather than
``gitea-infra-docs``). The alias contract (D1) is untouched — the
``path`` / ``title`` / summary halves of the line, the stored overview
row and its ``Sources: …`` inventory tail all keep the derived name;
only this model input changes. ``None`` / an empty map / no entry for
a source (or an entry with an empty name) → the pre-phase line,
byte-identical.
""" """
limit = max_chars if max_chars is not None else get_settings().overview_input_max_chars limit = max_chars if max_chars is not None else get_settings().overview_input_max_chars
lines: list[str] = [] lines: list[str] = []
for source, path, title, summary in rows: for source, path, title, summary in rows:
line = f"{source} — {path} — {title}" # Phase 191 (D4): the display name REPLACES the raw source name in
# the lite model's input only — never in the stored outline.
display = names.get(source) if names else None
line = f"{display or source} — {path} — {title}"
first = _first_summary_line(summary) first = _first_summary_line(summary)
if first: if first:
line += f" — {first}" line += f" — {first}"
@@ -220,6 +245,10 @@ async def regenerate_overview(
never sees the tail; the zero-doc and ``LLMError`` paths below never sees the tail; the zero-doc and ``LLMError`` paths below
leave a previous row byte-identical — an existing tail is part of leave a previous row byte-identical — an existing tail is part of
the stored content and is preserved on failure. the stored content and is preserved on failure.
* Phase 191 (D4): the ONE :func:`app.rag.source_labels.all_labels`
read happens AFTER the zero-doc gate (no label query when there is
nothing to outline) and feeds only the model input — the stored
content and its inventory tail stay derived-name text.
* :class:`LLMError` → ``overview: regeneration failed — …`` is * :class:`LLMError` → ``overview: regeneration failed — …`` is
logged and ``False`` is returned with the previous row intact — logged and ``False`` is returned with the previous row intact —
an old outline is better than none. an old outline is better than none.
@@ -249,7 +278,17 @@ async def regenerate_overview(
) )
return False return False
system, user = build_overview_prompt(rows) # Phase 191 (D4, task 05): ONE all_labels read for the whole run —
# the display-name map for the model input is derived from these
# views in this loop, never re-queried (the same single-read
# contract the chat turn holds, task 04/05). Rows without a
# display name are absent from the map, so their lines stay
# byte-identical to pre-phase.
labels = all_labels(session)
name_map = {
src: view.display_name for src, view in labels.items() if view.display_name
}
system, user = build_overview_prompt(rows, names=name_map)
try: try:
text = await llm.chat( text = await llm.chat(
[{"role": "system", "content": system}, {"role": "user", "content": user}], [{"role": "system", "content": system}, {"role": "user", "content": user}],
app/rag/prompts.py
+76 -2
View File
@@ -100,16 +100,32 @@ owner decision). The deterministic lever is the honesty gate itself
(A8 revised 2026-09-14: an FTS hit flips HIGH only when (A8 revised 2026-09-14: an FTS hit flips HIGH only when
``best_cosine >= lexical_support_floor``, so the misleading docs are ``best_cosine >= lexical_support_floor``, so the misleading docs are
no longer injected — see :mod:`app.api.chat`). no longer injected — see :mod:`app.api.chat`).
Source names, labels and tags (phase 191, D4): a ``<document>`` block can
carry the owner's label row for its source as two ADDITIVE attributes —
``name="…"`` (the source's ``display_name``) immediately after
``source="…"``, then ``label="…"`` (the one-line description) — so the
model learns the name→docs mapping ("the git server" is a source with
these documents). The attribute order is fixed
(``source`` → ``name?`` → ``label?`` → ``path`` → ``title`` → ``date``),
an unset field adds NOTHING, and no *names* map / no entry for the
source leaves the block byte-identical to pre-phase (the alias contract,
D1: the identity attributes stay the DERIVED name — a display name never
replaces ``source`` or ``path``). Only the two new values are XML-escaped
(they are free-form owner text); the identity attributes keep their
pre-phase unescaped bytes.
""" """
from __future__ import annotations from __future__ import annotations
import re import re
from collections.abc import Sequence from collections.abc import Mapping, Sequence
from typing import Any from typing import Any
from xml.sax.saxutils import escape
from app.config import Settings, get_settings from app.config import Settings, get_settings
from app.models import Document from app.models import Document
from app.rag.retriever import TRUNCATION_MARKER from app.rag.retriever import TRUNCATION_MARKER
from app.rag.source_labels import SourceLabelView
from app.schemas import HistoryTurn from app.schemas import HistoryTurn
#: PLAN §6 verbatim (line wrapping included); ``{relevance}`` is filled by #: PLAN §6 verbatim (line wrapping included); ``{relevance}`` is filled by
@@ -769,6 +785,47 @@ def _extract_context_line(image_context: str) -> str | None:
) )
def _attr(value: str) -> str:
"""XML-escape one attribute value (phase 191, D4) — ``&``/``<``/``>``
and the double quote (``&quot;``), the delimiter the block uses.
Applied ONLY to the two additive label attributes (``name`` /
``label``): they hold free-form OWNER text where a ``"`` is realistic
and must not break the attribute. The pre-existing identity
attributes (``source`` / ``path`` / ``title`` / ``date``) deliberately
keep their unescaped pre-phase bytes — escaping them here would move
every label-less block, which the phase's byte-identity criterion
forbids.
"""
return escape(value, {'"': "&quot;"})
def _source_label_attrs(
doc: Document, names: Mapping[str, SourceLabelView] | None
) -> str:
"""The additive `` name="…"`` / `` label="…"`` fragment of one
``<document>`` block (phase 191, D4) — the empty string unless the
source has a label row with that field set.
``None`` / an empty map / no entry for ``doc.source`` / a row with
neither field set all render ``""`` — the block is then byte-identical
to pre-phase, and a label-only row adds exactly the one ``label``
attribute (an unset field adds NOTHING, it is never an empty
attribute).
"""
if not names:
return ""
view = names.get(doc.source)
if view is None:
return ""
attrs = ""
if view.display_name:
attrs += f' name="{_attr(view.display_name)}"'
if view.label:
attrs += f' label="{_attr(view.label)}"'
return attrs
def _image_context_line(doc: Document) -> str | None: def _image_context_line(doc: Document) -> str | None:
"""The phase-125 context line for one suggested document (task 05, """The phase-125 context line for one suggested document (task 05,
LOCKED A4) — ``None`` in the common case (EVERY text doc, and every LOCKED A4) — ``None`` in the common case (EVERY text doc, and every
@@ -832,6 +889,7 @@ def build_high_prompt(
folder_lines: Sequence[str] = (), folder_lines: Sequence[str] = (),
show_image: bool = False, show_image: bool = False,
show_text: bool = False, show_text: bool = False,
names: Mapping[str, SourceLabelView] | None = None,
) -> str: ) -> str:
"""Grounded turn: locked persona (+ steering, + KB overview) + the """Grounded turn: locked persona (+ steering, + KB overview) + the
top-ranked documents' SUMMARY blocks + the ``<tools>`` instructions top-ranked documents' SUMMARY blocks + the ``<tools>`` instructions
@@ -917,6 +975,21 @@ def build_high_prompt(
(:func:`_document_body`; the NULL/whitespace-summary preview (:func:`_document_body`; the NULL/whitespace-summary preview
fallback is LOCKED A5; an enriched image doc's summary leads with fallback is LOCKED A5; an enriched image doc's summary leads with
the phase-125 one-line context line, LOCKED A4). the phase-125 one-line context line, LOCKED A4).
*names* (phase 191, D4): the source-name → :class:`SourceLabelView`
map of the owner's label rows (the SAME single ``all_labels`` read
that feeds the task-04 selection boost — one query per turn). When a
doc's source has a row, the block head gains `` name="{display_name}"``
immediately after ``source="…"`` and `` label="{label}"`` after that
(the fixed order ``source`` → ``name?`` → ``label?`` → ``path`` →
``title`` → ``date``); the two values are XML-escaped
(:func:`_attr`), the identity attributes are not. An unset field adds
NOTHING, so a label-only row gains exactly the one attribute, and
``None`` / ``{}`` / no entry for the source keeps the block
byte-identical to pre-phase (the default, and the kill pin in
:mod:`tests.unit.test_prompts`). This is surfacing only — the alias
contract (D1) keeps ``source`` / ``path`` as the document identity
the tools, citations and locators use.
""" """
# Phase 106 (D5): every document the model sees carries its # Phase 106 (D5): every document the model sees carries its
# creation date — the block's ``date`` attribute (the row's # creation date — the block's ``date`` attribute (the row's
@@ -926,7 +999,8 @@ def build_high_prompt(
# stays out of the prompt (the capped ``read`` tool is the only # stays out of the prompt (the capped ``read`` tool is the only
# full-text path). # full-text path).
blocks = [ blocks = [
f'<document source="{doc.source}" path="{doc.path}" title="{doc.title}" ' f'<document source="{doc.source}"{_source_label_attrs(doc, names)}'
f' path="{doc.path}" title="{doc.title}" '
f'date="{doc.created_at:%Y-%m-%d}">\n' f'date="{doc.created_at:%Y-%m-%d}">\n'
f"{_document_body(doc)}\n" f"{_document_body(doc)}\n"
"</document>" "</document>"
app/rag/retriever.py
+75 -17
View File
@@ -291,6 +291,16 @@ class RetrievedChunk:
:func:`fuse`'s ``replace()`` copies it (the double-hit merge ORs it :func:`fuse`'s ``replace()`` copies it (the double-hit merge ORs it
in — a vector row that is also the name hit's representative chunk in — a vector row that is also the name hit's representative chunk
keeps the flag). keeps the flag).
* ``source_name_hit`` — True for a chunk whose parent document's
SOURCE the question named (phase 191, D2/D3): the flag is set at
the retrieval boundary in ``app/api/chat.py`` from the turn's ONE
``all_labels`` read (the pure D2 rule per view — the same read that
feeds the ``name``/``label`` prompt attributes, task 05) — the
selection tier's SECOND bonus input (``source_name_hit_bonus``, a
separate knob). Default
``False`` keeps every ordinary construction valid; :func:`fuse`'s
``replace()`` copies it (the double-hit merge ORs it in, mirroring
``name_hit``).
""" """
chunk_id: uuid.UUID chunk_id: uuid.UUID
@@ -302,6 +312,7 @@ class RetrievedChunk:
fts_hit: bool = False fts_hit: bool = False
is_summary: bool = False is_summary: bool = False
name_hit: bool = False name_hit: bool = False
source_name_hit: bool = False
def lexical_tsquery(question: str) -> str | None: def lexical_tsquery(question: str) -> str | None:
@@ -339,9 +350,11 @@ def fuse(
Lexical-only hits (no vector rank) enter with ``cosine=0.0`` and Lexical-only hits (no vector rank) enter with ``cosine=0.0`` and
``fts_hit=True``; vector chunks matched by the lexical list get ``fts_hit=True``; vector chunks matched by the lexical list get
``fts_hit=True`` in place (the input objects are mutated — callers ``fts_hit=True`` in place (the input objects are mutated — callers
should not reuse them afterwards), and ``name_hit=True`` is ORed in should not reuse them afterwards), ``name_hit=True`` is ORed in
(a vector row that is also the name hit's representative chunk is a (a vector row that is also the name hit's representative chunk is a
name-hit row — the phase-119 selection tier must see it). name-hit row — the phase-119 selection tier must see it), and
``source_name_hit`` is ORed in the SAME way (phase 191, D3 — a
vector row merged with a flagged source-name row keeps the flag).
""" """
if k <= 0: if k <= 0:
raise ValueError("rrf k must be > 0") raise ValueError("rrf k must be > 0")
@@ -355,7 +368,10 @@ def fuse(
if rc.chunk_id in by_id: if rc.chunk_id in by_id:
existing = by_id[rc.chunk_id] existing = by_id[rc.chunk_id]
by_id[rc.chunk_id] = replace( by_id[rc.chunk_id] = replace(
existing, fts_hit=True, name_hit=existing.name_hit or rc.name_hit existing,
fts_hit=True,
name_hit=existing.name_hit or rc.name_hit,
source_name_hit=existing.source_name_hit or rc.source_name_hit,
) )
fused[rc.chunk_id] += term fused[rc.chunk_id] += term
else: else:
@@ -754,17 +770,23 @@ def retrieve(
def _selection_order( def _selection_order(
chunks: Sequence[RetrievedChunk], chunks: Sequence[RetrievedChunk],
bonus: float, bonus: float,
source_bonus: float = 0.0,
) -> list[tuple[Document, float, float, int]]: ) -> list[tuple[Document, float, float, int]]:
"""The shared selection walk of the phase-119 name-hit bonus (D2, """The shared selection walk of the phase-119 name-hit bonus (D2,
LOCKED A3) — one walk for ``select_suggested``, ``select_related``, LOCKED A3) — one walk for ``select_suggested``, ``select_related``,
and ``weak_hit_titles``. and ``weak_hit_titles``, carrying the phase-191 source-name bonus
(D3) as its SECOND, independent knob.
Returns ``(document, effective_score, best_cosine, first_seen_index)`` Returns ``(document, effective_score, best_cosine, first_seen_index)``
for each distinct document, where *effective_score* is the for each distinct document, where *effective_score* is the
document's best fused chunk score plus *bonus* when ANY of its document's best fused chunk score plus *bonus* when ANY of its
chunks carries ``name_hit`` (the D1 path match — the bonus is per chunks carries ``name_hit`` (the D1 path match — the bonus is per
DOCUMENT, applied ONCE no matter how many of the document's chunks DOCUMENT, applied ONCE no matter how many of the document's chunks
are name hits). are name hits), plus *source_bonus* when ANY of its chunks carries
``source_name_hit`` (the phase-191 D2 source-name match — the SAME
per-DOCUMENT, apply-ONCE rule; at most TWO bonuses per document
on the fused scale, bounded: name_hit_bonus + source_name_hit_bonus
= 0.010 at the shipped defaults, never more).
The walk keeps the EXISTING selection semantics (not just the loop): The walk keeps the EXISTING selection semantics (not just the loop):
the same stable score-descending order as the same stable score-descending order as
@@ -773,16 +795,18 @@ def _selection_order(
best cosine tracked across ALL of its chunks (the tiered walk's best cosine tracked across ALL of its chunks (the tiered walk's
tracking). tracking).
* *bonus* ``== 0`` (the kill switch) or no name-hit chunk present: * *bonus* ``== 0`` (the kill switch) or no name-hit chunk present,
AND *source_bonus* ``== 0`` or no source-name-hit chunk present:
the document order is IDENTICAL to the pre-phase walk — no re-sort the document order is IDENTICAL to the pre-phase walk — no re-sort
happens at all (byte-identical, LOCKED A3); happens at all (byte-identical, LOCKED A3, mirrored for the second
bonus by LOCKED A3 of phase 191);
* otherwise the documents are ordered by * otherwise the documents are ordered by
``(−effective_score, −best_cosine, document.path, ``(−effective_score, −best_cosine, document.path,
first_seen_index)`` — the bounded re-rank: a name-hit document first_seen_index)`` — the bounded re-rank: a name-hit document
gets a head start on the fused scale, and an effective-score tie gets a head start on the fused scale, and an effective-score tie
resolves by cosine, then path, then the pre-bonus rank. resolves by cosine, then path, then the pre-bonus rank.
The bonus lives in the SELECTION layer only: the chunk objects are The bonuses live in the SELECTION layer only: the chunk objects are
never modified — their ``score``/``cosine``/``fts_hit`` (the A8 never modified — their ``score``/``cosine``/``fts_hit`` (the A8
gate's inputs) and ``query_log.top_score`` are untouched. gate's inputs) and ``query_log.top_score`` are untouched.
""" """
@@ -790,6 +814,7 @@ def _selection_order(
best_score: dict[uuid.UUID, float] = {} best_score: dict[uuid.UUID, float] = {}
best_cosine: dict[uuid.UUID, float] = {} best_cosine: dict[uuid.UUID, float] = {}
any_name_hit: dict[uuid.UUID, bool] = {} any_name_hit: dict[uuid.UUID, bool] = {}
any_source_hit: dict[uuid.UUID, bool] = {}
first_seen: dict[uuid.UUID, int] = {} first_seen: dict[uuid.UUID, int] = {}
for idx, rc in enumerate(sorted(chunks, key=lambda c: c.score, reverse=True)): for idx, rc in enumerate(sorted(chunks, key=lambda c: c.score, reverse=True)):
doc = rc.document doc = rc.document
@@ -798,17 +823,24 @@ def _selection_order(
best_cosine[doc.id] = rc.cosine best_cosine[doc.id] = rc.cosine
if rc.name_hit: if rc.name_hit:
any_name_hit[doc.id] = True any_name_hit[doc.id] = True
if rc.source_name_hit:
any_source_hit[doc.id] = True
else: else:
order.append(doc) order.append(doc)
best_score[doc.id] = rc.score best_score[doc.id] = rc.score
best_cosine[doc.id] = rc.cosine best_cosine[doc.id] = rc.cosine
any_name_hit[doc.id] = rc.name_hit any_name_hit[doc.id] = rc.name_hit
any_source_hit[doc.id] = rc.source_name_hit
first_seen[doc.id] = idx first_seen[doc.id] = idx
effective = { effective = {
doc.id: best_score[doc.id] + (bonus if any_name_hit[doc.id] else 0.0) doc.id: best_score[doc.id]
+ (bonus if any_name_hit[doc.id] else 0.0)
+ (source_bonus if any_source_hit[doc.id] else 0.0)
for doc in order for doc in order
} }
if bonus > 0.0 and any(any_name_hit[doc.id] for doc in order): if (bonus > 0.0 and any(any_name_hit[doc.id] for doc in order)) or (
source_bonus > 0.0 and any(any_source_hit[doc.id] for doc in order)
):
order.sort( order.sort(
key=lambda doc: ( key=lambda doc: (
-effective[doc.id], -effective[doc.id],
@@ -826,17 +858,26 @@ def _selection_order(
def weak_hit_titles( def weak_hit_titles(
chunks: Sequence[RetrievedChunk], chunks: Sequence[RetrievedChunk],
bonus: float | None = None, bonus: float | None = None,
source_bonus: float | None = None,
) -> list[str]: ) -> list[str]:
"""Distinct parent-document titles of *chunks*, best SELECTION score """Distinct parent-document titles of *chunks*, best SELECTION score
first (the phase-119 name-hit bonus applied — see first (the phase-119 name-hit bonus and the phase-191 source-name
:func:`_selection_order`, D2, LOCKED A3). bonus applied — see :func:`_selection_order`, D2/D3, LOCKED A3).
Deflection mode (PLAN §6, A8) is built from these *titles only* — the Deflection mode (PLAN §6, A8) is built from these *titles only* — the
LOW prompt and the "Maybe try" chips never see document content. LOW prompt and the "Maybe try" chips never see document content.
*source_bonus* defaults to the ``BOR_SOURCE_NAME_HIT_BONUS`` setting
(0.005); ``0`` reproduces the pre-phase-191 walk byte-identically.
""" """
if bonus is None: if bonus is None:
bonus = get_settings().name_hit_bonus bonus = get_settings().name_hit_bonus
return [doc.title for doc, _eff, _cos, _idx in _selection_order(chunks, bonus)] if source_bonus is None:
source_bonus = get_settings().source_name_hit_bonus
return [
doc.title
for doc, _eff, _cos, _idx in _selection_order(chunks, bonus, source_bonus)
]
def select_documents_tiered( def select_documents_tiered(
@@ -953,6 +994,7 @@ def select_suggested(
chunks: Sequence[RetrievedChunk], chunks: Sequence[RetrievedChunk],
n: int | None = None, n: int | None = None,
bonus: float | None = None, bonus: float | None = None,
source_bonus: float | None = None,
) -> list[Document]: ) -> list[Document]:
"""Top-N distinct parent documents in SELECTION rank order — the """Top-N distinct parent documents in SELECTION rank order — the
phase-118 "start here" suggestion tier (LOCKED A3, re-revised by phase-118 "start here" suggestion tier (LOCKED A3, re-revised by
@@ -975,7 +1017,10 @@ def select_suggested(
*bonus* defaults to the ``BOR_NAME_HIT_BONUS`` setting (0.005 — the *bonus* defaults to the ``BOR_NAME_HIT_BONUS`` setting (0.005 — the
owner-tunable starting point); ``0`` reproduces the pre-phase walk owner-tunable starting point); ``0`` reproduces the pre-phase walk
byte-identically (the kill switch, LOCKED A3). byte-identically (the kill switch, LOCKED A3). *source_bonus*
defaults to the ``BOR_SOURCE_NAME_HIT_BONUS`` setting (phase 191,
D3 — the same bounded shape, the same kill switch, an independent
knob).
The returned rows carry the full document content, byte-identical — The returned rows carry the full document content, byte-identical —
the content is what the agent's ``read`` tool serves later (never the content is what the agent's ``read`` tool serves later (never
@@ -985,7 +1030,14 @@ def select_suggested(
top_n = n if n is not None else get_settings().suggested_docs top_n = n if n is not None else get_settings().suggested_docs
if bonus is None: if bonus is None:
bonus = get_settings().name_hit_bonus bonus = get_settings().name_hit_bonus
return [doc for doc, _eff, _cos, _idx in _selection_order(chunks, bonus)[:top_n]] if source_bonus is None:
source_bonus = get_settings().source_name_hit_bonus
return [
doc
for doc, _eff, _cos, _idx in _selection_order(chunks, bonus, source_bonus)[
:top_n
]
]
def select_related( def select_related(
@@ -993,6 +1045,7 @@ def select_related(
excluded_ids: set[uuid.UUID], excluded_ids: set[uuid.UUID],
cap: int, cap: int,
bonus: float | None = None, bonus: float | None = None,
source_bonus: float | None = None,
) -> list[Document]: ) -> list[Document]:
"""The documents ranked AFTER *excluded_ids* — the phase-118 related """The documents ranked AFTER *excluded_ids* — the phase-118 related
tier (rank 6+ for the contiguous top-5 suggestion set), up to *cap* tier (rank 6+ for the contiguous top-5 suggestion set), up to *cap*
@@ -1012,7 +1065,10 @@ def select_related(
*bonus* defaults to the ``BOR_NAME_HIT_BONUS`` setting (0.005 — the *bonus* defaults to the ``BOR_NAME_HIT_BONUS`` setting (0.005 — the
owner-tunable starting point); ``0`` reproduces the pre-phase walk owner-tunable starting point); ``0`` reproduces the pre-phase walk
byte-identically (the kill switch, LOCKED A3). byte-identically (the kill switch, LOCKED A3). *source_bonus*
defaults to the ``BOR_SOURCE_NAME_HIT_BONUS`` setting (phase 191,
D3 — the same bounded shape, the same kill switch, an independent
knob).
The returned rows carry the full document content, byte-identical The returned rows carry the full document content, byte-identical
(the tier is metadata for the ``done`` frame and the durable (the tier is metadata for the ``done`` frame and the durable
@@ -1020,8 +1076,10 @@ def select_related(
""" """
if bonus is None: if bonus is None:
bonus = get_settings().name_hit_bonus bonus = get_settings().name_hit_bonus
if source_bonus is None:
source_bonus = get_settings().source_name_hit_bonus
out: list[Document] = [] out: list[Document] = []
for doc, _eff, _cos, _idx in _selection_order(chunks, bonus): for doc, _eff, _cos, _idx in _selection_order(chunks, bonus, source_bonus):
if len(out) >= cap: if len(out) >= cap:
break break
if doc.id in excluded_ids: if doc.id in excluded_ids:
app/rag/source_labels.py
+254
View File
@@ -0,0 +1,254 @@
"""Source name/label/tags storage + the D2 source-name matching rule (phase 191, task 01).
``TODO.md`` L4: "Sources need a name, a label, tag(s) so you can focus
the model when prompting if you have a project name or a standard name
you reference things by." This module is the whole non-HTTP surface of
that feature: the upsert/get/delete helpers over the ``source_labels``
table (migration 0030) and the pure matching rule the retrieval boost
(task 04) consumes.
**The alias contract (D1, owner-confirmed):** a label row is an ALIAS
beside the derived source name — ``Document.source``, the ``source/path``
document identity, the read/grep/``show_text`` locators, the citations
and the KB tree are ALL untouched by it. Renaming a source never
invalidates a saved chat, a shared page or a verbatim locator. Every
function here keys on the DERIVED name (``Document.source``); the
display name is data, never identity.
**The matching rule (D2):** a source is a SOURCE-NAME HIT when at least
one :func:`app.rag.retriever.name_hit_tokens` candidate (the phase-119
class-agnostic token list, reused UNCHANGED) equals — in the
:func:`app.rag.retriever._normalize_name` form, so case-insensitive and
accent-folded (phase 162) — a token of the source's ``display_name``,
``label`` or any ``tags`` entry. The phase-119 digit-class rule does
NOT apply to user-given names (the owner's names are not paths —
"gitea" must hit even though it is digitless; the digit rule exists to
keep prose tokens from matching PATHS). The one filter is
:data:`app.rag.retriever.NAME_TOKEN_MIN_LEN` (4) applied to the SOURCE
side too — a 2-char tag never matches.
Session convention (the phase-53 ``sources_meta`` style): the write
helpers only *flush* — the **caller** commits.
"""
from __future__ import annotations
from collections.abc import Sequence
from dataclasses import dataclass
from typing import Final
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.models import SourceLabel
from app.rag.retriever import NAME_TOKEN_MIN_LEN, _normalize_name, name_hit_tokens
#: Boundary limits (phase 191 D1/D5) — the API (task 02) turns the
#: ``normalize_tags`` raises into the framework's standard 422; the
#: display-name / label caps are enforced there against these constants.
DISPLAY_NAME_MAX_CHARS: Final = 120
LABEL_MAX_CHARS: Final = 200
TAG_MAX_CHARS: Final = 32
TAGS_MAX_COUNT: Final = 10
class _Unset:
"""Marker type for "field omitted" in :func:`set_label`.
The partial-update shape (phase-168 D5): an OMITTED field leaves the
stored value untouched, an explicit ``None`` CLEARS it — the two are
indistinguishable with plain ``None`` defaults, so the sentinel owns
the difference (the API passes only the fields the request carried).
"""
__slots__ = ()
_UNSET: Final = _Unset()
@dataclass(frozen=True)
class SourceLabelView:
"""Immutable read-side snapshot of one label row (D1).
``tags`` is a ``tuple`` so the view stays hashable/mutable-proof
after the ORM row leaves the session; ``display_name`` / ``label``
are ``None`` when unset (never the empty string — the API trims
empty submissions to NULL).
"""
source: str
display_name: str | None
label: str | None
tags: tuple[str, ...]
def _view(row: SourceLabel) -> SourceLabelView:
return SourceLabelView(
source=row.source,
display_name=row.display_name,
label=row.label,
tags=tuple(row.tags or ()),
)
def get_label(db: Session, source: str) -> SourceLabel | None:
"""The ORM row for *source* (one PK read); ``None`` when absent."""
return db.get(SourceLabel, source)
def label_for(db: Session, source: str) -> SourceLabelView | None:
"""The frozen :class:`SourceLabelView` for *source* (one PK read);
``None`` when absent — the "no label row = pre-phase" contract."""
row = db.get(SourceLabel, source)
return None if row is None else _view(row)
def all_labels(db: Session) -> dict[str, SourceLabelView]:
"""Every label row as a frozen view, keyed by ``source``.
ONE query (the phase-119 one-query convention): the table carries a
handful of rows, so callers match in Python against it instead of
issuing a query per source. Ordered by ``source`` for determinism.
"""
rows = db.scalars(select(SourceLabel).order_by(SourceLabel.source)).all()
return {row.source: _view(row) for row in rows}
def set_label(
db: Session,
source: str,
*,
display_name: str | None | _Unset = _UNSET,
label: str | None | _Unset = _UNSET,
tags: Sequence[str] | None | _Unset = _UNSET,
) -> SourceLabel:
"""Upsert the label row for *source*; return the ORM row.
The partial-update shape (phase 191 D5, the phase-168 precedent):
a field left at the ``_UNSET`` sentinel keeps its stored value, an
explicitly passed value OVERWRITES it, an explicit ``None`` CLEARS
(``tags`` clears to ``[]`` — the column is NOT NULL). A missing row
is created with the passed values (omitted fields = their defaults:
``NULL`` / ``[]``). Values arrive ALREADY VALIDATED (the API layer
trims/enforces the caps via :func:`normalize_tags`); this helper
stores what it is given.
Flush-only (the phase-53 session convention): the **caller** commits.
"""
row = db.get(SourceLabel, source)
if row is None:
row = SourceLabel(
source=source,
display_name=None if isinstance(display_name, _Unset) else display_name,
label=None if isinstance(label, _Unset) else label,
tags=[] if isinstance(tags, _Unset) or tags is None else list(tags),
)
db.add(row)
else:
if not isinstance(display_name, _Unset):
row.display_name = display_name
if not isinstance(label, _Unset):
row.label = label
if not isinstance(tags, _Unset):
row.tags = [] if tags is None else list(tags)
db.flush()
return row
def delete_label(db: Session, source: str) -> bool:
"""Remove the label row for *source*; ``False`` when absent (the
API's 404 shape), ``True`` when a row was deleted. Flush-only."""
row = db.get(SourceLabel, source)
if row is None:
return False
db.delete(row)
db.flush()
return True
def normalize_tags(raw: Sequence[str]) -> list[str]:
"""Normalize one tag submission (pure, the API's 422 source).
* trim every entry, drop the ones that are empty afterwards
(``[]`` is legal — it clears the tags);
* raise :class:`ValueError` above :data:`TAGS_MAX_COUNT` entries
(counted on the trimmed non-empty list, BEFORE the case-insensitive
de-dupe: 11 typed tags are 11 rejected tags);
* raise :class:`ValueError` when any entry exceeds
:data:`TAG_MAX_CHARS` afterwards;
* de-duplicate case-insensitively, preserving the FIRST case seen
(``Gitea`` then ``gitea`` keeps ``Gitea``), order otherwise kept.
"""
cleaned = [tag.strip() for tag in raw]
cleaned = [tag for tag in cleaned if tag]
if len(cleaned) > TAGS_MAX_COUNT:
raise ValueError(f"at most {TAGS_MAX_COUNT} tags are allowed")
for tag in cleaned:
if len(tag) > TAG_MAX_CHARS:
raise ValueError(f"each tag must be at most {TAG_MAX_CHARS} characters")
out: list[str] = []
seen: set[str] = set()
for tag in cleaned:
key = tag.casefold()
if key in seen:
continue
seen.add(key)
out.append(tag)
return out
def _source_side_tokens(view: SourceLabelView) -> set[str]:
"""The normalized token set of one label row (the D2 source side).
Every whitespace token of ``display_name`` / ``label`` / every
``tags`` entry, folded through :func:`_normalize_name` (the single
normalization BOTH sides of name matching pass through, phase 162)
and kept at :data:`NAME_TOKEN_MIN_LEN` chars minimum — the 4-char
floor applies to the SOURCE side too (a 2-char tag never matches).
"""
tokens: set[str] = set()
fields: list[str] = [view.display_name or "", view.label or "", *view.tags]
for field in fields:
for raw in field.split():
norm = _normalize_name(raw)
if len(norm) >= NAME_TOKEN_MIN_LEN:
tokens.add(norm)
return tokens
def question_name_hits(view: SourceLabelView | None, question: str) -> bool:
"""The D2 rule (pure): does *question* name the source behind *view*?
``None`` (no label row) → ``False`` — the pre-phase contract. Any
:func:`name_hit_tokens` candidate EQUAL (in the normalized form) to
a source-side token is a hit: no digit-class rule, no prefix match
(user-given names have no prose-precision problem — the phase-119
digit rule guards PATHS, not names).
"""
if view is None:
return False
candidates = name_hit_tokens(question)
if not candidates:
return False
source_tokens = _source_side_tokens(view)
return any(candidate in source_tokens for candidate in candidates)
def question_name_hit_sources(db: Session, question: str) -> set[str]:
"""The derived source names the question names (D2) — the standalone
one-query entry point (no other per-turn work needed).
ONE :func:`all_labels` read + the pure rule per row (a handful of
rows, matched in Python — the phase-119 one-query convention). The
empty set is the common case: no label rows, or no name in the
question.
The chat turn does NOT call this: since task 05 it inlines the same
pair against its own single ``all_labels`` read — the read that also
feeds the prompt surfaces (D4) — so a turn issues ONE label query.
"""
return {
view.source
for view in all_labels(db).values()
if question_name_hits(view, question)
}
app/schemas.py
+115
View File
@@ -2246,3 +2246,118 @@ class AdminSettingsCleared(BaseModel):
deleted.""" deleted."""
cleared: int cleared: int
#: One focus tag (phase 191, D1/D5): the owner's short token, capped at
#: the ``app.rag.source_labels.TAG_MAX_CHARS`` 32 (the ``_Chip`` item-cap
#: precedent — an ``Annotated`` alias, so the JSON shape stays a plain
#: string and the cap rides on every list of tags, request AND response).
_Tag = Annotated[str, Field(min_length=1, max_length=32)]
class SourceLabelOut(BaseModel):
"""One source's name/label/tags (``GET /api/source-labels`` entry,
``PUT``/``DELETE`` response — phase 191, task 02).
``source`` is the DERIVED source name — the name ``Document.source``
carries (the clone dir / local dir / upload file stem): the row's
identity on the wire and the join key to the documents
(phase 191 D1). The display name is an ALIAS beside it: no locator,
citation or KB-tree node is renamed by a label row.
``display_name`` / ``label`` are ``null`` when the owner set nothing
(never the empty string — the request validator folds a
whitespace-only submission to ``None``), ``tags`` ``[]``. No row =
the same null/``[]`` shape: a source without a label row is
indistinguishable from one whose label was cleared, which is exactly
the pre-phase state (D1).
``extra="forbid"`` (the :class:`GitSourceOut` structural contract):
a key outside this shape cannot be built into a response — the
row's ``updated_at`` and anything else that leaks later raise at
construction instead of silently widening the wire shape.
"""
model_config = ConfigDict(extra="forbid")
#: The derived source name (``documents.source``, ``String(120)``).
source: str = Field(max_length=120)
display_name: str | None = Field(default=None, max_length=120)
label: str | None = Field(default=None, max_length=200)
#: At most 10 tags of at most 32 chars each (:data:`_Tag`).
tags: list[_Tag] = Field(default_factory=list, max_length=10)
class SourceLabelList(BaseModel):
"""``GET /api/source-labels`` response (phase 191, task 02): the
whole source catalogue — every ``git_sources`` row (git AND local,
or the ``BOR_GIT_SOURCES`` env names while the table is empty) plus
every indexed document source no row derives (the uploads), one
entry per name, de-duplicated and sorted by ``source`` (the stable
order the Sources page and the phase's E2E rely on). Each entry
carries its label row or the null/``[]`` defaults."""
labels: list[SourceLabelOut]
class SourceLabelUpdate(BaseModel):
"""``PUT /api/source-labels/{source}`` body (phase 191, task 02).
The partial-update shape (D5, the phase-168 precedent): an OMITTED
field leaves the stored value untouched, an explicit ``null`` CLEARS
it, a value SETS it — the two ``None`` legs are distinguishable
because the router reads ``model_fields_set``, not the value.
Every string is trimmed *before* the length constraints run (the
``GitSourceIn._trim_url`` precedent), so a value with surrounding
spaces stores clean and a whitespace-only ``display_name``/``label``
folds to ``None`` — the CLEAR leg, since storing an empty string
would make "no name" and "an empty name" two different things the
prompt builders (task 05) would have to distinguish.
``tags`` carries the RAW list the token input produced. Every entry
is trimmed by ``_trim_tags`` BEFORE the constraints run, so an entry
that is empty afterwards is the framework's standard 422 (an empty
focus token matches nothing — a silent drop would tell the owner
"saved" while their tag vanished); the count cap (10) and the entry
cap (32, :data:`_Tag`) are the same numbers
:func:`app.rag.source_labels.normalize_tags` guards, and the router
still runs the submission through it for the case-insensitive
de-dupe (first case wins) and maps its ``ValueError`` to a 422, so
the rule can never turn into a 500 if this shape is ever relaxed.
``[]`` is legal and clears.
``extra="forbid"`` rejects a key outside the shape (the phase-91
voice) — a client cannot half-write a field nobody declared.
"""
model_config = ConfigDict(extra="forbid")
display_name: str | None = Field(default=None, max_length=120)
label: str | None = Field(default=None, max_length=200)
tags: list[_Tag] | None = Field(default=None, max_length=10)
@field_validator("display_name", "label", mode="before")
@classmethod
def _trim_to_none(cls, v: object) -> object:
"""Trim, and fold an empty result to ``None`` (the CLEAR leg).
``mode='before'`` so the trim runs BEFORE ``max_length`` (the
``GitSourceIn._trim_url`` convention): 121 chars that trim to
120 are legal, 121 chars of text are not."""
if isinstance(v, str):
trimmed = v.strip()
return trimmed or None
return v
@field_validator("tags", mode="before")
@classmethod
def _trim_tags(cls, v: object) -> object:
"""Trim every entry BEFORE the item constraints run (the same
``GitSourceIn`` ordering as :meth:`_trim_to_none`): ``" gitea "``
passes as ``gitea``, ``" "`` becomes ``""`` and the item's
``min_length`` rejects it. Non-list / non-string input is passed
through untouched so pydantic reports it in its own voice."""
if isinstance(v, list):
return [tag.strip() if isinstance(tag, str) else tag for tag in v]
return v
frontend/assets/git-sources.js
+540
View File
@@ -228,6 +228,36 @@
* token-free with no per-site change; the UI must never re-embed a * token-free with no per-site change; the UI must never re-embed a
* credential. * credential.
* *
* Phase 191 (task 03) — the per-source name / label / tags editor (the
* owner-visible half of "Sources need a name, a label, tag(s) …"): a
* SECOND list on the page, its own card below the sources table, whose
* ROW LIST is the phase-191 catalogue GET /api/source-labels (task 02) —
* the union of every registry row's DERIVED source name and every indexed
* document source — so every kind (git, local, upload) gets an editor
* without a second API (loadLabels → renderLabelRows → makeLabelRow, the
* page's ONE <template id="source-label-row-template"> cloned per row).
* The mono identity cell is the derived name, textContent-only and never
* an input: a label row is an ALIAS (D1/A1) — renaming here touches no
* document, citation, KB-tree node, saved chat or source/path locator.
* Each row expands (aria-expanded + aria-controls) into the three fields
* (Name ≤120 / Label ≤200 / the Tags token input — Enter or comma adds a
* chip, Backspace drops the last, ≤10 chips of ≤32 chars, the caps of the
* API's normalize_tags mirrored so an over-long submission never
* round-trips as a 422) + Save / Reset + the row's own §7.4 save-state
* line (role=status): dirty → "unsaved changes" + the .is-dirty row mark
* → "Saving…"/"Resetting…" → "saved" or the ONE-LINE error note, whose
* 404 arm names itself ("That source no longer exists") because a retry
* cannot help — the source left the catalogue. ONE in-flight request per
* ROW (the phase-168 saveInFlight pattern scoped per row — a second click
* while a request is out is ignored); the PUT / DELETE response is the
* new truth (no second GET), Reset clears the fields and the stored label,
* and a collapse discards the draft instead of leaving the row claiming
* something the server never got. The fetch sits inside the existing
* admin-gated content reveal (the page's gate is this list's gate) and
* every source-list change (add / remove / a finished upload) refreshes
* it, so a new source is nameable without a reload. The existing sync /
* upload / git-row flows are untouched — the editor is purely additive.
*
* Phase 135 (task 04) — the branch pin (the field, the tri-state * Phase 135 (task 04) — the branch pin (the field, the tri-state
* asymmetry, the tag rule): the add form gains the optional * asymmetry, the tag rule): the add form gains the optional
* `#git-source-branch` (maxlength 200, autocomplete=off, the visible * `#git-source-branch` (maxlength 200, autocomplete=off, the visible
@@ -354,6 +384,15 @@ export async function mount(root) {
(the checkbox is a table-cell control — no dialog of its own — (the checkbox is a table-cell control — no dialog of its own —
so its failure announces here, next to the table). */ so its failure announces here, next to the table). */
const hiddenErrorEl = root.querySelector("#git-sources-hidden-error"); const hiddenErrorEl = root.querySelector("#git-sources-hidden-error");
/* Phase 191 (task 03): the name/label/tags catalogue — this page's
SECOND list (its own endpoint, its own union of every source kind),
rendered into its own bounded table card. The template is the ONE
row skeleton cloned per source (makeLabelRow). */
const labelsErrorEl = root.querySelector("#source-labels-error");
const labelsWrap = root.querySelector("#source-labels-wrap");
const labelsTbody = root.querySelector("#source-labels-tbody");
const labelsEmptyEl = root.querySelector("#source-labels-empty");
const labelRowTemplate = root.querySelector("#source-label-row-template");
/* Polite live region: the screen-reader confirmation for loads, adds, /* Polite live region: the screen-reader confirmation for loads, adds,
and removals (the phase-15 announcer pattern). */ and removals (the phase-15 announcer pattern). */
@@ -724,6 +763,7 @@ export async function mount(root) {
// app-managed files). // app-managed files).
closeRemoveConfirm(); // focus returns to the row's Remove button closeRemoveConfirm(); // focus returns to the row's Remove button
await loadSources(); // the row leaves the table await loadSources(); // the row leaves the table
await loadLabels(); // phase 191: and its label row leaves with it
// The removal confirmation is the LAST announcement: the // The removal confirmation is the LAST announcement: the
// reload's "N sources listed." must not overwrite it (the // reload's "N sources listed." must not overwrite it (the
// announcer is the screen-reader confirmation for the // announcer is the screen-reader confirmation for the
@@ -1013,6 +1053,497 @@ export async function mount(root) {
if (ignoreBackdrop) ignoreBackdrop.addEventListener("click", cancelIgnoreEditor); if (ignoreBackdrop) ignoreBackdrop.addEventListener("click", cancelIgnoreEditor);
if (ignoreSaveBtn) ignoreSaveBtn.addEventListener("click", saveIgnorePaths); if (ignoreSaveBtn) ignoreSaveBtn.addEventListener("click", saveIgnorePaths);
/* ---------- source names, labels and tags (/api/source-labels) — phase 191 task 03 ----------
* TODO.md L4: "Sources need a name, a label, tag(s) so you can focus the
* model when prompting if you have a project name or a standard name you
* reference things by." Tasks 01/02 stored and exposed that; this is the
* surface the owner types into.
*
* WHY A SECOND LIST (and not a column of #git-sources-table): the row list
* comes from GET /api/source-labels — the union of (a) every registry row's
* DERIVED source name and (b) every indexed document source — so a source
* is nameable even when it has no git_sources row of its own, and the page
* never has to guess the server's naming (the importer's derivation is the
* ONE owner, app/rag/source_removal.resolve_source_name). One extra fetch,
* inside the admin branch only (the anonymous path returns at the gate and
* this file never asks).
*
* WHY THE IDENTITY CELL IS READ-ONLY (D1/A1, owner-confirmed): the derived
* name is the wire identity of every document, citation, KB-tree node,
* saved chat, shared page and source/path locator. The name typed here is
* an ALIAS the prompt layers read (tasks 04/05) — never a rename — so
* nothing in this block writes, resolves or joins on it: it is looked up
* (encodeURIComponent in the path) exactly as the API's PUT/DELETE take it.
*
* THE SAVE CONTRACT (§7.4 never-stale, the phase-168 shape scoped PER ROW):
* an edit marks the row (.is-dirty) + "unsaved changes" + enables that row's
* Save; Save/Reset set the row's own `saveInFlight` (a second click while a
* request is out is IGNORED, and the in-flight label owns the status line),
* the row settles on "saved" or the one-line error note — never a stuck
* "Saving…" on ANY outcome (the finally always drops the flag). The PUT /
* DELETE RESPONSE is the new truth (no second GET — task 03): the stored
* label, the fields, the state tag and the summary all re-paint from the
* body the server just wrote, so the row can never show something the
* server does not have. Collapsing the editor DISCARDS the draft (no
* request) and refills from the stored label for the same reason. */
/* The caps of app/rag/source_labels (D1/D5), mirrored client-side so a
long tag never costs a round trip: the input's maxlength enforces the
32/120/200 char caps in the markup, this pair owns the chip count. */
const LABELS_ENDPOINT = "/api/source-labels";
const LABEL_TAG_MAX_CHARS = 32;
const LABEL_TAGS_MAX_COUNT = 10;
/* The save-state line's copy (role=status, one line per row). */
const LABEL_STATUS_SAVED = "saved";
const LABEL_STATUS_DIRTY = "unsaved changes";
const LABEL_STATUS_SAVING = "Saving…";
const LABEL_STATUS_RESETTING = "Resetting…";
/* The 404 arm NAMED (task 03): the source is gone from the catalogue
(removed elsewhere, or its last document pruned), so a retry cannot
help — the owner needs a reload, not a retype. */
const LABEL_NOTE_GONE = "That source no longer exists — reload the page.";
const LABEL_FAIL_SAVE = "Could not save the name — try again.";
const LABEL_FAIL_RESET = "Could not reset the name — try again.";
/* Per-clone id sequence: the template ships ONE id per field, and a
literal clone N times over would duplicate it (WCAG 1.3.1 — every
<label for> would aim at the first match). makeLabelRow rewrites the
input ids, the matching <label for>s and the editor id (the toggle's
aria-controls target) with this counter. */
let labelRowSeq = 0;
function storedFromEntry(entry) {
/* One label row in its SERVER shape → the row's stored state. Nulls /
missing keys fold to "" / [] (the API's "no row" shape is identical
to a cleared label — D1), so a row with nothing stored renders the
empty form and no Reset. */
const e = entry || {};
return {
display_name: typeof e.display_name === "string" ? e.display_name : "",
label: typeof e.label === "string" ? e.label : "",
tags: Array.isArray(e.tags) ? e.tags.map((tag) => String(tag)) : [],
};
}
function hasStoredLabel(stored) {
return Boolean(stored.display_name || stored.label || stored.tags.length);
}
/* The collapsed row's read-out of the stored label — name · label · the
tag list. "" renders nothing (the row shows the toggle instead). */
function labelSummaryText(stored) {
const parts = [];
if (stored.display_name) parts.push(stored.display_name);
if (stored.label) parts.push(stored.label);
if (stored.tags.length) parts.push(stored.tags.join(", "));
return parts.join(" · ");
}
/* The token input's chip list, re-painted from the row's draft array.
The chips are the phase-89 tag idiom (.source-label-tag joins the
.git-source-ignore-count / -hidden-count / -branch-count rule — TEXT +
background, never color alone), so NO new tag-input CSS is needed
beyond the list's own layout. At the cap the input disables (the tenth
chip is the last one the API accepts). */
function paintLabelTags(view) {
view.tagsList.replaceChildren(
...view.tags.map((tag) => {
const li = document.createElement("li");
li.className = "source-label-tag";
li.textContent = tag; // textContent only (the XSS contract)
return li;
}),
);
const full = view.tags.length >= LABEL_TAGS_MAX_COUNT;
view.tagsInput.disabled = full;
view.tagsInput.placeholder = full
? `${LABEL_TAGS_MAX_COUNT} tags at most`
: "gitea";
}
/* Type + Enter (or a comma) adds a chip. The rules are the API's
normalize_tags, applied client-side: trim, drop the empties, keep the
FIRST case of a duplicate, cap the count — a duplicate chip would
otherwise show the owner two chips the server de-duplicates to one. */
function addLabelTags(view, raw) {
for (const piece of String(raw).split(",")) {
const tag = piece.trim().slice(0, LABEL_TAG_MAX_CHARS);
if (!tag) continue;
if (view.tags.some((t) => t.toLowerCase() === tag.toLowerCase())) continue;
if (view.tags.length >= LABEL_TAGS_MAX_COUNT) break;
view.tags.push(tag);
}
view.tagsInput.value = "";
paintLabelTags(view);
updateLabelRowUI(view);
}
function readLabelForm(view) {
return {
display_name: view.nameInput.value.trim(),
label: view.labelInput.value.trim(),
tags: view.tags.slice(),
};
}
function fillLabelForm(view) {
view.nameInput.value = view.stored.display_name;
view.labelInput.value = view.stored.label;
view.tagsInput.value = "";
view.tags = view.stored.tags.slice();
paintLabelTags(view);
}
function labelFormDirty(view) {
const form = readLabelForm(view);
return (
form.display_name !== view.stored.display_name ||
form.label !== view.stored.label ||
form.tags.join("\n") !== view.stored.tags.join("\n")
);
}
function setLabelStatus(view, text, kind) {
view.statusEl.textContent = text;
view.statusEl.classList.toggle("is-error", kind === "error");
view.statusEl.classList.toggle("is-hint", kind === "hint");
}
/* The row's stored-state paint: the "named" tag, the summary line, the
Reset button (a row with nothing stored has nothing to reset — the
phase-168 per-row reset shape) and the toggle's label + aria-label
(the ONLY place the source name rides on the controls — setAttribute,
never innerHTML, the phase-89/105 idiom). */
function paintLabelRowState(view) {
const labelled = hasStoredLabel(view.stored);
if (view.stateTag) view.stateTag.hidden = !labelled;
if (view.summaryEl) {
const text = labelSummaryText(view.stored);
view.summaryEl.textContent = text;
view.summaryEl.hidden = text === "";
}
view.resetBtn.hidden = !labelled;
view.toggleBtn.textContent = labelled ? "Edit name" : "Name this source";
view.toggleBtn.setAttribute(
"aria-label",
`${labelled ? "Edit" : "Set"} the name, label and tags for source: ${view.source}`,
);
view.saveBtn.setAttribute(
"aria-label",
`Save the name, label and tags for source: ${view.source}`,
);
view.resetBtn.setAttribute(
"aria-label",
`Reset the name, label and tags of source: ${view.source}`,
);
}
/* Drop a failure note the moment the draft moves again: the note
explains the submission the server refused, and that submission is
gone as soon as the owner types. Called by the edit handlers ONLY —
never by updateLabelRowUI, which is exactly what a finishing request
calls (and would erase the note it just wrote). */
function clearLabelError(view) {
if (view.statusEl.classList.contains("is-error")) setLabelStatus(view, "", null);
}
/* The per-row dirty paint (the phase-168 .is-dirty shape, scoped to ONE
row instead of the panel). While a request is out the in-flight label
owns the status line — a keystroke mid-save must not overwrite it. */
function updateLabelRowUI(view) {
const dirty = labelFormDirty(view);
view.tr.classList.toggle("is-dirty", dirty);
if (view.saveInFlight) return;
view.saveBtn.disabled = !dirty;
/* A FAILED save's note survives this paint (§7.4): the dirty hint and
the note are the same one line, and the hint is the weaker claim —
silently replacing "That source no longer exists — reload the
page." with "unsaved changes" hides the one error the owner cannot
fix by retrying. The note clears on the next edit (clearLabelError)
or on a collapse, never by re-deriving the dirty state. */
if (view.statusEl.classList.contains("is-error")) return;
if (dirty) setLabelStatus(view, LABEL_STATUS_DIRTY, "hint");
else if (view.statusEl.classList.contains("is-hint")) {
setLabelStatus(view, "", null);
}
}
/* Expand / collapse. A collapse discards the draft (no request) and
refills from the stored label, so a closed row never claims an edit the
server never got; a save in flight freezes the row. */
function setLabelEditorOpen(view, open) {
if (view.saveInFlight) return;
view.editor.hidden = !open;
view.toggleBtn.setAttribute("aria-expanded", open ? "true" : "false");
if (!open) {
fillLabelForm(view);
setLabelStatus(view, "", null);
}
updateLabelRowUI(view);
}
/* The one-line error note. The 404 arm names itself (a retry cannot fix a
source that left the catalogue); every other non-2xx carries the server
detail (apiDetail, 422 shape-aware — the cap that was crossed names
itself there). */
async function labelFailureNote(r, fallback) {
if (r.status === 404) return LABEL_NOTE_GONE;
return await apiDetail(r, fallback);
}
/* PUT the whole form (the three keys always ride — an empty Name is the
deliberate CLEAR leg, the API folds "" to NULL; the tags array is the
chip list). 200 → the response body IS the row's new stored label. */
async function saveLabelRow(view) {
if (view.saveInFlight) return; // §7.4 — one request per row at a time
const form = readLabelForm(view);
view.saveInFlight = true;
view.saveBtn.disabled = true;
view.resetBtn.disabled = true;
setLabelStatus(view, LABEL_STATUS_SAVING, null);
try {
const r = await fetch(`${LABELS_ENDPOINT}/${encodeURIComponent(view.source)}`, {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(form),
});
if (r.ok) {
/* The response is the new truth — NO second GET (task 03). A body
that cannot be parsed degrades to what was just sent (never to
"unknown": the write succeeded either way). */
let saved = null;
try {
saved = await r.json();
} catch {
saved = null;
}
view.stored = storedFromEntry(
saved || {
display_name: form.display_name || null,
label: form.label || null,
tags: form.tags,
},
);
fillLabelForm(view);
paintLabelRowState(view);
hideLabelsError(); // a healed save heals the catalogue line too
setLabelStatus(view, LABEL_STATUS_SAVED, null);
return;
}
setLabelStatus(view, await labelFailureNote(r, LABEL_FAIL_SAVE), "error");
} catch {
setLabelStatus(view, `${LABEL_FAIL_SAVE} Is the app reachable?`, "error");
} finally {
// Never stale on ANY outcome (§7.4): the flag drops, the buttons
// come back, and the dirty paint re-decides Save.
view.saveInFlight = false;
view.resetBtn.disabled = false;
updateLabelRowUI(view);
}
}
/* DELETE the row — the fields clear, the stored label nulls, and the
row settles on the same "saved" line (the row now matches the server,
which is all the line has ever claimed). */
async function resetLabelRow(view) {
if (view.saveInFlight) return; // the SAME per-row guard covers Reset
view.saveInFlight = true;
view.saveBtn.disabled = true;
view.resetBtn.disabled = true;
setLabelStatus(view, LABEL_STATUS_RESETTING, null);
try {
const r = await fetch(`${LABELS_ENDPOINT}/${encodeURIComponent(view.source)}`, {
method: "DELETE",
});
if (r.ok) {
// The 200 body is the entry's cleared state (task 02) — the row
// re-reads it instead of assuming, and the fields clear with it.
let cleared = null;
try {
cleared = await r.json();
} catch {
cleared = null;
}
view.stored = storedFromEntry(cleared || {});
fillLabelForm(view);
paintLabelRowState(view);
hideLabelsError();
setLabelStatus(view, LABEL_STATUS_SAVED, null);
return;
}
setLabelStatus(view, await labelFailureNote(r, LABEL_FAIL_RESET), "error");
} catch {
setLabelStatus(view, `${LABEL_FAIL_RESET} Is the app reachable?`, "error");
} finally {
view.saveInFlight = false;
view.resetBtn.disabled = false;
updateLabelRowUI(view);
}
}
/* Clone the page's ONE row template for one catalogue entry. The
template is static markup (no data): every value lands through
textContent / .value, and the row's own object (`view`) carries its
state — stored label, chip draft, saveInFlight — closed over by its
listeners. */
function makeLabelRow(entry) {
const proto =
labelRowTemplate && labelRowTemplate.content
? labelRowTemplate.content.querySelector("tr")
: null;
if (!proto) return null; // defensive — the template ships with the page
const tr = proto.cloneNode(true);
labelRowSeq += 1;
const seq = labelRowSeq;
const pick = (sel) => tr.querySelector(sel);
const nameInput = pick(".source-label-name-input");
const labelInput = pick(".source-label-label-input");
const tagsInput = pick(".source-label-tags-input");
const tagsList = pick(".source-label-tags");
const editor = pick(".source-label-editor");
const toggleBtn = pick(".source-label-toggle");
const saveBtn = pick(".source-label-save");
const resetBtn = pick(".source-label-reset");
const statusEl = pick(".source-label-status");
if (!nameInput || !labelInput || !tagsInput || !tagsList || !editor ||
!toggleBtn || !saveBtn || !resetBtn || !statusEl) {
return null; // a template without the fields is a build error, not a row
}
/* Rewrite the template's static ids (and the <label for>s that point at
them) so every row keeps a unique id and a real label pair; the
editor id is the toggle's aria-controls target. */
for (const [input, base] of [
[nameInput, "source-label-name"],
[labelInput, "source-label-label"],
[tagsInput, "source-label-tags"],
]) {
input.id = `${base}-${seq}`;
const lbl = tr.querySelector(`label[for="${base}"]`);
if (lbl) lbl.setAttribute("for", input.id);
}
editor.id = `source-label-editor-${seq}`;
toggleBtn.setAttribute("aria-controls", editor.id);
const codeEl = pick(".source-label-source");
const source = String((entry && entry.source) || "");
if (codeEl) codeEl.textContent = source; // the identity: read-only text
const cell = pick(".source-label-id-cell");
if (cell) cell.title = source; // long names hover in full (house idiom)
const view = {
source,
tr,
stored: storedFromEntry(entry),
tags: [],
saveInFlight: false,
nameInput,
labelInput,
tagsInput,
tagsList,
editor,
toggleBtn,
saveBtn,
resetBtn,
statusEl,
stateTag: pick(".source-label-state"),
summaryEl: pick(".source-label-summary"),
};
fillLabelForm(view);
paintLabelRowState(view);
updateLabelRowUI(view);
for (const input of [nameInput, labelInput]) {
input.addEventListener("input", () => {
clearLabelError(view);
updateLabelRowUI(view);
});
}
tagsInput.addEventListener("keydown", (e) => {
clearLabelError(view);
// Enter or a comma commits the token; the comma never "types".
if (e.key === "Enter" || e.key === ",") {
e.preventDefault();
addLabelTags(view, tagsInput.value);
return;
}
// Backspace on an empty input drops the LAST chip (the token rule).
if (e.key === "Backspace" && tagsInput.value === "" && view.tags.length) {
e.preventDefault();
view.tags.pop();
paintLabelTags(view);
updateLabelRowUI(view);
}
});
// Leaving the field commits a half-typed token (nothing is lost).
tagsInput.addEventListener("blur", () => {
if (tagsInput.value.trim()) addLabelTags(view, tagsInput.value);
});
toggleBtn.addEventListener("click", () => setLabelEditorOpen(view, editor.hidden));
saveBtn.addEventListener("click", () => saveLabelRow(view));
resetBtn.addEventListener("click", () => resetLabelRow(view));
return tr;
}
/* GET the label catalogue (ONE fetch — the union the server assembles).
A failure is this list's own role=alert line: the sources table above
keeps working (the two lists are independent), and the table + empty
state hide while the error is up so the note is the only claim about
the rows (the #git-sources-load-error discipline). */
async function loadLabels() {
let r;
try {
r = await fetch(LABELS_ENDPOINT);
} catch {
showLabelsError("Could not reach the server — the source names are not shown.");
return;
}
if (!r.ok) {
showLabelsError(
await apiDetail(r, `The server could not list the source names (${r.status}).`),
);
return;
}
let data;
try {
data = await r.json();
} catch {
showLabelsError("The server sent an unreadable label list — try again.");
return;
}
hideLabelsError();
renderLabelRows(Array.isArray(data.labels) ? data.labels : []);
}
function renderLabelRows(entries) {
if (!labelsTbody) return;
labelsTbody.replaceChildren(); // the list is REPLACED, never appended
for (const entry of entries) {
const tr = makeLabelRow(entry);
if (tr) labelsTbody.appendChild(tr);
}
const hasRows = labelsTbody.children.length > 0;
if (labelsWrap) labelsWrap.hidden = !hasRows;
if (labelsEmptyEl) labelsEmptyEl.hidden = hasRows;
}
function showLabelsError(message) {
if (labelsErrorEl) {
labelsErrorEl.textContent = message;
labelsErrorEl.hidden = false;
}
// The list state is unknown — hide the table AND the empty state.
if (labelsWrap) labelsWrap.hidden = true;
if (labelsEmptyEl) labelsEmptyEl.hidden = true;
}
function hideLabelsError() {
if (labelsErrorEl) {
labelsErrorEl.textContent = "";
labelsErrorEl.hidden = true;
}
}
/* ---------- add (POST /api/git-sources) — the git form ---------- /* ---------- add (POST /api/git-sources) — the git form ----------
* wireAddForm gives the form the §7.4 never-stale lifecycle: while * wireAddForm gives the form the §7.4 never-stale lifecycle: while
* the request is out the button disables + relabels "Adding…" and * the request is out the button disables + relabels "Adding…" and
@@ -1065,6 +1596,7 @@ export async function mount(root) {
if (branchInput) branchInput.value = ""; // the branch is stored (NULL when blank) if (branchInput) branchInput.value = ""; // the branch is stored (NULL when blank)
announce(opts.addedMessage); announce(opts.addedMessage);
await loadSources(); // the new row lands in the table await loadSources(); // the new row lands in the table
await loadLabels(); // phase 191: and it is nameable straight away
focusNewRow(createdId); // a11y: land the caret on the new row focusNewRow(createdId); // a11y: land the caret on the new row
return; return;
} }
@@ -1333,6 +1865,7 @@ export async function mount(root) {
uploadFileInput.value = ""; uploadFileInput.value = "";
restoreUploadButton(); restoreUploadButton();
loadSources(); // the row lands / refreshes loadSources(); // the row lands / refreshes
loadLabels(); // phase 191: the uploaded source joins the name list
return; return;
} }
if (status.state === "failed") { if (status.state === "failed") {
@@ -1524,6 +2057,13 @@ export async function mount(root) {
the whoami gate passed: anonymous shows the gate and must never the whoami gate passed: anonymous shows the gate and must never
fetch /api/git-sources (the Sources-page gate pattern). */ fetch /api/git-sources (the Sources-page gate pattern). */
root.addEventListener("bor:view-refresh", () => loadSources()); root.addEventListener("bor:view-refresh", () => loadSources());
/* Phase 191 (task 03): the label catalogue rides the same re-show
refresh. A SECOND listener (rather than a call inside loadSources)
keeps the phase-77 listener above verbatim AND keeps the two lists
independent — the ignore-paths / hidden-folders saves reload the
source table without ever wiping a half-typed name draft. */
root.addEventListener("bor:view-refresh", () => loadLabels());
await loadLabels(); // phase 191: the names list (admin branch only)
await loadSources(); await loadSources();
// Phase 64 (task 05): re-attach a running run (a reload mid-run // Phase 64 (task 05): re-attach a running run (a reload mid-run
// resumes the bare Processing state) or re-render a terminal // resumes the bare Processing state) or re-render a terminal
frontend/assets/styles.css
+259 -4
View File
@@ -4349,9 +4349,16 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
.git-source-ignore-count idiom (TEXT + background, never .git-source-ignore-count idiom (TEXT + background, never
color alone — WCAG 1.4.1: --ink on --bg 16.7:1). Phase 135 (A6): color alone — WCAG 1.4.1: --ink on --bg 16.7:1). Phase 135 (A6):
the "branch: <name>" state tag joins the same rule — stored git the "branch: <name>" state tag joins the same rule — stored git
rows with a branch only, the text IS the signal. */ rows with a branch only, the text IS the signal.
Phase 191 (task 03): the TWO new tags of the name/label/tags row join
the SAME rule — .source-label-state ("named", beside the mono identity)
and .source-label-tag (one chip of the Tags token input). That is the
reuse the phase asks for: the chips are the house tag, not a new
widget, so no tag-input styling is added anywhere below. */
.git-source-hidden-count, .git-source-hidden-count,
.git-source-branch-count { .git-source-branch-count,
.source-label-state,
.source-label-tag {
display: inline-block; display: inline-block;
margin-left: 0.55rem; margin-left: 0.55rem;
padding: 0.08rem 0.5rem; padding: 0.08rem 0.5rem;
@@ -4376,8 +4383,11 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
} }
/* Empty state: the tuning page's quiet centered line at full table /* Empty state: the tuning page's quiet centered line at full table
width — no stored rows AND no env fallback to show. */ width — no stored rows AND no env fallback to show. Phase 191 (task
.git-sources-empty { 03): the name/label/tags list shares the line verbatim (its own
no-sources case, the same quiet card). */
.git-sources-empty,
.source-labels-empty {
margin: 0; margin: 0;
padding: 1.4rem 1rem; padding: 1.4rem 1rem;
text-align: center; text-align: center;
@@ -4389,6 +4399,251 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
box-shadow: var(--shadow); box-shadow: var(--shadow);
} }
/* ---------- Source names, labels and tags (phase 191, task 03) ----------
The Sources page's SECOND list: every source the knowledge base knows
(git, local, upload — the union the phase-191 catalogue GET returns),
each with the optional name / label / tags the model is told about
(D1: an ALIAS — the mono identity cell is read-only text, nothing here
renames a document, a citation or a locator).
Nothing new is invented: the card is the phase-156 .table-wrap (bounded
panel + sticky header, inherited by the class alone), the identity cell
is the .git-source-url-cell mono, the state tag and the tag CHIPS are
the phase-89 tag rule above, the field boxes are the .ignore-editor-
token / #ignore-editor-branch language at the 44px floor, Save is the
page's brand pill, Reset and the row's toggle are the neutral
.git-source-ignore language, and the dirty mark is the phase-168
the phase-168 dirty-row stripe. PALETTE VARIABLES ONLY — no new hue
(every pair is a pair this page already measures: ink 15.3:1 and
ink-soft 8.2:1 on --surface, brand-ink on brand-soft 12.4:1, the err
pair 9.3:1, --bg ink on --brand 5.2:1); :focus-visible rides the global
3px outline rule; no CDN, no blur. The editor expands UNDER its own row
(one row open at a time keeps a 27-source list scannable) and its three
fields stack in a single column below 60rem instead of pushing the card
into a horizontal scroll at 375px. */
.source-labels {
display: flex;
flex-direction: column;
gap: 0.75rem;
}
.source-labels-title {
margin: 0;
font-size: 1.25rem;
line-height: 1.3;
color: var(--ink);
}
/* The alias note (the contract in the owner's words) — the page-sub
language, ink-soft on the page canvas (8.2:1 on --surface). */
.source-labels-sub {
margin: 0;
max-width: 60rem;
color: var(--ink-soft);
font-size: 0.88rem;
}
/* The list's own load failure (role=alert) — the .git-source-error
pair (err-ink on err-bg 9.3:1). */
.source-labels-error {
margin: 0;
padding: 0.45rem 0.8rem;
background: var(--err-bg);
color: var(--err-ink);
border: 1px solid var(--err-line);
border-radius: var(--radius-sm);
font-size: 0.85rem;
font-weight: 600;
}
/* No min-width on purpose: the editor column has to SHRINK (the fields
stack), so this table never scrolls the card sideways at 375px — the
panel still bounds vertically (.table-wrap). */
.source-labels-table {
width: 100%;
border-collapse: collapse;
font-size: 0.93rem;
}
.source-labels-table th, .source-labels-table td {
text-align: left;
vertical-align: top;
padding: 0.7rem 1rem;
border-bottom: 1px solid var(--line);
}
.source-labels-table th {
color: var(--brand-ink);
font-size: 0.82rem;
text-transform: uppercase;
letter-spacing: 0.04em;
}
.source-labels-table tbody tr:last-child td { border-bottom: 0; }
/* The identity cell: the derived name, mono like the sources table's
location cell, wrapping (overflow-wrap) instead of widening the card —
it is text, never an input (D1/A1). */
.source-labels-table td.source-label-id-cell {
font-family: var(--mono);
font-size: 0.82rem;
overflow-wrap: anywhere;
}
.source-labels-table td.source-label-id-cell code { font-family: inherit; }
/* The dirty row mark — the phase-168 dirty-row stripe (the brand edge is
beside the row's own "unsaved changes" TEXT, so the state is never color
alone — WCAG 1.4.1). The literal is NOT written in this comment: the
phase-168 pin (test_settings_frontend_view.py) regex-matches that
selector to its own rule and a comment mention would steal the match. */
.source-label-row.is-dirty > td:first-child { box-shadow: inset 3px 0 0 var(--brand); }
/* The collapsed row's read-out of the stored label (name · label · tags).
textContent-only (a label is owner text — the XSS contract). */
.source-label-summary {
margin: 0 0 0.45rem;
color: var(--ink-soft);
font-size: 0.85rem;
overflow-wrap: anywhere;
}
/* The row's expand trigger — the .git-source-ignore language (neutral
secondary fill, >=44px, hovers to the brand pair 12.4:1). */
.source-label-toggle {
display: inline-flex;
align-items: center;
justify-content: center;
max-width: 100%;
min-height: 44px;
min-width: 44px;
padding: 0.35rem 0.7rem;
border: 1px solid var(--line);
border-radius: var(--radius-sm);
background: transparent;
color: var(--ink-soft);
font: inherit;
font-weight: 600;
font-size: 0.82rem;
white-space: nowrap;
cursor: pointer;
}
.source-label-toggle:hover:not(:disabled) {
background: var(--brand-soft);
color: var(--brand-ink);
}
/* The expanded editor panel — the row's own form card. The fields go in
one column narrow, three across from 60rem (the house grid). */
.source-label-editor {
display: grid;
gap: 0.9rem;
margin-top: 0.75rem;
padding: 0.9rem;
background: var(--bg);
border: 1px solid var(--line);
border-radius: var(--radius-sm);
}
@media (min-width: 60rem) {
.source-label-fields { grid-template-columns: repeat(3, minmax(0, 1fr)); }
}
.source-label-fields {
display: grid;
gap: 0.75rem;
}
.source-label-field {
display: block;
min-width: 0;
}
/* The field label — the .ignore-editor-label language (ink-soft 8.2:1 on
the panel, 600 weight, the hint span the .field-hint secondary read). */
.source-label-field-label {
display: block;
margin-bottom: 0.35rem;
color: var(--ink-soft);
font-size: 0.88rem;
font-weight: 600;
}
/* The field box — the .ignore-editor-token / #ignore-editor-branch box
language VERBATIM (full width, --bg… panel fill, the line border, mono)
at the 44px touch floor; the global [hidden] rule and the global
:focus-visible outline cover the rest. */
.source-label-input {
display: block;
width: 100%;
min-height: 44px;
padding: 0.55rem 0.65rem;
font-family: var(--mono);
font-size: 0.85rem;
color: var(--ink);
background: var(--surface);
border: 1px solid var(--line);
border-radius: var(--radius-sm);
}
/* The Tags token input's chip list — the LAYOUT only (the chips
themselves are the shared tag rule above, so the widget adds no tag
styling). list-style is dropped, so role="list" stays in the markup
(the VoiceOver list-semantics idiom the house uses). */
.source-label-tags {
display: flex;
flex-wrap: wrap;
gap: 0.35rem;
margin: 0 0 0.45rem;
padding: 0;
list-style: none;
}
.source-label-tags .source-label-tag { margin-left: 0; }
.source-label-actions {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: 0.6rem;
}
/* Save — the page's brand pill at the row scale (--bg ink on --brand,
5.2:1); disabled until the row is dirty, and again while the PUT is
out (§7.4 — the row's own in-flight state). */
.source-label-save {
display: inline-flex;
align-items: center;
justify-content: center;
min-height: 44px;
padding: 0.4rem 1.1rem;
border: 0;
border-radius: var(--radius-sm);
background: var(--brand);
color: var(--bg);
font: inherit;
font-weight: 700;
cursor: pointer;
}
.source-label-save:hover:not(:disabled) { background: var(--brand-hover); }
.source-label-save:disabled { opacity: 0.6; cursor: default; }
/* Reset — the neutral secondary button (the .git-source-ignore language):
it clears a STORED label, it is not the destructive row Remove. */
.source-label-reset {
display: inline-flex;
align-items: center;
justify-content: center;
min-height: 44px;
padding: 0.35rem 0.9rem;
border: 1px solid var(--line);
border-radius: var(--radius-sm);
background: transparent;
color: var(--ink-soft);
font: inherit;
font-weight: 600;
font-size: 0.82rem;
cursor: pointer;
}
.source-label-reset:hover:not(:disabled) {
background: var(--brand-soft);
color: var(--brand-ink);
}
.source-label-reset:disabled { opacity: 0.5; cursor: wait; }
/* The §7.4 save-state line (role=status): "Saving…" → "saved" / the
one-line error note. The hint (unsaved) is the muted italic; the error
takes the err pair (9.3:1) so the note reads as a failure, not as a
label — TEXT carries it either way, never color alone. */
.source-label-status {
color: var(--ink-soft);
font-size: 0.82rem;
}
.source-label-status.is-hint { font-style: italic; }
.source-label-status.is-error {
padding: 0.35rem 0.6rem;
color: var(--err-ink);
background: var(--err-bg);
border: 1px solid var(--err-line);
border-radius: var(--radius-sm);
font-weight: 600;
}
/* ---------- History refresh button (phase 77 task 03, TODO.md L3) ---------- /* ---------- History refresh button (phase 77 task 03, TODO.md L3) ----------
The #view-history page-head is a flex row: the title block left, the The #view-history page-head is a flex row: the title block left, the
actions slot right (the row wraps below 640px). Scoped to actions slot right (the row wraps below 640px). Scoped to
frontend/index.html
+118
View File
@@ -840,6 +840,124 @@
pruned on that run). pruned on that run).
</p> </p>
<!-- Phase 191 (task 03): the per-source NAME / LABEL / TAGS
editor — "Sources need a name, a label, tag(s) so you can
focus the model when prompting if you have a project name
or a standard name you reference things by."
The ROW LIST is the phase-191 catalogue
GET /api/source-labels (task 02) — the union of every
registry row's DERIVED source name and every indexed
document source — which is why this is its own table card
and not a column of #git-sources-table above: a source can
carry documents with no git_sources row of its own, and the
one GET is the single source of truth for the row list.
THE ALIAS CONTRACT (D1/A1, owner-confirmed): the mono cell
is the row's IDENTITY — the derived name, rendered with
textContent and never an input. Editing a name here
renames NOTHING: documents, citations, the KB tree, saved
chats, shared pages and every source/path locator keep the
derived name. A name and its tags only change what the
model is TOLD (tasks 04/05).
ONE shared <template> (source-label-row-template) is cloned
per row by git-sources.js: the three fields (Name ≤120,
Label ≤200, the Tags token input — Enter or comma adds a
chip, Backspace drops the last, ≤10 chips of ≤32 chars: the
caps of the API's normalize_tags, mirrored so a long
submission never round-trips as a 422) + Save / Reset + the
§7.4 never-stale save-state line (role=status: "Saving…" →
"saved" / the one-line error note, scoped PER ROW). The
section lives INSIDE the admin-gated #git-sources-content,
so an anonymous visitor never fetches it — the page's own
gate is this list's gate too. -->
<section class="source-labels" id="source-labels-section" aria-labelledby="source-labels-title">
<h2 class="source-labels-title" id="source-labels-title">Names, labels and tags</h2>
<p class="source-labels-sub">
A name is what <em>you</em> call a source (&ldquo;the git server&rdquo;)
and a tag is a focus token — the model is told both, so a
question that uses the name finds that source&rsquo;s documents.
Nothing here renames a source: the mono name below stays the
identity of every document, citation and link.
</p>
<!-- The catalogue's own load failure (role=alert). The table and
the empty state are hidden while it is up, so the error is
the only claim about the list (the #git-sources-load-error
discipline, applied to this page's second list). -->
<p class="source-labels-error" id="source-labels-error" role="alert" hidden></p>
<div class="table-wrap source-labels-wrap" id="source-labels-wrap" role="region"
aria-label="Source names, labels and tags" tabindex="0">
<table class="source-labels-table" id="source-labels-table">
<caption class="visually-hidden">Every source the knowledge base knows — git repositories, local directories and uploaded archives — each with the optional name, label and tags the model reads. The mono name in the first column is the source&rsquo;s identity and never changes.</caption>
<thead>
<tr>
<th scope="col">Source</th>
<th scope="col">Name, label and tags</th>
</tr>
</thead>
<tbody id="source-labels-tbody"></tbody>
</table>
</div>
<!-- Empty state — the catalogue answered with nothing at all
(no registry row and no indexed document). -->
<p class="source-labels-empty" id="source-labels-empty" hidden>No sources to name yet.</p>
<!-- ONE shared template, cloned per row by makeLabelRow.
The ids written here are the FIRST row&rsquo;s: the clone
loop rewrites each input&rsquo;s id, its <label for> and the
editor&rsquo;s id (the toggle&rsquo;s aria-controls target) —
a literal clone N times over would duplicate ids and make
every label ambiguous (WCAG 1.3.1; the fields keep real
<label for> pairs, never aria-label-only). Static markup
only: every value lands through textContent or .value in
git-sources.js (the XSS contract), and the editor ships
hidden — the row&rsquo;s toggle reveals it. -->
<template id="source-label-row-template">
<tr class="source-label-row">
<td class="source-label-id-cell">
<code class="source-label-source"></code>
<!-- The stored-label state tag: TEXT + a background, never
color alone (WCAG 1.4.1 — the
.git-source-ignore-count idiom, phase 89/105/135). -->
<span class="source-label-state" hidden>named</span>
</td>
<td class="source-label-edit-cell">
<p class="source-label-summary" hidden></p>
<button type="button" class="source-label-toggle" aria-expanded="false">Name this source</button>
<div class="source-label-editor" id="source-label-editor" hidden>
<div class="source-label-fields">
<div class="source-label-field">
<label class="source-label-field-label" for="source-label-name">Name <span class="field-hint">optional — what you call it</span></label>
<input class="source-label-input source-label-name-input" id="source-label-name"
type="text" maxlength="120" autocomplete="off"
placeholder="the git server">
</div>
<div class="source-label-field">
<label class="source-label-field-label" for="source-label-label">Label <span class="field-hint">optional — one line about it</span></label>
<input class="source-label-input source-label-label-input" id="source-label-label"
type="text" maxlength="200" autocomplete="off"
placeholder="self-hosted forge + Gitea">
</div>
<div class="source-label-field">
<label class="source-label-field-label" for="source-label-tags">Tags <span class="field-hint">optional — Enter or comma adds one</span></label>
<ul class="source-label-tags" role="list"></ul>
<input class="source-label-input source-label-tags-input" id="source-label-tags"
type="text" maxlength="32" autocomplete="off"
placeholder="gitea">
</div>
</div>
<div class="source-label-actions">
<button type="button" class="source-label-save" disabled>Save</button>
<button type="button" class="source-label-reset" hidden>Reset</button>
<span class="source-label-status" role="status" aria-live="polite"></span>
</div>
</div>
</td>
</tr>
</template>
</section>
<!-- Phase 69 (owner request 2026-09-02): the remove <!-- Phase 69 (owner request 2026-09-02): the remove
confirmation — a real in-app alertdialog (the native confirmation — a real in-app alertdialog (the native
confirm() retired): a row's Remove button opens it confirm() retired): a row's Remove button opens it
scripts/api_payload_audit.py
+3
View File
@@ -310,6 +310,9 @@ def run(args: argparse.Namespace) -> tuple[int, list[Row], list[str]]:
rows.append(Row(label="saved chat (one)", path="/api/chats/{id}", note="no saved chats")) rows.append(Row(label="saved chat (one)", path="/api/chats/{id}", note="no saved chats"))
rows.append(measure(client, "git sources", "/api/git-sources")) rows.append(measure(client, "git sources", "/api/git-sources"))
# phase 191: the label catalogue the Sources page loads (one entry
# per source — it grows with the registry, so it belongs in the audit)
rows.append(measure(client, "source labels (catalogue)", "/api/source-labels"))
rows.append(measure(client, "suggestions", "/api/suggestions")) rows.append(measure(client, "suggestions", "/api/suggestions"))
rows.append(measure(client, "sync status (polled)", "/api/sync/status")) rows.append(measure(client, "sync status (polled)", "/api/sync/status"))
rows.append(measure(client, "tokens", "/api/tokens")) rows.append(measure(client, "tokens", "/api/tokens"))
tests/e2e/test_source_name_labels.py
+1066
View File
File diff suppressed because it is too large Load Diff
tests/integration/test_chunks_hnsw_index.py
+17 -3
View File
@@ -236,7 +236,17 @@ def test_invalid_index_is_dropped_and_rebuilt(db: Session, alembic: Config) -> N
data, and timing a real interruption is nondeterministic; the dev/ data, and timing a real interruption is nondeterministic; the dev/
test DB role is a superuser, and a role without catalog UPDATE test DB role is a superuser, and a role without catalog UPDATE
rights skips cleanly instead of leaving the DB inconsistent (the rights skips cleanly instead of leaving the DB inconsistent (the
stamp that follows the forced state, never before it).""" stamp that follows the forced state, never before it).
Phase 191 fix: the stamp moves the version table ALL the way below
0029, so every LATER migration's objects must be genuinely dropped
first (``downgrade("0029")``) — a stamped-below head with a later
migration's table still physically present would make every
following ``upgrade(head)`` re-run that migration and die on
``relation already exists`` (observed with 0030's
``source_labels`` once it landed). The only lie that stays between
the stamp and the re-run is the corrupted index — exactly the
state this test is about."""
command.upgrade(alembic, "head") command.upgrade(alembic, "head")
assert _index_valid(db) is True, "precondition: a valid index to corrupt" assert _index_valid(db) is True, "precondition: a valid index to corrupt"
@@ -253,8 +263,12 @@ def test_invalid_index_is_dropped_and_rebuilt(db: Session, alembic: Config) -> N
db.rollback() db.rollback()
pytest.skip(f"forcing INVALID needs catalog UPDATE rights: {exc}") pytest.skip(f"forcing INVALID needs catalog UPDATE rights: {exc}")
# The stamp moves the version table WITHOUT touching the index, so # Drop every post-0029 migration's objects for real BEFORE the
# the next upgrade() re-runs 0029 against the corrupted state. # stamp (the phase-191 note above), then move the version table
# WITHOUT touching the index, so the next upgrade() re-runs 0029
# against the corrupted state and head is re-derivable cleanly.
db.rollback() # release the session before the DDL (house pattern)
command.downgrade(alembic, "0029")
command.stamp(alembic, "0028") command.stamp(alembic, "0028")
assert _version(db) == "0028" assert _version(db) == "0028"
assert _index_valid(db) is False, "the forced INVALID state must be visible" assert _index_valid(db) is False, "the forced INVALID state must be visible"
tests/integration/test_migration_0030.py
+311
View File
@@ -0,0 +1,311 @@
"""Integration: migration 0030 (source_labels) schema contract
(phase 191, task 01).
Drives the **real Alembic engine** against the live dev database
(``podman compose up -d db``), mirroring the house pattern of
``test_migration_0028.py`` (information_schema assertions on the state
the migration must leave). The tests target the 0029 → 0030 step
explicitly so later migrations cannot break them:
* upgrade 0029 → 0030 → the ``source_labels`` table exists with the
full column contract (D1) — ``source`` VARCHAR(120) NOT NULL and the
sole primary key (one label row per source; the join key is the
DERIVED ``documents.source`` name — an alias key, never a rename),
``display_name`` / ``label`` TEXT nullable (the ≤120 / ≤200 caps are
API-boundary concerns, the phase-121 plain-column pattern), ``tags``
JSONB NOT NULL with the ``'[]'`` server default (the phase-89
``ignore_paths`` shape — a source with no tags reads ``[]``, never
NULL), ``updated_at`` TIMESTAMPTZ NOT NULL with the now() server
default (house style) — while the 0029 schema
(``ix_chunks_embedding_hnsw``) survives;
* the NO-BACKFILL contract: right after the upgrade the table is EMPTY
(a source without a row behaves exactly as pre-phase);
* inserted rows round-trip (name/label/tags verbatim, ``updated_at``
stamped by the server default);
* a duplicate ``source`` raises ``IntegrityError`` (the PK pin) and a
NULL ``tags`` is rejected (NOT NULL — the reset is ``[]``, never
NULL);
* downgrade to 0029 → the table is GONE (A13 — reversible), the rest of
the schema (the HNSW index, ``app_settings``) survives;
* upgrade back to 0030 → the table is back (round-trip).
The ``alembic`` fixture guarantees the DB ends at head even if a test
fails or the process is interrupted.
"""
from __future__ import annotations
from collections.abc import Iterator
from typing import Any
import pytest
from alembic.config import Config
from sqlalchemy import text
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from alembic import command
from app.db import db_available
@pytest.fixture()
def alembic(db: Session) -> Iterator[Config]:
"""Real Alembic config bound to the dev DB (URL from app settings).
Starts at head (repairs an interrupted earlier run); teardown
upgrades to head no matter what happened, so the dev DB is never
left below head.
"""
if not db_available():
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
cfg = Config() # no alembic.ini file — env.py gets the URL from app config
cfg.set_main_option("script_location", "alembic")
command.upgrade(cfg, "head")
try:
yield cfg
finally:
# Release the test session's open transaction BEFORE the repair
# DDL (the 0028 fixture's lesson): psycopg3 keeps its snapshot
# alive while idle-in-transaction, and a later 0029
# CREATE INDEX CONCURRENTLY would wait on it forever.
db.rollback()
command.upgrade(cfg, "head")
def _version(db: Session) -> str | None:
return db.execute(text("SELECT version_num FROM alembic_version")).scalar()
def _table_exists(db: Session, table: str) -> bool:
return (
db.execute(
text("SELECT 1 FROM information_schema.tables WHERE table_name = :t"),
{"t": table},
).scalar()
is not None
)
def _column(db: Session, table: str, column: str) -> tuple[Any, ...] | None:
"""(data_type, is_nullable, column_default, character_maximum_length)
for one table column."""
row = db.execute(
text(
"SELECT data_type, is_nullable, column_default, character_maximum_length"
" FROM information_schema.columns"
" WHERE table_name = :t AND column_name = :c"
),
{"t": table, "c": column},
).fetchone()
return tuple(row) if row is not None else None
def _pk_columns(db: Session, table: str) -> list[str]:
"""The table's PRIMARY KEY columns in ordinal position."""
rows = db.execute(
text(
"SELECT kcu.column_name"
" FROM information_schema.table_constraints tc"
" JOIN information_schema.key_column_usage kcu"
" ON kcu.constraint_name = tc.constraint_name"
" AND kcu.table_name = tc.table_name"
" WHERE tc.table_name = :t"
" AND tc.constraint_type = 'PRIMARY KEY'"
" ORDER BY kcu.ordinal_position"
),
{"t": table},
).fetchall()
return [r[0] for r in rows]
def _hnsw_index_exists(db: Session) -> bool:
return (
db.execute(
text(
"SELECT 1 FROM pg_indexes"
" WHERE indexname = 'ix_chunks_embedding_hnsw'"
)
).scalar()
is not None
)
def _clear_rows(db: Session) -> None:
db.execute(text("DELETE FROM source_labels"))
db.commit()
def test_upgrade_to_0030_creates_source_labels(db: Session, alembic: Config) -> None:
"""Upgrade 0029 → 0030: the table exists with the full D1 column
contract, the table is ABSENT at 0029, and the 0029 HNSW index
survives the additive upgrade."""
command.downgrade(alembic, "0029") # start from the pre-0030 state
assert _version(db) == "0029"
assert not _table_exists(db, "source_labels"), (
"source_labels must be absent at 0029"
)
command.upgrade(alembic, "0030")
assert _version(db) == "0030", "alembic_version must be at 0030"
assert _table_exists(db, "source_labels"), "the table must exist at 0030"
source = _column(db, "source_labels", "source")
assert source is not None, "source_labels.source is missing"
assert source[0] == "character varying", "source must be VARCHAR"
assert source[1] == "NO", "source must be NOT NULL (PK)"
assert source[2] is None, "source must have no server default"
assert source[3] == 120, "source must be String(120) (mirrors documents.source)"
display = _column(db, "source_labels", "display_name")
assert display is not None, "source_labels.display_name is missing"
assert display[0] == "text", "display_name must be plain TEXT (caps are API-side)"
assert display[1] == "YES", "display_name must be nullable (NULL = no name)"
label = _column(db, "source_labels", "label")
assert label is not None, "source_labels.label is missing"
assert label[0] == "text", "label must be plain TEXT (caps are API-side)"
assert label[1] == "YES", "label must be nullable (NULL = no label)"
tags = _column(db, "source_labels", "tags")
assert tags is not None, "source_labels.tags is missing"
assert tags[0] == "jsonb", "tags must be JSONB (the ignore_paths shape)"
assert tags[1] == "NO", "tags must be NOT NULL"
assert tags[2] is not None and "[]" in str(tags[2]), (
"tags must carry the '[]' server default (no tags reads [], never NULL)"
)
updated = _column(db, "source_labels", "updated_at")
assert updated is not None, "source_labels.updated_at is missing"
assert updated[0] == "timestamp with time zone", "updated_at must be TIMESTAMPTZ"
assert updated[1] == "NO", "updated_at must be NOT NULL"
assert updated[2] is not None and "now" in str(updated[2]), (
"updated_at must carry the now() server default (house style)"
)
assert _pk_columns(db, "source_labels") == ["source"], (
"source must be the sole primary key (one label row per source)"
)
# The NO-BACKFILL contract (D1): the migration creates an EMPTY
# table — every source keeps its pre-phase "no row" behavior.
count = db.execute(text("SELECT count(*) FROM source_labels")).scalar()
assert count == 0, "0030 must not backfill any label rows"
# The 0029 schema survives the additive upgrade.
assert _hnsw_index_exists(db), "ix_chunks_embedding_hnsw (0029) must survive"
def test_inserted_rows_round_trip(db: Session, alembic: Config) -> None:
"""At head, the D1 values round-trip verbatim (name / label / tags)
and ``updated_at`` is stamped by the server default."""
command.upgrade(alembic, "head")
try:
db.execute(
text(
"INSERT INTO source_labels (source, display_name, label, tags)"
" VALUES ('gitea-forge', 'the git server',"
" 'self-hosted forge + Gitea', '[\"gitea\", \"forge\"]'::jsonb)"
)
)
db.execute(
text(
"INSERT INTO source_labels (source)"
" VALUES ('bare-row')"
)
)
db.commit()
rows = db.execute(
text(
"SELECT source, display_name, label, tags, updated_at"
" FROM source_labels ORDER BY source"
)
).fetchall()
assert len(rows) == 2, "both rows must be stored"
assert rows[0][0] == "bare-row"
assert rows[0][1] is None and rows[0][2] is None
assert rows[0][3] == [], "the server default must read as the empty list"
assert rows[0][4] is not None, "updated_at must be stamped (server default)"
assert rows[1][0] == "gitea-forge", "source must round-trip verbatim"
assert rows[1][1] == "the git server"
assert rows[1][2] == "self-hosted forge + Gitea"
assert rows[1][3] == ["gitea", "forge"], "the tags array must round-trip"
finally:
_clear_rows(db)
def test_duplicate_source_is_rejected(db: Session, alembic: Config) -> None:
"""The PK is the uniqueness guard: two label rows for one source
would make "which name wins" undefined, so the second insert must
raise ``IntegrityError``."""
command.upgrade(alembic, "head")
try:
db.execute(
text("INSERT INTO source_labels (source) VALUES ('homelab')")
)
db.commit()
with pytest.raises(IntegrityError):
db.execute(
text(
"INSERT INTO source_labels (source, display_name)"
" VALUES ('homelab', 'second name')"
)
)
db.rollback() # the IntegrityError aborts the open transaction
finally:
_clear_rows(db)
def test_null_tags_is_rejected(db: Session, alembic: Config) -> None:
"""``tags`` is NOT NULL: "no tags" is ``[]`` (the server default),
never NULL — readers never branch on the None case."""
command.upgrade(alembic, "head")
try:
with pytest.raises(IntegrityError):
db.execute(
text(
"INSERT INTO source_labels (source, tags)"
" VALUES ('nulltags', NULL)"
)
)
db.rollback() # the IntegrityError aborts the open transaction
stored = db.execute(text("SELECT count(*) FROM source_labels")).scalar()
assert stored == 0, "the rejected row must not be stored"
finally:
_clear_rows(db)
def test_downgrade_to_0029_drops_the_table(db: Session, alembic: Config) -> None:
"""Downgrade 0030 → 0029: the table is gone (A13 — fully
reversible) while the rest of the schema survives (the 0029 HNSW
index, ``app_settings``). The names/labels/tags are re-enterable
from the Sources page — nothing outside this table was mutated."""
command.downgrade(alembic, "0029")
assert _version(db) == "0029"
assert not _table_exists(db, "source_labels"), "source_labels must be dropped"
assert _hnsw_index_exists(db), "ix_chunks_embedding_hnsw (0029) must survive"
strip = _column(db, "documents", "page_strip_mode")
assert strip is not None and strip[3] == 16, (
"documents.page_strip_mode must survive the downgrade"
)
key = _column(db, "app_settings", "key")
assert key is not None and key[3] == 100, "app_settings.key must survive"
def test_upgrade_round_trip_restores_the_table(db: Session, alembic: Config) -> None:
"""Downgrade to 0029, then upgrade back to 0030: the table is back
with the column contract and PK intact (A13 round-trip)."""
command.downgrade(alembic, "0029")
command.upgrade(alembic, "0030")
assert _version(db) == "0030", "round-trip upgrade must land at 0030"
assert _table_exists(db, "source_labels"), "the table must be back"
source = _column(db, "source_labels", "source")
assert source is not None, "source_labels.source must be back"
assert source[0] == "character varying", "source must be VARCHAR after the round-trip"
assert source[1] == "NO", "source must be NOT NULL after the round-trip"
assert source[3] == 120, "source must be String(120) after the round-trip"
tags = _column(db, "source_labels", "tags")
assert tags is not None, "source_labels.tags must be back"
assert tags[0] == "jsonb", "tags must be JSONB after the round-trip"
assert tags[1] == "NO", "tags must be NOT NULL after the round-trip"
assert _pk_columns(db, "source_labels") == ["source"]
tests/integration/test_source_labels_api.py
+753
View File
@@ -0,0 +1,753 @@
"""Integration: the admin ``/api/source-labels`` surface (phase 191, task 02).
The wire half of "Sources need a name, a label, tag(s)" (``TODO.md`` L4)
— the catalogue the Sources page loads, the upsert its Save drives and
the reset its Reset button drives. Driven through the real app with the
house ``TestClient`` cookie jar (the ``test_ui_settings_api.py`` phase-91
fixture pattern: the admin login and the token-user matrix), against the
real Postgres (``podman compose up -d db``); no LLM involved.
Contract under test:
* **the gate** — an anonymous caller AND a signed-in token USER (role
``"user"``) get 403 ``{"detail": "admin only"}`` on all three routes
(the router-wide ``require_admin``, the phase-91/168 admin-surface
shape, one fixed detail with nothing to enumerate), and nothing lands;
* **the catalogue GET** — the union of the ``git_sources`` rows of BOTH
kinds and the indexed document sources no row derives (the uploads),
de-duplicated and sorted by ``source``, each entry exactly
``{source, display_name, label, tags}`` with its label row or the
``null``/``[]`` defaults; an env-only deployment (empty table) lists
the ``BOR_GIT_SOURCES`` names; three tables, three SELECTs — the
assembly never queries per source;
* **the PUT** — the round trip (set all three → 200 → the GET reads them
back), the partial-update shape (an omitted field is untouched, an
explicit ``null`` clears, ``[]`` clears the tags, an all-omitted body
is a 200 no-op), the trim-then-cap ordering, the case-insensitive tag
de-dupe preserving the first case, the framework's standard 422 for a
121-char name / 201-char label / 11 tags / 33-char tag / whitespace-only
tag / a key outside the shape — each leaving a pre-seeded row
BYTE-IDENTICAL — the 404 for a source that is neither a row name nor an
indexed document (nothing created), the 422 for a > 120-char path, and
the ``normalize_tags`` funnel (its ``ValueError`` is a 422, never a
500);
* **the DELETE** — 200 with the entry's post-reset state, the GET shows
the nulls after it, a second DELETE is 404;
* **the alias contract (D1)** — labelling a source touches
``source_labels`` and NOTHING else: ``documents.source``, the row's
derived name and the catalogue's identity are byte-identical after a
rename, which is what keeps a saved chat's ``source/path`` locator, a
shared page and a verbatim caption valid across a rename.
``source_labels``, ``git_sources`` and the catalogue are global state:
truncated around every test.
Requires: podman compose up -d db
"""
from __future__ import annotations
import uuid
from collections.abc import Callable, Iterator
from contextlib import contextmanager
from typing import Any
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import event, text
from sqlalchemy.orm import Session
from app.api import source_labels as source_labels_api
from app.config import Settings
from app.db import engine
from app.main import app as fastapi_app
from app.models import Document, GitSource
from tests.conftest import ADMIN_PASSWORD
#: The catalogue entry's exact wire shape (``extra="forbid"`` makes the
#: omission structural — the row's ``updated_at`` never rides along).
_ENTRY_KEYS = {"source", "display_name", "label", "tags"}
@pytest.fixture(autouse=True)
def clean_state(db: Session) -> Iterator[None]:
"""Every table this module reads or writes is global state (the
catalogue is a FULL scan of the registry and the documents, so
another suite's rows would move the pins)."""
_wipe(db)
yield
_wipe(db)
def _wipe(db: Session) -> None:
db.execute(text("DELETE FROM source_labels"))
db.execute(text("TRUNCATE git_sources"))
db.execute(text("TRUNCATE chunks, documents"))
db.execute(text("TRUNCATE api_tokens"))
db.commit()
def _login(client: TestClient) -> TestClient:
client.post("/api/login", json={"password": ADMIN_PASSWORD})
return client
@pytest.fixture()
def env_sources(
monkeypatch: pytest.MonkeyPatch,
) -> Callable[[str], None]:
"""Pin the ``BOR_GIT_SOURCES`` fallback the way
``test_git_sources_api.py`` does — with a fresh ``Settings`` that
ignores the operator's ``.env`` (a dev ``.env`` with a repo in it
would otherwise BE the catalogue while the table is empty, the
phase-35 fallback). Returns the setter so one test can name the list
it wants; the empty string is the "nothing configured" state."""
def _set(git_sources: str = "") -> None:
monkeypatch.setattr(
"app.rag.git_sources.get_settings",
lambda: Settings(
_env_file=None, # pyright: ignore[reportCallIssue]
git_sources=git_sources,
),
)
return _set
def _doc(db: Session, source: str, path: str) -> Document:
"""One indexed document — the catalogue's (b) arm reads
``documents.source`` and NOTHING else."""
doc = Document(
id=uuid.uuid4(),
source=source,
path=path,
full_path=f"/srv/{source}/{path}",
title=path.rsplit("/", 1)[-1],
content=f"body of {path}",
content_hash="0" * 64,
)
db.add(doc)
return doc
def _seed_three_kinds(db: Session) -> None:
"""One git row, one local row and one upload-shaped document.
The git row is a ``file://`` URL (the phase-135 E2E-fixture form) and
carries the ``.git`` suffix, so the pin also proves the name is the
importer's DERIVED one (``repo_name``) rather than the raw URL; a
document exists under that derived name too, which is what the
de-duplication has to collapse.
"""
db.add_all(
[
GitSource(url="file:///srv/repos/gitea-forge.git", kind="git"),
GitSource(
url="/srv/docs/media-vault",
kind="local",
path="/srv/docs/media-vault",
),
]
)
db.add(_doc(db, "gitea-forge", "docs/app.md"))
db.add(_doc(db, "corpus-upload", "notes/a.md"))
db.commit()
@contextmanager
def _captured_statements() -> Iterator[list[str]]:
"""Every SQL statement the ENGINE runs inside the block.
The house statement-capture idiom (``test_import_session_retention``
/ ``test_prune_cascade_statements``): a local listener on the shared
engine, removed in ``finally`` — the request runs on another thread
under ``TestClient``, so the engine (not the session) is the place
both sides see.
"""
captured: list[str] = []
def _listen(
conn, cursor, statement, parameters, context, executemany
) -> None: # the engine's own signature — no types to restate
captured.append(statement)
event.listen(engine, "before_cursor_execute", _listen)
try:
yield captured
finally:
event.remove(engine, "before_cursor_execute", _listen)
def _selects(statements: list[str]) -> list[str]:
"""The SELECTs only, whitespace-collapsed for comparability, with
the pool's ``SELECT 1`` health check dropped — it is a checkout
artifact, not a read this endpoint asked for, and pinning it would
make the count depend on how warm the pool happens to be."""
return [
collapsed
for statement in statements
if (collapsed := " ".join(statement.split())).upper().startswith("SELECT")
and collapsed.upper() != "SELECT 1"
]
def _row(db: Session, source: str) -> dict[str, Any]:
"""One ``source_labels`` row read RAW — the pin never trusts the
shape under test to report its own storage."""
db.expire_all()
found = db.execute(
text(
"SELECT source, display_name, label, tags FROM source_labels"
" WHERE source = :s"
),
{"s": source},
).first()
if found is None:
return {}
return {
"source": found[0],
"display_name": found[1],
"label": found[2],
"tags": found[3],
}
# ---------------------------------------------------------------------------
# The admin gate (the phase-91 shape)
# ---------------------------------------------------------------------------
def test_anonymous_is_403_on_every_route(client: TestClient, db: Session) -> None:
"""Router-wide ``require_admin``: one fixed detail on all three
routes, and nothing landed (the ``test_ui_settings_api`` contract)."""
r = client.get("/api/source-labels")
assert r.status_code == 403
assert r.json() == {"detail": "admin only"}
r = client.put("/api/source-labels/gitea-forge", json={"display_name": "nope"})
assert r.status_code == 403
assert r.json() == {"detail": "admin only"}
r = client.delete("/api/source-labels/gitea-forge")
assert r.status_code == 403
assert r.json() == {"detail": "admin only"}
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
def test_token_user_is_403_on_every_route(client: TestClient) -> None:
"""A signed-in token USER is not the admin: the owner's naming surface
is closed to them on all three methods (the phase-79 token matrix —
the same leg the phase-91/168 admin surfaces pin)."""
admin = _login(TestClient(fastapi_app))
r = admin.post("/api/tokens", json={"label": "alice"})
assert r.status_code == 201, r.text
token = r.json()["token"]
assert client.post("/api/token-auth", json={"token": token}).status_code == 204
assert client.get("/api/whoami").json() == {"authenticated": True, "role": "user"}
assert client.get("/api/source-labels").status_code == 403
r = client.put("/api/source-labels/gitea-forge", json={"display_name": "nope"})
assert r.status_code == 403
assert r.json() == {"detail": "admin only"}
assert client.delete("/api/source-labels/gitea-forge").status_code == 403
# ---------------------------------------------------------------------------
# GET — the catalogue
# ---------------------------------------------------------------------------
def test_get_empty_deployment_is_an_empty_catalogue(
admin_client: TestClient, env_sources: Callable[[str], None]
) -> None:
"""No rows and no env list: the response is the empty list, not a 404
and not a missing key — the page always has a shape to render."""
env_sources("")
r = admin_client.get("/api/source-labels")
assert r.status_code == 200, r.text
assert r.json() == {"labels": []}
def test_get_lists_git_local_and_upload_sources(
admin_client: TestClient, db: Session
) -> None:
"""The D5 union, sorted, de-duplicated, with the exact entry shape.
``gitea-forge`` (a git row, derived from the URL basename with the
``.git`` stripped) also has a document under that name — the entry
still appears exactly ONCE. ``corpus-upload`` has documents and no
row, so only arm (b) can list it (and the E2E needs it to).
"""
_seed_three_kinds(db)
r = admin_client.get("/api/source-labels")
assert r.status_code == 200, r.text
labels = r.json()["labels"]
assert [entry["source"] for entry in labels] == [
"corpus-upload",
"gitea-forge",
"media-vault",
]
for entry in labels:
assert set(entry) == _ENTRY_KEYS
assert entry["display_name"] is None
assert entry["label"] is None
assert entry["tags"] == []
def test_get_carries_the_stored_label_on_one_entry_only(
admin_client: TestClient, db: Session
) -> None:
"""One labelled row, two unlabelled ones: the label rides its own
entry and the others keep the null/``[]`` defaults (a label row is
data beside the name, never a rename — D1)."""
_seed_three_kinds(db)
assert (
admin_client.put(
"/api/source-labels/corpus-upload",
json={"display_name": "the uploads", "tags": ["upload"]},
).status_code
== 200
)
labels = admin_client.get("/api/source-labels").json()["labels"]
by_source = {entry["source"]: entry for entry in labels}
assert by_source["corpus-upload"] == {
"source": "corpus-upload",
"display_name": "the uploads",
"label": None,
"tags": ["upload"],
}
assert by_source["gitea-forge"]["display_name"] is None
assert by_source["media-vault"]["tags"] == []
def test_get_lists_env_source_names_while_the_table_is_empty(
admin_client: TestClient, env_sources: Callable[[str], None]
) -> None:
"""An env-only deployment (no ``git_sources`` rows): the
``BOR_GIT_SOURCES`` names are the catalogue, derived exactly as the
importer would name what it clones — and one of them is labelable."""
env_sources("https://a.example.com/one.git, git@b.example.com:two.git ,")
labels = admin_client.get("/api/source-labels").json()["labels"]
assert [entry["source"] for entry in labels] == ["one", "two"]
r = admin_client.put("/api/source-labels/one", json={"label": "the first repo"})
assert r.status_code == 200, r.text
assert r.json()["label"] == "the first repo"
def test_get_reads_three_tables_and_never_one_per_source(
admin_client: TestClient, db: Session
) -> None:
"""The one-query-per-table discipline: with four sources and one
label row the GET costs exactly three SELECTs — the registry, the
distinct document sources, the label rows. A per-source label lookup
(or a third existence query) fails this pin."""
_seed_three_kinds(db)
db.add(_doc(db, "third-source", "b.md"))
db.commit()
admin_client.put("/api/source-labels/media-vault", json={"display_name": "vault"})
with _captured_statements() as statements:
body = admin_client.get("/api/source-labels").json()
selects = _selects(statements)
assert [entry["source"] for entry in body["labels"]] == [
"corpus-upload",
"gitea-forge",
"media-vault",
"third-source",
]
assert len(selects) == 3, selects
assert sum(1 for s in selects if "FROM git_sources" in s) == 1
assert sum(1 for s in selects if "DISTINCT documents.source" in s) == 1
assert sum(1 for s in selects if "FROM source_labels" in s) == 1
# No whole-entity document read — the catalogue never hydrates content.
assert not any("documents.content" in s for s in selects), selects
# ---------------------------------------------------------------------------
# PUT — the upsert
# ---------------------------------------------------------------------------
def test_put_round_trips_all_three_fields(
admin_client: TestClient, db: Session
) -> None:
"""Set name + label + tags on a git source: 200 carries the stored
shape, the row exists, and the GET reads it back."""
_seed_three_kinds(db)
r = admin_client.put(
"/api/source-labels/gitea-forge",
json={
"display_name": "the git server",
"label": "self-hosted forge + Gitea",
"tags": ["gitea", "forge"],
},
)
assert r.status_code == 200, r.text
assert r.json() == {
"source": "gitea-forge",
"display_name": "the git server",
"label": "self-hosted forge + Gitea",
"tags": ["gitea", "forge"],
}
assert _row(db, "gitea-forge") == {
"source": "gitea-forge",
"display_name": "the git server",
"label": "self-hosted forge + Gitea",
"tags": ["gitea", "forge"],
}
entry = admin_client.get("/api/source-labels").json()["labels"]
assert entry[1]["display_name"] == "the git server"
def test_put_partial_update_shape(admin_client: TestClient, db: Session) -> None:
"""D5 three ways: a value SETS, an OMITTED field is untouched, an
explicit ``null`` CLEARS (and ``tags: []`` clears the tags while the
strings stay)."""
_seed_three_kinds(db)
admin_client.put(
"/api/source-labels/media-vault",
json={"display_name": "the media box", "label": "jellyfin + arrs", "tags": ["jellyfin"]},
)
# Only the label moves — the name and the tags keep their values.
r = admin_client.put("/api/source-labels/media-vault", json={"label": "arrs only"})
assert r.status_code == 200, r.text
assert r.json()["display_name"] == "the media box"
assert r.json()["label"] == "arrs only"
assert r.json()["tags"] == ["jellyfin"]
# An explicit null clears JUST that field.
r = admin_client.put("/api/source-labels/media-vault", json={"display_name": None})
assert r.status_code == 200, r.text
assert r.json() == {
"source": "media-vault",
"display_name": None,
"label": "arrs only",
"tags": ["jellyfin"],
}
# An empty tag list clears the tags; the label stays.
r = admin_client.put("/api/source-labels/media-vault", json={"tags": []})
assert r.json() == {
"source": "media-vault",
"display_name": None,
"label": "arrs only",
"tags": [],
}
assert _row(db, "media-vault")["tags"] == []
def test_put_all_omitted_body_is_a_200_no_op(
admin_client: TestClient, db: Session
) -> None:
"""The ``GitSourcePatchIn`` "all absent → 200 no-op" precedent: an
empty body changes nothing on a labelled row (and creates a defaults
row on an unlabelled one)."""
_seed_three_kinds(db)
admin_client.put(
"/api/source-labels/gitea-forge",
json={"display_name": "the git server", "tags": ["gitea"]},
)
r = admin_client.put("/api/source-labels/gitea-forge", json={})
assert r.status_code == 200, r.text
assert r.json() == {
"source": "gitea-forge",
"display_name": "the git server",
"label": None,
"tags": ["gitea"],
}
assert _row(db, "gitea-forge")["display_name"] == "the git server"
def test_put_trims_values_and_dedupes_tags(
admin_client: TestClient, db: Session
) -> None:
"""Trim before the caps (``GitSourceIn._trim_url``), and the ONE tag
rule: case-insensitive de-dupe with the FIRST case kept, order
otherwise preserved."""
_seed_three_kinds(db)
r = admin_client.put(
"/api/source-labels/gitea-forge",
json={
"display_name": " the git server ",
"label": " self-hosted forge ",
"tags": [" Gitea ", "gitea", "FORGE", "forge"],
},
)
assert r.status_code == 200, r.text
assert r.json()["display_name"] == "the git server"
assert r.json()["label"] == "self-hosted forge"
assert r.json()["tags"] == ["Gitea", "FORGE"]
@pytest.mark.parametrize(
("field", "value", "needle"),
[
("display_name", "x" * 121, "string_too_long"),
("label", "x" * 201, "string_too_long"),
("tags", ["tag"] * 11, "too_long"),
("tags", ["x" * 33], "string_too_long"),
("tags", [" "], "string_too_short"),
("tags", [""], "string_too_short"),
],
)
def test_put_validation_422s_leave_the_row_unchanged(
admin_client: TestClient,
db: Session,
field: str,
value: Any,
needle: str,
) -> None:
"""Every D5 cap is the framework's standard 422 (the pydantic body,
not a fixed ``detail``), names the offending field, and writes
NOTHING: the pre-seeded row is byte-identical afterwards.
The trim runs BEFORE the cap, so a 121-char value that trims to 120
is legal — the 120-char leg of each boundary is asserted too, or the
cap could silently be one short."""
_seed_three_kinds(db)
admin_client.put(
"/api/source-labels/gitea-forge",
json={"display_name": "the git server", "label": "forge", "tags": ["gitea"]},
)
before = _row(db, "gitea-forge")
if field == "display_name":
legal = "y" * 120
elif field == "label":
legal = "y" * 200
else:
legal = ["tag"] * 10 if len(value) == 11 else ["y" * 32]
r = admin_client.put("/api/source-labels/gitea-forge", json={field: value})
assert r.status_code == 422, r.text
detail = r.json()["detail"]
assert isinstance(detail, list), r.text
assert detail[0]["type"] == needle, detail
assert field in detail[0]["loc"], detail
assert _row(db, "gitea-forge") == before
assert admin_client.put(
"/api/source-labels/gitea-forge", json={field: legal}
).status_code == 200
def test_put_rejects_a_key_outside_the_shape(
admin_client: TestClient, db: Session
) -> None:
"""``extra="forbid"`` (the phase-91 voice): an undeclared key is a
422 and no row is created."""
_seed_three_kinds(db)
r = admin_client.put(
"/api/source-labels/media-vault",
json={"display_name": "vault", "source": "gitea-forge"},
)
assert r.status_code == 422, r.text
assert r.json()["detail"][0]["type"] == "extra_forbidden"
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
def test_put_rejects_a_malformed_tags_value(
admin_client: TestClient, db: Session
) -> None:
"""A ``tags`` value that is not a list of strings is pydantic's own
422, in its own voice (``list_type`` / ``string_type``): the trimmer
passes anything it does not understand straight through rather than
guessing, so the framework names the type and NOTHING is written."""
_seed_three_kinds(db)
r = admin_client.put("/api/source-labels/media-vault", json={"tags": "gitea"})
assert r.status_code == 422, r.text
assert r.json()["detail"][0]["type"] == "list_type"
r = admin_client.put(
"/api/source-labels/media-vault", json={"tags": ["gitea", 7]}
)
assert r.status_code == 422, r.text
assert r.json()["detail"][0]["type"] == "string_type"
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
def test_put_folds_empty_strings_to_null(admin_client: TestClient, db: Session) -> None:
"""An empty / whitespace-only ``display_name`` or ``label`` is the
CLEAR leg, not a stored empty string: "no name" and "an empty name"
must not be two different things for the prompt builders (task 05)."""
_seed_three_kinds(db)
admin_client.put(
"/api/source-labels/corpus-upload",
json={"display_name": "the uploads", "label": "the upload folder"},
)
r = admin_client.put(
"/api/source-labels/corpus-upload",
json={"display_name": " ", "label": ""},
)
assert r.status_code == 200, r.text
assert r.json()["display_name"] is None
assert r.json()["label"] is None
assert _row(db, "corpus-upload")["display_name"] is None
def test_put_404s_a_source_that_exists_nowhere(
admin_client: TestClient, db: Session
) -> None:
"""D5's no-orphan-row arm: neither a registry name nor an indexed
document → 404, and NOTHING is written (a dead label would boost
nothing and never appear in the UI to be cleared)."""
_seed_three_kinds(db)
r = admin_client.put("/api/source-labels/no-such-source", json={"label": "x"})
assert r.status_code == 404, r.text
assert r.json() == {"detail": "source not found"}
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
# A near-miss on the derived name is still unknown (no fuzzy leg).
r = admin_client.put("/api/source-labels/gitea-forg", json={"label": "x"})
assert r.status_code == 404, r.text
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
def test_put_422s_an_overlong_source_path(admin_client: TestClient) -> None:
"""The path param carries the ``documents.source`` 120 cap: a longer
name is the framework's standard 422 on the ``path`` location (it
never reaches the query)."""
r = admin_client.put(f"/api/source-labels/{'x' * 121}", json={"label": "x"})
assert r.status_code == 422, r.text
detail = r.json()["detail"]
assert detail[0]["type"] == "string_too_long"
assert detail[0]["loc"] == ["path", "source"]
def test_put_maps_the_tag_normalizer_error_to_422(
admin_client: TestClient,
db: Session,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The belt: ``normalize_tags`` is the ONE tag rule and its
``ValueError`` is the framework's 422 — never a 500. The request
shape caps the same numbers today, so the belt is pinned by
substituting the raising normalizer (the row stays byte-identical,
which is the part that must not regress)."""
_seed_three_kinds(db)
admin_client.put(
"/api/source-labels/gitea-forge", json={"display_name": "the git server"}
)
before = _row(db, "gitea-forge")
def _boom(_raw: list[str]) -> list[str]:
raise ValueError("at most 10 tags are allowed")
monkeypatch.setattr(source_labels_api, "normalize_tags", _boom)
r = admin_client.put("/api/source-labels/gitea-forge", json={"tags": ["gitea"]})
assert r.status_code == 422, r.text
assert r.json() == {"detail": "at most 10 tags are allowed"}
assert _row(db, "gitea-forge") == before
# ---------------------------------------------------------------------------
# DELETE — the reset
# ---------------------------------------------------------------------------
def test_delete_resets_then_404s(admin_client: TestClient, db: Session) -> None:
"""200 with the entry's post-reset state (the phase-168
``DELETE /api/admin/settings/{key}`` convention — the row re-renders
from the response it already has), the GET shows the nulls, and a
second DELETE is a 404 with one fixed detail."""
_seed_three_kinds(db)
admin_client.put(
"/api/source-labels/gitea-forge",
json={"display_name": "the git server", "label": "forge", "tags": ["gitea"]},
)
r = admin_client.delete("/api/source-labels/gitea-forge")
assert r.status_code == 200, r.text
assert r.json() == {
"source": "gitea-forge",
"display_name": None,
"label": None,
"tags": [],
}
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
entry = {e["source"]: e for e in admin_client.get("/api/source-labels").json()["labels"]}
assert entry["gitea-forge"] == {
"source": "gitea-forge",
"display_name": None,
"label": None,
"tags": [],
}
r = admin_client.delete("/api/source-labels/gitea-forge")
assert r.status_code == 404, r.text
assert r.json() == {"detail": "source label not found"}
def test_delete_works_for_a_source_that_has_vanished(
admin_client: TestClient, db: Session
) -> None:
"""The reset never consults the registry: a label whose source has
since been removed (phase 69's total removal) stays CLEARABLE, or a
label could never be taken back off a gone source."""
_seed_three_kinds(db)
admin_client.put("/api/source-labels/corpus-upload", json={"label": "the upload"})
db.execute(text("DELETE FROM documents WHERE source = 'corpus-upload'"))
db.commit()
r = admin_client.delete("/api/source-labels/corpus-upload")
assert r.status_code == 200, r.text
assert db.execute(text("SELECT count(*) FROM source_labels")).scalar_one() == 0
# ---------------------------------------------------------------------------
# The alias contract (D1)
# ---------------------------------------------------------------------------
def test_labelling_a_source_touches_nothing_but_the_label_row(
admin_client: TestClient, db: Session
) -> None:
"""The owner-confirmed alias contract, at the API: a rename writes
``source_labels`` and leaves ``documents.source`` (and with it every
``source/path`` locator, citation and KB-tree node) byte-identical —
and the catalogue's identity column stays the DERIVED name."""
_seed_three_kinds(db)
before = db.execute(
text("SELECT source, path FROM documents ORDER BY source, path")
).fetchall()
r = admin_client.put(
"/api/source-labels/gitea-forge", json={"display_name": "the git server"}
)
assert r.status_code == 200, r.text
db.expire_all()
after = db.execute(
text("SELECT source, path FROM documents ORDER BY source, path")
).fetchall()
assert after == before
assert [row[0] for row in after] == ["corpus-upload", "gitea-forge"]
catalogue = admin_client.get("/api/source-labels").json()["labels"]
assert [entry["source"] for entry in catalogue] == [
"corpus-upload",
"gitea-forge",
"media-vault",
]
assert catalogue[1]["display_name"] == "the git server"
# And the PUT still keys on the DERIVED name after the rename (the
# display name is never an address).
assert admin_client.put(
"/api/source-labels/the git server", json={"label": "nope"}
).status_code == 404
tests/unit/test_chat_gate.py
+113
View File
@@ -13,6 +13,7 @@ import json
import uuid import uuid
from collections.abc import Iterator, Sequence from collections.abc import Iterator, Sequence
from datetime import UTC, datetime from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
import pytest import pytest
@@ -27,6 +28,7 @@ from app.rag.agent import AGENT_TOOLS, SHOW_TEXT_TOOL
from app.rag.llm import StreamPiece, ToolCallPiece from app.rag.llm import StreamPiece, ToolCallPiece
from app.rag.prompts import SUGGEST_INTRO, build_deflect_prompt from app.rag.prompts import SUGGEST_INTRO, build_deflect_prompt
from app.rag.retriever import RetrievedChunk, weak_hit_titles from app.rag.retriever import RetrievedChunk, weak_hit_titles
from app.rag.source_labels import SourceLabelView
from app.rag.suggestions import MAX_SUGGESTIONS, derive_suggestions from app.rag.suggestions import MAX_SUGGESTIONS, derive_suggestions
from tests.conftest import ADMIN_PASSWORD from tests.conftest import ADMIN_PASSWORD
@@ -1817,3 +1819,114 @@ def test_deflected_turn_resolves_nothing_and_logs_zero(
done = [f for f in frames if f.get("type") == "done"] done = [f for f in frames if f.get("type") == "done"]
assert done and done[0]["deflected"] is True assert done and done[0]["deflected"] is True
assert "answer_text" not in done[0] assert "answer_text" not in done[0]
# ---------- phase 191 (D4, task 05): the turn's prompt learns the name ----------
def _label_view(
source: str = "Homelab",
display_name: str | None = None,
label: str | None = None,
) -> SourceLabelView:
return SourceLabelView(source=source, display_name=display_name, label=label, tags=())
def test_plan_turn_high_prompt_carries_the_source_name() -> None:
"""The HIGH branch hands the label views to ``build_high_prompt``: the
suggested block gains `` name="…"`` / `` label="…"`` right after
``source="…"`` — the model sees the name→docs mapping."""
doc = _doc("Gitea Upgrade", "GITEA_DOC_CONTENT", summary="Upgrading Gitea.")
names = {"Homelab": _label_view(display_name="the git server", label="self-hosted forge")}
plan = chat_api.plan_turn(
[_chunk(doc, 0.90, cosine=0.90)],
_settings(threshold=0.30),
names=names,
)
assert plan.deflected is False
assert 'source="Homelab" name="the git server" label="self-hosted forge" path=' in (
plan.system_prompt
)
assert "Upgrading Gitea." in plan.system_prompt # the body is untouched
def test_plan_turn_without_names_is_byte_identical() -> None:
"""THE kill pin on the call-through: ``names`` omitted, ``None`` and
``{}`` each build the exact pre-phase prompt (and every other TurnPlan
field is identical)."""
doc = _doc("Gitea Upgrade", "GITEA_DOC_CONTENT", summary="Upgrading Gitea.")
chunks = [_chunk(doc, 0.90, cosine=0.90)]
settings = _settings(threshold=0.30)
pre = chat_api.plan_turn(chunks, settings)
for names in (None, {}):
plan = chat_api.plan_turn(chunks, settings, names=names)
assert plan.system_prompt == pre.system_prompt
assert plan.deflected == pre.deflected
assert [d.id for d in plan.suggested_docs] == [d.id for d in pre.suggested_docs]
assert ' name="' not in pre.system_prompt
def test_plan_turn_low_prompt_ignores_names() -> None:
"""A deflected turn has no document blocks: the LOW prompt is
byte-identical whether or not the source is named (D4 touches the
suggested tier only)."""
doc = _doc("Mongolia Trivia", "MONGOLIA_CONTENT")
settings = _settings(threshold=0.62)
chunks = [_chunk(doc, 0.05, cosine=0.10, fts_hit=True)]
pre = chat_api.plan_turn(chunks, settings)
plan = chat_api.plan_turn(chunks, settings, names={"Homelab": _label_view("Homelab", "x")})
assert plan.deflected is True
assert plan.system_prompt == pre.system_prompt
# The pre-phase deflected build, unchanged, is what the LOW branch owes.
assert plan.system_prompt == build_deflect_prompt(
weak_hit_titles(chunks), notes=[], kb_overview=""
)
def test_chat_module_issues_exactly_one_label_read_per_turn() -> None:
"""The marker pin for the "one query per turn" contract: the endpoint
reads the label rows ONCE (``all_labels``) and derives both the D2 hit
set and the prompt map from that read — no second label query, and task
04's convenience wrapper (which would re-query) is gone from the
module."""
src = Path(chat_api.__file__).read_text(encoding="utf-8")
assert src.count("all_labels(") == 1
assert "question_name_hit_sources(" not in src
assert src.count("names=labels") == 1, "the single read is what the prompt receives"
assert "question_name_hits(" in src, "the pure D2 rule stays applied per view"
def test_endpoint_single_label_read_feeds_flags_and_prompt(
client: TestClient,
gate_env: tuple[_FakeSession, _CannedLLM],
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The one-read contract on the wire (task 04 + task 05): the ONE
``all_labels`` result of a turn sets the D2 boost flags AND the
``<document>`` name/label attributes — one query, two surfaces."""
_session, llm = gate_env
doc = _doc("Gitea Upgrade", "GITEA_DOC_CONTENT", summary="Upgrading Gitea.")
chunk = _chunk(doc, 0.90, cosine=0.90)
assert chunk.source_name_hit is False # the pre-flag state
monkeypatch.setattr(chat_api, "retrieve", _fake_retriever([chunk]))
views = {
"Homelab": SourceLabelView(
source="Homelab",
display_name="the git server",
label="self-hosted forge",
tags=("gitea",),
)
}
reads: list[int] = []
def _counting_all_labels(_db: Any) -> dict[str, SourceLabelView]:
reads.append(1)
return views
monkeypatch.setattr(chat_api, "all_labels", _counting_all_labels)
frames = _ask(client, "how do I restart the git server?")
assert [f for f in frames if f.get("type") == "done"], "the turn completed"
assert len(reads) == 1, "ONE label read for the whole turn"
assert chunk.source_name_hit is True, "the same read sets the task-04 boost flag"
system = llm.seen[0][0]["content"]
assert 'source="Homelab" name="the git server" label="self-hosted forge"' in system
tests/unit/test_config.py
+33
View File
@@ -45,6 +45,9 @@ def test_defaults_match_locked_decisions(monkeypatch: pytest.MonkeyPatch) -> Non
# selection-time document score is ON by default (0.005 — the # selection-time document score is ON by default (0.005 — the
# owner-tunable starting point; ``0`` is the kill switch). # owner-tunable starting point; ``0`` is the kill switch).
assert s.name_hit_bonus == 0.005 assert s.name_hit_bonus == 0.005
# Phase 191, D3: the SECOND bounded selection bonus (the source-name
# hit) ships ON at the same phase-119 starting point.
assert s.source_name_hit_bonus == 0.005
# Owner instruction 2026-08-22: answers may run up to 32 768 tokens. # Owner instruction 2026-08-22: answers may run up to 32 768 tokens.
assert s.max_output_tokens == 32_768 assert s.max_output_tokens == 32_768
# Phase 17: the model's thinking streams by default (kill-switch off). # Phase 17: the model's thinking streams by default (kill-switch off).
@@ -553,6 +556,36 @@ def test_name_hit_bonus_rejects_negative(monkeypatch: pytest.MonkeyPatch) -> Non
_settings() _settings()
def test_source_name_hit_bonus_default_and_env_override(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Phase 191, D3 (LOCKED A3): the bounded SOURCE-name bonus (the
second, independent selection knob over the ``source_labels`` row)
is ON by default at the phase-119 starting point (0.005) and
env-tunable so the owner re-tunes live (the ``BOR_NAME_HIT_BONUS``
pattern). ``0`` is legal — the byte-identical kill switch."""
monkeypatch.delenv("BOR_SOURCE_NAME_HIT_BONUS", raising=False)
s = _settings()
assert s.source_name_hit_bonus == 0.005
monkeypatch.setenv("BOR_SOURCE_NAME_HIT_BONUS", "0")
assert _settings().source_name_hit_bonus == 0.0
monkeypatch.setenv("BOR_SOURCE_NAME_HIT_BONUS", "0.01")
assert _settings().source_name_hit_bonus == 0.01
def test_source_name_hit_bonus_rejects_negative(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""``0`` is the byte-identical kill switch (the pre-phase-191
selection order) — a NEGATIVE bonus would demote source-named
documents (the exact opposite of D2/D3), so the validator fails
loudly at startup naming the field (the ``name_hit_bonus``
pattern, phase 191)."""
monkeypatch.setenv("BOR_SOURCE_NAME_HIT_BONUS", "-0.1")
with pytest.raises(ValidationError, match="source_name_hit_bonus"):
_settings()
def test_retrieval_ef_search_default_and_env_override( def test_retrieval_ef_search_default_and_env_override(
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
) -> None: ) -> None:
tests/unit/test_git_source_branch_ui.py
+7 -1
View File
@@ -373,7 +373,13 @@ def test_branch_field_css_rules_are_house_styled() -> None:
assert "var(--ink-soft)" in hint_rule and "font-size: 0.8rem" in hint_rule, ( assert "var(--ink-soft)" in hint_rule and "font-size: 0.8rem" in hint_rule, (
"the .field-hint values (the token field's hint styling)" "the .field-hint values (the token field's hint styling)"
) )
tag = css.find(".git-source-hidden-count,\n.git-source-branch-count {") # Phase 191 (task 03): the name/label/tags row's two tags ("named" and
# one tag chip) joined this SAME rule — the reuse that keeps the token
# input from shipping its own tag CSS.
tag = css.find(
".git-source-hidden-count,\n.git-source-branch-count,\n"
".source-label-state,\n.source-label-tag {"
)
assert tag != -1, "the row tag joins the 'hidden on' tag rule" assert tag != -1, "the row tag joins the 'hidden on' tag rule"
tag_rule = css[tag : css.find("}", tag)] tag_rule = css[tag : css.find("}", tag)]
assert "background: var(--bg)" in tag_rule and "border" in tag_rule, ( assert "background: var(--bg)" in tag_rule and "border" in tag_rule, (
tests/unit/test_hidden_folders_toggle.py
+8 -2
View File
@@ -398,8 +398,14 @@ def test_hidden_checkbox_css_rules_exist_with_accent_color() -> None:
) )
# Phase 135 (task 04): the "branch: <name>" state tag joined this # Phase 135 (task 04): the "branch: <name>" state tag joined this
# rule — the "hidden on" tag keeps the same idiom (text + a # rule — the "hidden on" tag keeps the same idiom (text + a
# distinct background, never color alone). # distinct background, never color alone). Phase 191 (task 03): the
tag = _css_rule(css, ".git-source-hidden-count,\n.git-source-branch-count") # name/label/tags row's "named" tag and its tag CHIPS joined the same
# rule (the token input reuses this tag, it ships none of its own).
tag = _css_rule(
css,
".git-source-hidden-count,\n.git-source-branch-count,\n"
".source-label-state,\n.source-label-tag",
)
assert "var(--ink)" in tag and "var(--bg)" in tag, ( assert "var(--ink)" in tag and "var(--bg)" in tag, (
"text + a distinct background (never color alone)" "text + a distinct background (never color alone)"
) )
tests/unit/test_overview.py
+217
View File
@@ -10,13 +10,22 @@ pins the deterministic ``Sources: …`` inventory tail (LOCKED A2): the
format/order/counts, the 1-doc survival, the happy-path stored content, format/order/counts, the 1-doc survival, the happy-path stored content,
the byte-identical zero-doc / ``LLMError`` paths, and the byte-identical the byte-identical zero-doc / ``LLMError`` paths, and the byte-identical
model input. model input.
And the phase-191 display names (task 05, D4): the INPUT lines lead with
the owner's ``display_name`` when the source has a label row (the raw
source name is replaced in the ``lite`` model's input ONLY — the stored
row and its ``Sources: …`` tail keep the derived name), byte-identical
for every unnamed source, and fed by exactly ONE ``all_labels`` read per
run (the marker pins at the end).
""" """
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import inspect
import logging import logging
import uuid import uuid
from datetime import UTC, datetime, timedelta from datetime import UTC, datetime, timedelta
from pathlib import Path
import pytest import pytest
from sqlalchemy import select, text from sqlalchemy import select, text
@@ -24,6 +33,7 @@ from sqlalchemy.orm import Session
from app.config import Settings, get_settings from app.config import Settings, get_settings
from app.models import Document, KbOverview from app.models import Document, KbOverview
from app.rag import overview as overview_module
from app.rag.llm import LLMError from app.rag.llm import LLMError
from app.rag.overview import ( from app.rag.overview import (
KB_OVERVIEW_MODE, KB_OVERVIEW_MODE,
@@ -35,6 +45,8 @@ from app.rag.overview import (
regenerate_overview, regenerate_overview,
) )
from app.rag.retriever import TRUNCATION_MARKER from app.rag.retriever import TRUNCATION_MARKER
from app.rag.source_labels import all_labels as real_all_labels
from app.rag.source_labels import set_label
REPLY = "- Homelab\n - Kubernetes (k3s)\n- Deployments\n - Borg backups" REPLY = "- Homelab\n - Kubernetes (k3s)\n- Deployments\n - Borg backups"
@@ -650,3 +662,208 @@ def test_regenerate_llm_error_preserves_an_existing_inventory_tail(
assert row is not None and row.content == previous assert row is not None and row.content == previous
finally: finally:
_truncate_documents(db) _truncate_documents(db)
# ---------------------------------------------------------------------------
# Phase 191 (D4, task 05): the input lines carry the owner's display name
# ---------------------------------------------------------------------------
#: The owner's alias for the ``gitea-infra`` source (phase 191 D1 — an
#: alias, never the identity).
GIT_SERVER = "the git server"
@pytest.fixture()
def clean_source_labels(db: Session):
"""Empty ``source_labels`` before and after (migration-0030 state)."""
db.execute(text("DELETE FROM source_labels"))
db.commit()
yield
db.execute(text("DELETE FROM source_labels"))
db.commit()
# ---------- build_overview_prompt: the names map (pure) ----------
@pytest.mark.parametrize(
"names",
[None, {}, {"Other": "another name"}],
ids=["names=None", "names={}", "no-entry-for-the-source"],
)
def test_input_lines_without_a_names_entry_are_byte_identical_to_pre_phase(
names: dict[str, str] | None,
) -> None:
"""THE kill pin (phase 191 D4): no map, an empty map and a map that
misses the row's source each build the pre-phase input byte-for-byte —
the same cap, the same separator, the same order."""
rows = [
("Homelab", "k8s.md", "Kubernetes", "A cluster.\nSource: Homelab/k8s.md"),
("Homelab", "borg.md", "Borg", None),
]
pre = build_overview_prompt(rows, max_chars=10_000)
assert build_overview_prompt(rows, max_chars=10_000, names=names) == pre
def test_input_line_leads_with_the_display_name_when_named() -> None:
"""A named source: the display name REPLACES the raw source name in the
lite model's input — the path / title / summary halves untouched."""
rows = [("gitea-infra", "docs/install.md", "Install", "The install notes.")]
_system, user = build_overview_prompt(rows, names={"gitea-infra": GIT_SERVER})
assert user == f"{GIT_SERVER} — docs/install.md — Install — The install notes."
assert "gitea-infra" not in user
def test_input_line_without_an_entry_keeps_the_raw_source_name() -> None:
"""Mixed catalogue: only the named source's lines change — every other
line stays exactly as the pre-phase builder wrote it."""
rows = [
("gitea-infra", "a.md", "A", "Sum A."),
("Homelab", "b.md", "B", "Sum B."),
]
_system, user = build_overview_prompt(rows, names={"gitea-infra": GIT_SERVER})
assert user.splitlines() == [
f"{GIT_SERVER} — a.md — A — Sum A.",
"Homelab — b.md — B — Sum B.",
]
def test_named_summary_less_line_has_no_dangling_dash() -> None:
"""The pre-phase "no summary → no trailing field" rule survives the
name swap."""
_system, user = build_overview_prompt(
[("gitea-infra", "a.md", "A", None)], names={"gitea-infra": GIT_SERVER}
)
assert user == f"{GIT_SERVER} — a.md — A"
def test_named_entry_with_an_empty_display_name_falls_back_to_the_source() -> None:
"""Defensive: an empty name in the map is "no name" (the API trims
empty submissions to NULL, so this never reaches the wire — the input
line must never lead with a blank)."""
_system, user = build_overview_prompt(
[("gitea-infra", "a.md", "A", None)], names={"gitea-infra": ""}
)
assert user == "gitea-infra — a.md — A"
def test_names_take_part_in_the_input_cap() -> None:
"""The cap counts the whole line (the name included) — the truncation
contract is unchanged: cut at the cap, marker on its own line."""
rows = [("gitea-infra", "a.md", "A", None)]
_system, plain = build_overview_prompt(rows, max_chars=10)
_system, named = build_overview_prompt(rows, max_chars=10, names={"gitea-infra": GIT_SERVER})
assert named != plain
assert named.endswith(TRUNCATION_MARKER)
assert named.startswith(GIT_SERVER[:10])
# ---------- regenerate_overview: the ONE all_labels read (DB) ----------
def test_regenerate_model_input_carries_the_display_name(
db: Session, clean_overview, clean_source_labels
) -> None:
"""The wire half of D4: the single read feeds the input lines, and the
unnamed source's line is untouched."""
_truncate_documents(db)
set_label(db, "gitea-infra", display_name=GIT_SERVER)
db.commit()
try:
base = datetime(2024, 6, 15, 12, 0, 0, tzinfo=UTC)
_add_doc(db, "gitea-infra", "a.md", "A", "Sum A.", created_at=base + timedelta(days=1))
_add_doc(db, "Homelab", "b.md", "B", "Sum B.", created_at=base)
llm = _FakeLLM()
assert asyncio.run(regenerate_overview(llm, db)) is True
user = llm.messages[1]["content"]
assert user.splitlines() == [
f"{GIT_SERVER} — a.md — A — Sum A.",
"Homelab — b.md — B — Sum B.",
]
finally:
_truncate_documents(db)
def test_regenerate_stored_content_keeps_the_derived_names(
db: Session, clean_overview, clean_source_labels
) -> None:
"""The ALIAS contract (D1) on the stored row: the deterministic
``Sources: …`` inventory tail names sources by their DERIVED name —
a rename never changes the identity the rest of the app keys on."""
_truncate_documents(db)
set_label(db, "gitea-infra", display_name=GIT_SERVER, label="self-hosted forge")
db.commit()
try:
_add_doc(db, "gitea-infra", "a.md", "A")
assert asyncio.run(regenerate_overview(_FakeLLM(), db)) is True
row = db.get(KbOverview, 1)
assert row is not None
assert row.content == REPLY + "\nSources: gitea-infra (1 doc)"
assert GIT_SERVER not in row.content
finally:
_truncate_documents(db)
def test_regenerate_without_label_rows_is_byte_identical_to_pre_phase(
db: Session, clean_overview, clean_source_labels
) -> None:
"""No label rows (the whole-KB kill pin): the model input and the
stored content are exactly the pre-phase bytes."""
_truncate_documents(db)
try:
_add_doc(db, "Homelab", "a.md", "A", "Sum A.")
llm = _FakeLLM()
assert asyncio.run(regenerate_overview(llm, db)) is True
assert llm.messages[1]["content"] == "Homelab — a.md — A — Sum A."
row = db.get(KbOverview, 1)
assert row is not None and row.content == REPLY + "\nSources: Homelab (1 doc)"
finally:
_truncate_documents(db)
def test_regenerate_reads_the_label_rows_once(
db: Session, clean_overview, clean_source_labels, monkeypatch: pytest.MonkeyPatch
) -> None:
"""ONE ``all_labels`` read per run (the task's single-query contract):
the regenerator calls it exactly once and derives the name map from
those views — no second query."""
_truncate_documents(db)
set_label(db, "gitea-infra", display_name=GIT_SERVER)
db.commit()
calls: list[int] = []
def _counting_all_labels(session: Session) -> dict:
calls.append(1)
return real_all_labels(session)
monkeypatch.setattr(
"app.rag.overview.all_labels",
_counting_all_labels,
)
try:
_add_doc(db, "gitea-infra", "a.md", "A")
_add_doc(db, "gitea-infra", "b.md", "B")
assert asyncio.run(regenerate_overview(_FakeLLM(), db)) is True
assert len(calls) == 1
finally:
_truncate_documents(db)
# ---------- the marker pins: one read, in the right function ----------
OVERVIEW_PY = Path(overview_module.__file__).resolve().read_text(encoding="utf-8")
def test_overview_module_issues_exactly_one_all_labels_read() -> None:
"""The marker pin: the ONE ``all_labels(` call in the module — the
overview builder never queries labels on its own."""
assert OVERVIEW_PY.count("all_labels(") == 1
def test_the_single_read_lives_in_regenerate_overview() -> None:
"""The read sits in the caller (which already holds the session), not
in the pure builder — so the builder stays a pure function of its
arguments and one read covers the whole run."""
src = inspect.getsource(regenerate_overview)
assert src.count("all_labels(") == 1
assert "all_labels(" not in inspect.getsource(build_overview_prompt)
tests/unit/test_prompts.py
+209
View File
@@ -39,6 +39,12 @@ phase-124 ``show_image`` clause through one mechanism, and every build
the flags do not select stays byte-identical (the constant's own sha pin the flags do not select stays byte-identical (the constant's own sha pin
lives in :mod:`tests.unit.test_prompt_lock`). lives in :mod:`tests.unit.test_prompt_lock`).
And the phase-191 source names (task 05, D4): a ``<document>`` block can
carry the owner's label row as the ADDITIVE ``name`` / ``label``
attributes (order ``source`` → ``name?`` → ``label?`` → ``path`` →
``title`` → ``date``, XML-escaped values), with no map / no entry for the
source byte-identical to pre-phase (the kill pin).
And the phase-119 cite discipline (task 04, LOCKED A5): the intro's And the phase-119 cite discipline (task 04, LOCKED A5): the intro's
final sentence ("cite the document(s) you used, by path", phase 118) final sentence ("cite the document(s) you used, by path", phase 118)
is REPLACED — the HIGH prompt carries the discipline sentence exactly is REPLACED — the HIGH prompt carries the discipline sentence exactly
@@ -65,6 +71,7 @@ from app.rag.prompts import (
SHOW_TEXT_CLAUSE, SHOW_TEXT_CLAUSE,
SUGGEST_INTRO, SUGGEST_INTRO,
TOOLS_SECTION, TOOLS_SECTION,
_attr,
_base, _base,
_document_body, _document_body,
_extract_context_line, _extract_context_line,
@@ -74,6 +81,7 @@ from app.rag.prompts import (
build_steering_section, build_steering_section,
) )
from app.rag.retriever import TRUNCATION_MARKER from app.rag.retriever import TRUNCATION_MARKER
from app.rag.source_labels import SourceLabelView
#: A small multi-line outline standing in for the lite-generated one. #: A small multi-line outline standing in for the lite-generated one.
OVERVIEW = "- Homelab\n - Kubernetes (k3s)\n- Deployments\n - Borg backups" OVERVIEW = "- Homelab\n - Kubernetes (k3s)\n- Deployments\n - Borg backups"
@@ -1944,3 +1952,204 @@ def test_high_prompt_pdf_image_doc_block_exact_shape() -> None:
assert prompt.index(line) < prompt.index("A diagram of the cluster.") assert prompt.index(line) < prompt.index("A diagram of the cluster.")
assert block_start < prompt.index(line) < prompt.index("</document>") assert block_start < prompt.index(line) < prompt.index("</document>")
assert "The second page of the guide" not in prompt assert "The second page of the guide" not in prompt
# ---------------------------------------------------------------------------
# Phase 191 (D4, task 05): the ``<document>`` block learns the owner's name
# ---------------------------------------------------------------------------
#: The owner's display name for the fixture source (phase 191 D1 — an
#: ALIAS beside the derived name, never a replacement for it).
GIT_SERVER = "the git server"
#: The owner's one-line description of the same source.
FORGE_LABEL = "self-hosted forge + Gitea"
def _label_view(
source: str = "Homelab",
display_name: str | None = None,
label: str | None = None,
tags: tuple[str, ...] = (),
) -> SourceLabelView:
"""One label-row view (task 01's frozen dataclass) for the pins."""
return SourceLabelView(
source=source,
display_name=display_name,
label=label,
tags=tags,
)
def _block_head(prompt: str) -> str:
"""The opening tag of the FIRST ``<document>`` block (attribute
order/escaping pins read it whole, so nothing is asserted loosely)."""
start = prompt.index("<document ")
return prompt[start : prompt.index(">", start) + 1]
@pytest.mark.parametrize(
"names",
[
None,
{},
{"Other": _label_view("Other", display_name=GIT_SERVER, label=FORGE_LABEL)},
],
ids=["names=None", "names={}", "no-entry-for-the-source"],
)
def test_high_prompt_without_a_label_entry_is_byte_identical_to_pre_phase(
names: dict[str, SourceLabelView] | None,
) -> None:
"""THE kill pin (phase 191 D4): no map, an empty map and a map with no
entry for the document's source each build a prompt byte-identical to
the pre-phase build — no ``name``, no ``label``, no stray space."""
doc = _doc(
"kubernetes.md",
"Talos Linux on three nodes.",
"Kubernetes Homelab Cluster",
summary="A Talos Linux cluster on three nodes.",
)
pre = build_high_prompt([doc])
assert build_high_prompt([doc], names=names) == pre
assert ' name="' not in pre and ' label="' not in pre
def test_no_label_block_keeps_the_exact_pre_phase_head() -> None:
"""The block head for a label-less source, byte for byte (the shape the
phase-106/118/125 pins rely on: ``source`` → ``path`` → ``title`` →
``date``, single-spaced)."""
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes Homelab Cluster", summary="S.")
assert _block_head(build_high_prompt([doc], names={})) == (
'<document source="Homelab" path="kubernetes.md" '
'title="Kubernetes Homelab Cluster" date="2024-06-15">'
)
def test_document_block_name_attribute_sits_immediately_after_source() -> None:
"""A ``display_name``-only row: exactly `` name="…"`` added, directly
after ``source="…"`` and before ``path`` (the attribute-order pin)."""
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
prompt = build_high_prompt([doc], names={"Homelab": _label_view(display_name=GIT_SERVER)})
assert _block_head(prompt) == (
'<document source="Homelab" name="the git server" path="kubernetes.md" '
'title="Kubernetes" date="2024-06-15">'
)
assert ' label="' not in prompt
def test_document_block_label_only_view_adds_exactly_the_label_attribute() -> None:
"""A ``label``-only row gains ONE attribute — an unset field adds
NOTHING (never an empty ``name=""``)."""
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
prompt = build_high_prompt([doc], names={"Homelab": _label_view(label=FORGE_LABEL)})
assert _block_head(prompt) == (
'<document source="Homelab" label="self-hosted forge + Gitea" '
'path="kubernetes.md" title="Kubernetes" date="2024-06-15">'
)
assert ' name="' not in prompt
def test_document_block_row_with_neither_field_is_byte_identical() -> None:
"""A row that exists but carries neither name nor label (the owner
saved only tags) leaves the block untouched — tags never render in the
block (they are a retrieval input, D2/D3)."""
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
pre = build_high_prompt([doc])
names = {"Homelab": _label_view(tags=("gitea", "forge"))}
built = build_high_prompt([doc], names=names)
assert built == pre
assert "gitea" not in built
def test_document_block_carries_both_attributes_in_the_locked_order() -> None:
"""Both fields set → both attributes, in the fixed order
``source`` → ``name`` → ``label`` → ``path`` → ``title`` → ``date``."""
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
names = {"Homelab": _label_view(display_name=GIT_SERVER, label=FORGE_LABEL)}
head = _block_head(build_high_prompt([doc], names=names))
order = ['source="', 'name="', 'label="', 'path="', 'title="', 'date="']
positions = [head.index(attr) for attr in order]
assert positions == sorted(positions)
assert head == (
'<document source="Homelab" name="the git server" '
'label="self-hosted forge + Gitea" path="kubernetes.md" '
'title="Kubernetes" date="2024-06-15">'
)
def test_document_block_name_and_label_are_xml_escaped() -> None:
"""The escaping pin (task 05 work item 1): a ``"`` in owner text renders
as ``&quot;`` so the attribute cannot break; ``&``/``<`` fold too."""
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
names = {
"Homelab": _label_view(
display_name='Reese "forge" & <co>',
label='say "hi"',
)
}
prompt = build_high_prompt([doc], names=names)
assert ' name="Reese &quot;forge&quot; &amp; &lt;co&gt;"' in prompt
assert ' label="say &quot;hi&quot;"' in prompt
# The unescaped form never reaches the wire: the attribute stays closed.
assert ' name="Reese "forge"' not in prompt
head = _block_head(prompt)
assert head.count('"') % 2 == 0, "an unescaped quote would break the attribute"
def test_attribute_escaping_helper_covers_the_xml_five() -> None:
"""``_attr`` is the ONE escaper for the new attribute values: the XML
specials go, the single quote stays (the delimiter is the double
quote, and ``Reese's`` must stay readable to the model)."""
assert _attr('a & b < c > d " e') == "a &amp; b &lt; c &gt; d &quot; e"
assert _attr("Reese's forge") == "Reese's forge"
def test_names_apply_per_document_source_and_never_touch_the_body() -> None:
"""Only the blocks whose source has an entry gain the attribute; the
summary body (and the other source's block) are untouched."""
named = _doc("a.md", "CONTENT_A", "Title A", summary="SUMMARY A.")
other = _doc("b.md", "CONTENT_B", "Title B", summary="SUMMARY B.")
other.source = "Other"
prompt = build_high_prompt(
[named, other],
names={"Homelab": _label_view(display_name=GIT_SERVER)},
)
assert 'source="Homelab" name="the git server" path="a.md"' in prompt
assert 'source="Other" path="b.md"' in prompt
assert prompt.count(GIT_SERVER) == 1, "the name rides the attribute, nothing else"
assert "SUMMARY A." in prompt and "SUMMARY B." in prompt
def test_names_on_an_empty_documents_build_change_nothing() -> None:
"""No blocks → nothing to annotate: the honest "(no documents
matched…)" build is byte-identical with a label row in hand."""
names = {"Homelab": _label_view(display_name=GIT_SERVER, label=FORGE_LABEL)}
assert build_high_prompt([], names=names) == build_high_prompt([])
def test_names_compose_with_folder_lines_and_display_clauses() -> None:
"""The additive attributes ride alongside the phase-119 folder lines and
the phase-124/182 display clauses — the section order and the intro are
untouched (the folder lines stay plain, no new markup)."""
doc = _doc("kubernetes.md", "CONTENT", "Kubernetes", summary="S.")
names = {"Homelab": _label_view(display_name=GIT_SERVER)}
prompt = build_high_prompt(
[doc],
folder_lines=["docs/gitea.md — Gitea install"],
show_image=True,
show_text=True,
names=names,
)
assert 'source="Homelab" name="the git server" path="kubernetes.md"' in prompt
assert prompt.index(SUGGEST_INTRO) < prompt.index("docs/gitea.md — Gitea install")
assert prompt.index("docs/gitea.md — Gitea install") < prompt.index("<document ")
assert prompt.endswith(TOOLS_SECTION[: -len("</tools>")]
+ SHOW_IMAGE_CLAUSE + "\n\n" + SHOW_TEXT_CLAUSE + "\n</tools>")
def test_deflect_prompt_has_no_name_or_label_surface() -> None:
"""D4 touches the suggested-tier block only: the deflected prompt is
byte-identical (it carries no document blocks to annotate)."""
prompt = build_deflect_prompt(["Kubernetes Homelab Cluster"])
assert ' name="' not in prompt and ' label="' not in prompt
assert GIT_SERVER not in prompt
tests/unit/test_retriever.py
+174 -9
View File
@@ -1260,9 +1260,12 @@ def _bonus_rc(
cosine: float, cosine: float,
position: int = 0, position: int = 0,
name_hit: bool = False, name_hit: bool = False,
source_name_hit: bool = False,
) -> RetrievedChunk: ) -> RetrievedChunk:
"""One fused-list candidate (name-hit rows follow the D1 lexical """One fused-list candidate (name-hit rows follow the D1 lexical
convention: ``cosine=0.0``, ``fts_hit=True``).""" convention: ``cosine=0.0``, ``fts_hit=True``). The
``source_name_hit`` flag (phase 191, D2) is set at the retrieval
boundary in chat.py — here directly, on the fused row."""
return RetrievedChunk( return RetrievedChunk(
chunk_id=uuid.uuid4(), chunk_id=uuid.uuid4(),
position=position, position=position,
@@ -1272,6 +1275,7 @@ def _bonus_rc(
cosine=cosine, cosine=cosine,
fts_hit=cosine == 0.0, fts_hit=cosine == 0.0,
name_hit=name_hit, name_hit=name_hit,
source_name_hit=source_name_hit,
) )
@@ -1479,24 +1483,185 @@ def test_weak_hit_titles_bonus_adjusted_order() -> None:
def test_bonus_lives_in_the_selection_layer_only() -> None: def test_bonus_lives_in_the_selection_layer_only() -> None:
"""LOCKED A3: the bonus never touches the chunk objects — """LOCKED A3 (mirrored by phase 191 for the SECOND bonus): the
bonuses never touch the chunk objects —
``score``/``cosine``/``fts_hit`` (the A8 gate's inputs — ``score``/``cosine``/``fts_hit`` (the A8 gate's inputs —
``query_log.top_score`` is the best fused chunk score, the same ``query_log.top_score`` is the best fused chunk score, the same
values) are unchanged after every selection walk, even with the values) and the ``name_hit``/``source_name_hit`` flags themselves
bonus lifting a document.""" are unchanged after every selection walk, even with the bonuses
lifting a document."""
chunks = _bonus_chunks() chunks = _bonus_chunks()
chunks[-1].source_name_hit = True
before = { before = {
rc.chunk_id: (rc.score, rc.cosine, rc.fts_hit, rc.name_hit) for rc in chunks rc.chunk_id: (
rc.score,
rc.cosine,
rc.fts_hit,
rc.name_hit,
rc.source_name_hit,
)
for rc in chunks
} }
select_suggested(chunks, n=5, bonus=0.005) select_suggested(chunks, n=5, bonus=0.005, source_bonus=0.005)
select_related(chunks, set(), cap=5, bonus=0.005) select_related(chunks, set(), cap=5, bonus=0.005, source_bonus=0.005)
weak_hit_titles(chunks, bonus=0.005) weak_hit_titles(chunks, bonus=0.005, source_bonus=0.005)
after = { after = {
rc.chunk_id: (rc.score, rc.cosine, rc.fts_hit, rc.name_hit) for rc in chunks rc.chunk_id: (
rc.score,
rc.cosine,
rc.fts_hit,
rc.name_hit,
rc.source_name_hit,
)
for rc in chunks
} }
assert before == after assert before == after
# ---------------------------------------------------------------------------
# Phase 191, D3 — the bounded SOURCE-name bonus (the second, independent
# knob; the phase-119 mirror: kill switch, bounded per-document apply,
# the fuse OR-merge, the settings defaults)
# ---------------------------------------------------------------------------
def test_selection_order_source_bonus_zero_is_byte_identical_to_pre_phase() -> None:
"""The kill switch (LOCKED A3, phase-119 mirror): with
*source_bonus* ``0`` the walk is byte-identical to the
pre-phase-191 walk even when EVERY chunk carries the flag — same
order, same effective scores (the phase-119 name bonus alone). With
BOTH knobs ``0`` the pre-phase golden walk returns byte-identical."""
chunks = _bonus_chunks()
for rc in chunks:
rc.source_name_hit = True
out = _selection_order(chunks, 0.005, 0.0)
assert [d.path for d, _eff, _cos, _idx in out] == [
"gitea/README.md", "a.md", "b.md", "c.md", "d.md"
]
# No +0.005 rides the effective scores — the name bonus alone.
assert out[0][1] == pytest.approx(0.016 + 0.005)
off = _selection_order(chunks, 0.0, 0.0)
assert [d.path for d, *_ in off] == GOLDEN_PRE_PHASE_ORDER
assert [eff for _d, eff, _cos, _idx in off] == [
0.0200, 0.0160, 0.0150, 0.0150, 0.0100
]
def test_selection_order_source_bonus_inert_without_source_hits() -> None:
"""No ``source_name_hit`` chunk present → the second bonus cannot
fire: with both knobs on the walk equals the phase-119 name-bonus
walk exactly (the flagged-signal presence, not just the knob, gates
the re-sort)."""
out = _selection_order(_bonus_chunks(), 0.005, 0.005)
assert [d.path for d, _eff, _cos, _idx in out] == [
"gitea/README.md", "a.md", "b.md", "c.md", "d.md"
]
assert out[0][1] == pytest.approx(0.016 + 0.005)
def test_selection_order_source_bonus_lifts_named_doc_over_equal_fused() -> None:
"""The boost (D3): a document whose source is a D2 hit
(``source_name_hit=True``) ranks ABOVE an equal-fused-score
non-hit — the fused tie normally resolves on cosine (a's 0.9 beats
the hit's 0.0), the 0.005 head start decides instead."""
gitserver = _doc("gitserver.md", "G" * 50)
a = _doc("a.md", "A" * 50)
chunks = [
_bonus_rc(a, 0.0200, 0.9),
_bonus_rc(gitserver, 0.0200, 0.0, source_name_hit=True),
]
assert [d.path for d, *_ in _selection_order(chunks, 0.0, 0.0)] == [
"a.md", "gitserver.md"
]
out = _selection_order(chunks, 0.0, 0.005)
assert [d.path for d, *_ in out] == ["gitserver.md", "a.md"]
assert out[0][1] == pytest.approx(0.0200 + 0.005)
def test_selection_order_two_bonuses_bounded_at_two_per_document() -> None:
"""A document carrying BOTH flags gets EXACTLY +0.010 (the bounded
two-bonus pin — no amplification): every one of the doc's THREE
chunks is flagged for both signals, yet each bonus applies ONCE per
document — best score + bonus + source_bonus, never more."""
gitserver = _doc("gitserver.md", "G" * 50)
a = _doc("a.md", "A" * 50)
chunks = [
_bonus_rc(a, 0.0300, 0.9),
_bonus_rc(gitserver, 0.0260, 0.0, 0, name_hit=True, source_name_hit=True),
_bonus_rc(gitserver, 0.0200, 0.0, 1, name_hit=True, source_name_hit=True),
_bonus_rc(gitserver, 0.0100, 0.0, 2, name_hit=True, source_name_hit=True),
]
out = _selection_order(chunks, 0.005, 0.005)
assert [d.path for d, *_ in out] == ["gitserver.md", "a.md"]
assert out[0][1] == pytest.approx(0.0260 + 0.010)
def test_selection_helpers_pull_the_source_bonus_from_settings(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""All three selection helpers default *source_bonus* to the LIVE
``BOR_SOURCE_NAME_HIT_BONUS`` setting (the phase-119 ``bonus``
pattern): the settings default 0.005 lifts the flagged doc, a
settings kill switch (``0``) and an explicit ``source_bonus=0``
both reproduce the pre-phase-191 walk."""
chunks = _bonus_chunks()
chunks[-1].score = 0.0199 # d.md just UNDER a.md's 0.0200
chunks[-1].source_name_hit = True
on = Settings(_env_file=None) # pyright: ignore[reportCallIssue]
assert on.source_name_hit_bonus == 0.005 # the production default
monkeypatch.setattr(retriever, "get_settings", lambda: on)
lifted = ["d.md", "gitea/README.md", "a.md", "b.md", "c.md"]
assert [d.path for d in select_suggested(chunks, n=5)] == lifted
assert [d.path for d in select_related(chunks, set(), cap=5)] == lifted
assert weak_hit_titles(chunks) == lifted
# Explicit kill switch — the pre-phase-191 (name-bonus-only) walk.
assert [d.path for d in select_suggested(chunks, n=5, source_bonus=0.0)] == [
"gitea/README.md", "a.md", "d.md", "b.md", "c.md"
]
# Settings kill switch (BOR_SOURCE_NAME_HIT_BONUS=0) — the same walk.
off = Settings(_env_file=None, source_name_hit_bonus=0.0) # pyright: ignore[reportCallIssue]
monkeypatch.setattr(retriever, "get_settings", lambda: off)
pre = ["gitea/README.md", "a.md", "d.md", "b.md", "c.md"]
assert [d.path for d in select_suggested(chunks, n=5)] == pre
assert [d.path for d in select_related(chunks, set(), cap=5)] == pre
assert weak_hit_titles(chunks) == pre
def test_fuse_or_s_source_name_hit_on_double_hit() -> None:
"""A vector row merged with a flagged source-name row keeps
``source_name_hit=True`` — the merge ORs the flag in (the
``name_hit`` mirror), so the selection tier still sees the hit on
the fused list."""
v = _rc("gitserver.md", cosine=0.9)
l1 = _rc("gitserver.md", cosine=0.1) # the lexical copy of the same chunk
l1.chunk_id = v.chunk_id
l1.source_name_hit = True
out = fuse([v], [l1], k=60)
assert len(out) == 1
assert out[0].source_name_hit is True
assert out[0].fts_hit is True
assert out[0].score == pytest.approx(2 / 61)
def test_fuse_keeps_source_name_hit_on_lexical_only_hit() -> None:
"""A source-name-hit row with no vector rank keeps
``source_name_hit=True`` through the fusion (the ``replace()``
copy carries the field — the ``name_hit`` mirror)."""
row = _rc("gitserver.md")
row.source_name_hit = True
out = fuse([], [row], k=60)
assert len(out) == 1
assert out[0].source_name_hit is True
def test_fuse_default_source_name_hit_stays_false_for_ordinary_rows() -> None:
"""Neither list flagged ⇒ fusion never invents a source-name hit —
ordinary vector and FTS rows are ``source_name_hit=False``."""
out = fuse([_rc("a.md", cosine=0.8)], [_rc("b.md", fts_hit=True)], k=60)
assert len(out) == 2
assert all(rc.source_name_hit is False for rc in out)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Phase 162, D1 — the accent-folded name normalization (LOCKED A1): the # Phase 162, D1 — the accent-folded name normalization (LOCKED A1): the
# 2026-10-07 live gap, the ASCII filename vs the accented content layer # 2026-10-07 live gap, the ASCII filename vs the accented content layer
tests/unit/test_settings_catalog.py
+23
View File
@@ -253,6 +253,29 @@ def test_sync_session_bound_field_types() -> None:
assert Settings.model_fields["sync_prune_batch"].default == 500 assert Settings.model_fields["sync_prune_batch"].default == 500
def test_source_name_hit_bonus_row_is_a_plain_live_overridable_row() -> None:
"""Phase 191 (task 04, D3): the second selection-bonus row sits in
``Retrieval & honesty`` (right beside ``name_hit_bonus`` — the two
independently kill-able knobs) with the ``Source name hit bonus``
label and mirrors the ``name_hit_bonus`` entry's flags exactly:
plain — not ``secret`` (a float), not ``restart_required`` (the
selection walk reads ``get_settings()`` per turn, so a saved
override applies on the next question), overridable (the overlay is
generic over ``float`` fields). ``0`` is the byte-identical kill
switch; a negative value fails startup, not the panel."""
meta = CATALOG["source_name_hit_bonus"]
assert meta.group == "Retrieval & honesty"
assert meta.label == "Source name hit bonus"
assert meta.secret is False
assert meta.restart_required is False
assert meta.overridable is True
assert "source_name_hit_bonus" not in SECRETS
assert "source_name_hit_bonus" not in RESTART_REQUIRED
assert "source_name_hit_bonus" not in NON_OVERRIDABLE
assert Settings.model_fields["source_name_hit_bonus"].annotation is float
assert Settings.model_fields["source_name_hit_bonus"].default == 0.005
def test_labels_nonempty_and_unique() -> None: def test_labels_nonempty_and_unique() -> None:
labels = [meta.label for meta in CATALOG.values()] labels = [meta.label for meta in CATALOG.values()]
assert all(label.strip() for label in labels), "empty label in the catalogue" assert all(label.strip() for label in labels), "empty label in the catalogue"
tests/unit/test_source_labels.py
+370
View File
@@ -0,0 +1,370 @@
"""Unit: source name/label/tags helpers + the D2 matching rule (phase 191, task 01).
The pure half (``normalize_tags``, ``question_name_hits``) needs no DB;
the session helpers run against the local compose Postgres
(``podman compose up -d db``) — the house DB-test pattern of
``test_sources_meta.py``: the helpers are thin session wrappers whose
contract (upsert, partial-update shape, flush-not-commit, absent-row
``None``/``False``) only holds against a real database. Skips with clear
instructions when the stack is not up.
The D2 rule is the heart of the phase — every branch is pinned: the
name/label/tag hit classes, the case-insensitive ``_normalize_name``
form (the phase-162 accent fold included), the 4-char minimum on BOTH
sides, the NO digit-class rule ("gitea" hits digitless), ``None`` →
``False``, and the no-overlap baseline.
A fixture truncates ``source_labels`` before and after every test so
the suite leaves the dev DB exactly as migration 0030 left it (empty).
"""
from __future__ import annotations
import pytest
from sqlalchemy import text
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from app.db import SessionLocal
from app.rag.source_labels import (
TAGS_MAX_COUNT,
SourceLabelView,
all_labels,
delete_label,
get_label,
label_for,
normalize_tags,
question_name_hit_sources,
question_name_hits,
set_label,
)
@pytest.fixture(autouse=True)
def clean_source_labels(db: Session):
"""Empty ``source_labels`` before and after every test (the
migration-0030 state)."""
db.execute(text("DELETE FROM source_labels"))
db.commit()
yield
db.execute(text("DELETE FROM source_labels"))
db.commit()
# ---------------------------------------------------------------------------
# The model round-trip + the upsert / partial-update shape (D1)
# ---------------------------------------------------------------------------
def test_set_label_creates_row_with_stamp(db: Session) -> None:
"""All three fields set: the row is created verbatim and
``updated_at`` carries the server stamp (the house style)."""
row = set_label(
db,
"gitea-forge",
display_name="the git server",
label="self-hosted forge + Gitea",
tags=["gitea", "forge"],
)
db.commit()
assert row.source == "gitea-forge"
stored = get_label(db, "gitea-forge")
assert stored is not None
assert stored.display_name == "the git server"
assert stored.label == "self-hosted forge + Gitea"
assert stored.tags == ["gitea", "forge"] # stored exactly as normalized
assert stored.updated_at is not None, "updated_at must be server-stamped"
def test_set_label_defaults_omitted_fields(db: Session) -> None:
"""A brand-new row created with only ``display_name`` carries the
D1 defaults: ``label`` NULL and ``tags`` ``[]`` (NOT NULL column)."""
set_label(db, "uploads", display_name="the uploads")
db.commit()
stored = get_label(db, "uploads")
assert stored is not None
assert stored.display_name == "the uploads"
assert stored.label is None
assert stored.tags == []
def test_partial_update_omitted_field_untouched(db: Session) -> None:
"""The partial-update shape: an OMITTED field leaves the stored
value untouched (the ``_UNSET`` sentinel owns the difference)."""
set_label(
db, "homelab", display_name="the lab", label="first", tags=["homelab"]
)
db.commit()
set_label(db, "homelab", label="second")
db.commit()
stored = get_label(db, "homelab")
assert stored is not None
assert stored.label == "second", "the passed field overwrites"
assert stored.display_name == "the lab", "the omitted field is untouched"
assert stored.tags == ["homelab"], "the omitted tags are untouched"
def test_partial_update_explicit_none_clears(db: Session) -> None:
"""An explicit ``None`` CLEARS the field (the D5 shape); ``tags``
clears to ``[]`` — the column stays NOT NULL."""
set_label(
db, "homelab", display_name="the lab", label="first", tags=["homelab"]
)
db.commit()
set_label(db, "homelab", display_name=None, label=None, tags=None)
db.commit()
stored = get_label(db, "homelab")
assert stored is not None
assert stored.display_name is None
assert stored.label is None
assert stored.tags == []
def test_set_label_flushes_without_committing(db: Session) -> None:
"""Flush-only (the phase-53 session convention): a rolled-back
session rolls the upsert back with it."""
set_label(db, "homelab", display_name="the lab")
db.rollback()
other = SessionLocal()
try:
assert get_label(other, "homelab") is None, "the uncommitted row must roll back"
finally:
other.close()
def test_source_is_the_sole_pk(db: Session) -> None:
"""One label row per source: the raw duplicate insert raises."""
db.execute(text("INSERT INTO source_labels (source) VALUES ('dup')"))
db.commit()
with pytest.raises(IntegrityError):
db.execute(text("INSERT INTO source_labels (source) VALUES ('dup')"))
db.rollback()
def test_tags_server_default_is_empty_list(db: Session) -> None:
"""A row inserted without ``tags`` reads ``[]`` (the phase-89
``ignore_paths`` server-default shape — no NULL ever surfaces)."""
db.execute(text("INSERT INTO source_labels (source) VALUES ('nodefault')"))
db.commit()
stored = get_label(db, "nodefault")
assert stored is not None
assert stored.tags == []
# ---------------------------------------------------------------------------
# The read side: label_for / all_labels / delete_label
# ---------------------------------------------------------------------------
def test_label_for_returns_frozen_view(db: Session) -> None:
"""``label_for`` maps the row to the frozen view; ``tags`` arrives
as a tuple."""
set_label(db, "homelab", display_name="the lab", tags=["a", "b"])
db.commit()
view = label_for(db, "homelab")
assert view is not None
assert view == SourceLabelView(
source="homelab", display_name="the lab", label=None, tags=("a", "b")
)
assert isinstance(view.tags, tuple)
def test_label_for_absent_returns_none(db: Session) -> None:
"""No row → ``None`` — the "no label row = pre-phase" contract."""
assert label_for(db, "never-named") is None
def test_all_labels_keyed_by_source(db: Session) -> None:
"""One query, every row, keyed by ``source`` (the views the boost
and the API iterate)."""
set_label(db, "alpha", display_name="the alpha")
set_label(db, "beta", label="the beta row")
db.commit()
labels = all_labels(db)
assert set(labels) == {"alpha", "beta"}
assert labels["alpha"].display_name == "the alpha"
assert labels["beta"].label == "the beta row"
assert labels["alpha"].tags == ()
def test_all_labels_empty_table(db: Session) -> None:
"""The pre-phase state: an empty table reads as an empty mapping."""
assert all_labels(db) == {}
def test_delete_label_removes_row(db: Session) -> None:
"""The reset path: the row is gone, the helper reports ``True``."""
set_label(db, "homelab", display_name="the lab")
db.commit()
assert delete_label(db, "homelab") is True
db.commit()
assert get_label(db, "homelab") is None
def test_delete_label_absent_returns_false(db: Session) -> None:
"""An absent row is ``False`` — the API maps that to 404 (D5)."""
assert delete_label(db, "never-named") is False
def test_delete_label_flushes_without_committing(db: Session) -> None:
"""Flush-only both ways: a rolled-back delete keeps the row."""
set_label(db, "homelab", display_name="the lab")
db.commit()
assert delete_label(db, "homelab") is True
db.rollback()
other = SessionLocal()
try:
assert get_label(other, "homelab") is not None, "the rollback must restore"
finally:
other.close()
# ---------------------------------------------------------------------------
# normalize_tags (the API's 422 source, D5)
# ---------------------------------------------------------------------------
def test_normalize_tags_trims_and_drops_empties() -> None:
"""Entries are trimmed; entries that are empty afterwards dropped;
``[]`` is legal (it clears)."""
assert normalize_tags([" gitea ", "", " ", "forge"]) == ["gitea", "forge"]
assert normalize_tags([]) == []
assert normalize_tags(["", " "]) == []
def test_normalize_tags_dedupes_case_insensitively_first_case_wins() -> None:
"""Case-insensitive de-dupe preserving the FIRST case seen, order
otherwise kept."""
assert normalize_tags(["Gitea", "gitea", "GITEA", "Forge"]) == ["Gitea", "Forge"]
def test_normalize_tags_rejects_too_many_tags() -> None:
"""> 10 entries raise (counted on the trimmed non-empty list)."""
with pytest.raises(ValueError):
normalize_tags([f"tag{i}" for i in range(TAGS_MAX_COUNT + 1)])
# exactly 10 is legal
assert len(normalize_tags([f"tag{i}" for i in range(TAGS_MAX_COUNT)])) == 10
def test_normalize_tags_rejects_overlong_tag() -> None:
"""A > 32-char entry raises (the cap is checked after the trim)."""
with pytest.raises(ValueError):
normalize_tags(["gitea", "x" * 33])
assert normalize_tags(["x" * 32]) == ["x" * 32] # exactly 32 is legal
# ---------------------------------------------------------------------------
# question_name_hits — the D2 rule (every branch pinned)
# ---------------------------------------------------------------------------
_VIEW = SourceLabelView(
source="gitea-forge",
display_name="the git server",
label="self-hosted forge + Gitea",
tags=("gitea", "forgejo"),
)
def test_hit_on_display_name_token() -> None:
"""A question token equal to a display-name token hits — and the
phase-119 digit-class rule does NOT apply ("server" is digitless)."""
assert question_name_hits(_VIEW, "restart the SERVER properly") is True
def test_hit_on_label_token() -> None:
"""A token of the ``label`` field hits (``+Gitea`` normalizes to
``gitea``)."""
assert question_name_hits(_VIEW, "upgrade gitea today") is True
def test_hit_on_tag_entry() -> None:
"""A tag entry hits even when neither name nor label mentions it."""
view = SourceLabelView(
source="s", display_name=None, label=None, tags=("forgejo",)
)
assert question_name_hits(view, "mirror forgejo repos") is True
def test_hit_is_accent_folded_both_sides() -> None:
"""The phase-162 fold rides ``_normalize_name``: the accented source
name answers to the ASCII question token (and vice versa)."""
view = SourceLabelView(
source="besta", display_name="BESTÅ", label=None, tags=()
)
assert question_name_hits(view, "how tall is the besta shelf") is True
assert question_name_hits(view, "BESTÅ shelf height") is True
def test_digitless_name_hits_without_digit_rule() -> None:
"""The 2026-09-16 finding, name side: "gitea" must hit even though
it carries no digit (the path digit rule is NOT reused here)."""
view = SourceLabelView(
source="gitea-forge", display_name="Gitea", label=None, tags=()
)
assert question_name_hits(view, "How do I deploy gitea?") is True
def test_short_source_side_token_never_matches() -> None:
"""The 4-char minimum applies to the SOURCE side: a 3-char tag can
never match, even against an identical question token."""
view = SourceLabelView(
source="ops", display_name="gke", label=None, tags=("gke",)
)
assert question_name_hits(view, "set up gke today") is False
def test_short_question_token_never_matches() -> None:
"""The phase-119 ``NAME_TOKEN_MIN_LEN`` side: a 3-char question
token is no candidate against a matching source token."""
view = SourceLabelView(
source="ops", display_name="Gke Cluster", label=None, tags=()
)
assert question_name_hits(view, "gke deploy") is False
def test_none_view_never_hits() -> None:
"""No label row → ``False`` — the byte-identical pre-phase contract."""
assert question_name_hits(None, "restart the git server") is False
def test_question_without_candidates_never_hits() -> None:
"""A question with NO qualifying candidate (every token under the
4-char floor) short-circuits to ``False`` without touching the
source side."""
assert question_name_hits(_VIEW, "ai ox up") is False
def test_no_overlap_question_never_hits() -> None:
"""The baseline: a question sharing no normalized token with any
field is not a hit."""
assert question_name_hits(_VIEW, "bake a birthday cake") is False
def test_dotted_and_versioned_candidates_still_match() -> None:
"""The reused phase-119 candidate list: a dotted token (kept whole,
``llama.cpp`` → ``llamacpp``) matches the equal source token."""
view = SourceLabelView(
source="infer", display_name="llama.cpp", label=None, tags=()
)
assert question_name_hits(view, "tune llama.cpp flags") is True
# ---------------------------------------------------------------------------
# question_name_hit_sources — the per-turn entry point (one read)
# ---------------------------------------------------------------------------
def test_hit_sources_matches_per_row(db: Session) -> None:
"""The DB half: every labelled row is tested against the ONE
``all_labels`` read; only the named source comes back."""
set_label(db, "gitea-forge", display_name="the git server", tags=["gitea"])
set_label(db, "media", display_name="the media box", tags=["jellyfin"])
set_label(db, "plain") # a row with nothing set can never hit
db.commit()
assert question_name_hit_sources(db, "restart jellyfin tonight") == {"media"}
assert question_name_hit_sources(db, "upgrade gitea") == {"gitea-forge"}
assert question_name_hit_sources(db, "nothing relevant here") == set()
def test_hit_sources_empty_table(db: Session) -> None:
"""Pre-phase state: no rows → the empty set (the boost never fires)."""
assert question_name_hit_sources(db, "restart the git server") == set()
tests/unit/test_source_labels_editor_ui.py
+746
View File
@@ -0,0 +1,746 @@
"""Unit: the per-source name/label/tags editor on the Sources page
(phase 191, task 03).
``TODO.md`` L4: "Sources need a name, a label, tag(s) so you can focus
the model when prompting if you have a project name or a standard name
you reference things by." Tasks 01/02 stored that (``source_labels``,
migration 0030) and exposed it (``/api/source-labels``); this task is the
surface the owner types into — a SECOND list on ``/git-sources.html``
whose row list is the catalogue ``GET /api/source-labels`` (the union of
every registry row's DERIVED name and every indexed document source, so
git, local AND upload sources all get an editor without a second API):
* the markup — the ``#source-labels-section`` card inside the admin-gated
``#git-sources-content`` (the page's gate is this list's gate: the
anonymous path returns at the gate and never fetches it), its own
``.table-wrap`` (the phase-156 bounded panel + sticky header — the
class alone, no divergent card rule), its ``role="alert"`` load-error
line, its empty state, and ONE shared
``<template id="source-label-row-template">`` cloned per row: the
read-only mono identity cell, the three fields (Name ``maxlength=120``,
Label ``maxlength=200``, the Tags token input ``maxlength=32``), the
``type="button"`` Save / Reset pair and the ``role="status"`` save-state
line;
* THE ALIAS CONTRACT (D1/A1) — the identity cell is a ``<code>`` and the
template carries NO input there: a label row never renames
``Document.source``, a locator, a citation or a saved chat;
* the JS lifecycle — ``loadLabels`` → ``renderLabelRows`` → ``makeLabelRow``
(``template.content`` + ``cloneNode``, then the per-clone id rewrite so
every ``<label for>`` stays unique), the token input (Enter / comma adds
a chip, Backspace drops the last, the ``normalize_tags`` caps mirrored),
the dirty paint (``.is-dirty`` + "unsaved changes", per ROW), and Save /
Reset on the row's OWN ``saveInFlight`` flag — the ``PUT`` / ``DELETE``
response is the new truth (NO second GET), the 404 arm names itself
("That source no longer exists") and the ``finally`` settles every
outcome so "Saving…" can never stick (§7.4);
* styles.css — the chips and the state tag JOIN the phase-89 tag rule
(``.git-source-hidden-count`` / ``.git-source-branch-count`` /
``.source-label-state`` / ``.source-label-tag`` are ONE rule — no new
tag-input CSS), the fields take the ``.ignore-editor-token`` box
language, the buttons keep the ≥44px floor, the empty state is shared
with ``.git-sources-empty``, and the new block stays on palette
variables (no raw hex, no new hue).
The browser behaviour is E2E-gated by the phase-191 story suite
(``tests/e2e/test_source_name_labels.py``, task 06); like
``test_source_ignore_paths.py`` / ``test_git_source_branch_ui.py`` (the
house pattern for this page), this module pins the source-level contract a
silent regression would break.
"""
from __future__ import annotations
import re
from pathlib import Path
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
SHELL_HTML = FRONTEND / "index.html"
JS = FRONTEND / "assets" / "git-sources.js"
CSS = FRONTEND / "assets" / "styles.css"
#: The static ids the section ships (the no-collision guard keys on these
#: — the ids inside the template are the FIRST row's and makeLabelRow
#: rewrites the clones').
SECTION_IDS = (
"source-labels-section",
"source-labels-title",
"source-labels-error",
"source-labels-wrap",
"source-labels-table",
"source-labels-tbody",
"source-labels-empty",
"source-label-row-template",
)
#: The template's field ids (rewritten per clone to ``<base>-<n>``).
TEMPLATE_FIELD_IDS = (
"source-label-name",
"source-label-label",
"source-label-tags",
"source-label-editor",
)
#: The save-state line's copy, pinned verbatim (§7.4 + the 404 arm).
STATUS_SAVED = "saved"
STATUS_DIRTY = "unsaved changes"
STATUS_SAVING = "Saving…"
STATUS_RESETTING = "Resetting…"
NOTE_GONE = "That source no longer exists — reload the page."
#: The caps of ``app/rag/source_labels`` mirrored client-side.
TAG_MAX_CHARS = 32
TAGS_MAX_COUNT = 10
#: The functions this module inspects (all live inside ``mount(root)``).
LABEL_FNS = (
"storedFromEntry",
"hasStoredLabel",
"labelSummaryText",
"paintLabelTags",
"addLabelTags",
"readLabelForm",
"fillLabelForm",
"labelFormDirty",
"clearLabelError",
"setLabelStatus",
"paintLabelRowState",
"updateLabelRowUI",
"setLabelEditorOpen",
"labelFailureNote",
"saveLabelRow",
"resetLabelRow",
"makeLabelRow",
"loadLabels",
"renderLabelRows",
"showLabelsError",
"hideLabelsError",
)
def _read(path: Path) -> str:
return path.read_text(encoding="utf-8")
def _html() -> str:
return _read(SHELL_HTML)
def _js() -> str:
return _read(JS)
def _css() -> str:
return _read(CSS)
def _fn(js: str, name: str) -> str:
"""The source of a (possibly async) function via balanced-brace
counting (the ``test_source_ignore_paths.py`` helper)."""
for prefix in ("async function ", "function "):
start = js.find(f"{prefix}{name}(")
if start != -1:
depth = 0
for i in range(js.find("{", start), len(js)):
if js[i] == "{":
depth += 1
elif js[i] == "}":
depth -= 1
if depth == 0:
return js[start : i + 1]
raise AssertionError(f"unbalanced braces in {name}()")
raise AssertionError(f"{name}() must exist in git-sources.js")
def _input_tag(html: str, class_attr: str) -> str:
m = re.search(rf"<input[^>]*{class_attr}[^>]*>", html, re.S)
assert m, f"missing <input class*=…{class_attr}> in the shell"
return re.sub(r"\s+", " ", m.group(0))
def _element_by_id(html: str, id_attr: str) -> str:
m = re.search(rf'<[a-z]+ [^>]*id="{id_attr}"[^>]*>', html, re.S)
assert m, f"missing the #{id_attr} element in the shell"
return re.sub(r"\s+", " ", m.group(0))
def _div_span(html: str, open_tag: str) -> str:
"""The substring from an opening ``<div …>`` to its matching
``</div>`` (balanced div-tag counting — the phase-174 helper)."""
start = html.index(open_tag)
depth = 0
for m in re.finditer(r"<div\b|</div>", html[start:]):
if m.group(0) == "</div>":
depth -= 1
if depth == 0:
return html[start : start + m.end()]
else:
depth += 1
raise AssertionError(f"unbalanced divs from {open_tag!r}")
def _label_for(html: str, field_id: str) -> str:
m = re.search(rf'<label[^>]*for="{field_id}"[^>]*>(.*?)</label>', html, re.S)
assert m, f"missing <label for={field_id}> (WCAG — never aria-label-only)"
return re.sub(r"\s+", " ", m.group(0))
def _css_rule(css: str, selector: str) -> str:
"""The BODY of one CSS rule (these rules carry no nested braces)."""
m = re.search(re.escape(selector) + r"\s*\{([^}]*)\}", css)
assert m is not None, f"rule for {selector!r} not found in styles.css"
return " ".join(m.group(1).split())
def _css_selector_list(css: str, needle: str) -> str:
"""The SELECTOR LIST of the rule that styles ``needle`` (the last
selector before the ``{``), so a shared rule is detectable."""
i = css.find(needle)
assert i != -1, f"{needle!r} is not in styles.css"
open_brace = css.find("{", i)
head = css.rfind("}", 0, i)
start = head + 1 if head != -1 else 0
return " ".join(css[start:open_brace].split())
def _template_block(html: str) -> str:
start = html.index('<template id="source-label-row-template">')
end = html.index("</template>", start)
return html[start:end]
# ---------------------------------------------------------------------------
# The section scaffold (index.html)
# ---------------------------------------------------------------------------
def test_section_ships_inside_the_admin_gated_content() -> None:
"""The card exists, is labelled by its own heading, and sits INSIDE
``#git-sources-content`` — the reveal git-sources.js performs only in
the ADMIN branch, which is what keeps the anonymous visitor from ever
fetching ``/api/source-labels`` (the page's gate is this list's gate)."""
html = _html()
section = _element_by_id(html, "source-labels-section")
assert 'class="source-labels"' in section
assert 'aria-labelledby="source-labels-title"' in section
content = _div_span(html, '<div id="git-sources-content" hidden>')
assert 'id="source-labels-section"' in content, (
"the section must live inside #git-sources-content (anonymous ships hidden)"
)
assert 'id="source-labels-title"' in content
# The heading is real copy about the alias, not a placeholder.
title = re.search(
r'<h2 class="source-labels-title" id="source-labels-title">(.*?)</h2>', html, re.S
)
assert title and "Names, labels and tags" in title.group(1)
def test_section_carries_its_own_error_line_and_empty_state() -> None:
"""``#source-labels-error`` is the list's own ``role="alert"`` line
(hidden until the GET fails) and ``#source-labels-empty`` the empty
state (hidden by default) — the two states a failed or empty catalogue
may reach, so the page is never stuck (the ``#git-sources-load-error``
discipline applied to this page's second list)."""
html = _html()
err = _element_by_id(html, "source-labels-error")
assert 'role="alert"' in err, "a failed catalogue is an alert, not silence"
assert " hidden" in err, "the error line ships hidden"
empty = _element_by_id(html, "source-labels-empty")
assert " hidden" in empty, "the empty state ships hidden"
assert "No sources to name yet." in html
def test_table_is_a_bounded_sticky_panel_with_scoped_columns() -> None:
"""The wrapper carries the shared ``.table-wrap`` class (the ONE
bounded-panel + sticky-header card rule, phase 156 — an unbounded
bespoke card would be the regression) and the table has two scoped
column headers, the second one naming all three fields."""
html = _html()
wrap = _element_by_id(html, "source-labels-wrap")
assert "table-wrap" in wrap, "the shared bounded/sticky card class"
assert 'role="region"' in wrap and 'tabindex="0"' in wrap, (
"a scroll region is keyboard-reachable and named (WCAG 2.1.1/1.3.1)"
)
assert 'id="source-labels-tbody"' in html
table_i = html.index('id="source-labels-table"')
tail = html[table_i : html.index("</table>", table_i)]
assert '<th scope="col">Source</th>' in tail
assert '<th scope="col">Name, label and tags</th>' in tail
assert '<caption class="visually-hidden">' in tail
def test_new_ids_appear_exactly_once() -> None:
"""The no-collision guard (the phase-46/76 contract): every static id
this task adds appears EXACTLY ONCE in the shell."""
html = _html()
for id_attr in SECTION_IDS + TEMPLATE_FIELD_IDS:
n = len(re.findall(rf'id="{re.escape(id_attr)}"', html))
assert n == 1, f'id="{id_attr}" appears {n} times in index.html (must be 1)'
# ---------------------------------------------------------------------------
# The shared row template + the alias contract
# ---------------------------------------------------------------------------
def test_template_ships_one_row_with_the_three_capped_fields() -> None:
"""ONE shared template, cloned per row: the Name (≤120), Label (≤200)
and Tags (≤32) inputs — ``type="text"`` + ``autocomplete="off"`` +
the ``maxlength`` mirroring the API caps (so an over-long submission
never round-trips as a 422) — each with a real visible
``<label for>`` (WCAG, never aria-label-only)."""
tpl = _template_block(_html())
assert "<tr" in tpl, "the template holds one row (a <tr> parses in a template)"
for cls, cap, field_id in (
("source-label-name-input", 120, "source-label-name"),
("source-label-label-input", 200, "source-label-label"),
("source-label-tags-input", 32, "source-label-tags"),
):
tag = _input_tag(tpl, cls)
assert 'type="text"' in tag, f"{cls} is a plain text field"
assert f'maxlength="{cap}"' in tag, f"{cls} mirrors the {cap}-char API cap"
assert 'autocomplete="off"' in tag, f"{cls} offers no browser autofill"
assert f'id="{field_id}"' in tag, f"{cls} carries the template id JS rewrites"
assert _label_for(tpl, field_id), f"{cls} has a visible <label for>"
def test_identity_cell_is_read_only_text_never_an_input() -> None:
"""D1/A1 pinned in markup: the identity cell is a ``<code>`` — the
derived source name — and the ONLY element the row's identity area
contains; the editor is a sibling cell, so a label can never become a
rename of ``Document.source``, a citation or a ``source/path`` locator."""
tpl = _template_block(_html())
m = re.search(
r'<td class="source-label-id-cell">(.*?)</td>', tpl, re.S
)
assert m, "the identity cell exists"
cell = m.group(1)
assert '<code class="source-label-source">' in cell
assert "<input" not in cell, "the identity cell has NO input (an alias, not a rename)"
assert 'class="source-label-state"' in cell, "the stored-label state tag (text, 1.4.1)"
id_i = tpl.index('class="source-label-id-cell"')
edit_i = tpl.index('class="source-label-edit-cell"')
assert id_i < edit_i, "the identity cell precedes the editor (it is not replaced)"
def test_editor_ships_closed_with_the_a11y_pair_and_state_line() -> None:
"""The expandable block: the toggle is a real ``type="button"`` with
``aria-expanded="false"``, the editor div ships ``hidden`` (JS sets
``aria-controls`` to the editor's per-clone id), Save ships
``disabled`` (nothing is dirty on paint) and Reset ships ``hidden``
(a row with no stored label has nothing to reset — the phase-168
per-row reset shape). The save-state line is a live region."""
tpl = _template_block(_html())
toggle = re.search(r'<button[^>]*class="source-label-toggle"[^>]*>', tpl, re.S)
assert toggle and 'type="button"' in toggle.group(0)
assert 'aria-expanded="false"' in toggle.group(0), "collapsed by default"
editor = re.search(r'<div class="source-label-editor"[^>]*>', tpl, re.S)
assert editor and " hidden" in editor.group(0), "the editor ships closed"
save = re.search(r'<button[^>]*class="source-label-save"[^>]*>', tpl, re.S)
assert save and 'type="button"' in save.group(0) and "disabled" in save.group(0)
reset = re.search(r'<button[^>]*class="source-label-reset"[^>]*>', tpl, re.S)
assert reset and 'type="button"' in reset.group(0) and " hidden" in reset.group(0)
status = re.search(r'<span class="source-label-status"[^>]*>', tpl, re.S)
assert status and 'role="status"' in status.group(0)
assert 'aria-live="polite"' in status.group(0), "the save state announces"
# ---------------------------------------------------------------------------
# The load path
# ---------------------------------------------------------------------------
def test_load_labels_fetches_the_catalogue_once_and_heals() -> None:
"""``loadLabels`` is ONE ``GET /api/source-labels`` (no method key =
GET), renders from ``data.labels``, and heals its own error line on
the success path BEFORE rendering (an error followed by a good
refresh clears it — the phase-77 rule for this page's lists). A
network blip and a non-2xx both land on the role=alert line."""
js = _js()
body = _fn(js, "loadLabels")
assert 'const LABELS_ENDPOINT = "/api/source-labels";' in js
assert "fetch(LABELS_ENDPOINT)" in body
assert 'method: "' not in body, "the catalogue read is a GET"
assert body.count("fetch(") == 1, "exactly one fetch per load"
assert "Array.isArray(data.labels)" in body, "renders from the labels array"
heal_i = body.find("hideLabelsError();")
render_i = body.find("renderLabelRows(")
assert -1 < heal_i < render_i, "the success path heals the error before rendering"
assert body.find("showLabelsError(") < body.find("hideLabelsError();")
for fallback in (
"Could not reach the server — the source names are not shown.",
"The server could not list the source names",
"The server sent an unreadable label list — try again.",
):
assert fallback in body, f"missing the failure line {fallback!r}"
def test_render_replaces_rows_and_syncs_the_list_states() -> None:
"""A re-render REPLACES the rows (never appends — a re-show must not
duplicate), and the table card / empty state flip off the row count."""
js = _js()
body = _fn(js, "renderLabelRows")
clear_i = body.find("labelsTbody.replaceChildren()")
append_i = body.find("labelsTbody.appendChild(")
assert -1 < clear_i < append_i, "the clear precedes the fill"
assert 'labelsWrap.hidden = !hasRows' in body
assert 'labelsEmptyEl.hidden = hasRows' in body
def test_error_state_hides_the_table_and_the_empty_state() -> None:
"""While the error is up it is the ONLY claim about the list (the
``showLoadError`` discipline): both the table and the empty state go
hidden, and ``hideLabelsError`` clears the text on the way out."""
js = _js()
show = _fn(js, "showLabelsError")
assert "labelsErrorEl.textContent = message" in show
assert "labelsErrorEl.hidden = false" in show
assert "labelsWrap.hidden = true" in show
assert "labelsEmptyEl.hidden = true" in show
hide = _fn(js, "hideLabelsError")
assert 'labelsErrorEl.textContent = ""' in hide
assert "labelsErrorEl.hidden = true" in hide
def test_labels_fetch_is_admin_only_and_refreshes_the_list_flows() -> None:
"""Every ``loadLabels()`` call site sits AFTER the ``fetchIsAdmin``
gate (anonymous never asks), the boot loads it alongside the sources,
and each flow that CHANGES the source list (add / remove / a finished
upload) refreshes it — so a new source is nameable without a reload."""
js = _js()
gate = js.find("const admin = await fetchIsAdmin();")
assert gate != -1
anon = js[gate : js.find("return;", gate)]
assert "loadLabels" not in anon, "the anonymous branch never fetches labels"
# The gate sits at the END of mount (the boot), so the pin is the
# branch itself + the boot's own call after it: nothing before the
# gate runs for an anonymous visitor, and the mount tail is where the
# first catalogue read happens.
boot = js[gate:]
assert "await loadLabels();" in boot, "the admin boot loads the catalogue"
assert boot.index("await loadLabels();") < boot.index("await loadSources();"), (
"the two lists load side by side at boot (neither waits on the other)"
)
# add / remove / upload settle paths
add = _fn(js, "wireAddForm")
assert "await loadSources(); // the new row lands in the table" in add
assert js.count("await loadLabels();") >= 3, "add + remove + boot refresh labels"
assert "loadLabels(); // phase 191: the uploaded source joins the name list" in js
# the re-show refresh keeps the phase-77 listener verbatim + a second
# listener for the second list (so an unrelated table reload never
# wipes a half-typed name draft)
refresh_listeners = re.findall(
r'addEventListener\("bor:view-refresh", \(\) => (\w+)\(\)\)', js
)
assert refresh_listeners == ["loadSources", "loadLabels"], refresh_listeners
# ---------------------------------------------------------------------------
# Save / Reset — the §7.4 contract, per row
# ---------------------------------------------------------------------------
def test_save_puts_the_whole_form_and_uses_the_response_as_truth() -> None:
"""Save: ``PUT /api/source-labels/<source>`` (``encodeURIComponent`` —
a source name is a path segment), the three keys always ride (an empty
Name is the deliberate CLEAR leg), and on 200 the ROW RE-READS ITS
STORED LABEL FROM THE RESPONSE — no second GET anywhere in the save
path (task 03: "the response is the new truth")."""
js = _js()
body = _fn(js, "saveLabelRow")
fetch_i = body.find(
"fetch(`${LABELS_ENDPOINT}/${encodeURIComponent(view.source)}`"
)
assert fetch_i != -1, "the PUT targets the per-source path"
assert 'method: "PUT"' in body[fetch_i : fetch_i + 220]
assert "body: JSON.stringify(form)" in body, "the whole form is sent"
form_i = body.find("const form = readLabelForm(view);")
assert -1 < form_i < fetch_i, "the form is read before the request"
ok_i = body.find("if (r.ok)")
stored_i = body.find("view.stored = storedFromEntry(", ok_i)
json_i = body.find("await r.json()", ok_i)
assert -1 < ok_i < json_i < stored_i, "the response body IS the new truth"
assert "loadLabels()" not in body, "no second GET after a save"
assert "loadSources()" not in body, "a save never reloads the sources table"
assert "fillLabelForm(view)" in body[stored_i:], "the fields re-paint from the response"
assert "setLabelStatus(view, LABEL_STATUS_SAVED, null)" in body
def test_save_and_reset_guard_on_the_rows_own_in_flight_flag() -> None:
"""The §7.4 guard scoped PER ROW (the phase-168 ``saveInFlight``
pattern): the early return precedes everything, the flag is set BEFORE
the fetch, and the ``finally`` drops it on every outcome — a second
click while a request is out is ignored and no row is ever left
claiming "Saving…" after a terminal frame."""
js = _js()
for name, label in (
("saveLabelRow", "LABEL_STATUS_SAVING"),
("resetLabelRow", "LABEL_STATUS_RESETTING"),
):
body = _fn(js, name)
guard_i = body.find("if (view.saveInFlight) return;")
set_i = body.find("view.saveInFlight = true;")
status_i = body.find(f"setLabelStatus(view, {label}, null);")
fetch_i = body.find("fetch(`")
finally_i = body.find("} finally {")
clear_i = body.find("view.saveInFlight = false;", finally_i)
assert -1 < guard_i < set_i < status_i < fetch_i, (
f"{name}: guard → flag → in-flight label → fetch"
)
assert finally_i != -1 and clear_i != -1, f"{name}: the finally must drop the flag"
assert body.index("view.saveBtn.disabled = true") < fetch_i
assert body.index("view.resetBtn.disabled = true") < fetch_i
assert "updateLabelRowUI(view);" in body[clear_i:], (
f"{name}: the settle re-decides Save from the dirty state"
)
def test_reset_deletes_and_clears_the_row() -> None:
"""Reset: ``DELETE /api/source-labels/<source>`` → the stored label is
nulled from the response and the fields clear with it, settling on the
same "saved" line (the row now matches the server)."""
body = _fn(_js(), "resetLabelRow")
fetch_i = body.find(
"fetch(`${LABELS_ENDPOINT}/${encodeURIComponent(view.source)}`"
)
assert fetch_i != -1
assert 'method: "DELETE"' in body[fetch_i : fetch_i + 200]
ok_i = body.find("if (r.ok)")
stored_i = body.find("view.stored = storedFromEntry(cleared || {})", ok_i)
assert stored_i != -1, "the stored label is nulled from the response"
assert body.find("fillLabelForm(view)", stored_i) != -1, "the fields clear"
assert "setLabelStatus(view, LABEL_STATUS_SAVED, null)" in body
assert "loadLabels()" not in body, "no second GET after a reset"
def test_the_404_note_names_the_arm() -> None:
"""A 404 means the source left the catalogue — a retry cannot help —
so the one-line note names it (task 03), while every other non-2xx
carries the server's own detail (``apiDetail``, 422 shape-aware)."""
js = _js()
assert f'const LABEL_NOTE_GONE = "{NOTE_GONE}";' in js
body = _fn(js, "labelFailureNote")
gone_i = body.find("if (r.status === 404) return LABEL_NOTE_GONE;")
detail_i = body.find("return await apiDetail(r, fallback);")
assert -1 < gone_i < detail_i, "the 404 arm first, then the server detail"
for name in ("saveLabelRow", "resetLabelRow"):
assert "labelFailureNote(r," in _fn(js, name), f"{name} uses the shared note"
def test_a_failed_saves_note_survives_the_dirty_paint() -> None:
"""A failed save leaves the row DIRTY, and the request's ``finally``
calls ``updateLabelRowUI`` — so the dirty hint used to overwrite the
note the same request had just written, and the one error whose advice
is a reload rather than a retype ("That source no longer exists —
reload the page.") disappeared in the tick that produced it (the
defect the phase-191 story suite walked into). Pinned: the error line
returns from the paint untouched, Save keeps its dirty-driven
``disabled`` decision (a retry is legitimate for every arm but the
404), and the ONLY thing that clears a note is the owner editing
again — the settle path never does."""
js = _js()
paint = _fn(js, "updateLabelRowUI")
disable_i = paint.find("view.saveBtn.disabled = !dirty;")
guard_i = paint.find('if (view.statusEl.classList.contains("is-error")) return;')
hint_i = paint.find('setLabelStatus(view, LABEL_STATUS_DIRTY, "hint")')
assert -1 < disable_i < guard_i < hint_i, (
"the error guard sits after the Save paint, before the hint"
)
assert "setLabelStatus" not in paint[guard_i:hint_i], "the note is untouched"
clear = _fn(js, "clearLabelError")
assert 'if (view.statusEl.classList.contains("is-error"))' in clear
assert 'setLabelStatus(view, "", null);' in clear
row = _fn(js, "makeLabelRow")
for opener in (
'input.addEventListener("input", () => {',
'tagsInput.addEventListener("keydown", (e) => {',
):
i = row.find(opener)
assert i != -1, f"missing {opener}"
assert "clearLabelError(view);" in row[i : i + 220], f"{opener} clears the stale note"
# A collapse clears it the same way (it refills from the stored label).
assert 'setLabelStatus(view, "", null);' in _fn(js, "setLabelEditorOpen")
def test_status_line_copy_is_pinned_verbatim() -> None:
"""The save-state line's whole vocabulary (§7.4 + the dirty hint) —
pinned so a silent copy change cannot break the E2E and the
screen-reader contract with it."""
js = _js()
for literal in (
f'const LABEL_STATUS_SAVED = "{STATUS_SAVED}";',
f'const LABEL_STATUS_DIRTY = "{STATUS_DIRTY}";',
f'const LABEL_STATUS_SAVING = "{STATUS_SAVING}";',
f'const LABEL_STATUS_RESETTING = "{STATUS_RESETTING}";',
):
assert literal in js, f"missing {literal!r}"
dirty_paint = _fn(js, "updateLabelRowUI")
assert 'setLabelStatus(view, LABEL_STATUS_DIRTY, "hint")' in dirty_paint
assert 'view.tr.classList.toggle("is-dirty", dirty)' in dirty_paint
assert "if (view.saveInFlight) return;" in dirty_paint, (
"a keystroke mid-request never overwrites the in-flight label"
)
# ---------------------------------------------------------------------------
# The token input
# ---------------------------------------------------------------------------
def test_token_input_adds_on_enter_or_comma_and_drops_the_last_on_backspace() -> None:
"""The token input's widget rules (task 03): Enter or a comma commits
the token (``preventDefault`` — a comma never types, Enter never
submits), Backspace on an empty input drops the LAST chip, and leaving
the field commits a half-typed token."""
js = _js()
body = _fn(js, "makeLabelRow")
enter_i = body.find('if (e.key === "Enter" || e.key === ",") {')
assert enter_i != -1
assert "e.preventDefault();" in body[enter_i : enter_i + 200]
add_i = body.find("addLabelTags(view, tagsInput.value);", enter_i)
back_i = body.find(
'if (e.key === "Backspace" && tagsInput.value === "" && view.tags.length)'
)
assert -1 < enter_i < add_i < back_i, "Enter/comma precedes the Backspace rule"
pop_i = body.find("view.tags.pop();", back_i)
assert -1 < back_i < pop_i < body.find("paintLabelTags(view);", back_i)
blur_i = body.find('tagsInput.addEventListener("blur"')
assert blur_i > back_i, "blur commits a pending token"
def test_tag_caps_are_the_servers_and_the_first_case_wins() -> None:
"""The caps are ``normalize_tags``' numbers mirrored client-side
(32 chars, 10 chips) and the de-dupe keeps the FIRST case — the chip
list the owner sees is the list the API stores."""
js = _js()
assert f"const LABEL_TAG_MAX_CHARS = {TAG_MAX_CHARS};" in js
assert f"const LABEL_TAGS_MAX_COUNT = {TAGS_MAX_COUNT};" in js
body = _fn(js, "addLabelTags")
assert ".slice(0, LABEL_TAG_MAX_CHARS)" in body
assert "if (!tag) continue;" in body, "an empty token is dropped, never stored"
assert "t.toLowerCase() === tag.toLowerCase()" in body, "case-insensitive de-dupe"
assert "view.tags.length >= LABEL_TAGS_MAX_COUNT" in body, "the 10-chip cap"
assert "LABEL_TAGS_MAX_COUNT" in _fn(js, "paintLabelTags")
def test_chips_render_as_text_in_the_shared_tag_class() -> None:
"""Each chip is an ``<li class="source-label-tag">`` written with
``textContent`` (the XSS contract — owner text is never HTML) and the
class is the one the shared phase-89 tag rule styles, so the widget
adds NO tag-input CSS (task 03 step 1)."""
js = _js()
body = _fn(js, "paintLabelTags")
assert 'li.className = "source-label-tag";' in body
assert "li.textContent = tag;" in body
assert "innerHTML" not in body
selector = _css_selector_list(_css(), ".source-label-tag {")
assert ".git-source-hidden-count" in selector, (
"the chip joins the phase-89 tag rule (no new tag-input CSS)"
)
assert ".source-label-state" in selector, "the state tag shares it too"
# ---------------------------------------------------------------------------
# makeLabelRow — the clone
# ---------------------------------------------------------------------------
def test_make_label_row_clones_the_template_and_rewrites_the_ids() -> None:
"""One shared template, cloned per row: ``template.content`` +
``cloneNode(true)``, then the per-clone id rewrite (input id + the
matching ``<label for>``) and the ``aria-controls`` link — without it
every clone would duplicate the template's ids and every label would
aim at the first match (WCAG 1.3.1)."""
js = _js()
body = _fn(js, "makeLabelRow")
assert "labelRowTemplate.content" in body
assert "proto.cloneNode(true)" in body
assert 'input.id = `${base}-${seq}`;' in body
assert 'lbl.setAttribute("for", input.id);' in body
assert 'editor.id = `source-label-editor-${seq}`;' in body
assert 'toggleBtn.setAttribute("aria-controls", editor.id);' in body
assert 'codeEl.textContent = source;' in body, "the identity is textContent"
for fn in LABEL_FNS:
assert "innerHTML" not in _fn(js, fn), f"{fn} must never touch innerHTML"
def test_row_state_paint_names_the_source_on_every_control() -> None:
"""The state paint: the "named" tag + summary appear only when a label
is stored, Reset is visible only then (the phase-168 per-row reset),
and every control's ``aria-label`` names the row's source (the
phase-89/105 idiom — the clipped label stays reachable)."""
js = _js()
body = _fn(js, "paintLabelRowState")
assert "view.stateTag.hidden = !labelled" in body
assert "view.resetBtn.hidden = !labelled" in body
assert 'labelled ? "Edit name" : "Name this source"' in body
for label in (
"${labelled ? \"Edit\" : \"Set\"} the name, label and tags for source: ${view.source}",
"Save the name, label and tags for source: ${view.source}",
"Reset the name, label and tags of source: ${view.source}",
):
assert label in body, f"missing the aria-label template {label!r}"
def test_collapse_discards_the_draft_instead_of_staling_it() -> None:
"""Closing the editor refills it from the STORED label and clears the
status line (no request) — a closed row never claims an edit the
server never got; a request in flight freezes the row."""
js = _js()
body = _fn(js, "setLabelEditorOpen")
assert "if (view.saveInFlight) return;" in body
assert 'view.toggleBtn.setAttribute("aria-expanded", open ? "true" : "false");' in body
open_i = body.find("view.editor.hidden = !open;")
fill_i = body.find("fillLabelForm(view);")
clear_i = body.find('setLabelStatus(view, "", null);')
assert -1 < open_i < fill_i, "a collapse refills from the stored label"
assert fill_i < clear_i < body.find("updateLabelRowUI(view);")
def test_stored_shape_folds_nulls_to_the_empty_form() -> None:
"""``storedFromEntry`` folds the API's nulls / missing keys to ``\"\"``
and ``[]`` (a missing row and a cleared label are ONE state — D1), and
``hasStoredLabel`` is what drives the "named" tag and Reset."""
js = _js()
body = _fn(js, "storedFromEntry")
assert 'typeof e.display_name === "string"' in body
assert "Array.isArray(e.tags)" in body
stored = _fn(js, "hasStoredLabel")
assert "stored.display_name || stored.label || stored.tags.length" in stored
# ---------------------------------------------------------------------------
# styles.css — the reuse contract
# ---------------------------------------------------------------------------
def test_new_block_reuses_the_page_language() -> None:
"""The new rules stay on the page's own language: the field box is the
``.ignore-editor-token`` box (mono, the line border, the 44px floor),
the row's three buttons keep the ≥44px touch target, the empty state is
SHARED with ``.git-sources-empty``, and the dirty mark is the phase-168
``.is-dirty`` stripe."""
css = _css()
field = _css_rule(css, ".source-label-input")
assert "min-height: 44px" in field
assert "font-family: var(--mono)" in field
assert "border: 1px solid var(--line)" in field
for selector in (".source-label-toggle", ".source-label-save", ".source-label-reset"):
assert "min-height: 44px" in _css_rule(css, selector), f"{selector} lost the floor"
selector = _css_selector_list(css, ".source-labels-empty {")
assert ".git-sources-empty" in selector, "the empty-state rule is shared, not copied"
dirty = _css_rule(css, ".source-label-row.is-dirty > td:first-child")
assert "box-shadow: inset 3px 0 0 var(--brand)" in dirty, "the phase-168 stripe"
def test_new_css_uses_palette_variables_only() -> None:
"""No new hue, no CDN: every colour in the block is a palette
variable (the phase-08/92 monochrome invariant — a raw hex would not
follow a saved theme)."""
css = _css()
start = css.index("/* ---------- Source names, labels and tags (phase 191, task 03)")
end = css.index("/* ---------- History refresh button", start)
block = css[start:end]
assert block, "the labelled block exists"
offenders = re.findall(r"#[0-9a-fA-F]{3,8}\b", block)
assert not offenders, f"raw hex colours in the phase-191 block: {offenders}"
assert "url(" not in block, "no external asset (the no-CDN rule)"
assert "blur(" not in block, "the phase-08 no-blur anchor"