phase: 191_source_name_labels_tags
Build and Push Containers / build-and-push-app (push) Successful in 7m24s
Build and Push Containers / build-and-push-db (push) Successful in 36s

All gates re-verified against a stable tree. Report:

**Verified (all 6 tasks in `complete/`; this was the final pass)**
- `SourceLabel`+alembic `0030` (dev DB `alembic current` = `0030 (head)`, table present, 0 rows); admin-only `GET/PUT/DELETE /api/source-labels` (`require_admin` router-wide, `resolve_source_name`, partial-update caps/404s); Sources-page "Names, labels and tags" card (read-only derived-name `<code>`, per-row `saveInFlight`, 404 note that survives the settle, chips reuse `.git-source-hidden-count`); `question_name_hits` D2 rule; ONE `all_labels` read per turn in `chat.py` feeding both the `source_name_hit` flag and the prompt surfaces; `source_name_hit_bonus=0.005` in `_selection_order`; ` name=`/` label=` in `build_high_prompt` + `build_overview_prompt(names=…)`.
- **Completion criteria:** E2E `tests/e2e/test_source_name_labels.py` ✅ 7 passed (arms a–e: editor all kinds, control/named question, rename-never-breaks-identity, 404 note, reset) · full suite ✅ · coverage ✅ · README/AGENTS ✅ · phase dir in `complete/` ✅ · commit left to the harness.
- **Gates:** `uv run pytest --cov=app --cov-report=term-missing` → **exit 0, TOTAL 7160 stmts / 16 missed = 99%** (>90%) · `uv run pytest tests/e2e/test_source_name_labels.py -v --no-cov` → **7 passed, exit 0** · regression E2E isolated, exit 0: `test_document_viewer`, `test_admin_auth`, `test_api_tokens`, `test_git_sources_admin`, `test_kb_tree`, `test_local_directory_sources` · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors.
- **Battery (task 04 criterion):** `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` at default vs `BOR_SOURCE_NAME_HIT_BONUS=0` → **`diff` empty (line-for-line identical)**, matching the recorded arms (dev KB is empty, so every question deflects; the observable boost is unit/E2E-pinned).
- **Documentation:** README — fixed the `/git-sources.html` rows (nav label is **Sources**) and cross-linked the phase-191 card from that section; full audit otherwise clean: all 86 `BOR_*` settings documented with defaults matching `app/config.py`, 0 broken file/dir references and 0 broken anchors in README+AGENTS, AGENTS rule 17 re-checked line-by-line against the code.
- **Notable / deviations:** (1) a **concurrent pipeline process** (`phased-execution/progress.mjs` for `191_…_00_phase`) was mutating this same working tree mid-pass — it rewrote `tests/e2e/test_source_name_labels.py` (11→7 tests), `frontend/assets/git-sources.js`, `tests/unit/test_source_labels_editor_ui.py` and `AGENTS.md` at 05:27; every gate above was (re-)run after those edits with the tree no source file newer than the green full run. (2) The task-06 report/phase text claims regression suites `test_chat_token_gate.py`, `test_git_sources_page.py`, `test_rag_browser.py` are green — **those files have never existed in git history** (those runs exit 4); I ran the real equivalents listed above instead (phase files are not mine to edit). (3) **No live changes**: the VM DB already has `source_labels` (0 rows) but the deployed app predates this uncommitted code, so deploying `/api/source-labels` to the live service is the owner's post-commit step, not part of this phase's criteria.
- **Next pending phase:** none — `191_source_name_labels_tags` is the only entry in `.agents/phases/todo/`.
This commit is contained in:
2026-10-19 06:13:32 -04:00
parent c6ffa050a6
commit 4f20c256d7
70 changed files with 7378 additions and 50 deletions
@@ -1,72 +0,0 @@
# Phase 191 — Sources carry a name, a label and tags — and the name focuses retrieval
**Source:** `TODO.md` L4 — "Sources need a name, a label, tag(s) so you can focus the model when prompting if you have a project name or a standard name you reference things by."
**Story:** n/a (owner directive)
**Depends on:** none functionally (builds on the completed phase-119 name-signal selection bonus, the phase-31/142 KB overview, the phase-76/35/38/49/90 Sources page).
## Design (shared by all tasks — the executor reads this, not the chat)
- **D1 — the data (task 01).** A new single-purpose table `source_labels` (alembic **0030**, additive, no backfill — a source without a row has no name/label/tags and behaves exactly as pre-phase):
- `source` — `String(120)` **PK**: the source name EXACTLY as `Document.source` carries it (the derived basename — the clone dir / local dir / upload file stem); the join key to the documents and to the `git_sources` row (via `_url_basename` / the local path basename);
- `display_name` — `Text`, nullable, ≤ 120 chars (the trimmed value; the owner's name for the source — "the git server");
- `label` — `Text`, nullable, ≤ 200 chars (the short one-line description/category — "self-hosted forge + Gitea");
- `tags` — `JSONB`, `NOT NULL`, server default `'[]'` (the focus tokens, each ≤ 32 chars, ≤ 10 tags — the owner's "tag(s)");
- `updated_at` — `DateTime(timezone=True)`, server default now, onupdate now.
- **The display name is an ALIAS ONLY** (owner-confirmed assumption): `Document.source`, the `source/path` document identity, the read/grep/`show_text` locators, the citations and the KB tree are ALL untouched by a label row — renaming a source never invalidates a saved chat, a shared page or a verbatim locator. The derived name stays the wire identity; the display name rides beside it.
- The ORM model `SourceLabel` (the `SourcesMeta` single-column style, `app/models.py`) + `app/rag/source_labels.py` (the new module — `app/rag/sources_meta.py` is the phase-53 version counter, NOT this feature; no name collision): `get_label(db, source)`, `set_label(db, source, display_name, label, tags)` (upsert, flush-only — the caller commits, the phase-53 session convention), `all_labels(db)`, `label_for(db, source)` → a frozen `SourceLabelView` (display_name / label / tags, `None` when absent), `question_name_hits(labels, question)` → the matching rule below.
- **D2 — the matching rule (task 04).** A source is a SOURCE-NAME HIT when at least one `name_hit_tokens(question)` candidate (the phase-119 class-agnostic token list, reused UNCHANGED — `app/rag/retriever.py` L191) equals, case-insensitively (the `_normalize_name` form), a token of the source's `display_name`, `label` or any `tags` entry. The phase-119 digit-class rule does NOT apply to user-given names (the owner's names are not paths — "gitea" must hit even though it is digitless; the phase-119 digit rule exists to keep prose tokens from matching PATHS, and user names have no prose-precision problem: a 4-char minimum token length (`NAME_TOKEN_MIN_LEN`, reused) is the only filter, applied to the SOURCE side too — a 2-char tag never matches). One query per turn: the label rows (a handful) are loaded once and the match is in Python (the phase-119 one-query convention).
- **D3 — the boost (task 04).** A NEW bounded selection-time bonus, the phase-119 `name_hit_bonus` pattern end to end, a SEPARATE knob (the two signals stay independently kill-able):
- `source_name_hit_bonus: float = 0.005` (default = the phase-119 starting point; validator `>= 0`, `0` = off; `BOR_SOURCE_NAME_HIT_BONUS` env; the `settings_catalog` metadata entry — the catalogue-completeness pin is extended, NOT waived);
- `RetrievedChunk` gains the additive field `source_name_hit: bool = False` (the `name_hit` docstring pattern: default `False` keeps every ordinary construction valid; `fuse`'s `replace()` copies it; the double-hit merge ORs it in — a vector row that is also the source-name hit keeps the flag);
- the flag is set at the retrieval boundary in `app/api/chat.py` (the call sites where `select_suggested` / `select_related` / `weak_hit_titles` run — L537–572 and L875 — the question is in scope): for every chunk whose `document.source` is a D2 hit, `rc.source_name_hit = True` (a loop over the few dozen retrieved chunks against the handful of hit sources — never a new query);
- `_selection_order` (L754) gains the second bonus parameter: `effective_score = best_fused + (bonus if any name_hit) + (source_bonus if any source_name_hit)` — each bonus per DOCUMENT, applied ONCE, at most two bonuses per document (bounded: 0.010 on the fused scale, the same scale phase 119 proved); `source_bonus == 0` (or no flagged chunk) → the walk is BYTE-IDENTICAL to the pre-phase walk (the phase-119 kill-switch contract, LOCKED A3 — a new unit pin mirrors the phase-119 one);
- the three application sites (`select_suggested` L537/L875, `select_related` L538, `weak_hit_titles` L572) pass `get_settings().source_name_hit_bonus`; the bonus lives in the SELECTION layer only — `score` / `cosine` / `fts_hit` / `query_log.top_score` are untouched (the phase-119 LOCKED A3 clause, mirrored).
- **D4 — the prompt surfacing (task 05).** The model learns the name→docs mapping from the prompt, not from retrieval alone:
- the KB overview's INPUT lines (`app/rag/overview.py` `build_overview_prompt`, L148 — `source — path — title — summary`) render `display_name — path — …` when set (the raw source name is REPLACED in the lite model's input only — the stored overview text is the lite model's prose, which then naturally uses the owner's name; the overview row's `source` field in the DB is untouched);
- the suggested-tier `<documents>` block (the per-doc line in `app/rag/prompts.py`, the phase-31 section builder) renders `display_name (label)` before the path when set — e.g. `the git server (self-hosted forge) — docs/gitea.md — …` (the parenthetical only when BOTH are set; one of the two → just the present one; neither → the pre-phase line, byte-identical).
- **D5 — the API (task 02).** `app/api/source_labels.py` (NEW router, the phase-91 admin gate — `require_admin` on every route; registered in `app/main.py` alongside the other admin routers):
- `GET /api/source-labels` → `{"labels": [ {source, display_name, label, tags} … ]}` — the FULL source catalogue: the union of (a) every `git_sources` row's derived source name (git + local, the `effective_sources` rows — an env-only deployment lists the env source names) and (b) every `SELECT DISTINCT source FROM documents` name NOT covered by (a) (the uploads) — one query for (b), the handful of `git_sources` rows for (a); each entry carries its label row or nulls;
- `PUT /api/source-labels/{source}` (path param ≤ 120) body `{display_name?, label?, tags?}` → the upsert; validation (the phase-121 boundary style — the framework's standard 422): `display_name` trimmed ≤ 120 (empty string / whitespace-only = NULL), `label` trimmed ≤ 200 (same), `tags` ≤ 10 entries, each a trimmed ≤ 32 non-empty token, de-duplicated case-insensitively preserving first case, `[]` legal (clears); an explicit `null` clears that field (an omitted field leaves it — the phase-168 partial-update shape); a `source` that matches NO git row and NO document → **404** (no orphan label rows — a label for a source that does not exist would be a dead boost input);
- `DELETE /api/source-labels/{source}` → row removed (404 when absent) — the reset.
- **D6 — the UI (task 03).** The Sources page (`#view-git-sources`, `git-sources.js`) — per-source name/label/tags editor for EVERY source kind (git, local, upload — the upload rows are the same table's extra section, phase 49/90):
- one expandable row per source (the house pattern — the phase-121 token field is the closest precedent): three fields (Name / Label / Tags, the last a comma-or-Enter token input, the phase-89 ignore-paths token input is the precedent) + Save / Reset, the §7.4 never-stale save state ("Saving…" → "saved" / the one-line error note, the phase-168 shape);
- the saved state renders back from the GET (the pre-population — the phase-91 convention); the derived source name is the row's identity (read-only, the `git-sources.js` name cell — the display name is a SEPARATE field, D1);
- the chat page's source cluster chips and the RAG page's source tree keep the DERIVED name (D1 — no identity change anywhere); the display name surfaces in the chat only through the prompt (D4) and the verbatim panel caption is untouched (it names `source/path` — the identity).
## ASSUMPTIONS (owner-confirmed 2026-10-18)
- **A1 (D1):** the display name is an alias — the `Document.source` / `source/path` identity is untouched.
- **A2 (D2/D3):** the focus mechanism is the new bounded `source_name_hit_bonus` selection boost (default 0.005, `0` = off) + the KB-overview / suggested-tier surfacing (D4).
- **A3 (D5):** admin-gated like the phase-91/168 admin surfaces; labels for non-existent sources are rejected (404).
## Context
- `app/models.py` (the `SourcesMeta` L— single-row precedent; the `GitSource` row — the `url`/`path`/`kind` columns the derived names come from), `app/rag/git_sources.py` (`_url_basename` L89, `effective_sources` L285 — the git/local source-name derivation), alembic `0029` (the last migration — 0030 is next), `app/rag/retriever.py` (`RetrievedChunk` L276+, `name_hit_tokens` L191, `_name_hit_chunks` L576, `_selection_order` L754, the three bonus application sites L838/L987/L1022), `app/api/chat.py` (the selection call sites L537–572, L875), `app/config.py` (`name_hit_bonus` L431 + validator L964 — the knob pattern), `app/core/settings_catalog.py` (L225 — the metadata entry pattern), `app/rag/overview.py` (`build_overview_prompt` L148 — the input lines), `app/rag/prompts.py` (the `<knowledge_base>` section L561, the suggested-tier doc lines), `app/api/git_sources.py` (the router pattern — admin gate, the upload status), `frontend/assets/git-sources.js` (the Sources page), `tests/fixtures/retrieval_battery.txt` + `scripts/eval_retrieval.py` (the battery), `tests/integration/` (the phase-119 pins — the kill-switch pattern to mirror).
## Tasks
1. `01_source_labels_table.md` — the `SourceLabel` model, alembic 0030, the `app/rag/source_labels.py` helpers (D1/D2 pure half).
2. `02_source_labels_api.md` — the admin-gated router (D5): the catalogue GET, the validated PUT upsert, the 404s, the DELETE.
3. `03_source_labels_ui.md` — the per-source name/label/tags editor on the Sources page (D6), all three source kinds.
4. `04_source_name_boost.md` — the `source_name_hit_bonus` knob + the `source_name_hit` flag + the `_selection_order` second bonus (D3), the kill-switch pins, the battery run.
5. `05_prompt_surfacing.md` — the KB-overview input lines + the suggested-tier doc lines carry the display name / label (D4).
6. `06_e2e_and_gates.md` — the dedicated Playwright suite (the owner-visible arm: a named source's document is suggested when the question uses the name, the alias never breaks a locator), the full gates, the docs pass, the atomic commit.
## Testing & Quality
- **Unit:** `tests/unit/test_source_labels.py` (NEW) — the D2 matching rule (name/label/tag hits, the case-insensitive `_normalize_name` form, the 4-char minimum on both sides, the no-hit baseline), the D1 helpers (upsert, the partial-update shape, the 404 shape at the helper level); `tests/unit/test_retriever.py` — the phase-119 mirror pins for the SECOND bonus (the kill-switch byte-identity, the bounded per-document application, the OR-merge through `fuse`).
- **Integration:** `tests/integration/test_source_labels_api.py` (NEW) — the admin gate (401/403 the phase-91 shape), the catalogue GET (git + local + upload union, no dupes), the PUT validation 422s (121-char name, 201-char label, 11 tags, a 33-char tag, a whitespace tag), the partial-update shape (an omitted field is untouched, an explicit null clears), the 404s (no such source; DELETE of an absent row), the round-trip.
- **E2E (AGENTS rule 4, task 06):** `tests/e2e/test_source_name_labels.py`, run in isolation — the owner-visible arm through the real UI (set a name + tag on a fixture source → a chat question using the name surfaces that source's document in the suggested tier via the mock/real battery — the phase-119 E2E is the precedent), the alias contract (a read/grep locator for the renamed source still resolves — the derived name is untouched), the 404 arm (a label for a non-existent source is rejected in the UI with the one-line note).
- **Battery:** `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` — the pre/post run at the default bonus (0.005) and at `0` (byte-identical to pre-phase) is recorded in the phase file (the phase-119 convention).
- **Coverage:** **>90%** on `app/`; the frontend change is marker-pinned per the house JS convention.
- **Lint/types:** `uv run ruff check . && uv run pyright`.
## Documentation & Consistency
- Final pass reviews `README.md` and `AGENTS.md` in their entirety against the post-phase app and fixes every inconsistency found, pre-existing ones included.
- `README.md`: the sources section gains the naming feature (the name/label/tags editor, the alias-only contract, the new knob `BOR_SOURCE_NAME_HIT_BONUS` with its `0` = off meaning) (task 06). `AGENTS.md`: the phase-119 name-signal clause gains the parallel source-name clause if it names the bonus (verify, do not add if absent).
## Completion Criteria
- [ ] alembic 0030 applied cleanly; `SourceLabel` round-trips (unit); the alias contract holds (a `Document.source` / locator is untouched by a label — the E2E arm).
- [ ] `GET /api/source-labels` lists git + local + upload sources with their labels (integration + E2E); the PUT validation 422s and 404s pinned; the DELETE resets.
- [ ] the Sources page edits name/label/tags for all three source kinds (E2E), save state per the §7.4 contract.
- [ ] `source_name_hit_bonus = 0` → the selection walk byte-identical to pre-phase (unit pin); at 0.005 the battery shows the named source surfacing (battery record); the three application sites pass the knob (unit pins).
- [ ] the KB-overview input lines and the suggested-tier doc lines carry `display_name (label)` when set, byte-identical otherwise (unit pins on both builders).
- [ ] test suite green, coverage >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] no behavior change in completed phases (no label row → every surface byte-identical to pre-phase).
- [ ] one atomic Conventional Commits commit with `--no-gpg-sign`.
@@ -1,41 +0,0 @@
# Task 01 — The `source_labels` table and helpers (D1/D2 pure half)
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "Sources need a name, a label, tag(s) …"`
**Story:** n/a
## Objective
The additive data layer: the `SourceLabel` ORM model + alembic 0030 (no backfill) + `app/rag/source_labels.py` with the upsert/get helpers and the D2 `question_name_hits` matching rule — so a source without a row is byte-identical to pre-phase everywhere.
## Work
1. `app/models.py` — NEW `SourceLabel(Base)` (the `SourcesMeta` style, tabulated docstring — the D1 column spec from the phase file):
- `source: Mapped[str] = mapped_column(String(120), primary_key=True)` (the `Document.source` name — the join key);
- `display_name: Mapped[str | None] = mapped_column(Text, default=None)` (≤ 120 trimmed — enforced at the API, the column is plain Text, the phase-121 pattern);
- `label: Mapped[str | None] = mapped_column(Text, default=None)` (≤ 200);
- `tags: Mapped[list] = mapped_column(JSONB, default=list, server_default=text("'[]'"), nullable=False)`;
- `updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())`.
2. `alembic/versions/0030_source_labels.py` — NEW, revision `0030`, down_revision `0029` (verify the 0029 rev id — it is the current head): `CREATE TABLE source_labels` (the columns above, the PK, the JSONB server default), no backfill (an empty table — every source's pre-phase behavior is "no row"); the downgrade `DROP TABLE` (the house migration style — `0028` is the shape to copy).
3. `app/rag/source_labels.py` (NEW module — the docstring names the feature and the D1 alias contract):
- `@dataclass(frozen=True) SourceLabelView` — `source, display_name, label, tags` (tags: `tuple[str, …]`); `label_for(db, source) -> SourceLabelView | None` (one PK read; `None` when absent);
- `get_label(db, source) -> SourceLabel | None` (one PK read);
- `set_label(db, source, *, display_name, label, tags) -> SourceLabel` — upsert (the `SourcesMeta` upsert shape): an existing row is updated in place (only the provided non-`None` fields are overwritten — the partial-update shape; an explicit `None` clears), a new row is created; **flush-only — the caller commits** (the phase-53 session convention);
- `all_labels(db) -> dict[str, SourceLabelView]` (one query, keyed by `source`);
- `delete_label(db, source) -> bool` (one PK delete; `False` when absent; flush-only);
- `normalize_tags(raw: list[str]) -> list[str]` (pure: trim, drop empties, ≤ 32-char guard raises `ValueError`, ≤ 10 guard raises `ValueError`, case-insensitive de-dupe preserving first case — the API's 422 comes from these raises);
- `question_name_hits(view: SourceLabelView | None, question: str) -> bool` (the D2 rule, pure): `None` → `False`; else any `name_hit_tokens(question)` candidate (imported from `app.rag.retriever` — the phase-119 list, UNCHANGED) equals, in `_normalize_name` form, a token of `display_name` / `label` / any `tags` entry — the SOURCE side tokenized the same way (whitespace-split, `_normalize_name`, `NAME_TOKEN_MIN_LEN` (4) minimum on BOTH sides);
- `question_name_hit_sources(db, question: str) -> set[str]` (one `all_labels` read + the pure rule per row — a handful of rows; the per-turn entry point task 04 uses).
4. `tests/unit/test_source_labels.py` (NEW):
- the model round-trip (create via `set_label`, the `updated_at` stamp, the partial-update shape — an omitted field untouched, an explicit `None` clears);
- the D2 rule: a digitless name hit (`"gitea"` question token vs `display_name "Gitea"` — case-insensitive via `_normalize_name`), a label hit, a tag hit, a 3-char source-side token NEVER matches (the 4-char minimum), a 3-char QUESTION token never matches (the phase-119 `NAME_TOKEN_MIN_LEN` side), `None` view → `False`, a no-overlap question → `False`;
- `normalize_tags` (the 11-tag ValueError, the 33-char ValueError, the whitespace drop, the case-insensitive de-dupe keeping first case, `[]` legal);
- the migration 0030 up/down (the `tests/unit` or `tests/integration` migration-pin convention — verify where the phase-168 0028 pin lives and mirror it).
5. Verify: `uv run alembic upgrade head` clean on the dev DB; `uv run pytest tests/unit/test_source_labels.py -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the pins above (the D2 rule is the heart of the phase — pin every branch).
- Coverage: **>90%** on `app/` (the new module is fully exercised by its unit module).
## Completion Criteria
- [ ] alembic 0030 applies and reverses cleanly; the `SourceLabel` model matches the D1 column spec.
- [ ] `tests/unit/test_source_labels.py` green — the D2 rule, the partial-update shape, the `normalize_tags` guards.
- [ ] full test suite **100% passing** (the empty table changes nothing).
- [ ] no behavior change in completed work (no row → no surface reads it yet).
@@ -1,35 +0,0 @@
# Task 02 — The admin `source-labels` API (D5)
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "Sources need a name, a label, tag(s) …"`
**Story:** n/a
## Objective
The wire surface: `app/api/source_labels.py` — the catalogue `GET` (every source, all three kinds, with its label or nulls), the validated `PUT` upsert (the framework's standard 422; a 404 for a source that exists nowhere), the `DELETE` reset — admin-gated like the phase-91/168 admin surfaces, registered in `app/main.py`.
## Work
1. `app/schemas.py` — NEW shapes (the phase-91 `extra="forbid"` style):
- `SourceLabelOut` — `source: str` (≤ 120), `display_name: str | None` (≤ 120), `label: str | None` (≤ 200), `tags: list[str]` (≤ 10 items, each ≤ 32);
- `SourceLabelList` — `labels: list[SourceLabelOut]`;
- `SourceLabelUpdate` — `display_name: str | None = Field(default=None, max_length=120)`, `label: str | None = Field(default=None, max_length=200)`, `tags: list[str] | None = Field(default=None, max_length=10)` (each item ≤ 32 — an item-level `Field(max_length=32)` or a model validator; verify the house item-cap convention in `SavedChatCreate`'s messages and mirror it) — `extra="forbid"`; the OMITTED-vs-NULL distinction is the partial-update shape (D5): omitted → untouched, `null` → cleared, a value → set (the validator trims; a whitespace-only `display_name`/`label` → `None` at the validator boundary).
2. `app/api/source_labels.py` (NEW router, `APIRouter(prefix="/api/source-labels", tags=["source-labels"], dependencies=[Depends(require_admin)])` — the phase-91 gate line):
- `GET ""` → `SourceLabelList`: (a) the `git_sources` rows — `effective_sources(db)` (the phase-35/38 helper, L285): each row's derived source name (git: `_url_basename(row.url)`; local: the stored path's basename — the SAME expression the importer uses for `Document.source` — verify the importer's local-name expression and reuse it, do not re-derive it); an env-only deployment (no DB rows, the `env` source state) lists the env source names (the `effective_sources` `source` return is `"db"`/`"env"` — for `env`, parse the env list the phase-35 fallback uses); (b) `select(Document.source).distinct()` (ONE query, no `content` — the phase-126 load discipline) minus the names from (a) (the uploads); de-duplicated, sorted by `source` (stable order — the UI and the E2E rely on it); each entry = its label row via `all_labels(db)` (one query) or nulls;
- `PUT "/{source}"` (path param, the router's `max_length` via the path conversion — a >120 path → the framework's 422) → the D5 404 arm FIRST: `source` matches no `git_sources` row name AND is not in the distinct document sources (the SAME two queries the GET runs — no third) → 404 (the standard body, the phase-91 shape); else `normalize_tags` (422 on its raises — a `ValueError` → the framework's 422 via the validator, NOT a 500), `set_label` (flush + `db.commit()` — the router owns the session here, the phase-91 commit convention), 200 `SourceLabelOut`;
- `DELETE "/{source}"` → the 404 arm (no label row → 404), else `delete_label` + commit, 200 (the phase-91 DELETE shape — verify its body: an empty object or a message; mirror the sibling admin router).
3. `app/main.py` — `app.include_router(source_labels.router)` (the admin routers' block — verify the exact registration list and the order convention).
4. `tests/integration/test_source_labels_api.py` (NEW — the phase-91 `test_ui_settings_api.py` fixture pattern):
- the gate: an anonymous GET/PUT/DELETE → the phase-91 gate shape (403 / the redirect-to-login — verify the phase-91 gate pin and mirror it EXACTLY);
- the catalogue GET: a module app with one git row (a `file://` URL — the phase-35 fixture), one local row, one upload (a document row with a source name no row derives) + a label on the upload → the response lists all three (sorted, no dupes), the upload carries its label, the others nulls;
- the PUT: a valid upsert round-trips (set name+label+tags → GET shows them; a second PUT with only `label` → the name/tags untouched; a PUT with explicit `null` `display_name` → cleared, the label kept); the 422s (a 121-char `display_name`, a 201-char `label`, 11 `tags`, a 33-char tag, a whitespace-only tag — each with the framework's standard body and NO row change: a pre-seeded row is byte-identical after the 422); the 404s (a `source` that is neither a row name nor a document source — the row table unchanged);
- the DELETE: a labelled source → 200, the GET shows nulls; a second DELETE → 404.
5. Verify: `uv run pytest tests/integration/test_source_labels_api.py -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Integration: the pins above — the gate, the union catalogue, the validation 422s (no partial write), the 404s, the round-trip.
- Coverage: **>90%** on `app/` (the router is fully exercised by its integration module).
## Completion Criteria
- [ ] `GET /api/source-labels` lists git + local + upload sources (integration pin) with their labels; admin-gated.
- [ ] `PUT /api/source-labels/{source}` upserts with the D5 partial-update shape; every 422 leaves the row unchanged; a 404 for a non-existent source creates nothing.
- [ ] `DELETE` resets; a second DELETE 404s.
- [ ] full test suite **100% passing**.
- [ ] no behavior change in completed work (the `git_sources` API, the `documents` API, the phase-91/168 admin surfaces all green unedited).
@@ -1,35 +0,0 @@
# Task 03 — The per-source name/label/tags editor on the Sources page (D6)
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "Sources need a name, a label, tag(s) …"`
**Story:** n/a
## Objective
The owner-visible surface: every source on the Sources page (git, local, upload — the catalogue `GET /api/source-labels` is the single source of truth for the row list, so uploads appear without a second API) gets an inline name/label/tags editor with the §7.4 never-stale save state — the phase-121 token field and the phase-89 ignore-paths token input are the house precedents.
## Work
1. `frontend/index.html` — `#view-git-sources` (L646+): the sources table area (the `#git-sources-table-wrap` region, L792) gains the per-row expandable label block markup — ONE shared template (a `<template id="source-label-row-template">`) the JS clones per row:
- three fields in one row (the house grid): **Name** (`input`, ≤ 120, `maxlength="120"`), **Label** (`input`, ≤ 200, `maxlength="200"`), **Tags** (the token input — the phase-89 ignore-paths input's exact widget: type + Enter/comma adds a chip, Backspace deletes the last, ≤ 10 chips, each ≤ 32 chars — reuse its CSS classes, no new styles beyond the phase-89 ones);
- **Save** / **Reset** buttons (the phase-168 button shapes) + the save-state line (`role="status"` — "Saving…" → "saved" / the one-line error note; Reset visible only when the row has a stored label, the phase-168 per-row reset shape);
- the row's IDENTITY cell (the derived source name) is UNCHANGED and read-only (D1/A1 — the editor sits beside it, never replaces it).
2. `frontend/assets/git-sources.js`:
- the load path: after the existing catalogue load, `GET /api/source-labels` (one extra fetch — the gate is the SAME as the page's: the anonymous gate hides the whole page, the admin sees it; the fetch runs inside the existing admin-gated content reveal); merge the labels into the row data (by `source` name — the GET is the union, so the row list for the label editor comes from the GET, not from the `git-sources` GET — the upload rows have no `git-sources` row);
- the editor state per row: the three fields pre-populated from the label (nulls → empty); dirty tracking (a field change enables Save — the phase-168 `.is-dirty` + "unsaved changes" hint shape, scoped per row);
- **Save**: `PUT /api/source-labels/<source>` (the trimmed values; the tags array from the chips) → success: the "saved" state, the row's label re-read from the response (no second GET — the response is the new truth); failure: the one-line error note (the framework's detail — the 422/404 arm: a 404 shows "that source no longer exists" — the note text names the 404 arm, the phase-168 error-note shape); the §7.4 guard: one in-flight save per row (a second click while saving is ignored, the phase-168 `saveInFlight` pattern scoped per row);
- **Reset**: `DELETE /api/source-labels/<source>` → the "saved" state, the fields cleared, the row's stored label nulled;
- the page's existing sync / upload / git-row flows are UNTOUCHED (the editor is additive — the row's existing action buttons keep their behavior).
3. `tests/unit/` — the JS marker pins (the module that marker-pins `git-sources.js` — verify the name; the `test_save_chat_ui.py` convention):
- the editor's save path references `PUT /api/source-labels/` and the response (not a second GET) as the new truth (source marker);
- the per-row `saveInFlight` guard marker; the 404 note text marker;
- the token input reuses the phase-89 widget classes (a marker naming the class reuse — no new tag-input CSS).
4. Verify: `uv run pytest tests/unit/<js-pin-module> -v` green; `uv run playwright`-free local check: the page renders the editor for a seeded label row (the E2E covers this in task 06 — a manual `uv run uvicorn` smoke is optional, the E2E is the gate).
## Testing & Quality
- Unit: the marker pins above.
- E2E: task 06 (the owner-visible arm).
- Coverage: **>90%** on `app/` (unchanged — the task is frontend; the app-side gate stays green).
## Completion Criteria
- [ ] every source row (git, local, upload) renders the name/label/tags editor; the derived name cell is read-only and untouched (D6/A1).
- [ ] Save/Reset hit the phase-02 endpoints; the save state follows the §7.4 contract (never a stuck "Saving…", the 404 note names the arm) — marker pins.
- [ ] the existing sources-page flows (add git/local source, upload, sync) are green unedited.
- [ ] no behavior change in completed work.
@@ -1,42 +0,0 @@
# Task 04 — The `source_name_hit_bonus` selection boost (D2/D3)
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "… so you can focus the model when prompting …"`
**Story:** n/a
## Objective
The focus mechanism: a NEW bounded, kill-able selection-time bonus that lifts a document when the question names its source (D2 rule) — the phase-119 `name_hit_bonus` pattern end to end, a separate knob, the SAME shared walk, `0` = byte-identical pre-phase.
## Work
1. `app/config.py` — the knob (the `name_hit_bonus` L431 pattern, directly below it):
- `source_name_hit_bonus: float = 0.005` (the docstring: the D3 spec — default = the phase-119 starting point; `0` = off, byte-identical walk; bounded per-document, at most two bonuses on the fused scale);
- the `field_validator` (the L964 pattern): `>= 0`, the message `"source_name_hit_bonus must be >= 0 (0 = off)"`.
2. `app/core/settings_catalog.py` — the `SettingMeta` entry (the L225 `name_hit_bonus` pattern): `"source_name_hit_bonus"` in the **"Retrieval & honesty"** section, label "Source name hit bonus", the restart-required flag FALSE (the phase-168 runtime knob shape — verify the `name_hit_bonus` entry's flags and mirror them). Extend the catalogue-completeness pin (the unit test asserting every `Settings` field has a `SettingMeta` entry — find the module; it starts failing when the field lands, the entry is the fix).
3. `app/rag/retriever.py`:
- `RetrievedChunk` (L276+): the additive field `source_name_hit: bool = False` + the docstring line (the `name_hit` line pattern — default `False` keeps every ordinary construction valid; `fuse`'s `replace()` copies it automatically — the frozen dataclass; the double-hit merge at L342–358 gains `source_name_hit=existing.source_name_hit or rc.source_name_hit` beside the existing `name_hit` OR);
- `_selection_order` (L754): signature `_selection_order(chunks, bonus, source_bonus: float = 0.0)` — the walk gains `any_source_hit[doc.id]` (the `any_name_hit` pattern, L792/L799/L804); the effective-score dict: `best_score[doc.id] + (bonus if any_name_hit[doc.id] else 0.0) + (source_bonus if any_source_hit[doc.id] else 0.0)`; the re-sort condition: `(bonus > 0.0 and any_name_hit…) or (source_bonus > 0.0 and any_source_hit…)` (the pre-phase condition extended — when neither bonus is active, the walk is byte-identical: no re-sort, the same order list, the same `+ 0.0`); the docstring's D2/LOCKED-A3 paragraph is extended for the second bonus (each bonus per DOCUMENT, applied ONCE, at most two per document — bounded 0.010; the bonus lives in the SELECTION layer only — `score` / `cosine` / `fts_hit` / `query_log.top_score` untouched);
- `weak_hit_titles` (L825), `select_suggested` (L952), `select_related` (L1000): a `source_bonus: float | None = None` parameter each; `if source_bonus is None: source_bonus = get_settings().source_name_hit_bonus`; passed to `_selection_order`.
4. `app/api/chat.py` — the flag-set at the retrieval boundary (the call sites L537–572 and L875 — the question is in scope, the session is in scope):
- ONE call per turn: `hits = question_name_hit_sources(db, question)` (task 01's per-turn entry point — the single `all_labels` read) BEFORE the first selection call;
- the flag loop: `for rc in chunks: if rc.document.source in hits: rc.source_name_hit = True` (the few dozen retrieved chunks against the handful of hit sources — never a per-chunk query); the selection calls keep their existing arguments (the functions pull the knob from settings themselves — the call sites gain NO new positional arguments);
- the per-turn log line (PLAN §9) is UNCHANGED (the `verbatim_blocks=` / `scaffold_stripped=` fields are untouched — the boost is invisible to the log; no new log field).
5. `tests/unit/test_retriever.py` (or the phase-119 pin module — verify the name):
- the KILL-SWITCH pin (the phase-119 mirror): `source_name_hit_bonus = 0` (and no flagged chunks) → the selection order is byte-identical to the pre-phase walk (the same fixture, the same expectation list);
- the boost pin: a document whose source is a D2 hit (a `source_name_hit=True` chunk) ranks ABOVE an equal-fused-score non-hit when the bonus is on (the 0.005 head start), and a document carrying BOTH flags gets exactly +0.010 (the bounded two-bonus pin — no amplification);
- the `fuse` merge pin: a vector row merged with a flagged source-name row keeps `source_name_hit=True`;
- the settings pins: the validator (`-0.1` → the ValueError), the env name (`BOR_SOURCE_NAME_HIT_BONUS`), the catalogue-completeness pin extended.
6. The battery (the phase-119 convention): `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` — run it TWICE and record both in the phase file: (a) pre-change (the baseline), (b) post-change at `source_name_hit_bonus = 0` (must match (a) line-for-line — the kill switch) and at the default 0.005 (no label rows exist in the battery app → also identical to (a) — the no-row baseline).
7. Verify: `uv run pytest tests/unit/test_retriever.py tests/unit/test_settings*.py -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the kill-switch byte-identity, the bounded two-bonus application, the `fuse` OR-merge, the settings/catalogue pins.
- Integration: the E2E task's app fixture exercises the flag-set (no separate integration module — the flag loop is covered by the E2E arm + the unit pins).
- Battery: the three-run record above.
- Coverage: **>90%** on `app/`.
## Completion Criteria
- [ ] `source_name_hit_bonus = 0` → the selection walk byte-identical to pre-phase (the unit pin + the battery match).
- [ ] at the default 0.005, a D2-hit document gets exactly the 0.005 head start (0.010 with both flags) — the bounded pin; no chunk field is modified (the untouched-fields pin).
- [ ] the three selection helpers pull the knob from settings by default (the unit pins with a settings override).
- [ ] `tests/fixtures/retrieval_battery.txt` battery: the kill-switch run matches the baseline line-for-line (recorded in the phase file).
- [ ] full test suite **100% passing**; `uv run ruff check . && uv run pyright` clean.
- [ ] no behavior change in completed work (the phase-119 bonus, the A8 gate inputs, the per-turn log line all untouched).
@@ -1,37 +0,0 @@
# Task 05 — The prompt learns the name: the `<document>` block + the KB overview (D4)
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "… so you can focus the model when prompting …"`
**Story:** n/a
## Objective
The model learns the name→docs mapping from the prompt: a suggested `<document>` block carries the source's `name` / `label` as additive attributes (when set), and the KB overview's INPUT lines carry the display name — both byte-identical for a source without a label row. ONE `all_labels` read per turn feeds the task-04 boost flags AND these surfaces.
## Work
1. `app/rag/prompts.py` — `build_high_prompt` (L828):
- a new optional parameter `names: Mapping[str, "SourceLabelView"] | None = None` (the source-name → view map; `None` = pre-phase, byte-identical — the `show_image`/`show_text` optional-parameter pattern);
- the block builder (L934–939): when `names.get(doc.source)` has a `display_name`, the block head gains ` name="{display_name}"` immediately after the `source="…"` attribute; when it has a `label`, it gains ` label="{label}"` after the `name` (the attribute order `source` → `name?` → `label?` → `path` → `title` → `date`; an unset field adds NOTHING — a no-label document's block is byte-identical to pre-phase, and a label-only document gets exactly the one `label` attribute);
- the values are XML-escaped with the SAME helper the block already uses for `source`/`path`/`title` (verify the escaping at L934 — a `"` in a display name must not break the attribute; the phase-106 date formatting is untouched).
2. `app/rag/overview.py` — `build_overview_prompt` (L148):
- a new optional parameter `names: Mapping[str, str] | None = None` (source → display_name); the line builder (L173): `line = f"{names.get(source, source)} — {path} — {title}"` (the display name REPLACES the raw source name in the lite model's input only — the stored overview row and the DB are untouched); no `names` / no entry → the pre-phase line, byte-identical.
- `regenerate_overview` (the caller — verify the signature): it already holds a session; it gains the ONE `all_labels(db)` read (task 01's helper) and passes (a) the display-name map to `build_overview_prompt` and (b) the same read is NOT re-queried — the map is derived from the views in the same loop.
3. `app/api/chat.py` — the task-04 call site is REFACTORED to the shared read (the "one query per turn" contract):
- `labels = all_labels(db)` (the single read, before the first selection call — replacing task 04's `question_name_hit_sources` call, whose internals it inlines);
- `hits = {v.source for v in labels.values() if question_name_hits(v, question)}` (the pure D2 rule per view — the flag loop of task 04 is unchanged, `hits` is now local);
- the `build_high_prompt` call (the grounded plan — verify the exact call site in the module) passes `names=labels` (the `SourceLabelView` map — `build_high_prompt` accepts the views directly; the overview's `str` map is the overview's own derivation, a different surface, no conversion shared).
4. `tests/unit/test_prompts.py` (or the prompt-lock module — verify the name):
- the KILL pin: `names=None` and `names={}` and a map with no entry for the doc's source → the built prompt is byte-identical to the pre-phase build (the existing pre-task pins stay green UNEDITED — they are the kill pin);
- the attribute pins: a `display_name`-only view → exactly ` name="…"` after `source="…"` (the attribute-order pin); a `label`-only view → exactly the `label` attribute (no `name`); both → both attributes in order; the XML-escaping pin (a `"` in a display name renders as the escaped entity in the attribute);
- the overview pins: `names=None` → the pre-phase lines byte-identical; a map entry → the line uses the display name (the `path` / `title` / summary halves untouched); a missing entry → the raw source name.
5. Verify: `uv run pytest tests/unit/test_prompts.py tests/unit/test_overview.py -v` green (verify the overview test module name); `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the byte-identical kill pins (both builders), the attribute-order + escaping pins, the overview line-swap pins.
- Integration: the E2E task's arm covers the wire (the system prompt of a turn with a named source — the mock LLM's captured prompt, the phase-119 E2E precedent).
- Coverage: **>90%** on `app/`.
## Completion Criteria
- [ ] a named source's suggested document renders `name="…"` / `label="…"` (the unit pins); an unnamed source's block and overview line are byte-identical to pre-phase (the unedited pre-task pins + the kill pins).
- [ ] ONE `all_labels` read per turn feeds the boost flags and the prompt surfaces (the marker pin on the chat.py call site — no second label query in the turn).
- [ ] `regenerate_overview` uses the same single read (the marker pin — no second query).
- [ ] full test suite **100% passing**; `uv run ruff check . && uv run pyright` clean.
- [ ] no behavior change in completed work (the `<document>` block for label-less sources, the KB overview text for label-less sources, the phase-106 date attribute all byte-identical).
@@ -1,37 +0,0 @@
# Task 06 — The phase E2E suite, the full gates, the docs pass, the commit
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5`
**Story:** n/a
## Objective
The dedicated Playwright suite proves the owner-visible arm through the real UI — a named source's document surfaces when the question uses the name, the alias never breaks a locator, the 404 arm is visible — run in isolation per AGENTS rule 4; the full gates pass; the docs are consistent; one atomic commit.
## Work
1. `tests/e2e/test_source_name_labels.py` (NEW — the module docstring names the TODO item and the D1 alias contract) — the module-app fixture pattern (the phase-119/168 E2E convention — verify in `tests/e2e/conftest.py` / the sibling suites; the mock LLM from `tests/e2e/mock_llm.py` for the prompt-capture arm):
- the fixture: the standard small KB (the phase-119 fixture convention — a handful of documents in a couple of sources) + one uploaded document (the phase-49/90 upload flow or a direct document-row seed — verify the sibling suites' seeding helper).
- **(a) the editor, all three kinds:** as the admin, open the Sources page; set a Name ("the git server"), a Label ("self-hosted forge") and a tag ("gitea") on a git source row, a Name on a local source row, a Name + tag on the upload row → each save shows "saved" (the §7.4 state) and a reload pre-populates the saved values (the phase-91 pre-population convention); the derived-name cells are unchanged (the A1 pin — the row identity cell text is the pre-save text).
- **(b) the focus, owner-visible:** a chat question that uses the NAME ("what does the git server run?") with the mock LLM → the suggested tier (the `done` frame's `sources` / the UI's source cluster) includes the named source's document (the D3 boost's owner-visible arm — the phase-119 E2E is the precedent; the mock's captured system prompt asserts the `<document name="the git server" label="self-hosted forge" …>` attributes — the D4 arm at the wire); the same question phrased WITHOUT the name (the pre-baseline question from the battery) shows the pre-phase suggested set (the no-signal control — the boost is the only difference).
- **(c) the alias contract:** after the rename, a verbatim-fenced answer (or a direct `page.request` read of the document API) for `source/path` (the DERIVED name) still resolves — the locator is untouched (the A1 pin through the wire); the KB tree / RAG page still lists the derived name (the D1 surface pin).
- **(d) the 404 arm:** the editor's PUT for a source that no longer exists (delete the underlying git row via the API mid-test, then save a label through the UI) → the one-line error note names the 404 arm (the task-03 pin, owner-visible).
- **(e) the reset:** the editor's Reset on the named source → "saved", the fields cleared, a reload shows empty, and the chat's suggested set for the named question reverts to the control (the label's absence is the pre-phase state end to end).
2. Regression, run in isolation: `tests/e2e/test_name_signal.py` (the phase-119 suite — verify the exact name in `tests/e2e/`), `tests/e2e/test_settings_panel.py`, `tests/e2e/test_git_sources.py` (the phase-76 suite — verify the name) — green unedited.
3. The full gates, in order:
- `uv run pytest` (unit + integration) — 100% passing;
- `uv run pytest --cov=app --cov-report=term-missing` — **>90%** on `app/`;
- `uv run pytest tests/e2e/test_source_name_labels.py -v --no-cov` — green in isolation;
- `uv run ruff check . && uv run pyright` — clean.
4. Documentation pass (the phase file's Documentation section): `README.md` + `AGENTS.md` reviewed in their entirety against the post-phase app; the sources section gains the naming feature (the editor, the alias-only contract, `BOR_SOURCE_NAME_HIT_BONUS` with its `0` = off meaning); `AGENTS.md` corrected only where it names the phase-119 bonus as the ONLY name signal.
5. Commit: one atomic, professional commit (Conventional Commits, `--no-gpg-sign`), e.g.
`feat(sources): per-source name, label and tags focus retrieval and the prompt (phase 191)` — the body names the D1 alias contract, the D2 rule, the D3 knob, the D4 surfaces, the new table/migration 0030, and the battery record location. `.agents/phases/todo/191_source_name_labels_tags/` moves to `.agents/phases/complete/` in the SAME commit (AGENTS rule 8).
## Testing & Quality
- E2E: the suite above, isolated; the regression suites green.
- Battery record: task 04's three-run record is in the phase file (the kill-switch run matches the baseline).
- Coverage: **>90%** on `app/`.
- Lint/types: clean.
## Completion Criteria
- [ ] `tests/e2e/test_source_name_labels.py` green in isolation — (a) the editor for git/local/upload with the saved-state contract, (b) the named question surfaces the named source's document + the prompt attributes, (c) the derived-name locator is untouched by a rename, (d) the 404 note, (e) the reset reverts end to end.
- [ ] `uv run pytest` 100% passing; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] `README.md` / `AGENTS.md` consistent (the naming feature + the knob).
- [ ] the phase directory is in `.agents/phases/complete/`; one atomic commit with `--no-gpg-sign`.