From 47456a44dff61834d93fa5e90e82ce138b8218b6 Mon Sep 17 00:00:00 2001 From: ducoterra Date: Sat, 17 Oct 2026 19:35:55 -0400 Subject: [PATCH] chore(agent): phase roadmap from TODO.md, 4 phases (190-193) Convert the four open TODO items into executable phases (append mode, starting at 190 after the completed 189): 190_app_name_settings_fix - the Settings-panel app name wins over the Theme tab; the three brand strings stripped from the Theme tab (colors-only); live brand re-apply on save, no reload 191_source_name_labels_tags - source_labels table + admin API + Sources page editor; the separate bounded source_name_hit_bonus retrieval boost; label surfacing in the blocks and KB overview 192_verbatim_inline_midturn - an additive verbatim_block SSE frame at the show_text success moment; in-place mid-turn fence hydration; the done/restore per-locator dedupe; the done frame, record key, restore and shared page stay byte-identical 193_sync_remaining_count - the running sync/upload label counts down: "N remaining" (N = total - done), hidden at the final file; the chunk/image/summaries/terminal parts untouched TODO.md is cleared (its items now live in .agents/phases/todo/). LLM-Generated: true --- .../190_app_name_settings_fix/00_phase.md | 51 +++++++++++++ .../01_reproduce_shadow_and_staleness.md | 31 ++++++++ .../02_precedence_and_strip_theme_tab.md | 45 ++++++++++++ .../03_live_brand_on_save.md | 37 ++++++++++ .../todo/190_app_name_settings_fix/04_e2e.md | 35 +++++++++ .../191_source_name_labels_tags/00_phase.md | 72 +++++++++++++++++++ .../01_source_labels_table.md | 41 +++++++++++ .../02_source_labels_api.md | 35 +++++++++ .../03_source_labels_ui.md | 35 +++++++++ .../04_source_name_boost.md | 42 +++++++++++ .../05_prompt_surfacing.md | 37 ++++++++++ .../191_source_name_labels_tags/06_e2e.md | 37 ++++++++++ .../192_verbatim_inline_midturn/00_phase.md | 58 +++++++++++++++ .../01_verbatim_block_sse.md | 60 ++++++++++++++++ .../02_midturn_hydration_and_append.md | 66 +++++++++++++++++ .../03_done_and_restore_dedupe.md | 36 ++++++++++ .../192_verbatim_inline_midturn/04_e2e.md | 36 ++++++++++ .../todo/193_sync_remaining_count/00_phase.md | 55 ++++++++++++++ .../01_remaining_label.md | 31 ++++++++ .../02_update_pinned_e2e.md | 25 +++++++ .../todo/193_sync_remaining_count/03_e2e.md | 34 +++++++++ TODO.md | 2 - 22 files changed, 899 insertions(+), 2 deletions(-) create mode 100644 .agents/phases/todo/190_app_name_settings_fix/00_phase.md create mode 100644 .agents/phases/todo/190_app_name_settings_fix/01_reproduce_shadow_and_staleness.md create mode 100644 .agents/phases/todo/190_app_name_settings_fix/02_precedence_and_strip_theme_tab.md create mode 100644 .agents/phases/todo/190_app_name_settings_fix/03_live_brand_on_save.md create mode 100644 .agents/phases/todo/190_app_name_settings_fix/04_e2e.md create mode 100644 .agents/phases/todo/191_source_name_labels_tags/00_phase.md create mode 100644 .agents/phases/todo/191_source_name_labels_tags/01_source_labels_table.md create mode 100644 .agents/phases/todo/191_source_name_labels_tags/02_source_labels_api.md create mode 100644 .agents/phases/todo/191_source_name_labels_tags/03_source_labels_ui.md create mode 100644 .agents/phases/todo/191_source_name_labels_tags/04_source_name_boost.md create mode 100644 .agents/phases/todo/191_source_name_labels_tags/05_prompt_surfacing.md create mode 100644 .agents/phases/todo/191_source_name_labels_tags/06_e2e.md create mode 100644 .agents/phases/todo/192_verbatim_inline_midturn/00_phase.md create mode 100644 .agents/phases/todo/192_verbatim_inline_midturn/01_verbatim_block_sse.md create mode 100644 .agents/phases/todo/192_verbatim_inline_midturn/02_midturn_hydration_and_append.md create mode 100644 .agents/phases/todo/192_verbatim_inline_midturn/03_done_and_restore_dedupe.md create mode 100644 .agents/phases/todo/192_verbatim_inline_midturn/04_e2e.md create mode 100644 .agents/phases/todo/193_sync_remaining_count/00_phase.md create mode 100644 .agents/phases/todo/193_sync_remaining_count/01_remaining_label.md create mode 100644 .agents/phases/todo/193_sync_remaining_count/02_update_pinned_e2e.md create mode 100644 .agents/phases/todo/193_sync_remaining_count/03_e2e.md diff --git a/.agents/phases/todo/190_app_name_settings_fix/00_phase.md b/.agents/phases/todo/190_app_name_settings_fix/00_phase.md new file mode 100644 index 0000000..857c6b7 --- /dev/null +++ b/.agents/phases/todo/190_app_name_settings_fix/00_phase.md @@ -0,0 +1,51 @@ +# Phase 190 — The Settings-panel app name is the app name again (and saves apply live) + +**Source:** `TODO.md` L3 — "changing the App name in settings broke, it no longer updates the name." +**Story:** n/a (owner report) +**Depends on:** none functionally (builds on the completed phase-168 admin Settings panel and phase-91/92 admin Theme tab). + +## Evidence (diagnosed from the code, read this not the chat) + +1. **Two writers for one setting.** `app_name` (and `input_placeholder` / `footer_text`) is now writable from TWO admin surfaces: the Settings panel (`/api/admin/settings/app_name` → `app_settings` table → the `app.config` DB-over-env overlay, phase 168) and the Theme tab (`/api/ui-settings` PUT → the single `ui_settings` row, phase 91). +2. **The Theme tab row WINS.** `app.core.theming.effective_settings` (L144–177) resolves each string as: the `ui_settings` row value when it is a non-empty string, **else** `settings.` (the env value, which — via the phase-168 overlay — is the Settings-panel row value when one exists). So the order is Theme-tab row → Settings-panel row → env. +3. **A Theme-tab save bakes the strings in.** `theme.js` `collectBody()` (L526) collects ALL 20 fields — the 3 string inputs are pre-populated with the EFFECTIVE value on load — and `PUT /api/ui-settings` writes every non-empty string into the row. So the moment an admin saves ANY color from the Theme tab, the current app name is baked into `ui_settings.app_name`, and from then on **every later Settings-panel app-name change is shadowed** by that baked row. This is exactly "it no longer updates the name" — the Settings panel shows the new value with its `db` badge, but `/api/config` (which serves `effective_settings`) reports the Theme tab's baked name. +4. **No live brand update, on either surface.** `settings.js` re-GETs the catalogue after a save but never re-applies the brand strings to the open document; `theme.js` after a save runs `applyServedTheme(settings)` which reconciles the COLOR palette (phase 92) but never touches `document.title`, `.brand-text`, the footer or the composer placeholder. `brand.js` runs the brand path once at boot from `/api/config`. So even when the server-side value is right, the open page keeps showing the old name until a manual reload. + +## Objective +The Settings panel is the single writable owner of the three brand strings and always wins; the Theme tab becomes colors-only (its PUT no longer writes the strings, its form drops them — for legacy baked-in rows the panel still wins); and a save in either surface re-applies the brand strings to the open document immediately, so the change is visible the moment "save" is clicked (the phase-168 contract extended to the display). + +## ASSUMPTIONS (owner-confirmed 2026-10-18) +- **A1 — Panel wins, strings stripped from the Theme tab.** The Settings-panel row beats the Theme-tab row for `app_name` / `input_placeholder` / `footer_text` (no data migration — existing baked-in rows simply stop shadowing the panel; with no panel row they still apply as the documented fallback). The Theme tab's form drops the three string inputs and its PUT body no longer carries them (the server ignores string fields in the PUT body defensively — an old client's save can no longer re-bake a string). The Theme tab's GET keeps reporting the effective strings (read-only, old-client compat). +- **A2 — Live brand on save.** After a successful Settings-panel save/clear or Theme-tab save/reset, the open document's `document.title`, `.brand-text`, the footer and the composer placeholder are re-applied from a fresh `/api/config` — no reload. The boot path (`brand.js`) is byte-identical. + +## Context +- `app/core/theming.py` — `effective_settings` (L144–177: the string resolution to change), `STRING_FIELDS`, `COLOR_FIELDS`, `BUILTIN_COLORS`; `app/api/config.py` — `GET /api/config` (serves `effective_settings` + `source` per string); `app/api/ui_settings.py` — the phase-91 admin API (GET reports 20 fields, PUT writes 3 strings + 17 colors, DELETE resets the row); `app/core/settings_overlay.py` + `app/config.py` `invalidate_settings` (L1129) — the phase-168 DB-over-env overlay (the panel rows; `settings.app_name` already resolves through it); `app/api/admin_settings.py` (phase 168, the panel surface). +- `frontend/assets/brand.js` — the boot-time brand path (title / `.brand-text` / footer / placeholder from `/api/config`); `frontend/assets/theme.js` — `FIELDS` (L183–185 the 3 string fields), `collectBody` (L526), `saveTheme` (L580, `applyServedTheme` L589), `resetTheme` (L597); `frontend/assets/settings.js` — the phase-168 panel (its header names the phase-168 contract "the change should take effect on clicking 'save'"); `frontend/index.html` — the Theme-tab markup (`#theme-app-name` L1354 et al., the three string rows to remove). +- `tests/unit/test_theming.py` (the `effective_settings` pins), `tests/integration/test_ui_settings_api.py` (the phase-91 API pins), `tests/e2e/test_admin_theme_tab.py` (fills `#theme-app-name` — must move with the strip), `tests/e2e/test_settings_panel.py` (the phase-168 pins), `tests/unit/test_save_chat_ui.py` (JS marker-pin convention). + +## Tasks +1. `01_reproduce_shadow_and_staleness.md` — failing tests pinning both defects (the shadowing through `effective_settings` / `/api/config`, the Theme-tab PUT string bake, the no-live-update behavior). +2. `02_precedence_and_strip_theme_tab.md` — A1: the resolution order flips to panel → tab → env; the Theme tab drops the 3 string inputs and its PUT stops writing them; the pinned theme-tab suites move. +3. `03_live_brand_on_save.md` — A2: one shared `applyBrandFromConfig` pass (the `brand.js` DOM path extracted, boot behavior byte-identical) re-applied after a Settings-panel save/clear and a Theme-tab save/reset. +4. `04_e2e_and_gates.md` — the dedicated Playwright suite (the incident regression through the real UI), the full gates, the docs pass, the atomic commit. + +## Testing & Quality +- **Unit:** `tests/unit/test_theming.py` — the NEW resolution order pins (panel row beats a conflicting `ui_settings` row; with no panel row the tab row still applies — the A1 fallback); the pre-phase pins that do not involve a panel row stay green. +- **Integration:** `tests/integration/test_ui_settings_api.py` — a Theme-tab PUT with string fields in the body **ignores** them (the stored row's string columns are untouched / the response no longer round-trips a changed string); `tests/integration/` Settings-panel pins (phase 168) green — the panel write + `invalidate_settings` path is untouched. +- **E2E (AGENTS rule 4, task 04):** `tests/e2e/test_app_name_settings_fix.py`, run in isolation — (a) the incident regression: set the name in the Settings panel → save a color in the Theme tab → the name still shows the panel's value (pre-fix: the baked tab value); (b) live apply: change the app name in the Settings panel → `document.title`, `.brand-text`, the footer and the composer placeholder update WITHOUT a reload; (c) a Theme-tab save no longer bakes the name (a later panel change is visible on reload); (d) a panel reset (DELETE) falls back to env live. +- **Regression:** `tests/e2e/test_settings_panel.py`, `tests/e2e/test_admin_theme_tab.py` (moved in task 02), `tests/e2e/test_theme_save_and_coverage.py` green. +- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`); the frontend change is marker-pinned per the house JS convention. +- **Lint/types:** `uv run ruff check . && uv run pyright`. + +## Documentation & Consistency +- Final pass reviews `README.md` and `AGENTS.md` in their entirety against the post-phase app and fixes every inconsistency found, pre-existing ones included. +- `README.md`: the settings/theme section notes the single-writer ownership (the Settings panel owns the brand strings; the Theme tab is colors-only) and the live-apply behavior (task 04). `AGENTS.md`: the phase-168 "settings overlay" description gains the resolution-order clause if it names one (verify, do not add if absent). + +## Completion Criteria +- [ ] `effective_settings` resolves the 3 strings panel-row → tab-row → env (unit pins, both branches). +- [ ] `PUT /api/ui-settings` with string fields in the body does not write them (integration pin); the Theme-tab form no longer renders the 3 inputs (E2E pin). +- [ ] A Settings-panel app-name change is visible in `document.title`, `.brand-text`, the footer and the placeholder without a reload (E2E); the Theme-tab color save no longer shadows it (E2E regression of the incident). +- [ ] the `brand.js` boot path is byte-identical (marker pin) — no double-apply at boot. +- [ ] test suite green, coverage >90%; `uv run ruff check . && uv run pyright` clean. +- [ ] no behavior change in completed phases (the 17-color palette, the phase-168 panel, the env fallback all behave as before). +- [ ] one atomic Conventional Commits commit with `--no-gpg-sign`. diff --git a/.agents/phases/todo/190_app_name_settings_fix/01_reproduce_shadow_and_staleness.md b/.agents/phases/todo/190_app_name_settings_fix/01_reproduce_shadow_and_staleness.md new file mode 100644 index 0000000..97ae0e8 --- /dev/null +++ b/.agents/phases/todo/190_app_name_settings_fix/01_reproduce_shadow_and_staleness.md @@ -0,0 +1,31 @@ +# Task 01 — Reproduce and pin the two defects (shadowing + no live update) + +**Phase:** `190_app_name_settings_fix` · **Source:** `TODO.md:4 — "changing the App name in settings broke, it no longer updates the name."` +**Story:** n/a + +## Objective +Before touching behavior: pin BOTH defects with failing tests — (a) the `ui_settings` row shadows the Settings-panel row in `effective_settings` and therefore in `GET /api/config`, and a Theme-tab PUT bakes the effective strings into the row (the mechanism of "it no longer updates"); (b) a Settings-panel save never re-applies the brand strings to the open document (marker pin on the ABSENT behavior, inverted in task 03). + +## Work +1. `tests/unit/test_theming.py` — NEW pins (failing until task 02): + - `test_effective_app_name_prefers_settings_panel_row_over_ui_settings_row` — an `app_settings` row (via the phase-168 overlay / a `Settings` with `app_name="Panel Name"`) + a `ui_settings` row with `app_name="Tab Name"` → `effective_settings["app_name"] == "Panel Name"` (today: `"Tab Name"` — the documented A1 order). Same pair for `input_placeholder` and `footer_text`. + - `test_effective_strings_fall_back_to_ui_settings_row_without_panel_row` — `ui_settings` row set, no panel row (the `Settings` at the env default) → the tab value applies (the A1 fallback this pin protects — it must stay GREEN pre-phase and post-phase). + - The pre-existing pins that assume tab-over-env with NO panel row stay green unedited. +2. `tests/integration/test_ui_settings_api.py` — NEW pins (failing until task 02): + - `test_put_ignores_string_fields_in_body` — a `PUT /api/ui-settings` whose body carries `app_name` / `input_placeholder` / `footer_text` DIFFERENT from the stored row's values and from the effective value → 200; the stored row's string columns are UNCHANGED (re-GET shows the old values); the 17 colors in the same body ARE stored (the colors half of the contract survives). + - `test_get_reports_effective_strings_read_only` — with a conflicting panel row, `GET /api/ui-settings` reports the EFFECTIVE (panel) value for the strings (read-only view; the A1 order reaches the API). +3. `tests/unit/test_brand_live.py` (NEW module) — marker pins on the CURRENT (absent) behavior, documented as "inverted in task 03": + - `settings.js` contains NO call that re-applies `document.title` / `.brand-text` after a save (assert the marker is ABSENT pre-phase — the test asserts the file's save path does not reference the brand DOM ids; task 03 inverts the expectation); + - `theme.js` `saveTheme`'s success path calls `applyServedTheme` (colors only) and does NOT touch the brand DOM (marker pin — stays green through the phase; the live-apply added in task 03 is asserted there, not here). +4. Verify: `uv run pytest tests/unit/test_theming.py tests/integration/test_ui_settings_api.py tests/unit/test_brand_live.py -v` — the NEW pins FAIL (documented), everything else green. + +## Testing & Quality +- Unit/integration: the pins above — the shadowing mechanism, the string-bake, the fallback (green pre-phase), the absent-live-behavior markers. +- Coverage: **>90%** on `app/` (no app code changes this task — the pins exercise existing code). + +## Completion Criteria +- [ ] the 3 `effective_settings` order pins FAIL pre-fix (documented in the test docstrings: "fixed by task 02"). +- [ ] the string-bake integration pin FAILS pre-fix (documented); the colors-half pin is GREEN pre-fix. +- [ ] the live-behavior marker pins pass (asserting the ABSENT behavior, inverted in task 03). +- [ ] full test suite **100% passing** except the documented failing pins (the failing pins are the task's artifact — the phase's E2E gate runs them post-fix). +- [ ] no behavior change in completed work (app/ untouched). diff --git a/.agents/phases/todo/190_app_name_settings_fix/02_precedence_and_strip_theme_tab.md b/.agents/phases/todo/190_app_name_settings_fix/02_precedence_and_strip_theme_tab.md new file mode 100644 index 0000000..4b2fe17 --- /dev/null +++ b/.agents/phases/todo/190_app_name_settings_fix/02_precedence_and_strip_theme_tab.md @@ -0,0 +1,45 @@ +# Task 02 — The panel wins; the Theme tab becomes colors-only (A1) + +**Phase:** `190_app_name_settings_fix` · **Source:** `TODO.md:4 — "changing the App name in settings broke, it no longer updates the name."` +**Story:** n/a + +## Objective +Flip the string resolution order to Settings-panel row → Theme-tab row → env (no data migration — existing baked-in rows stop shadowing the panel), and strip the three string inputs from the Theme tab: the form drops them, `collectBody` stops collecting them, and the server's `PUT /api/ui-settings` persists ONLY the 17 colors (string fields in the body are ignored defensively — an old client's save can no longer re-bake a string; string columns are left untouched, not cleared, so the A1 fallback survives). + +## Work +1. `app/core/theming.py` — `effective_settings` (L170–173), the string loop becomes a three-way resolution per field: + - **panel row** — `getattr(settings, field)` (the `Settings` passed by the route; the phase-168 overlay makes it the panel value when an `app_settings` row exists, else env) is used when a panel row exists for that key. The panel presence is read through the SAME overlay source the `Settings` object carries: the overlay's `app_settings` row map (`app.core.settings_overlay.load_overrides()` — the one query the overlay already caches per process; the route passes the settled `Settings`, so the check is `field in the override map` — implement as `has_override(field)` on the overlay module, one cached read, no second query per field); + - **else the tab row** — the `ui_settings` row value when it is a non-empty string (the phase-91 fallback, now second); + - **else the env value** — `getattr(settings, field)` (identical expression to today's fallback). + - The 17-color half is UNTOUCHED (row → `BUILTIN_COLORS`). The docstring's resolution table is rewritten (the old "DB value when non-empty, else env" becomes the three-way table). + - `app/api/config.py` — the per-string `source` badge (`db` / `env`): `db` now means the PANEL row; the tab row yields the existing non-`db` value (verify the current `source` vocabulary — if it is a two-value `db`/`env` pair, `db` = any stored row wins, unchanged; do NOT grow the vocabulary). +2. `app/api/ui_settings.py` — the PUT handler: + - persist ONLY the 17 `COLOR_FIELDS` from the body; the three string columns are NEVER written (the request model keeps accepting the fields — old-client compat — the handler drops them before persisting); the stored row's string values are left exactly as stored (a legacy baked-in row keeps its value; the A1 order decides visibility); + - the DELETE (reset) path: UNCHANGED (it nulls the whole row — including the strings; a reset is a user decision, and the A1 fallback then falls through to env, which is the documented reset behavior). + - the GET: unchanged (reports the effective strings — with the new order, a conflicting panel value is what the tab page pre-populates its (now removed) inputs with in an old client; the read-only report stays). +3. `frontend/assets/theme.js`: + - `FIELDS` (L183–185): the three `kind: "string"` entries are REMOVED — `FIELDS` becomes the 17 color fields only; everything FIELDS-driven (live preview, `collectBody`, `applyServedTheme`'s tag content) stays in `COLOR_FIELDS` order (the comment at L180–181 already says so — it becomes true); + - `collectBody` (L526): the body is the 17 colors only (no string keys at all); + - the string-specific docstring lines (L183–185 region + any "20 fields" counts in the module header) become "17 colors." +4. `frontend/index.html` — the Theme-tab markup: the three string rows (`#theme-app-name` L1354 et al., the placeholder + footer rows) are REMOVED from the form; the section count / any "3 strings + 17 colors" comment becomes colors-only. +5. `tests/unit/test_theming.py` — the task-01 failing pins now PASS unedited (they name the A1 order); the no-panel-row fallback pin stays green. +6. `tests/integration/test_ui_settings_api.py` — the task-01 string-bake pin now PASSES (body strings ignored, colors stored, string columns untouched). +7. The pinned theme-tab E2E suites that fill the removed inputs move: + - `tests/e2e/test_admin_theme_tab.py` — every step that fills `#theme-app-name` / `#theme-placeholder` / `#theme-footer` (or asserts their presence) is RETARGETED: the tab renders 17 color inputs only (a new pin: the three string inputs are ABSENT from the tab DOM — `page.locator("#theme-app-name")` count 0); a save still round-trips the 17 colors through the GET pre-population. + - `tests/e2e/test_theme_save_and_coverage.py` / any other suite that seeds a string through the tab: the string is now seeded through the Settings panel (the `test_settings_panel.py` helper) or the API (the integration test covers the wire; the E2E keeps its color assertions). +8. `tests/unit/test_theme_ui.py` (or the module that marker-pins `theme.js` — verify the name in `tests/unit/`) — marker pins: `FIELDS` has no `kind: "string"` entry; `collectBody`'s body has no string keys (the source-marker convention); the `saveTheme` success path still calls `applyServedTheme`. +9. Verify: `uv run pytest tests/unit/test_theming.py tests/integration/test_ui_settings_api.py -v` green; `uv run pytest tests/e2e/test_admin_theme_tab.py tests/e2e/test_settings_panel.py -v --no-cov` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean. + +## Testing & Quality +- Unit: the A1 order pins (both branches) + the marker pins (tab has no string fields). +- Integration: string-ignored PUT pin + the phase-91 color pins green unedited. +- E2E (regression, run in isolation): `test_admin_theme_tab.py` (moved) + `test_settings_panel.py` green. +- Coverage: **>90%** on `app/`. + +## Completion Criteria +- [ ] `effective_settings` resolves the 3 strings panel → tab → env (both-branch unit pins green). +- [ ] `PUT /api/ui-settings` with string fields in the body stores no string change and stores the colors (integration pin green). +- [ ] the Theme-tab DOM renders 17 color inputs only; the 3 string inputs are absent (E2E pin). +- [ ] the phase-91 color contract (GET pre-population, save, reset) is green unedited. +- [ ] full test suite **100% passing**. +- [ ] no behavior change in completed work (the 17 colors, the phase-168 panel, the env fallback, the DELETE-reset behavior all as before). diff --git a/.agents/phases/todo/190_app_name_settings_fix/03_live_brand_on_save.md b/.agents/phases/todo/190_app_name_settings_fix/03_live_brand_on_save.md new file mode 100644 index 0000000..58bb3f1 --- /dev/null +++ b/.agents/phases/todo/190_app_name_settings_fix/03_live_brand_on_save.md @@ -0,0 +1,37 @@ +# Task 03 — The save is visible without a reload (A2) + +**Phase:** `190_app_name_settings_fix` · **Source:** `TODO.md:4 — "changing the App name in settings broke, it no longer updates the name."` (the phase-168 header contract: "the change should take effect on clicking 'save'") +**Story:** n/a + +## Objective +A successful save in the Settings panel (or a per-row reset) and a save/reset in the Theme tab re-applies the brand strings to the OPEN document immediately — `document.title`, `.brand-text`, the footer and the composer placeholder — from a fresh `GET /api/config`, with no reload. The boot path in `brand.js` is extracted into the shared pass byte-identically, so boot and live-apply are ONE function and the never-stale contract (PLAN §7.4) covers the strings the way it already covers the palette (phase 92 `applyServedTheme`). + +## Work +1. `frontend/assets/brand.js` — extract the DOM path (title / `.brand-text` / footer / placeholder assignment from the `/api/config` payload) into ONE named function `applyBrandFromConfig(cfg)` (same file, same module scope — no new file, no new CDN; `brand.js` stays the single owner of the brand DOM): + - the BOOT call site (`initBrand` / the boot fetch-then-apply) calls `applyBrandFromConfig(cfg)` — the observable boot behavior is byte-identical (same request, same assignments, same order); + - the function is exported to the module's existing export surface (the `window` / module pattern `brand.js` already uses — verify the convention; `theme.js` and `settings.js` import from the same pattern). +2. `frontend/assets/settings.js` — the save path: + - after a SUCCESSFUL per-key PUT (`PUT /api/admin/settings/`, the phase-168 success note) and after a SUCCESSFUL per-row reset (DELETE) and the reset-all: `await fetch("/api/config")` → `applyBrandFromConfig(cfg)` — ONLY when the saved/removed key is one of the three brand strings (`app_name` / `input_placeholder` / `footer_text`); a save of any other key does NOT re-apply the brand (no wasted request, no DOM churn on 72 other rows); + - the §7.4 guard: while the re-apply fetch is in flight, a second save does not stack a second fetch (the existing `saveInFlight` guard's scope — extend, do not fork); + - the existing post-save re-GET of the catalogue is UNTOUCHED (it stays the catalogue's own refresh; the brand re-apply is additive and independent). +3. `frontend/assets/theme.js` — the save/reset path: + - `saveTheme` success (after the existing `applyServedTheme(settings)` at L589) and `resetTheme` success: `await fetch("/api/config")` → `applyBrandFromConfig(cfg)`. (The tab no longer edits the strings, but a RESET nulls the stored row — with a legacy baked-in string that reset makes the effective string FALL BACK (panel row → env), so the open document must re-read it; the save path keeps the same call — one code path for both, the body change is the colors.) + - the fetch-failure path: the brand keeps its last applied state (never a blanked title/footer — the phase-168 "never a blanked panel" precedent, applied to the brand: a failed re-apply leaves the DOM as-is; the save's own error note is the visibility). +4. `tests/unit/test_brand_live.py` — the task-01 absent-behavior marker pins are INVERTED: + - `settings.js`'s save path references `applyBrandFromConfig` (the marker is now PRESENT) and gates the re-apply on the three string keys (the source marker names the key check); + - `theme.js`'s `saveTheme`/`resetTheme` success paths reference `applyBrandFromConfig` (marker present); + - `brand.js` exports `applyBrandFromConfig` and the boot call site calls it (marker — the extraction, not a duplication: `initBrand`/boot contains no second copy of the four DOM assignments — they exist only in the shared function). +5. Verify: `uv run pytest tests/unit/test_brand_live.py -v` green; `uv run pytest tests/e2e/test_settings_panel.py -v --no-cov` green (the phase-168 suite — its save flow now also triggers the brand re-apply; its pins must stay green — the re-apply is additive); `uv run pytest` green. + +## Testing & Quality +- Unit: the inverted marker pins (shared function exists; both surfaces call it; the boot path is the extraction — single copy). +- Integration: none (pure frontend — the wire contract is task 02's; `/api/config` is unchanged). +- E2E: covered by task 04 (the owner-visible arm). +- Coverage: **>90%** on `app/` (unchanged — the task is frontend; the app-side gate stays green). + +## Completion Criteria +- [ ] `applyBrandFromConfig` is ONE function in `brand.js`; the boot call site uses it; the four DOM assignments exist in exactly one place (marker pins). +- [ ] `settings.js` re-applies the brand after a save/reset of a brand-string key only; `theme.js` re-applies after save and reset (marker pins). +- [ ] a failed re-apply leaves the DOM at its last applied state (marker pin on the failure path). +- [ ] `test_settings_panel.py` green unedited. +- [ ] no behavior change in completed work (the boot path, the phase-168 catalogue refresh, the phase-92 palette sync all as before). diff --git a/.agents/phases/todo/190_app_name_settings_fix/04_e2e.md b/.agents/phases/todo/190_app_name_settings_fix/04_e2e.md new file mode 100644 index 0000000..a680d74 --- /dev/null +++ b/.agents/phases/todo/190_app_name_settings_fix/04_e2e.md @@ -0,0 +1,35 @@ +# Task 04 — The phase E2E suite, the full gates, the docs pass, the commit + +**Phase:** `190_app_name_settings_fix` · **Source:** `TODO.md:4` +**Story:** n/a + +## Objective +The dedicated Playwright suite proves the incident regression and the live-apply contract through the real UI (run in isolation per AGENTS rule 4); the full gates pass; `README.md` / `AGENTS.md` are consistent with the post-phase app; one atomic commit. + +## Work +1. `tests/e2e/test_app_name_settings_fix.py` (NEW — the module docstring names the incident: "a Theme-tab save used to bake the effective name into `ui_settings` and shadow every later Settings-panel change") — the app under test is the standard module-app pattern (the phase-168/91 suites' fixture — verify the convention in `tests/e2e/conftest.py` / the sibling suites): + - **(a) the incident regression (server-side, at the wire):** with an env default name, the admin sets the app name via the Settings panel UI (type + save, the phase-168 flow) → `GET /api/config` (via `page.request`) reports the panel value with the `db` source; then the admin opens the Theme tab, changes ONE color and saves → `GET /api/config` STILL reports the panel value (pre-fix: the baked tab value); the stored `ui_settings` row's string columns are unchanged (the API-level assertion — `page.request` GET `/api/ui-settings`). + - **(b) live apply, panel:** change the app name in the Settings panel (a brand-string row) → after the save note, WITHOUT a reload: `document.title`, `.brand-text` (the header), the footer text and the composer placeholder all carry the new name (the four assertions in one test — the A2 contract). + - **(c) live apply, tab reset:** with a legacy `ui_settings` row pre-seeded (the API seeds the row directly — a baked `app_name`), a Theme-tab Reset → the open document's brand falls back (panel row or env — seed NO panel row so the fallback is env) WITHOUT a reload. + - **(d) the boot path is byte-identical:** a fresh page load (no save) shows the env/panel name exactly as the pre-phase `brand.js` did (the `test_admin_theme_tab.py` boot pin, moved here if the moved suite no longer covers it). + - **(e) the tab DOM is colors-only:** the Theme tab renders 17 color inputs and NO `#theme-app-name` / placeholder / footer string inputs (the task-02 pin, asserted in the isolated suite as well). +2. Regression, run in isolation: `tests/e2e/test_admin_theme_tab.py` (moved in task 02), `tests/e2e/test_settings_panel.py`, `tests/e2e/test_theme_save_and_coverage.py` — all green unedited after the moves. +3. The full gates, in order: + - `uv run pytest` (unit + integration) — 100% passing; + - `uv run pytest --cov=app --cov-report=term-missing` — **>90%** on `app/`; + - `uv run pytest tests/e2e/test_app_name_settings_fix.py -v --no-cov` — green in isolation; + - `uv run ruff check . && uv run pyright` — clean. +4. Documentation pass (the phase file's Documentation section): `README.md` + `AGENTS.md` reviewed in their entirety against the post-phase app; the settings/theme section notes the single-writer ownership (Settings panel owns the 3 brand strings, panel-row → tab-row → env order) and the live-apply behavior; `AGENTS.md` corrected only where it names the old order. +5. Commit: one atomic, professional commit (Conventional Commits, `--no-gpg-sign`), e.g. + `fix(settings): the Settings-panel app name wins over the Theme tab and saves apply live (phase 190)` — the body names the two defects (the `ui_settings` shadow bake, the absent live re-apply), the A1/A2 decisions, and the moved theme-tab suites. `.agents/phases/todo/190_app_name_settings_fix/` moves to `.agents/phases/complete/` in the SAME commit (AGENTS rule 8). + +## Testing & Quality +- E2E: the suite above, isolated; the three regression suites green. +- Coverage: **>90%** on `app/`. +- Lint/types: clean. + +## Completion Criteria +- [ ] `tests/e2e/test_app_name_settings_fix.py` green in isolation — (a) the incident regression (the panel value survives a Theme-tab color save), (b) the four-DOM live apply without reload, (c) the tab-reset fallback live, (d) the byte-identical boot, (e) the colors-only tab DOM. +- [ ] `uv run pytest` 100% passing; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean. +- [ ] `README.md` / `AGENTS.md` consistent (the single-writer + live-apply clauses). +- [ ] the phase directory is in `.agents/phases/complete/`; one atomic commit with `--no-gpg-sign`. diff --git a/.agents/phases/todo/191_source_name_labels_tags/00_phase.md b/.agents/phases/todo/191_source_name_labels_tags/00_phase.md new file mode 100644 index 0000000..91d16f7 --- /dev/null +++ b/.agents/phases/todo/191_source_name_labels_tags/00_phase.md @@ -0,0 +1,72 @@ +# Phase 191 — Sources carry a name, a label and tags — and the name focuses retrieval + +**Source:** `TODO.md` L4 — "Sources need a name, a label, tag(s) so you can focus the model when prompting if you have a project name or a standard name you reference things by." +**Story:** n/a (owner directive) +**Depends on:** none functionally (builds on the completed phase-119 name-signal selection bonus, the phase-31/142 KB overview, the phase-76/35/38/49/90 Sources page). + +## Design (shared by all tasks — the executor reads this, not the chat) + +- **D1 — the data (task 01).** A new single-purpose table `source_labels` (alembic **0030**, additive, no backfill — a source without a row has no name/label/tags and behaves exactly as pre-phase): + - `source` — `String(120)` **PK**: the source name EXACTLY as `Document.source` carries it (the derived basename — the clone dir / local dir / upload file stem); the join key to the documents and to the `git_sources` row (via `_url_basename` / the local path basename); + - `display_name` — `Text`, nullable, ≤ 120 chars (the trimmed value; the owner's name for the source — "the git server"); + - `label` — `Text`, nullable, ≤ 200 chars (the short one-line description/category — "self-hosted forge + Gitea"); + - `tags` — `JSONB`, `NOT NULL`, server default `'[]'` (the focus tokens, each ≤ 32 chars, ≤ 10 tags — the owner's "tag(s)"); + - `updated_at` — `DateTime(timezone=True)`, server default now, onupdate now. + - **The display name is an ALIAS ONLY** (owner-confirmed assumption): `Document.source`, the `source/path` document identity, the read/grep/`show_text` locators, the citations and the KB tree are ALL untouched by a label row — renaming a source never invalidates a saved chat, a shared page or a verbatim locator. The derived name stays the wire identity; the display name rides beside it. + - The ORM model `SourceLabel` (the `SourcesMeta` single-column style, `app/models.py`) + `app/rag/source_labels.py` (the new module — `app/rag/sources_meta.py` is the phase-53 version counter, NOT this feature; no name collision): `get_label(db, source)`, `set_label(db, source, display_name, label, tags)` (upsert, flush-only — the caller commits, the phase-53 session convention), `all_labels(db)`, `label_for(db, source)` → a frozen `SourceLabelView` (display_name / label / tags, `None` when absent), `question_name_hits(labels, question)` → the matching rule below. +- **D2 — the matching rule (task 04).** A source is a SOURCE-NAME HIT when at least one `name_hit_tokens(question)` candidate (the phase-119 class-agnostic token list, reused UNCHANGED — `app/rag/retriever.py` L191) equals, case-insensitively (the `_normalize_name` form), a token of the source's `display_name`, `label` or any `tags` entry. The phase-119 digit-class rule does NOT apply to user-given names (the owner's names are not paths — "gitea" must hit even though it is digitless; the phase-119 digit rule exists to keep prose tokens from matching PATHS, and user names have no prose-precision problem: a 4-char minimum token length (`NAME_TOKEN_MIN_LEN`, reused) is the only filter, applied to the SOURCE side too — a 2-char tag never matches). One query per turn: the label rows (a handful) are loaded once and the match is in Python (the phase-119 one-query convention). +- **D3 — the boost (task 04).** A NEW bounded selection-time bonus, the phase-119 `name_hit_bonus` pattern end to end, a SEPARATE knob (the two signals stay independently kill-able): + - `source_name_hit_bonus: float = 0.005` (default = the phase-119 starting point; validator `>= 0`, `0` = off; `BOR_SOURCE_NAME_HIT_BONUS` env; the `settings_catalog` metadata entry — the catalogue-completeness pin is extended, NOT waived); + - `RetrievedChunk` gains the additive field `source_name_hit: bool = False` (the `name_hit` docstring pattern: default `False` keeps every ordinary construction valid; `fuse`'s `replace()` copies it; the double-hit merge ORs it in — a vector row that is also the source-name hit keeps the flag); + - the flag is set at the retrieval boundary in `app/api/chat.py` (the call sites where `select_suggested` / `select_related` / `weak_hit_titles` run — L537–572 and L875 — the question is in scope): for every chunk whose `document.source` is a D2 hit, `rc.source_name_hit = True` (a loop over the few dozen retrieved chunks against the handful of hit sources — never a new query); + - `_selection_order` (L754) gains the second bonus parameter: `effective_score = best_fused + (bonus if any name_hit) + (source_bonus if any source_name_hit)` — each bonus per DOCUMENT, applied ONCE, at most two bonuses per document (bounded: 0.010 on the fused scale, the same scale phase 119 proved); `source_bonus == 0` (or no flagged chunk) → the walk is BYTE-IDENTICAL to the pre-phase walk (the phase-119 kill-switch contract, LOCKED A3 — a new unit pin mirrors the phase-119 one); + - the three application sites (`select_suggested` L537/L875, `select_related` L538, `weak_hit_titles` L572) pass `get_settings().source_name_hit_bonus`; the bonus lives in the SELECTION layer only — `score` / `cosine` / `fts_hit` / `query_log.top_score` are untouched (the phase-119 LOCKED A3 clause, mirrored). +- **D4 — the prompt surfacing (task 05).** The model learns the name→docs mapping from the prompt, not from retrieval alone: + - the KB overview's INPUT lines (`app/rag/overview.py` `build_overview_prompt`, L148 — `source — path — title — summary`) render `display_name — path — …` when set (the raw source name is REPLACED in the lite model's input only — the stored overview text is the lite model's prose, which then naturally uses the owner's name; the overview row's `source` field in the DB is untouched); + - the suggested-tier `` block (the per-doc line in `app/rag/prompts.py`, the phase-31 section builder) renders `display_name (label)` before the path when set — e.g. `the git server (self-hosted forge) — docs/gitea.md — …` (the parenthetical only when BOTH are set; one of the two → just the present one; neither → the pre-phase line, byte-identical). +- **D5 — the API (task 02).** `app/api/source_labels.py` (NEW router, the phase-91 admin gate — `require_admin` on every route; registered in `app/main.py` alongside the other admin routers): + - `GET /api/source-labels` → `{"labels": [ {source, display_name, label, tags} … ]}` — the FULL source catalogue: the union of (a) every `git_sources` row's derived source name (git + local, the `effective_sources` rows — an env-only deployment lists the env source names) and (b) every `SELECT DISTINCT source FROM documents` name NOT covered by (a) (the uploads) — one query for (b), the handful of `git_sources` rows for (a); each entry carries its label row or nulls; + - `PUT /api/source-labels/{source}` (path param ≤ 120) body `{display_name?, label?, tags?}` → the upsert; validation (the phase-121 boundary style — the framework's standard 422): `display_name` trimmed ≤ 120 (empty string / whitespace-only = NULL), `label` trimmed ≤ 200 (same), `tags` ≤ 10 entries, each a trimmed ≤ 32 non-empty token, de-duplicated case-insensitively preserving first case, `[]` legal (clears); an explicit `null` clears that field (an omitted field leaves it — the phase-168 partial-update shape); a `source` that matches NO git row and NO document → **404** (no orphan label rows — a label for a source that does not exist would be a dead boost input); + - `DELETE /api/source-labels/{source}` → row removed (404 when absent) — the reset. +- **D6 — the UI (task 03).** The Sources page (`#view-git-sources`, `git-sources.js`) — per-source name/label/tags editor for EVERY source kind (git, local, upload — the upload rows are the same table's extra section, phase 49/90): + - one expandable row per source (the house pattern — the phase-121 token field is the closest precedent): three fields (Name / Label / Tags, the last a comma-or-Enter token input, the phase-89 ignore-paths token input is the precedent) + Save / Reset, the §7.4 never-stale save state ("Saving…" → "saved" / the one-line error note, the phase-168 shape); + - the saved state renders back from the GET (the pre-population — the phase-91 convention); the derived source name is the row's identity (read-only, the `git-sources.js` name cell — the display name is a SEPARATE field, D1); + - the chat page's source cluster chips and the RAG page's source tree keep the DERIVED name (D1 — no identity change anywhere); the display name surfaces in the chat only through the prompt (D4) and the verbatim panel caption is untouched (it names `source/path` — the identity). + +## ASSUMPTIONS (owner-confirmed 2026-10-18) +- **A1 (D1):** the display name is an alias — the `Document.source` / `source/path` identity is untouched. +- **A2 (D2/D3):** the focus mechanism is the new bounded `source_name_hit_bonus` selection boost (default 0.005, `0` = off) + the KB-overview / suggested-tier surfacing (D4). +- **A3 (D5):** admin-gated like the phase-91/168 admin surfaces; labels for non-existent sources are rejected (404). + +## Context +- `app/models.py` (the `SourcesMeta` L— single-row precedent; the `GitSource` row — the `url`/`path`/`kind` columns the derived names come from), `app/rag/git_sources.py` (`_url_basename` L89, `effective_sources` L285 — the git/local source-name derivation), alembic `0029` (the last migration — 0030 is next), `app/rag/retriever.py` (`RetrievedChunk` L276+, `name_hit_tokens` L191, `_name_hit_chunks` L576, `_selection_order` L754, the three bonus application sites L838/L987/L1022), `app/api/chat.py` (the selection call sites L537–572, L875), `app/config.py` (`name_hit_bonus` L431 + validator L964 — the knob pattern), `app/core/settings_catalog.py` (L225 — the metadata entry pattern), `app/rag/overview.py` (`build_overview_prompt` L148 — the input lines), `app/rag/prompts.py` (the `` section L561, the suggested-tier doc lines), `app/api/git_sources.py` (the router pattern — admin gate, the upload status), `frontend/assets/git-sources.js` (the Sources page), `tests/fixtures/retrieval_battery.txt` + `scripts/eval_retrieval.py` (the battery), `tests/integration/` (the phase-119 pins — the kill-switch pattern to mirror). + +## Tasks +1. `01_source_labels_table.md` — the `SourceLabel` model, alembic 0030, the `app/rag/source_labels.py` helpers (D1/D2 pure half). +2. `02_source_labels_api.md` — the admin-gated router (D5): the catalogue GET, the validated PUT upsert, the 404s, the DELETE. +3. `03_source_labels_ui.md` — the per-source name/label/tags editor on the Sources page (D6), all three source kinds. +4. `04_source_name_boost.md` — the `source_name_hit_bonus` knob + the `source_name_hit` flag + the `_selection_order` second bonus (D3), the kill-switch pins, the battery run. +5. `05_prompt_surfacing.md` — the KB-overview input lines + the suggested-tier doc lines carry the display name / label (D4). +6. `06_e2e_and_gates.md` — the dedicated Playwright suite (the owner-visible arm: a named source's document is suggested when the question uses the name, the alias never breaks a locator), the full gates, the docs pass, the atomic commit. + +## Testing & Quality +- **Unit:** `tests/unit/test_source_labels.py` (NEW) — the D2 matching rule (name/label/tag hits, the case-insensitive `_normalize_name` form, the 4-char minimum on both sides, the no-hit baseline), the D1 helpers (upsert, the partial-update shape, the 404 shape at the helper level); `tests/unit/test_retriever.py` — the phase-119 mirror pins for the SECOND bonus (the kill-switch byte-identity, the bounded per-document application, the OR-merge through `fuse`). +- **Integration:** `tests/integration/test_source_labels_api.py` (NEW) — the admin gate (401/403 the phase-91 shape), the catalogue GET (git + local + upload union, no dupes), the PUT validation 422s (121-char name, 201-char label, 11 tags, a 33-char tag, a whitespace tag), the partial-update shape (an omitted field is untouched, an explicit null clears), the 404s (no such source; DELETE of an absent row), the round-trip. +- **E2E (AGENTS rule 4, task 06):** `tests/e2e/test_source_name_labels.py`, run in isolation — the owner-visible arm through the real UI (set a name + tag on a fixture source → a chat question using the name surfaces that source's document in the suggested tier via the mock/real battery — the phase-119 E2E is the precedent), the alias contract (a read/grep locator for the renamed source still resolves — the derived name is untouched), the 404 arm (a label for a non-existent source is rejected in the UI with the one-line note). +- **Battery:** `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` — the pre/post run at the default bonus (0.005) and at `0` (byte-identical to pre-phase) is recorded in the phase file (the phase-119 convention). +- **Coverage:** **>90%** on `app/`; the frontend change is marker-pinned per the house JS convention. +- **Lint/types:** `uv run ruff check . && uv run pyright`. + +## Documentation & Consistency +- Final pass reviews `README.md` and `AGENTS.md` in their entirety against the post-phase app and fixes every inconsistency found, pre-existing ones included. +- `README.md`: the sources section gains the naming feature (the name/label/tags editor, the alias-only contract, the new knob `BOR_SOURCE_NAME_HIT_BONUS` with its `0` = off meaning) (task 06). `AGENTS.md`: the phase-119 name-signal clause gains the parallel source-name clause if it names the bonus (verify, do not add if absent). + +## Completion Criteria +- [ ] alembic 0030 applied cleanly; `SourceLabel` round-trips (unit); the alias contract holds (a `Document.source` / locator is untouched by a label — the E2E arm). +- [ ] `GET /api/source-labels` lists git + local + upload sources with their labels (integration + E2E); the PUT validation 422s and 404s pinned; the DELETE resets. +- [ ] the Sources page edits name/label/tags for all three source kinds (E2E), save state per the §7.4 contract. +- [ ] `source_name_hit_bonus = 0` → the selection walk byte-identical to pre-phase (unit pin); at 0.005 the battery shows the named source surfacing (battery record); the three application sites pass the knob (unit pins). +- [ ] the KB-overview input lines and the suggested-tier doc lines carry `display_name (label)` when set, byte-identical otherwise (unit pins on both builders). +- [ ] test suite green, coverage >90%; `uv run ruff check . && uv run pyright` clean. +- [ ] no behavior change in completed phases (no label row → every surface byte-identical to pre-phase). +- [ ] one atomic Conventional Commits commit with `--no-gpg-sign`. diff --git a/.agents/phases/todo/191_source_name_labels_tags/01_source_labels_table.md b/.agents/phases/todo/191_source_name_labels_tags/01_source_labels_table.md new file mode 100644 index 0000000..8d88bbf --- /dev/null +++ b/.agents/phases/todo/191_source_name_labels_tags/01_source_labels_table.md @@ -0,0 +1,41 @@ +# Task 01 — The `source_labels` table and helpers (D1/D2 pure half) + +**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "Sources need a name, a label, tag(s) …"` +**Story:** n/a + +## Objective +The additive data layer: the `SourceLabel` ORM model + alembic 0030 (no backfill) + `app/rag/source_labels.py` with the upsert/get helpers and the D2 `question_name_hits` matching rule — so a source without a row is byte-identical to pre-phase everywhere. + +## Work +1. `app/models.py` — NEW `SourceLabel(Base)` (the `SourcesMeta` style, tabulated docstring — the D1 column spec from the phase file): + - `source: Mapped[str] = mapped_column(String(120), primary_key=True)` (the `Document.source` name — the join key); + - `display_name: Mapped[str | None] = mapped_column(Text, default=None)` (≤ 120 trimmed — enforced at the API, the column is plain Text, the phase-121 pattern); + - `label: Mapped[str | None] = mapped_column(Text, default=None)` (≤ 200); + - `tags: Mapped[list] = mapped_column(JSONB, default=list, server_default=text("'[]'"), nullable=False)`; + - `updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())`. +2. `alembic/versions/0030_source_labels.py` — NEW, revision `0030`, down_revision `0029` (verify the 0029 rev id — it is the current head): `CREATE TABLE source_labels` (the columns above, the PK, the JSONB server default), no backfill (an empty table — every source's pre-phase behavior is "no row"); the downgrade `DROP TABLE` (the house migration style — `0028` is the shape to copy). +3. `app/rag/source_labels.py` (NEW module — the docstring names the feature and the D1 alias contract): + - `@dataclass(frozen=True) SourceLabelView` — `source, display_name, label, tags` (tags: `tuple[str, …]`); `label_for(db, source) -> SourceLabelView | None` (one PK read; `None` when absent); + - `get_label(db, source) -> SourceLabel | None` (one PK read); + - `set_label(db, source, *, display_name, label, tags) -> SourceLabel` — upsert (the `SourcesMeta` upsert shape): an existing row is updated in place (only the provided non-`None` fields are overwritten — the partial-update shape; an explicit `None` clears), a new row is created; **flush-only — the caller commits** (the phase-53 session convention); + - `all_labels(db) -> dict[str, SourceLabelView]` (one query, keyed by `source`); + - `delete_label(db, source) -> bool` (one PK delete; `False` when absent; flush-only); + - `normalize_tags(raw: list[str]) -> list[str]` (pure: trim, drop empties, ≤ 32-char guard raises `ValueError`, ≤ 10 guard raises `ValueError`, case-insensitive de-dupe preserving first case — the API's 422 comes from these raises); + - `question_name_hits(view: SourceLabelView | None, question: str) -> bool` (the D2 rule, pure): `None` → `False`; else any `name_hit_tokens(question)` candidate (imported from `app.rag.retriever` — the phase-119 list, UNCHANGED) equals, in `_normalize_name` form, a token of `display_name` / `label` / any `tags` entry — the SOURCE side tokenized the same way (whitespace-split, `_normalize_name`, `NAME_TOKEN_MIN_LEN` (4) minimum on BOTH sides); + - `question_name_hit_sources(db, question: str) -> set[str]` (one `all_labels` read + the pure rule per row — a handful of rows; the per-turn entry point task 04 uses). +4. `tests/unit/test_source_labels.py` (NEW): + - the model round-trip (create via `set_label`, the `updated_at` stamp, the partial-update shape — an omitted field untouched, an explicit `None` clears); + - the D2 rule: a digitless name hit (`"gitea"` question token vs `display_name "Gitea"` — case-insensitive via `_normalize_name`), a label hit, a tag hit, a 3-char source-side token NEVER matches (the 4-char minimum), a 3-char QUESTION token never matches (the phase-119 `NAME_TOKEN_MIN_LEN` side), `None` view → `False`, a no-overlap question → `False`; + - `normalize_tags` (the 11-tag ValueError, the 33-char ValueError, the whitespace drop, the case-insensitive de-dupe keeping first case, `[]` legal); + - the migration 0030 up/down (the `tests/unit` or `tests/integration` migration-pin convention — verify where the phase-168 0028 pin lives and mirror it). +5. Verify: `uv run alembic upgrade head` clean on the dev DB; `uv run pytest tests/unit/test_source_labels.py -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean. + +## Testing & Quality +- Unit: the pins above (the D2 rule is the heart of the phase — pin every branch). +- Coverage: **>90%** on `app/` (the new module is fully exercised by its unit module). + +## Completion Criteria +- [ ] alembic 0030 applies and reverses cleanly; the `SourceLabel` model matches the D1 column spec. +- [ ] `tests/unit/test_source_labels.py` green — the D2 rule, the partial-update shape, the `normalize_tags` guards. +- [ ] full test suite **100% passing** (the empty table changes nothing). +- [ ] no behavior change in completed work (no row → no surface reads it yet). diff --git a/.agents/phases/todo/191_source_name_labels_tags/02_source_labels_api.md b/.agents/phases/todo/191_source_name_labels_tags/02_source_labels_api.md new file mode 100644 index 0000000..a7c6bb2 --- /dev/null +++ b/.agents/phases/todo/191_source_name_labels_tags/02_source_labels_api.md @@ -0,0 +1,35 @@ +# Task 02 — The admin `source-labels` API (D5) + +**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "Sources need a name, a label, tag(s) …"` +**Story:** n/a + +## Objective +The wire surface: `app/api/source_labels.py` — the catalogue `GET` (every source, all three kinds, with its label or nulls), the validated `PUT` upsert (the framework's standard 422; a 404 for a source that exists nowhere), the `DELETE` reset — admin-gated like the phase-91/168 admin surfaces, registered in `app/main.py`. + +## Work +1. `app/schemas.py` — NEW shapes (the phase-91 `extra="forbid"` style): + - `SourceLabelOut` — `source: str` (≤ 120), `display_name: str | None` (≤ 120), `label: str | None` (≤ 200), `tags: list[str]` (≤ 10 items, each ≤ 32); + - `SourceLabelList` — `labels: list[SourceLabelOut]`; + - `SourceLabelUpdate` — `display_name: str | None = Field(default=None, max_length=120)`, `label: str | None = Field(default=None, max_length=200)`, `tags: list[str] | None = Field(default=None, max_length=10)` (each item ≤ 32 — an item-level `Field(max_length=32)` or a model validator; verify the house item-cap convention in `SavedChatCreate`'s messages and mirror it) — `extra="forbid"`; the OMITTED-vs-NULL distinction is the partial-update shape (D5): omitted → untouched, `null` → cleared, a value → set (the validator trims; a whitespace-only `display_name`/`label` → `None` at the validator boundary). +2. `app/api/source_labels.py` (NEW router, `APIRouter(prefix="/api/source-labels", tags=["source-labels"], dependencies=[Depends(require_admin)])` — the phase-91 gate line): + - `GET ""` → `SourceLabelList`: (a) the `git_sources` rows — `effective_sources(db)` (the phase-35/38 helper, L285): each row's derived source name (git: `_url_basename(row.url)`; local: the stored path's basename — the SAME expression the importer uses for `Document.source` — verify the importer's local-name expression and reuse it, do not re-derive it); an env-only deployment (no DB rows, the `env` source state) lists the env source names (the `effective_sources` `source` return is `"db"`/`"env"` — for `env`, parse the env list the phase-35 fallback uses); (b) `select(Document.source).distinct()` (ONE query, no `content` — the phase-126 load discipline) minus the names from (a) (the uploads); de-duplicated, sorted by `source` (stable order — the UI and the E2E rely on it); each entry = its label row via `all_labels(db)` (one query) or nulls; + - `PUT "/{source}"` (path param, the router's `max_length` via the path conversion — a >120 path → the framework's 422) → the D5 404 arm FIRST: `source` matches no `git_sources` row name AND is not in the distinct document sources (the SAME two queries the GET runs — no third) → 404 (the standard body, the phase-91 shape); else `normalize_tags` (422 on its raises — a `ValueError` → the framework's 422 via the validator, NOT a 500), `set_label` (flush + `db.commit()` — the router owns the session here, the phase-91 commit convention), 200 `SourceLabelOut`; + - `DELETE "/{source}"` → the 404 arm (no label row → 404), else `delete_label` + commit, 200 (the phase-91 DELETE shape — verify its body: an empty object or a message; mirror the sibling admin router). +3. `app/main.py` — `app.include_router(source_labels.router)` (the admin routers' block — verify the exact registration list and the order convention). +4. `tests/integration/test_source_labels_api.py` (NEW — the phase-91 `test_ui_settings_api.py` fixture pattern): + - the gate: an anonymous GET/PUT/DELETE → the phase-91 gate shape (403 / the redirect-to-login — verify the phase-91 gate pin and mirror it EXACTLY); + - the catalogue GET: a module app with one git row (a `file://` URL — the phase-35 fixture), one local row, one upload (a document row with a source name no row derives) + a label on the upload → the response lists all three (sorted, no dupes), the upload carries its label, the others nulls; + - the PUT: a valid upsert round-trips (set name+label+tags → GET shows them; a second PUT with only `label` → the name/tags untouched; a PUT with explicit `null` `display_name` → cleared, the label kept); the 422s (a 121-char `display_name`, a 201-char `label`, 11 `tags`, a 33-char tag, a whitespace-only tag — each with the framework's standard body and NO row change: a pre-seeded row is byte-identical after the 422); the 404s (a `source` that is neither a row name nor a document source — the row table unchanged); + - the DELETE: a labelled source → 200, the GET shows nulls; a second DELETE → 404. +5. Verify: `uv run pytest tests/integration/test_source_labels_api.py -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean. + +## Testing & Quality +- Integration: the pins above — the gate, the union catalogue, the validation 422s (no partial write), the 404s, the round-trip. +- Coverage: **>90%** on `app/` (the router is fully exercised by its integration module). + +## Completion Criteria +- [ ] `GET /api/source-labels` lists git + local + upload sources (integration pin) with their labels; admin-gated. +- [ ] `PUT /api/source-labels/{source}` upserts with the D5 partial-update shape; every 422 leaves the row unchanged; a 404 for a non-existent source creates nothing. +- [ ] `DELETE` resets; a second DELETE 404s. +- [ ] full test suite **100% passing**. +- [ ] no behavior change in completed work (the `git_sources` API, the `documents` API, the phase-91/168 admin surfaces all green unedited). diff --git a/.agents/phases/todo/191_source_name_labels_tags/03_source_labels_ui.md b/.agents/phases/todo/191_source_name_labels_tags/03_source_labels_ui.md new file mode 100644 index 0000000..d9d9aea --- /dev/null +++ b/.agents/phases/todo/191_source_name_labels_tags/03_source_labels_ui.md @@ -0,0 +1,35 @@ +# Task 03 — The per-source name/label/tags editor on the Sources page (D6) + +**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "Sources need a name, a label, tag(s) …"` +**Story:** n/a + +## Objective +The owner-visible surface: every source on the Sources page (git, local, upload — the catalogue `GET /api/source-labels` is the single source of truth for the row list, so uploads appear without a second API) gets an inline name/label/tags editor with the §7.4 never-stale save state — the phase-121 token field and the phase-89 ignore-paths token input are the house precedents. + +## Work +1. `frontend/index.html` — `#view-git-sources` (L646+): the sources table area (the `#git-sources-table-wrap` region, L792) gains the per-row expandable label block markup — ONE shared template (a `