chore(agent): phase roadmap from TODO.md, 4 phases (190-193)

Convert the four open TODO items into executable phases (append mode,
starting at 190 after the completed 189):

  190_app_name_settings_fix  - the Settings-panel app name wins over the
      Theme tab; the three brand strings stripped from the Theme tab
      (colors-only); live brand re-apply on save, no reload
  191_source_name_labels_tags - source_labels table + admin API + Sources
      page editor; the separate bounded source_name_hit_bonus retrieval
      boost; label surfacing in the <document> blocks and KB overview
  192_verbatim_inline_midturn - an additive verbatim_block SSE frame at
      the show_text success moment; in-place mid-turn fence hydration;
      the done/restore per-locator dedupe; the done frame, record key,
      restore and shared page stay byte-identical
  193_sync_remaining_count - the running sync/upload label counts down:
      "N remaining" (N = total - done), hidden at the final file; the
      chunk/image/summaries/terminal parts untouched

TODO.md is cleared (its items now live in .agents/phases/todo/).

LLM-Generated: true
This commit is contained in:
2026-10-17 19:35:55 -04:00
parent 140a343b92
commit 47456a44df
22 changed files with 899 additions and 2 deletions
@@ -0,0 +1,51 @@
# Phase 190 — The Settings-panel app name is the app name again (and saves apply live)
**Source:** `TODO.md` L3 — "changing the App name in settings broke, it no longer updates the name."
**Story:** n/a (owner report)
**Depends on:** none functionally (builds on the completed phase-168 admin Settings panel and phase-91/92 admin Theme tab).
## Evidence (diagnosed from the code, read this not the chat)
1. **Two writers for one setting.** `app_name` (and `input_placeholder` / `footer_text`) is now writable from TWO admin surfaces: the Settings panel (`/api/admin/settings/app_name` → `app_settings` table → the `app.config` DB-over-env overlay, phase 168) and the Theme tab (`/api/ui-settings` PUT → the single `ui_settings` row, phase 91).
2. **The Theme tab row WINS.** `app.core.theming.effective_settings` (L144–177) resolves each string as: the `ui_settings` row value when it is a non-empty string, **else** `settings.<field>` (the env value, which — via the phase-168 overlay — is the Settings-panel row value when one exists). So the order is Theme-tab row → Settings-panel row → env.
3. **A Theme-tab save bakes the strings in.** `theme.js` `collectBody()` (L526) collects ALL 20 fields — the 3 string inputs are pre-populated with the EFFECTIVE value on load — and `PUT /api/ui-settings` writes every non-empty string into the row. So the moment an admin saves ANY color from the Theme tab, the current app name is baked into `ui_settings.app_name`, and from then on **every later Settings-panel app-name change is shadowed** by that baked row. This is exactly "it no longer updates the name" — the Settings panel shows the new value with its `db` badge, but `/api/config` (which serves `effective_settings`) reports the Theme tab's baked name.
4. **No live brand update, on either surface.** `settings.js` re-GETs the catalogue after a save but never re-applies the brand strings to the open document; `theme.js` after a save runs `applyServedTheme(settings)` which reconciles the COLOR palette (phase 92) but never touches `document.title`, `.brand-text`, the footer or the composer placeholder. `brand.js` runs the brand path once at boot from `/api/config`. So even when the server-side value is right, the open page keeps showing the old name until a manual reload.
## Objective
The Settings panel is the single writable owner of the three brand strings and always wins; the Theme tab becomes colors-only (its PUT no longer writes the strings, its form drops them — for legacy baked-in rows the panel still wins); and a save in either surface re-applies the brand strings to the open document immediately, so the change is visible the moment "save" is clicked (the phase-168 contract extended to the display).
## ASSUMPTIONS (owner-confirmed 2026-10-18)
- **A1 — Panel wins, strings stripped from the Theme tab.** The Settings-panel row beats the Theme-tab row for `app_name` / `input_placeholder` / `footer_text` (no data migration — existing baked-in rows simply stop shadowing the panel; with no panel row they still apply as the documented fallback). The Theme tab's form drops the three string inputs and its PUT body no longer carries them (the server ignores string fields in the PUT body defensively — an old client's save can no longer re-bake a string). The Theme tab's GET keeps reporting the effective strings (read-only, old-client compat).
- **A2 — Live brand on save.** After a successful Settings-panel save/clear or Theme-tab save/reset, the open document's `document.title`, `.brand-text`, the footer and the composer placeholder are re-applied from a fresh `/api/config` — no reload. The boot path (`brand.js`) is byte-identical.
## Context
- `app/core/theming.py` — `effective_settings` (L144–177: the string resolution to change), `STRING_FIELDS`, `COLOR_FIELDS`, `BUILTIN_COLORS`; `app/api/config.py` — `GET /api/config` (serves `effective_settings` + `source` per string); `app/api/ui_settings.py` — the phase-91 admin API (GET reports 20 fields, PUT writes 3 strings + 17 colors, DELETE resets the row); `app/core/settings_overlay.py` + `app/config.py` `invalidate_settings` (L1129) — the phase-168 DB-over-env overlay (the panel rows; `settings.app_name` already resolves through it); `app/api/admin_settings.py` (phase 168, the panel surface).
- `frontend/assets/brand.js` — the boot-time brand path (title / `.brand-text` / footer / placeholder from `/api/config`); `frontend/assets/theme.js` — `FIELDS` (L183–185 the 3 string fields), `collectBody` (L526), `saveTheme` (L580, `applyServedTheme` L589), `resetTheme` (L597); `frontend/assets/settings.js` — the phase-168 panel (its header names the phase-168 contract "the change should take effect on clicking 'save'"); `frontend/index.html` — the Theme-tab markup (`#theme-app-name` L1354 et al., the three string rows to remove).
- `tests/unit/test_theming.py` (the `effective_settings` pins), `tests/integration/test_ui_settings_api.py` (the phase-91 API pins), `tests/e2e/test_admin_theme_tab.py` (fills `#theme-app-name` — must move with the strip), `tests/e2e/test_settings_panel.py` (the phase-168 pins), `tests/unit/test_save_chat_ui.py` (JS marker-pin convention).
## Tasks
1. `01_reproduce_shadow_and_staleness.md` — failing tests pinning both defects (the shadowing through `effective_settings` / `/api/config`, the Theme-tab PUT string bake, the no-live-update behavior).
2. `02_precedence_and_strip_theme_tab.md` — A1: the resolution order flips to panel → tab → env; the Theme tab drops the 3 string inputs and its PUT stops writing them; the pinned theme-tab suites move.
3. `03_live_brand_on_save.md` — A2: one shared `applyBrandFromConfig` pass (the `brand.js` DOM path extracted, boot behavior byte-identical) re-applied after a Settings-panel save/clear and a Theme-tab save/reset.
4. `04_e2e_and_gates.md` — the dedicated Playwright suite (the incident regression through the real UI), the full gates, the docs pass, the atomic commit.
## Testing & Quality
- **Unit:** `tests/unit/test_theming.py` — the NEW resolution order pins (panel row beats a conflicting `ui_settings` row; with no panel row the tab row still applies — the A1 fallback); the pre-phase pins that do not involve a panel row stay green.
- **Integration:** `tests/integration/test_ui_settings_api.py` — a Theme-tab PUT with string fields in the body **ignores** them (the stored row's string columns are untouched / the response no longer round-trips a changed string); `tests/integration/` Settings-panel pins (phase 168) green — the panel write + `invalidate_settings` path is untouched.
- **E2E (AGENTS rule 4, task 04):** `tests/e2e/test_app_name_settings_fix.py`, run in isolation — (a) the incident regression: set the name in the Settings panel → save a color in the Theme tab → the name still shows the panel's value (pre-fix: the baked tab value); (b) live apply: change the app name in the Settings panel → `document.title`, `.brand-text`, the footer and the composer placeholder update WITHOUT a reload; (c) a Theme-tab save no longer bakes the name (a later panel change is visible on reload); (d) a panel reset (DELETE) falls back to env live.
- **Regression:** `tests/e2e/test_settings_panel.py`, `tests/e2e/test_admin_theme_tab.py` (moved in task 02), `tests/e2e/test_theme_save_and_coverage.py` green.
- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`); the frontend change is marker-pinned per the house JS convention.
- **Lint/types:** `uv run ruff check . && uv run pyright`.
## Documentation & Consistency
- Final pass reviews `README.md` and `AGENTS.md` in their entirety against the post-phase app and fixes every inconsistency found, pre-existing ones included.
- `README.md`: the settings/theme section notes the single-writer ownership (the Settings panel owns the brand strings; the Theme tab is colors-only) and the live-apply behavior (task 04). `AGENTS.md`: the phase-168 "settings overlay" description gains the resolution-order clause if it names one (verify, do not add if absent).
## Completion Criteria
- [ ] `effective_settings` resolves the 3 strings panel-row → tab-row → env (unit pins, both branches).
- [ ] `PUT /api/ui-settings` with string fields in the body does not write them (integration pin); the Theme-tab form no longer renders the 3 inputs (E2E pin).
- [ ] A Settings-panel app-name change is visible in `document.title`, `.brand-text`, the footer and the placeholder without a reload (E2E); the Theme-tab color save no longer shadows it (E2E regression of the incident).
- [ ] the `brand.js` boot path is byte-identical (marker pin) — no double-apply at boot.
- [ ] test suite green, coverage >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] no behavior change in completed phases (the 17-color palette, the phase-168 panel, the env fallback all behave as before).
- [ ] one atomic Conventional Commits commit with `--no-gpg-sign`.
@@ -0,0 +1,31 @@
# Task 01 — Reproduce and pin the two defects (shadowing + no live update)
**Phase:** `190_app_name_settings_fix` · **Source:** `TODO.md:4 — "changing the App name in settings broke, it no longer updates the name."`
**Story:** n/a
## Objective
Before touching behavior: pin BOTH defects with failing tests — (a) the `ui_settings` row shadows the Settings-panel row in `effective_settings` and therefore in `GET /api/config`, and a Theme-tab PUT bakes the effective strings into the row (the mechanism of "it no longer updates"); (b) a Settings-panel save never re-applies the brand strings to the open document (marker pin on the ABSENT behavior, inverted in task 03).
## Work
1. `tests/unit/test_theming.py` — NEW pins (failing until task 02):
- `test_effective_app_name_prefers_settings_panel_row_over_ui_settings_row` — an `app_settings` row (via the phase-168 overlay / a `Settings` with `app_name="Panel Name"`) + a `ui_settings` row with `app_name="Tab Name"` → `effective_settings["app_name"] == "Panel Name"` (today: `"Tab Name"` — the documented A1 order). Same pair for `input_placeholder` and `footer_text`.
- `test_effective_strings_fall_back_to_ui_settings_row_without_panel_row` — `ui_settings` row set, no panel row (the `Settings` at the env default) → the tab value applies (the A1 fallback this pin protects — it must stay GREEN pre-phase and post-phase).
- The pre-existing pins that assume tab-over-env with NO panel row stay green unedited.
2. `tests/integration/test_ui_settings_api.py` — NEW pins (failing until task 02):
- `test_put_ignores_string_fields_in_body` — a `PUT /api/ui-settings` whose body carries `app_name` / `input_placeholder` / `footer_text` DIFFERENT from the stored row's values and from the effective value → 200; the stored row's string columns are UNCHANGED (re-GET shows the old values); the 17 colors in the same body ARE stored (the colors half of the contract survives).
- `test_get_reports_effective_strings_read_only` — with a conflicting panel row, `GET /api/ui-settings` reports the EFFECTIVE (panel) value for the strings (read-only view; the A1 order reaches the API).
3. `tests/unit/test_brand_live.py` (NEW module) — marker pins on the CURRENT (absent) behavior, documented as "inverted in task 03":
- `settings.js` contains NO call that re-applies `document.title` / `.brand-text` after a save (assert the marker is ABSENT pre-phase — the test asserts the file's save path does not reference the brand DOM ids; task 03 inverts the expectation);
- `theme.js` `saveTheme`'s success path calls `applyServedTheme` (colors only) and does NOT touch the brand DOM (marker pin — stays green through the phase; the live-apply added in task 03 is asserted there, not here).
4. Verify: `uv run pytest tests/unit/test_theming.py tests/integration/test_ui_settings_api.py tests/unit/test_brand_live.py -v` — the NEW pins FAIL (documented), everything else green.
## Testing & Quality
- Unit/integration: the pins above — the shadowing mechanism, the string-bake, the fallback (green pre-phase), the absent-live-behavior markers.
- Coverage: **>90%** on `app/` (no app code changes this task — the pins exercise existing code).
## Completion Criteria
- [ ] the 3 `effective_settings` order pins FAIL pre-fix (documented in the test docstrings: "fixed by task 02").
- [ ] the string-bake integration pin FAILS pre-fix (documented); the colors-half pin is GREEN pre-fix.
- [ ] the live-behavior marker pins pass (asserting the ABSENT behavior, inverted in task 03).
- [ ] full test suite **100% passing** except the documented failing pins (the failing pins are the task's artifact — the phase's E2E gate runs them post-fix).
- [ ] no behavior change in completed work (app/ untouched).
@@ -0,0 +1,45 @@
# Task 02 — The panel wins; the Theme tab becomes colors-only (A1)
**Phase:** `190_app_name_settings_fix` · **Source:** `TODO.md:4 — "changing the App name in settings broke, it no longer updates the name."`
**Story:** n/a
## Objective
Flip the string resolution order to Settings-panel row → Theme-tab row → env (no data migration — existing baked-in rows stop shadowing the panel), and strip the three string inputs from the Theme tab: the form drops them, `collectBody` stops collecting them, and the server's `PUT /api/ui-settings` persists ONLY the 17 colors (string fields in the body are ignored defensively — an old client's save can no longer re-bake a string; string columns are left untouched, not cleared, so the A1 fallback survives).
## Work
1. `app/core/theming.py` — `effective_settings` (L170–173), the string loop becomes a three-way resolution per field:
- **panel row** — `getattr(settings, field)` (the `Settings` passed by the route; the phase-168 overlay makes it the panel value when an `app_settings` row exists, else env) is used when a panel row exists for that key. The panel presence is read through the SAME overlay source the `Settings` object carries: the overlay's `app_settings` row map (`app.core.settings_overlay.load_overrides()` — the one query the overlay already caches per process; the route passes the settled `Settings`, so the check is `field in the override map` — implement as `has_override(field)` on the overlay module, one cached read, no second query per field);
- **else the tab row** — the `ui_settings` row value when it is a non-empty string (the phase-91 fallback, now second);
- **else the env value** — `getattr(settings, field)` (identical expression to today's fallback).
- The 17-color half is UNTOUCHED (row → `BUILTIN_COLORS`). The docstring's resolution table is rewritten (the old "DB value when non-empty, else env" becomes the three-way table).
- `app/api/config.py` — the per-string `source` badge (`db` / `env`): `db` now means the PANEL row; the tab row yields the existing non-`db` value (verify the current `source` vocabulary — if it is a two-value `db`/`env` pair, `db` = any stored row wins, unchanged; do NOT grow the vocabulary).
2. `app/api/ui_settings.py` — the PUT handler:
- persist ONLY the 17 `COLOR_FIELDS` from the body; the three string columns are NEVER written (the request model keeps accepting the fields — old-client compat — the handler drops them before persisting); the stored row's string values are left exactly as stored (a legacy baked-in row keeps its value; the A1 order decides visibility);
- the DELETE (reset) path: UNCHANGED (it nulls the whole row — including the strings; a reset is a user decision, and the A1 fallback then falls through to env, which is the documented reset behavior).
- the GET: unchanged (reports the effective strings — with the new order, a conflicting panel value is what the tab page pre-populates its (now removed) inputs with in an old client; the read-only report stays).
3. `frontend/assets/theme.js`:
- `FIELDS` (L183–185): the three `kind: "string"` entries are REMOVED — `FIELDS` becomes the 17 color fields only; everything FIELDS-driven (live preview, `collectBody`, `applyServedTheme`'s tag content) stays in `COLOR_FIELDS` order (the comment at L180–181 already says so — it becomes true);
- `collectBody` (L526): the body is the 17 colors only (no string keys at all);
- the string-specific docstring lines (L183–185 region + any "20 fields" counts in the module header) become "17 colors."
4. `frontend/index.html` — the Theme-tab markup: the three string rows (`#theme-app-name` L1354 et al., the placeholder + footer rows) are REMOVED from the form; the section count / any "3 strings + 17 colors" comment becomes colors-only.
5. `tests/unit/test_theming.py` — the task-01 failing pins now PASS unedited (they name the A1 order); the no-panel-row fallback pin stays green.
6. `tests/integration/test_ui_settings_api.py` — the task-01 string-bake pin now PASSES (body strings ignored, colors stored, string columns untouched).
7. The pinned theme-tab E2E suites that fill the removed inputs move:
- `tests/e2e/test_admin_theme_tab.py` — every step that fills `#theme-app-name` / `#theme-placeholder` / `#theme-footer` (or asserts their presence) is RETARGETED: the tab renders 17 color inputs only (a new pin: the three string inputs are ABSENT from the tab DOM — `page.locator("#theme-app-name")` count 0); a save still round-trips the 17 colors through the GET pre-population.
- `tests/e2e/test_theme_save_and_coverage.py` / any other suite that seeds a string through the tab: the string is now seeded through the Settings panel (the `test_settings_panel.py` helper) or the API (the integration test covers the wire; the E2E keeps its color assertions).
8. `tests/unit/test_theme_ui.py` (or the module that marker-pins `theme.js` — verify the name in `tests/unit/`) — marker pins: `FIELDS` has no `kind: "string"` entry; `collectBody`'s body has no string keys (the source-marker convention); the `saveTheme` success path still calls `applyServedTheme`.
9. Verify: `uv run pytest tests/unit/test_theming.py tests/integration/test_ui_settings_api.py -v` green; `uv run pytest tests/e2e/test_admin_theme_tab.py tests/e2e/test_settings_panel.py -v --no-cov` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the A1 order pins (both branches) + the marker pins (tab has no string fields).
- Integration: string-ignored PUT pin + the phase-91 color pins green unedited.
- E2E (regression, run in isolation): `test_admin_theme_tab.py` (moved) + `test_settings_panel.py` green.
- Coverage: **>90%** on `app/`.
## Completion Criteria
- [ ] `effective_settings` resolves the 3 strings panel → tab → env (both-branch unit pins green).
- [ ] `PUT /api/ui-settings` with string fields in the body stores no string change and stores the colors (integration pin green).
- [ ] the Theme-tab DOM renders 17 color inputs only; the 3 string inputs are absent (E2E pin).
- [ ] the phase-91 color contract (GET pre-population, save, reset) is green unedited.
- [ ] full test suite **100% passing**.
- [ ] no behavior change in completed work (the 17 colors, the phase-168 panel, the env fallback, the DELETE-reset behavior all as before).
@@ -0,0 +1,37 @@
# Task 03 — The save is visible without a reload (A2)
**Phase:** `190_app_name_settings_fix` · **Source:** `TODO.md:4 — "changing the App name in settings broke, it no longer updates the name."` (the phase-168 header contract: "the change should take effect on clicking 'save'")
**Story:** n/a
## Objective
A successful save in the Settings panel (or a per-row reset) and a save/reset in the Theme tab re-applies the brand strings to the OPEN document immediately — `document.title`, `.brand-text`, the footer and the composer placeholder — from a fresh `GET /api/config`, with no reload. The boot path in `brand.js` is extracted into the shared pass byte-identically, so boot and live-apply are ONE function and the never-stale contract (PLAN §7.4) covers the strings the way it already covers the palette (phase 92 `applyServedTheme`).
## Work
1. `frontend/assets/brand.js` — extract the DOM path (title / `.brand-text` / footer / placeholder assignment from the `/api/config` payload) into ONE named function `applyBrandFromConfig(cfg)` (same file, same module scope — no new file, no new CDN; `brand.js` stays the single owner of the brand DOM):
- the BOOT call site (`initBrand` / the boot fetch-then-apply) calls `applyBrandFromConfig(cfg)` — the observable boot behavior is byte-identical (same request, same assignments, same order);
- the function is exported to the module's existing export surface (the `window` / module pattern `brand.js` already uses — verify the convention; `theme.js` and `settings.js` import from the same pattern).
2. `frontend/assets/settings.js` — the save path:
- after a SUCCESSFUL per-key PUT (`PUT /api/admin/settings/<key>`, the phase-168 success note) and after a SUCCESSFUL per-row reset (DELETE) and the reset-all: `await fetch("/api/config")` → `applyBrandFromConfig(cfg)` — ONLY when the saved/removed key is one of the three brand strings (`app_name` / `input_placeholder` / `footer_text`); a save of any other key does NOT re-apply the brand (no wasted request, no DOM churn on 72 other rows);
- the §7.4 guard: while the re-apply fetch is in flight, a second save does not stack a second fetch (the existing `saveInFlight` guard's scope — extend, do not fork);
- the existing post-save re-GET of the catalogue is UNTOUCHED (it stays the catalogue's own refresh; the brand re-apply is additive and independent).
3. `frontend/assets/theme.js` — the save/reset path:
- `saveTheme` success (after the existing `applyServedTheme(settings)` at L589) and `resetTheme` success: `await fetch("/api/config")` → `applyBrandFromConfig(cfg)`. (The tab no longer edits the strings, but a RESET nulls the stored row — with a legacy baked-in string that reset makes the effective string FALL BACK (panel row → env), so the open document must re-read it; the save path keeps the same call — one code path for both, the body change is the colors.)
- the fetch-failure path: the brand keeps its last applied state (never a blanked title/footer — the phase-168 "never a blanked panel" precedent, applied to the brand: a failed re-apply leaves the DOM as-is; the save's own error note is the visibility).
4. `tests/unit/test_brand_live.py` — the task-01 absent-behavior marker pins are INVERTED:
- `settings.js`'s save path references `applyBrandFromConfig` (the marker is now PRESENT) and gates the re-apply on the three string keys (the source marker names the key check);
- `theme.js`'s `saveTheme`/`resetTheme` success paths reference `applyBrandFromConfig` (marker present);
- `brand.js` exports `applyBrandFromConfig` and the boot call site calls it (marker — the extraction, not a duplication: `initBrand`/boot contains no second copy of the four DOM assignments — they exist only in the shared function).
5. Verify: `uv run pytest tests/unit/test_brand_live.py -v` green; `uv run pytest tests/e2e/test_settings_panel.py -v --no-cov` green (the phase-168 suite — its save flow now also triggers the brand re-apply; its pins must stay green — the re-apply is additive); `uv run pytest` green.
## Testing & Quality
- Unit: the inverted marker pins (shared function exists; both surfaces call it; the boot path is the extraction — single copy).
- Integration: none (pure frontend — the wire contract is task 02's; `/api/config` is unchanged).
- E2E: covered by task 04 (the owner-visible arm).
- Coverage: **>90%** on `app/` (unchanged — the task is frontend; the app-side gate stays green).
## Completion Criteria
- [ ] `applyBrandFromConfig` is ONE function in `brand.js`; the boot call site uses it; the four DOM assignments exist in exactly one place (marker pins).
- [ ] `settings.js` re-applies the brand after a save/reset of a brand-string key only; `theme.js` re-applies after save and reset (marker pins).
- [ ] a failed re-apply leaves the DOM at its last applied state (marker pin on the failure path).
- [ ] `test_settings_panel.py` green unedited.
- [ ] no behavior change in completed work (the boot path, the phase-168 catalogue refresh, the phase-92 palette sync all as before).
@@ -0,0 +1,35 @@
# Task 04 — The phase E2E suite, the full gates, the docs pass, the commit
**Phase:** `190_app_name_settings_fix` · **Source:** `TODO.md:4`
**Story:** n/a
## Objective
The dedicated Playwright suite proves the incident regression and the live-apply contract through the real UI (run in isolation per AGENTS rule 4); the full gates pass; `README.md` / `AGENTS.md` are consistent with the post-phase app; one atomic commit.
## Work
1. `tests/e2e/test_app_name_settings_fix.py` (NEW — the module docstring names the incident: "a Theme-tab save used to bake the effective name into `ui_settings` and shadow every later Settings-panel change") — the app under test is the standard module-app pattern (the phase-168/91 suites' fixture — verify the convention in `tests/e2e/conftest.py` / the sibling suites):
- **(a) the incident regression (server-side, at the wire):** with an env default name, the admin sets the app name via the Settings panel UI (type + save, the phase-168 flow) → `GET /api/config` (via `page.request`) reports the panel value with the `db` source; then the admin opens the Theme tab, changes ONE color and saves → `GET /api/config` STILL reports the panel value (pre-fix: the baked tab value); the stored `ui_settings` row's string columns are unchanged (the API-level assertion — `page.request` GET `/api/ui-settings`).
- **(b) live apply, panel:** change the app name in the Settings panel (a brand-string row) → after the save note, WITHOUT a reload: `document.title`, `.brand-text` (the header), the footer text and the composer placeholder all carry the new name (the four assertions in one test — the A2 contract).
- **(c) live apply, tab reset:** with a legacy `ui_settings` row pre-seeded (the API seeds the row directly — a baked `app_name`), a Theme-tab Reset → the open document's brand falls back (panel row or env — seed NO panel row so the fallback is env) WITHOUT a reload.
- **(d) the boot path is byte-identical:** a fresh page load (no save) shows the env/panel name exactly as the pre-phase `brand.js` did (the `test_admin_theme_tab.py` boot pin, moved here if the moved suite no longer covers it).
- **(e) the tab DOM is colors-only:** the Theme tab renders 17 color inputs and NO `#theme-app-name` / placeholder / footer string inputs (the task-02 pin, asserted in the isolated suite as well).
2. Regression, run in isolation: `tests/e2e/test_admin_theme_tab.py` (moved in task 02), `tests/e2e/test_settings_panel.py`, `tests/e2e/test_theme_save_and_coverage.py` — all green unedited after the moves.
3. The full gates, in order:
- `uv run pytest` (unit + integration) — 100% passing;
- `uv run pytest --cov=app --cov-report=term-missing` — **>90%** on `app/`;
- `uv run pytest tests/e2e/test_app_name_settings_fix.py -v --no-cov` — green in isolation;
- `uv run ruff check . && uv run pyright` — clean.
4. Documentation pass (the phase file's Documentation section): `README.md` + `AGENTS.md` reviewed in their entirety against the post-phase app; the settings/theme section notes the single-writer ownership (Settings panel owns the 3 brand strings, panel-row → tab-row → env order) and the live-apply behavior; `AGENTS.md` corrected only where it names the old order.
5. Commit: one atomic, professional commit (Conventional Commits, `--no-gpg-sign`), e.g.
`fix(settings): the Settings-panel app name wins over the Theme tab and saves apply live (phase 190)` — the body names the two defects (the `ui_settings` shadow bake, the absent live re-apply), the A1/A2 decisions, and the moved theme-tab suites. `.agents/phases/todo/190_app_name_settings_fix/` moves to `.agents/phases/complete/` in the SAME commit (AGENTS rule 8).
## Testing & Quality
- E2E: the suite above, isolated; the three regression suites green.
- Coverage: **>90%** on `app/`.
- Lint/types: clean.
## Completion Criteria
- [ ] `tests/e2e/test_app_name_settings_fix.py` green in isolation — (a) the incident regression (the panel value survives a Theme-tab color save), (b) the four-DOM live apply without reload, (c) the tab-reset fallback live, (d) the byte-identical boot, (e) the colors-only tab DOM.
- [ ] `uv run pytest` 100% passing; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] `README.md` / `AGENTS.md` consistent (the single-writer + live-apply clauses).
- [ ] the phase directory is in `.agents/phases/complete/`; one atomic commit with `--no-gpg-sign`.
@@ -0,0 +1,72 @@
# Phase 191 — Sources carry a name, a label and tags — and the name focuses retrieval
**Source:** `TODO.md` L4 — "Sources need a name, a label, tag(s) so you can focus the model when prompting if you have a project name or a standard name you reference things by."
**Story:** n/a (owner directive)
**Depends on:** none functionally (builds on the completed phase-119 name-signal selection bonus, the phase-31/142 KB overview, the phase-76/35/38/49/90 Sources page).
## Design (shared by all tasks — the executor reads this, not the chat)
- **D1 — the data (task 01).** A new single-purpose table `source_labels` (alembic **0030**, additive, no backfill — a source without a row has no name/label/tags and behaves exactly as pre-phase):
- `source` — `String(120)` **PK**: the source name EXACTLY as `Document.source` carries it (the derived basename — the clone dir / local dir / upload file stem); the join key to the documents and to the `git_sources` row (via `_url_basename` / the local path basename);
- `display_name` — `Text`, nullable, ≤ 120 chars (the trimmed value; the owner's name for the source — "the git server");
- `label` — `Text`, nullable, ≤ 200 chars (the short one-line description/category — "self-hosted forge + Gitea");
- `tags` — `JSONB`, `NOT NULL`, server default `'[]'` (the focus tokens, each ≤ 32 chars, ≤ 10 tags — the owner's "tag(s)");
- `updated_at` — `DateTime(timezone=True)`, server default now, onupdate now.
- **The display name is an ALIAS ONLY** (owner-confirmed assumption): `Document.source`, the `source/path` document identity, the read/grep/`show_text` locators, the citations and the KB tree are ALL untouched by a label row — renaming a source never invalidates a saved chat, a shared page or a verbatim locator. The derived name stays the wire identity; the display name rides beside it.
- The ORM model `SourceLabel` (the `SourcesMeta` single-column style, `app/models.py`) + `app/rag/source_labels.py` (the new module — `app/rag/sources_meta.py` is the phase-53 version counter, NOT this feature; no name collision): `get_label(db, source)`, `set_label(db, source, display_name, label, tags)` (upsert, flush-only — the caller commits, the phase-53 session convention), `all_labels(db)`, `label_for(db, source)` → a frozen `SourceLabelView` (display_name / label / tags, `None` when absent), `question_name_hits(labels, question)` → the matching rule below.
- **D2 — the matching rule (task 04).** A source is a SOURCE-NAME HIT when at least one `name_hit_tokens(question)` candidate (the phase-119 class-agnostic token list, reused UNCHANGED — `app/rag/retriever.py` L191) equals, case-insensitively (the `_normalize_name` form), a token of the source's `display_name`, `label` or any `tags` entry. The phase-119 digit-class rule does NOT apply to user-given names (the owner's names are not paths — "gitea" must hit even though it is digitless; the phase-119 digit rule exists to keep prose tokens from matching PATHS, and user names have no prose-precision problem: a 4-char minimum token length (`NAME_TOKEN_MIN_LEN`, reused) is the only filter, applied to the SOURCE side too — a 2-char tag never matches). One query per turn: the label rows (a handful) are loaded once and the match is in Python (the phase-119 one-query convention).
- **D3 — the boost (task 04).** A NEW bounded selection-time bonus, the phase-119 `name_hit_bonus` pattern end to end, a SEPARATE knob (the two signals stay independently kill-able):
- `source_name_hit_bonus: float = 0.005` (default = the phase-119 starting point; validator `>= 0`, `0` = off; `BOR_SOURCE_NAME_HIT_BONUS` env; the `settings_catalog` metadata entry — the catalogue-completeness pin is extended, NOT waived);
- `RetrievedChunk` gains the additive field `source_name_hit: bool = False` (the `name_hit` docstring pattern: default `False` keeps every ordinary construction valid; `fuse`'s `replace()` copies it; the double-hit merge ORs it in — a vector row that is also the source-name hit keeps the flag);
- the flag is set at the retrieval boundary in `app/api/chat.py` (the call sites where `select_suggested` / `select_related` / `weak_hit_titles` run — L537–572 and L875 — the question is in scope): for every chunk whose `document.source` is a D2 hit, `rc.source_name_hit = True` (a loop over the few dozen retrieved chunks against the handful of hit sources — never a new query);
- `_selection_order` (L754) gains the second bonus parameter: `effective_score = best_fused + (bonus if any name_hit) + (source_bonus if any source_name_hit)` — each bonus per DOCUMENT, applied ONCE, at most two bonuses per document (bounded: 0.010 on the fused scale, the same scale phase 119 proved); `source_bonus == 0` (or no flagged chunk) → the walk is BYTE-IDENTICAL to the pre-phase walk (the phase-119 kill-switch contract, LOCKED A3 — a new unit pin mirrors the phase-119 one);
- the three application sites (`select_suggested` L537/L875, `select_related` L538, `weak_hit_titles` L572) pass `get_settings().source_name_hit_bonus`; the bonus lives in the SELECTION layer only — `score` / `cosine` / `fts_hit` / `query_log.top_score` are untouched (the phase-119 LOCKED A3 clause, mirrored).
- **D4 — the prompt surfacing (task 05).** The model learns the name→docs mapping from the prompt, not from retrieval alone:
- the KB overview's INPUT lines (`app/rag/overview.py` `build_overview_prompt`, L148 — `source — path — title — summary`) render `display_name — path — …` when set (the raw source name is REPLACED in the lite model's input only — the stored overview text is the lite model's prose, which then naturally uses the owner's name; the overview row's `source` field in the DB is untouched);
- the suggested-tier `<documents>` block (the per-doc line in `app/rag/prompts.py`, the phase-31 section builder) renders `display_name (label)` before the path when set — e.g. `the git server (self-hosted forge) — docs/gitea.md — …` (the parenthetical only when BOTH are set; one of the two → just the present one; neither → the pre-phase line, byte-identical).
- **D5 — the API (task 02).** `app/api/source_labels.py` (NEW router, the phase-91 admin gate — `require_admin` on every route; registered in `app/main.py` alongside the other admin routers):
- `GET /api/source-labels` → `{"labels": [ {source, display_name, label, tags} … ]}` — the FULL source catalogue: the union of (a) every `git_sources` row's derived source name (git + local, the `effective_sources` rows — an env-only deployment lists the env source names) and (b) every `SELECT DISTINCT source FROM documents` name NOT covered by (a) (the uploads) — one query for (b), the handful of `git_sources` rows for (a); each entry carries its label row or nulls;
- `PUT /api/source-labels/{source}` (path param ≤ 120) body `{display_name?, label?, tags?}` → the upsert; validation (the phase-121 boundary style — the framework's standard 422): `display_name` trimmed ≤ 120 (empty string / whitespace-only = NULL), `label` trimmed ≤ 200 (same), `tags` ≤ 10 entries, each a trimmed ≤ 32 non-empty token, de-duplicated case-insensitively preserving first case, `[]` legal (clears); an explicit `null` clears that field (an omitted field leaves it — the phase-168 partial-update shape); a `source` that matches NO git row and NO document → **404** (no orphan label rows — a label for a source that does not exist would be a dead boost input);
- `DELETE /api/source-labels/{source}` → row removed (404 when absent) — the reset.
- **D6 — the UI (task 03).** The Sources page (`#view-git-sources`, `git-sources.js`) — per-source name/label/tags editor for EVERY source kind (git, local, upload — the upload rows are the same table's extra section, phase 49/90):
- one expandable row per source (the house pattern — the phase-121 token field is the closest precedent): three fields (Name / Label / Tags, the last a comma-or-Enter token input, the phase-89 ignore-paths token input is the precedent) + Save / Reset, the §7.4 never-stale save state ("Saving…" → "saved" / the one-line error note, the phase-168 shape);
- the saved state renders back from the GET (the pre-population — the phase-91 convention); the derived source name is the row's identity (read-only, the `git-sources.js` name cell — the display name is a SEPARATE field, D1);
- the chat page's source cluster chips and the RAG page's source tree keep the DERIVED name (D1 — no identity change anywhere); the display name surfaces in the chat only through the prompt (D4) and the verbatim panel caption is untouched (it names `source/path` — the identity).
## ASSUMPTIONS (owner-confirmed 2026-10-18)
- **A1 (D1):** the display name is an alias — the `Document.source` / `source/path` identity is untouched.
- **A2 (D2/D3):** the focus mechanism is the new bounded `source_name_hit_bonus` selection boost (default 0.005, `0` = off) + the KB-overview / suggested-tier surfacing (D4).
- **A3 (D5):** admin-gated like the phase-91/168 admin surfaces; labels for non-existent sources are rejected (404).
## Context
- `app/models.py` (the `SourcesMeta` L— single-row precedent; the `GitSource` row — the `url`/`path`/`kind` columns the derived names come from), `app/rag/git_sources.py` (`_url_basename` L89, `effective_sources` L285 — the git/local source-name derivation), alembic `0029` (the last migration — 0030 is next), `app/rag/retriever.py` (`RetrievedChunk` L276+, `name_hit_tokens` L191, `_name_hit_chunks` L576, `_selection_order` L754, the three bonus application sites L838/L987/L1022), `app/api/chat.py` (the selection call sites L537–572, L875), `app/config.py` (`name_hit_bonus` L431 + validator L964 — the knob pattern), `app/core/settings_catalog.py` (L225 — the metadata entry pattern), `app/rag/overview.py` (`build_overview_prompt` L148 — the input lines), `app/rag/prompts.py` (the `<knowledge_base>` section L561, the suggested-tier doc lines), `app/api/git_sources.py` (the router pattern — admin gate, the upload status), `frontend/assets/git-sources.js` (the Sources page), `tests/fixtures/retrieval_battery.txt` + `scripts/eval_retrieval.py` (the battery), `tests/integration/` (the phase-119 pins — the kill-switch pattern to mirror).
## Tasks
1. `01_source_labels_table.md` — the `SourceLabel` model, alembic 0030, the `app/rag/source_labels.py` helpers (D1/D2 pure half).
2. `02_source_labels_api.md` — the admin-gated router (D5): the catalogue GET, the validated PUT upsert, the 404s, the DELETE.
3. `03_source_labels_ui.md` — the per-source name/label/tags editor on the Sources page (D6), all three source kinds.
4. `04_source_name_boost.md` — the `source_name_hit_bonus` knob + the `source_name_hit` flag + the `_selection_order` second bonus (D3), the kill-switch pins, the battery run.
5. `05_prompt_surfacing.md` — the KB-overview input lines + the suggested-tier doc lines carry the display name / label (D4).
6. `06_e2e_and_gates.md` — the dedicated Playwright suite (the owner-visible arm: a named source's document is suggested when the question uses the name, the alias never breaks a locator), the full gates, the docs pass, the atomic commit.
## Testing & Quality
- **Unit:** `tests/unit/test_source_labels.py` (NEW) — the D2 matching rule (name/label/tag hits, the case-insensitive `_normalize_name` form, the 4-char minimum on both sides, the no-hit baseline), the D1 helpers (upsert, the partial-update shape, the 404 shape at the helper level); `tests/unit/test_retriever.py` — the phase-119 mirror pins for the SECOND bonus (the kill-switch byte-identity, the bounded per-document application, the OR-merge through `fuse`).
- **Integration:** `tests/integration/test_source_labels_api.py` (NEW) — the admin gate (401/403 the phase-91 shape), the catalogue GET (git + local + upload union, no dupes), the PUT validation 422s (121-char name, 201-char label, 11 tags, a 33-char tag, a whitespace tag), the partial-update shape (an omitted field is untouched, an explicit null clears), the 404s (no such source; DELETE of an absent row), the round-trip.
- **E2E (AGENTS rule 4, task 06):** `tests/e2e/test_source_name_labels.py`, run in isolation — the owner-visible arm through the real UI (set a name + tag on a fixture source → a chat question using the name surfaces that source's document in the suggested tier via the mock/real battery — the phase-119 E2E is the precedent), the alias contract (a read/grep locator for the renamed source still resolves — the derived name is untouched), the 404 arm (a label for a non-existent source is rejected in the UI with the one-line note).
- **Battery:** `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` — the pre/post run at the default bonus (0.005) and at `0` (byte-identical to pre-phase) is recorded in the phase file (the phase-119 convention).
- **Coverage:** **>90%** on `app/`; the frontend change is marker-pinned per the house JS convention.
- **Lint/types:** `uv run ruff check . && uv run pyright`.
## Documentation & Consistency
- Final pass reviews `README.md` and `AGENTS.md` in their entirety against the post-phase app and fixes every inconsistency found, pre-existing ones included.
- `README.md`: the sources section gains the naming feature (the name/label/tags editor, the alias-only contract, the new knob `BOR_SOURCE_NAME_HIT_BONUS` with its `0` = off meaning) (task 06). `AGENTS.md`: the phase-119 name-signal clause gains the parallel source-name clause if it names the bonus (verify, do not add if absent).
## Completion Criteria
- [ ] alembic 0030 applied cleanly; `SourceLabel` round-trips (unit); the alias contract holds (a `Document.source` / locator is untouched by a label — the E2E arm).
- [ ] `GET /api/source-labels` lists git + local + upload sources with their labels (integration + E2E); the PUT validation 422s and 404s pinned; the DELETE resets.
- [ ] the Sources page edits name/label/tags for all three source kinds (E2E), save state per the §7.4 contract.
- [ ] `source_name_hit_bonus = 0` → the selection walk byte-identical to pre-phase (unit pin); at 0.005 the battery shows the named source surfacing (battery record); the three application sites pass the knob (unit pins).
- [ ] the KB-overview input lines and the suggested-tier doc lines carry `display_name (label)` when set, byte-identical otherwise (unit pins on both builders).
- [ ] test suite green, coverage >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] no behavior change in completed phases (no label row → every surface byte-identical to pre-phase).
- [ ] one atomic Conventional Commits commit with `--no-gpg-sign`.
@@ -0,0 +1,41 @@
# Task 01 — The `source_labels` table and helpers (D1/D2 pure half)
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "Sources need a name, a label, tag(s) …"`
**Story:** n/a
## Objective
The additive data layer: the `SourceLabel` ORM model + alembic 0030 (no backfill) + `app/rag/source_labels.py` with the upsert/get helpers and the D2 `question_name_hits` matching rule — so a source without a row is byte-identical to pre-phase everywhere.
## Work
1. `app/models.py` — NEW `SourceLabel(Base)` (the `SourcesMeta` style, tabulated docstring — the D1 column spec from the phase file):
- `source: Mapped[str] = mapped_column(String(120), primary_key=True)` (the `Document.source` name — the join key);
- `display_name: Mapped[str | None] = mapped_column(Text, default=None)` (≤ 120 trimmed — enforced at the API, the column is plain Text, the phase-121 pattern);
- `label: Mapped[str | None] = mapped_column(Text, default=None)` (≤ 200);
- `tags: Mapped[list] = mapped_column(JSONB, default=list, server_default=text("'[]'"), nullable=False)`;
- `updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())`.
2. `alembic/versions/0030_source_labels.py` — NEW, revision `0030`, down_revision `0029` (verify the 0029 rev id — it is the current head): `CREATE TABLE source_labels` (the columns above, the PK, the JSONB server default), no backfill (an empty table — every source's pre-phase behavior is "no row"); the downgrade `DROP TABLE` (the house migration style — `0028` is the shape to copy).
3. `app/rag/source_labels.py` (NEW module — the docstring names the feature and the D1 alias contract):
- `@dataclass(frozen=True) SourceLabelView` — `source, display_name, label, tags` (tags: `tuple[str, …]`); `label_for(db, source) -> SourceLabelView | None` (one PK read; `None` when absent);
- `get_label(db, source) -> SourceLabel | None` (one PK read);
- `set_label(db, source, *, display_name, label, tags) -> SourceLabel` — upsert (the `SourcesMeta` upsert shape): an existing row is updated in place (only the provided non-`None` fields are overwritten — the partial-update shape; an explicit `None` clears), a new row is created; **flush-only — the caller commits** (the phase-53 session convention);
- `all_labels(db) -> dict[str, SourceLabelView]` (one query, keyed by `source`);
- `delete_label(db, source) -> bool` (one PK delete; `False` when absent; flush-only);
- `normalize_tags(raw: list[str]) -> list[str]` (pure: trim, drop empties, ≤ 32-char guard raises `ValueError`, ≤ 10 guard raises `ValueError`, case-insensitive de-dupe preserving first case — the API's 422 comes from these raises);
- `question_name_hits(view: SourceLabelView | None, question: str) -> bool` (the D2 rule, pure): `None` → `False`; else any `name_hit_tokens(question)` candidate (imported from `app.rag.retriever` — the phase-119 list, UNCHANGED) equals, in `_normalize_name` form, a token of `display_name` / `label` / any `tags` entry — the SOURCE side tokenized the same way (whitespace-split, `_normalize_name`, `NAME_TOKEN_MIN_LEN` (4) minimum on BOTH sides);
- `question_name_hit_sources(db, question: str) -> set[str]` (one `all_labels` read + the pure rule per row — a handful of rows; the per-turn entry point task 04 uses).
4. `tests/unit/test_source_labels.py` (NEW):
- the model round-trip (create via `set_label`, the `updated_at` stamp, the partial-update shape — an omitted field untouched, an explicit `None` clears);
- the D2 rule: a digitless name hit (`"gitea"` question token vs `display_name "Gitea"` — case-insensitive via `_normalize_name`), a label hit, a tag hit, a 3-char source-side token NEVER matches (the 4-char minimum), a 3-char QUESTION token never matches (the phase-119 `NAME_TOKEN_MIN_LEN` side), `None` view → `False`, a no-overlap question → `False`;
- `normalize_tags` (the 11-tag ValueError, the 33-char ValueError, the whitespace drop, the case-insensitive de-dupe keeping first case, `[]` legal);
- the migration 0030 up/down (the `tests/unit` or `tests/integration` migration-pin convention — verify where the phase-168 0028 pin lives and mirror it).
5. Verify: `uv run alembic upgrade head` clean on the dev DB; `uv run pytest tests/unit/test_source_labels.py -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the pins above (the D2 rule is the heart of the phase — pin every branch).
- Coverage: **>90%** on `app/` (the new module is fully exercised by its unit module).
## Completion Criteria
- [ ] alembic 0030 applies and reverses cleanly; the `SourceLabel` model matches the D1 column spec.
- [ ] `tests/unit/test_source_labels.py` green — the D2 rule, the partial-update shape, the `normalize_tags` guards.
- [ ] full test suite **100% passing** (the empty table changes nothing).
- [ ] no behavior change in completed work (no row → no surface reads it yet).
@@ -0,0 +1,35 @@
# Task 02 — The admin `source-labels` API (D5)
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "Sources need a name, a label, tag(s) …"`
**Story:** n/a
## Objective
The wire surface: `app/api/source_labels.py` — the catalogue `GET` (every source, all three kinds, with its label or nulls), the validated `PUT` upsert (the framework's standard 422; a 404 for a source that exists nowhere), the `DELETE` reset — admin-gated like the phase-91/168 admin surfaces, registered in `app/main.py`.
## Work
1. `app/schemas.py` — NEW shapes (the phase-91 `extra="forbid"` style):
- `SourceLabelOut` — `source: str` (≤ 120), `display_name: str | None` (≤ 120), `label: str | None` (≤ 200), `tags: list[str]` (≤ 10 items, each ≤ 32);
- `SourceLabelList` — `labels: list[SourceLabelOut]`;
- `SourceLabelUpdate` — `display_name: str | None = Field(default=None, max_length=120)`, `label: str | None = Field(default=None, max_length=200)`, `tags: list[str] | None = Field(default=None, max_length=10)` (each item ≤ 32 — an item-level `Field(max_length=32)` or a model validator; verify the house item-cap convention in `SavedChatCreate`'s messages and mirror it) — `extra="forbid"`; the OMITTED-vs-NULL distinction is the partial-update shape (D5): omitted → untouched, `null` → cleared, a value → set (the validator trims; a whitespace-only `display_name`/`label` → `None` at the validator boundary).
2. `app/api/source_labels.py` (NEW router, `APIRouter(prefix="/api/source-labels", tags=["source-labels"], dependencies=[Depends(require_admin)])` — the phase-91 gate line):
- `GET ""` → `SourceLabelList`: (a) the `git_sources` rows — `effective_sources(db)` (the phase-35/38 helper, L285): each row's derived source name (git: `_url_basename(row.url)`; local: the stored path's basename — the SAME expression the importer uses for `Document.source` — verify the importer's local-name expression and reuse it, do not re-derive it); an env-only deployment (no DB rows, the `env` source state) lists the env source names (the `effective_sources` `source` return is `"db"`/`"env"` — for `env`, parse the env list the phase-35 fallback uses); (b) `select(Document.source).distinct()` (ONE query, no `content` — the phase-126 load discipline) minus the names from (a) (the uploads); de-duplicated, sorted by `source` (stable order — the UI and the E2E rely on it); each entry = its label row via `all_labels(db)` (one query) or nulls;
- `PUT "/{source}"` (path param, the router's `max_length` via the path conversion — a >120 path → the framework's 422) → the D5 404 arm FIRST: `source` matches no `git_sources` row name AND is not in the distinct document sources (the SAME two queries the GET runs — no third) → 404 (the standard body, the phase-91 shape); else `normalize_tags` (422 on its raises — a `ValueError` → the framework's 422 via the validator, NOT a 500), `set_label` (flush + `db.commit()` — the router owns the session here, the phase-91 commit convention), 200 `SourceLabelOut`;
- `DELETE "/{source}"` → the 404 arm (no label row → 404), else `delete_label` + commit, 200 (the phase-91 DELETE shape — verify its body: an empty object or a message; mirror the sibling admin router).
3. `app/main.py` — `app.include_router(source_labels.router)` (the admin routers' block — verify the exact registration list and the order convention).
4. `tests/integration/test_source_labels_api.py` (NEW — the phase-91 `test_ui_settings_api.py` fixture pattern):
- the gate: an anonymous GET/PUT/DELETE → the phase-91 gate shape (403 / the redirect-to-login — verify the phase-91 gate pin and mirror it EXACTLY);
- the catalogue GET: a module app with one git row (a `file://` URL — the phase-35 fixture), one local row, one upload (a document row with a source name no row derives) + a label on the upload → the response lists all three (sorted, no dupes), the upload carries its label, the others nulls;
- the PUT: a valid upsert round-trips (set name+label+tags → GET shows them; a second PUT with only `label` → the name/tags untouched; a PUT with explicit `null` `display_name` → cleared, the label kept); the 422s (a 121-char `display_name`, a 201-char `label`, 11 `tags`, a 33-char tag, a whitespace-only tag — each with the framework's standard body and NO row change: a pre-seeded row is byte-identical after the 422); the 404s (a `source` that is neither a row name nor a document source — the row table unchanged);
- the DELETE: a labelled source → 200, the GET shows nulls; a second DELETE → 404.
5. Verify: `uv run pytest tests/integration/test_source_labels_api.py -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Integration: the pins above — the gate, the union catalogue, the validation 422s (no partial write), the 404s, the round-trip.
- Coverage: **>90%** on `app/` (the router is fully exercised by its integration module).
## Completion Criteria
- [ ] `GET /api/source-labels` lists git + local + upload sources (integration pin) with their labels; admin-gated.
- [ ] `PUT /api/source-labels/{source}` upserts with the D5 partial-update shape; every 422 leaves the row unchanged; a 404 for a non-existent source creates nothing.
- [ ] `DELETE` resets; a second DELETE 404s.
- [ ] full test suite **100% passing**.
- [ ] no behavior change in completed work (the `git_sources` API, the `documents` API, the phase-91/168 admin surfaces all green unedited).
@@ -0,0 +1,35 @@
# Task 03 — The per-source name/label/tags editor on the Sources page (D6)
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "Sources need a name, a label, tag(s) …"`
**Story:** n/a
## Objective
The owner-visible surface: every source on the Sources page (git, local, upload — the catalogue `GET /api/source-labels` is the single source of truth for the row list, so uploads appear without a second API) gets an inline name/label/tags editor with the §7.4 never-stale save state — the phase-121 token field and the phase-89 ignore-paths token input are the house precedents.
## Work
1. `frontend/index.html` — `#view-git-sources` (L646+): the sources table area (the `#git-sources-table-wrap` region, L792) gains the per-row expandable label block markup — ONE shared template (a `<template id="source-label-row-template">`) the JS clones per row:
- three fields in one row (the house grid): **Name** (`input`, ≤ 120, `maxlength="120"`), **Label** (`input`, ≤ 200, `maxlength="200"`), **Tags** (the token input — the phase-89 ignore-paths input's exact widget: type + Enter/comma adds a chip, Backspace deletes the last, ≤ 10 chips, each ≤ 32 chars — reuse its CSS classes, no new styles beyond the phase-89 ones);
- **Save** / **Reset** buttons (the phase-168 button shapes) + the save-state line (`role="status"` — "Saving…" → "saved" / the one-line error note; Reset visible only when the row has a stored label, the phase-168 per-row reset shape);
- the row's IDENTITY cell (the derived source name) is UNCHANGED and read-only (D1/A1 — the editor sits beside it, never replaces it).
2. `frontend/assets/git-sources.js`:
- the load path: after the existing catalogue load, `GET /api/source-labels` (one extra fetch — the gate is the SAME as the page's: the anonymous gate hides the whole page, the admin sees it; the fetch runs inside the existing admin-gated content reveal); merge the labels into the row data (by `source` name — the GET is the union, so the row list for the label editor comes from the GET, not from the `git-sources` GET — the upload rows have no `git-sources` row);
- the editor state per row: the three fields pre-populated from the label (nulls → empty); dirty tracking (a field change enables Save — the phase-168 `.is-dirty` + "unsaved changes" hint shape, scoped per row);
- **Save**: `PUT /api/source-labels/<source>` (the trimmed values; the tags array from the chips) → success: the "saved" state, the row's label re-read from the response (no second GET — the response is the new truth); failure: the one-line error note (the framework's detail — the 422/404 arm: a 404 shows "that source no longer exists" — the note text names the 404 arm, the phase-168 error-note shape); the §7.4 guard: one in-flight save per row (a second click while saving is ignored, the phase-168 `saveInFlight` pattern scoped per row);
- **Reset**: `DELETE /api/source-labels/<source>` → the "saved" state, the fields cleared, the row's stored label nulled;
- the page's existing sync / upload / git-row flows are UNTOUCHED (the editor is additive — the row's existing action buttons keep their behavior).
3. `tests/unit/` — the JS marker pins (the module that marker-pins `git-sources.js` — verify the name; the `test_save_chat_ui.py` convention):
- the editor's save path references `PUT /api/source-labels/` and the response (not a second GET) as the new truth (source marker);
- the per-row `saveInFlight` guard marker; the 404 note text marker;
- the token input reuses the phase-89 widget classes (a marker naming the class reuse — no new tag-input CSS).
4. Verify: `uv run pytest tests/unit/<js-pin-module> -v` green; `uv run playwright`-free local check: the page renders the editor for a seeded label row (the E2E covers this in task 06 — a manual `uv run uvicorn` smoke is optional, the E2E is the gate).
## Testing & Quality
- Unit: the marker pins above.
- E2E: task 06 (the owner-visible arm).
- Coverage: **>90%** on `app/` (unchanged — the task is frontend; the app-side gate stays green).
## Completion Criteria
- [ ] every source row (git, local, upload) renders the name/label/tags editor; the derived name cell is read-only and untouched (D6/A1).
- [ ] Save/Reset hit the phase-02 endpoints; the save state follows the §7.4 contract (never a stuck "Saving…", the 404 note names the arm) — marker pins.
- [ ] the existing sources-page flows (add git/local source, upload, sync) are green unedited.
- [ ] no behavior change in completed work.
@@ -0,0 +1,42 @@
# Task 04 — The `source_name_hit_bonus` selection boost (D2/D3)
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "… so you can focus the model when prompting …"`
**Story:** n/a
## Objective
The focus mechanism: a NEW bounded, kill-able selection-time bonus that lifts a document when the question names its source (D2 rule) — the phase-119 `name_hit_bonus` pattern end to end, a separate knob, the SAME shared walk, `0` = byte-identical pre-phase.
## Work
1. `app/config.py` — the knob (the `name_hit_bonus` L431 pattern, directly below it):
- `source_name_hit_bonus: float = 0.005` (the docstring: the D3 spec — default = the phase-119 starting point; `0` = off, byte-identical walk; bounded per-document, at most two bonuses on the fused scale);
- the `field_validator` (the L964 pattern): `>= 0`, the message `"source_name_hit_bonus must be >= 0 (0 = off)"`.
2. `app/core/settings_catalog.py` — the `SettingMeta` entry (the L225 `name_hit_bonus` pattern): `"source_name_hit_bonus"` in the **"Retrieval & honesty"** section, label "Source name hit bonus", the restart-required flag FALSE (the phase-168 runtime knob shape — verify the `name_hit_bonus` entry's flags and mirror them). Extend the catalogue-completeness pin (the unit test asserting every `Settings` field has a `SettingMeta` entry — find the module; it starts failing when the field lands, the entry is the fix).
3. `app/rag/retriever.py`:
- `RetrievedChunk` (L276+): the additive field `source_name_hit: bool = False` + the docstring line (the `name_hit` line pattern — default `False` keeps every ordinary construction valid; `fuse`'s `replace()` copies it automatically — the frozen dataclass; the double-hit merge at L342–358 gains `source_name_hit=existing.source_name_hit or rc.source_name_hit` beside the existing `name_hit` OR);
- `_selection_order` (L754): signature `_selection_order(chunks, bonus, source_bonus: float = 0.0)` — the walk gains `any_source_hit[doc.id]` (the `any_name_hit` pattern, L792/L799/L804); the effective-score dict: `best_score[doc.id] + (bonus if any_name_hit[doc.id] else 0.0) + (source_bonus if any_source_hit[doc.id] else 0.0)`; the re-sort condition: `(bonus > 0.0 and any_name_hit…) or (source_bonus > 0.0 and any_source_hit…)` (the pre-phase condition extended — when neither bonus is active, the walk is byte-identical: no re-sort, the same order list, the same `+ 0.0`); the docstring's D2/LOCKED-A3 paragraph is extended for the second bonus (each bonus per DOCUMENT, applied ONCE, at most two per document — bounded 0.010; the bonus lives in the SELECTION layer only — `score` / `cosine` / `fts_hit` / `query_log.top_score` untouched);
- `weak_hit_titles` (L825), `select_suggested` (L952), `select_related` (L1000): a `source_bonus: float | None = None` parameter each; `if source_bonus is None: source_bonus = get_settings().source_name_hit_bonus`; passed to `_selection_order`.
4. `app/api/chat.py` — the flag-set at the retrieval boundary (the call sites L537–572 and L875 — the question is in scope, the session is in scope):
- ONE call per turn: `hits = question_name_hit_sources(db, question)` (task 01's per-turn entry point — the single `all_labels` read) BEFORE the first selection call;
- the flag loop: `for rc in chunks: if rc.document.source in hits: rc.source_name_hit = True` (the few dozen retrieved chunks against the handful of hit sources — never a per-chunk query); the selection calls keep their existing arguments (the functions pull the knob from settings themselves — the call sites gain NO new positional arguments);
- the per-turn log line (PLAN §9) is UNCHANGED (the `verbatim_blocks=` / `scaffold_stripped=` fields are untouched — the boost is invisible to the log; no new log field).
5. `tests/unit/test_retriever.py` (or the phase-119 pin module — verify the name):
- the KILL-SWITCH pin (the phase-119 mirror): `source_name_hit_bonus = 0` (and no flagged chunks) → the selection order is byte-identical to the pre-phase walk (the same fixture, the same expectation list);
- the boost pin: a document whose source is a D2 hit (a `source_name_hit=True` chunk) ranks ABOVE an equal-fused-score non-hit when the bonus is on (the 0.005 head start), and a document carrying BOTH flags gets exactly +0.010 (the bounded two-bonus pin — no amplification);
- the `fuse` merge pin: a vector row merged with a flagged source-name row keeps `source_name_hit=True`;
- the settings pins: the validator (`-0.1` → the ValueError), the env name (`BOR_SOURCE_NAME_HIT_BONUS`), the catalogue-completeness pin extended.
6. The battery (the phase-119 convention): `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` — run it TWICE and record both in the phase file: (a) pre-change (the baseline), (b) post-change at `source_name_hit_bonus = 0` (must match (a) line-for-line — the kill switch) and at the default 0.005 (no label rows exist in the battery app → also identical to (a) — the no-row baseline).
7. Verify: `uv run pytest tests/unit/test_retriever.py tests/unit/test_settings*.py -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the kill-switch byte-identity, the bounded two-bonus application, the `fuse` OR-merge, the settings/catalogue pins.
- Integration: the E2E task's app fixture exercises the flag-set (no separate integration module — the flag loop is covered by the E2E arm + the unit pins).
- Battery: the three-run record above.
- Coverage: **>90%** on `app/`.
## Completion Criteria
- [ ] `source_name_hit_bonus = 0` → the selection walk byte-identical to pre-phase (the unit pin + the battery match).
- [ ] at the default 0.005, a D2-hit document gets exactly the 0.005 head start (0.010 with both flags) — the bounded pin; no chunk field is modified (the untouched-fields pin).
- [ ] the three selection helpers pull the knob from settings by default (the unit pins with a settings override).
- [ ] `tests/fixtures/retrieval_battery.txt` battery: the kill-switch run matches the baseline line-for-line (recorded in the phase file).
- [ ] full test suite **100% passing**; `uv run ruff check . && uv run pyright` clean.
- [ ] no behavior change in completed work (the phase-119 bonus, the A8 gate inputs, the per-turn log line all untouched).
@@ -0,0 +1,37 @@
# Task 05 — The prompt learns the name: the `<document>` block + the KB overview (D4)
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5 — "… so you can focus the model when prompting …"`
**Story:** n/a
## Objective
The model learns the name→docs mapping from the prompt: a suggested `<document>` block carries the source's `name` / `label` as additive attributes (when set), and the KB overview's INPUT lines carry the display name — both byte-identical for a source without a label row. ONE `all_labels` read per turn feeds the task-04 boost flags AND these surfaces.
## Work
1. `app/rag/prompts.py` — `build_high_prompt` (L828):
- a new optional parameter `names: Mapping[str, "SourceLabelView"] | None = None` (the source-name → view map; `None` = pre-phase, byte-identical — the `show_image`/`show_text` optional-parameter pattern);
- the block builder (L934–939): when `names.get(doc.source)` has a `display_name`, the block head gains ` name="{display_name}"` immediately after the `source="…"` attribute; when it has a `label`, it gains ` label="{label}"` after the `name` (the attribute order `source` → `name?` → `label?` → `path` → `title` → `date`; an unset field adds NOTHING — a no-label document's block is byte-identical to pre-phase, and a label-only document gets exactly the one `label` attribute);
- the values are XML-escaped with the SAME helper the block already uses for `source`/`path`/`title` (verify the escaping at L934 — a `"` in a display name must not break the attribute; the phase-106 date formatting is untouched).
2. `app/rag/overview.py` — `build_overview_prompt` (L148):
- a new optional parameter `names: Mapping[str, str] | None = None` (source → display_name); the line builder (L173): `line = f"{names.get(source, source)} — {path} — {title}"` (the display name REPLACES the raw source name in the lite model's input only — the stored overview row and the DB are untouched); no `names` / no entry → the pre-phase line, byte-identical.
- `regenerate_overview` (the caller — verify the signature): it already holds a session; it gains the ONE `all_labels(db)` read (task 01's helper) and passes (a) the display-name map to `build_overview_prompt` and (b) the same read is NOT re-queried — the map is derived from the views in the same loop.
3. `app/api/chat.py` — the task-04 call site is REFACTORED to the shared read (the "one query per turn" contract):
- `labels = all_labels(db)` (the single read, before the first selection call — replacing task 04's `question_name_hit_sources` call, whose internals it inlines);
- `hits = {v.source for v in labels.values() if question_name_hits(v, question)}` (the pure D2 rule per view — the flag loop of task 04 is unchanged, `hits` is now local);
- the `build_high_prompt` call (the grounded plan — verify the exact call site in the module) passes `names=labels` (the `SourceLabelView` map — `build_high_prompt` accepts the views directly; the overview's `str` map is the overview's own derivation, a different surface, no conversion shared).
4. `tests/unit/test_prompts.py` (or the prompt-lock module — verify the name):
- the KILL pin: `names=None` and `names={}` and a map with no entry for the doc's source → the built prompt is byte-identical to the pre-phase build (the existing pre-task pins stay green UNEDITED — they are the kill pin);
- the attribute pins: a `display_name`-only view → exactly ` name="…"` after `source="…"` (the attribute-order pin); a `label`-only view → exactly the `label` attribute (no `name`); both → both attributes in order; the XML-escaping pin (a `"` in a display name renders as the escaped entity in the attribute);
- the overview pins: `names=None` → the pre-phase lines byte-identical; a map entry → the line uses the display name (the `path` / `title` / summary halves untouched); a missing entry → the raw source name.
5. Verify: `uv run pytest tests/unit/test_prompts.py tests/unit/test_overview.py -v` green (verify the overview test module name); `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the byte-identical kill pins (both builders), the attribute-order + escaping pins, the overview line-swap pins.
- Integration: the E2E task's arm covers the wire (the system prompt of a turn with a named source — the mock LLM's captured prompt, the phase-119 E2E precedent).
- Coverage: **>90%** on `app/`.
## Completion Criteria
- [ ] a named source's suggested document renders `name="…"` / `label="…"` (the unit pins); an unnamed source's block and overview line are byte-identical to pre-phase (the unedited pre-task pins + the kill pins).
- [ ] ONE `all_labels` read per turn feeds the boost flags and the prompt surfaces (the marker pin on the chat.py call site — no second label query in the turn).
- [ ] `regenerate_overview` uses the same single read (the marker pin — no second query).
- [ ] full test suite **100% passing**; `uv run ruff check . && uv run pyright` clean.
- [ ] no behavior change in completed work (the `<document>` block for label-less sources, the KB overview text for label-less sources, the phase-106 date attribute all byte-identical).
@@ -0,0 +1,37 @@
# Task 06 — The phase E2E suite, the full gates, the docs pass, the commit
**Phase:** `191_source_name_labels_tags` · **Source:** `TODO.md:5`
**Story:** n/a
## Objective
The dedicated Playwright suite proves the owner-visible arm through the real UI — a named source's document surfaces when the question uses the name, the alias never breaks a locator, the 404 arm is visible — run in isolation per AGENTS rule 4; the full gates pass; the docs are consistent; one atomic commit.
## Work
1. `tests/e2e/test_source_name_labels.py` (NEW — the module docstring names the TODO item and the D1 alias contract) — the module-app fixture pattern (the phase-119/168 E2E convention — verify in `tests/e2e/conftest.py` / the sibling suites; the mock LLM from `tests/e2e/mock_llm.py` for the prompt-capture arm):
- the fixture: the standard small KB (the phase-119 fixture convention — a handful of documents in a couple of sources) + one uploaded document (the phase-49/90 upload flow or a direct document-row seed — verify the sibling suites' seeding helper).
- **(a) the editor, all three kinds:** as the admin, open the Sources page; set a Name ("the git server"), a Label ("self-hosted forge") and a tag ("gitea") on a git source row, a Name on a local source row, a Name + tag on the upload row → each save shows "saved" (the §7.4 state) and a reload pre-populates the saved values (the phase-91 pre-population convention); the derived-name cells are unchanged (the A1 pin — the row identity cell text is the pre-save text).
- **(b) the focus, owner-visible:** a chat question that uses the NAME ("what does the git server run?") with the mock LLM → the suggested tier (the `done` frame's `sources` / the UI's source cluster) includes the named source's document (the D3 boost's owner-visible arm — the phase-119 E2E is the precedent; the mock's captured system prompt asserts the `<document name="the git server" label="self-hosted forge" …>` attributes — the D4 arm at the wire); the same question phrased WITHOUT the name (the pre-baseline question from the battery) shows the pre-phase suggested set (the no-signal control — the boost is the only difference).
- **(c) the alias contract:** after the rename, a verbatim-fenced answer (or a direct `page.request` read of the document API) for `source/path` (the DERIVED name) still resolves — the locator is untouched (the A1 pin through the wire); the KB tree / RAG page still lists the derived name (the D1 surface pin).
- **(d) the 404 arm:** the editor's PUT for a source that no longer exists (delete the underlying git row via the API mid-test, then save a label through the UI) → the one-line error note names the 404 arm (the task-03 pin, owner-visible).
- **(e) the reset:** the editor's Reset on the named source → "saved", the fields cleared, a reload shows empty, and the chat's suggested set for the named question reverts to the control (the label's absence is the pre-phase state end to end).
2. Regression, run in isolation: `tests/e2e/test_name_signal.py` (the phase-119 suite — verify the exact name in `tests/e2e/`), `tests/e2e/test_settings_panel.py`, `tests/e2e/test_git_sources.py` (the phase-76 suite — verify the name) — green unedited.
3. The full gates, in order:
- `uv run pytest` (unit + integration) — 100% passing;
- `uv run pytest --cov=app --cov-report=term-missing` — **>90%** on `app/`;
- `uv run pytest tests/e2e/test_source_name_labels.py -v --no-cov` — green in isolation;
- `uv run ruff check . && uv run pyright` — clean.
4. Documentation pass (the phase file's Documentation section): `README.md` + `AGENTS.md` reviewed in their entirety against the post-phase app; the sources section gains the naming feature (the editor, the alias-only contract, `BOR_SOURCE_NAME_HIT_BONUS` with its `0` = off meaning); `AGENTS.md` corrected only where it names the phase-119 bonus as the ONLY name signal.
5. Commit: one atomic, professional commit (Conventional Commits, `--no-gpg-sign`), e.g.
`feat(sources): per-source name, label and tags focus retrieval and the prompt (phase 191)` — the body names the D1 alias contract, the D2 rule, the D3 knob, the D4 surfaces, the new table/migration 0030, and the battery record location. `.agents/phases/todo/191_source_name_labels_tags/` moves to `.agents/phases/complete/` in the SAME commit (AGENTS rule 8).
## Testing & Quality
- E2E: the suite above, isolated; the regression suites green.
- Battery record: task 04's three-run record is in the phase file (the kill-switch run matches the baseline).
- Coverage: **>90%** on `app/`.
- Lint/types: clean.
## Completion Criteria
- [ ] `tests/e2e/test_source_name_labels.py` green in isolation — (a) the editor for git/local/upload with the saved-state contract, (b) the named question surfaces the named source's document + the prompt attributes, (c) the derived-name locator is untouched by a rename, (d) the 404 note, (e) the reset reverts end to end.
- [ ] `uv run pytest` 100% passing; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] `README.md` / `AGENTS.md` consistent (the naming feature + the knob).
- [ ] the phase directory is in `.agents/phases/complete/`; one atomic commit with `--no-gpg-sign`.
@@ -0,0 +1,58 @@
# Phase 192 — Verbatim panels land while the answer is still streaming
**Source:** `TODO.md` L5 — "The verbatim tag appear at the end and only when the llm is done answering which makes the chat read strangely."
**Story:** n/a (owner report)
**Depends on:** none functionally (builds on the completed phase-182 verbatim display, phase-95 tool_result frames, phase-158/164 saved-chat restore).
## Evidence (diagnosed from the code, read this not the chat)
1. **Panels materialize only on the `done` frame.** `app/api/chat.py` resolves the turn's verbatim blocks at done-time (L1319–1347: `resolve_answer_blocks` over `answer_text` → the done frame's additive `answer_text` list); `app.js` hydrates at done (L4506–4511: `hydrateVerbatim(bubble, ev.answer_text, …)` + `appendUnlocatedVerbatim(wrap, ev.answer_text, …)`) and at restore (L3044–3046, the record's `m.answer_text`).
2. **The `show_text` result already exists mid-turn — it just isn't sent.** The agent's show_text branch (app/rag/agent.py L2238–2340) appends the resolved `VerbatimBlock` to `holder.shown_texts` (call order, the `show_text_max_blocks` budget) the instant the call succeeds, and the result text rides the round's `tool` message — but the block itself reaches the client ONLY in the done frame.
3. **So the reading order inverts.** The prose streams first; every verbatim panel (fenced or not) pops in at the END, after the model has already finished talking about the bytes: "…here is the doc: [model finishes] [panel appears]". The unfenced blocks are the worst case — `appendUnlocatedVerbatim` appends them at the answer's tail (the `answer_image` position), detached from the sentence that introduced them.
4. **The fix is additive.** The block is already in hand at the success point (the `shown_texts` append, L2322 area); the round loop already has the snapshot-then-yield pattern for phase-95 truncations (`trunc_before`, L2640 → the `ToolResultPiece` yield at L2690). One new piece + one new SSE event + one client branch; the `done` frame, the record key, the restore and the shared page keep their exact contract.
## Design (shared by all tasks — the executor reads this, not the chat)
- **A1 — the mid-turn frame (task 01).** A new additive SSE event `verbatim_block`, emitted as soon as a `show_text` call succeeds:
- `app/rag/agent.py`: a new `@dataclass VerbatimBlockPiece` (the `ToolResultPiece` pattern, beside it; the `block: VerbatimBlock` field — the class is already imported at L343). In the round loop: `shown_before = len(holder.shown_texts)` BEFORE `_execute_tool` (the `trunc_before` pattern, L2640); after the round's `ToolResultPiece` yield (L2690), `for block in holder.shown_texts[shown_before:]: yield VerbatimBlockPiece(block=block)` — the frame lands AFTER the round's `tool_result` frame and BEFORE the next model round. SUCCESS-ONLY by construction: every refusal path returns before the `shown_texts.append` (L2322), and the already-shown dedupe is a refusal — a refused call yields no piece. The generator union types (L928/L992) gain the piece.
- `app/schemas.py`: `ChatVerbatimBlockEvent` (the `ChatToolResultEvent` pattern, L321) — `type: Literal["verbatim_block"]` + the EXACT `VerbatimBlockRef` field set (L185: `source` ≤ 120 / `path` ≤ 1000 / `title` ≤ 500 / `start_line` / `end_line` / `total_lines` / `text` ≤ 600 000 / `truncated` / `locator` ≤ 2100) — the client's `isVerbatimBlock` check (verbatim.js L70) accepts the event as-is; the framework's `extra` convention of the sibling events is mirrored.
- `app/api/chat.py`: the new dispatch branch after the `ToolResultPiece` branch (L1104): `if isinstance(piece, VerbatimBlockPiece): yield sse_event(ChatVerbatimBlockEvent(**asdict-style fields).model_dump()); continue`. The done-frame `answer_text` resolution (L1319–1347) and the per-turn log line (`verbatim_blocks=N`, done-frame count) are UNTOUCHED — the mid-turn frames are not counted there (the pin stays green).
- KILL-SWITCH: `BOR_ANSWER_TEXT=0` → the tool is not offered (the phase-182 D8 posture) → no success → NO piece, NO frame: the frame sequence is byte-identical to pre-phase (a wire pin, task 01 + the E2E arm).
- **A2 — the client lands the panel mid-stream (task 02).**
- `frontend/assets/verbatim.js`: a NEW `hydrateVerbatimNow(root, block, opts)` — walks the PENDING placeholders (`.verbatim-pending[data-bor-locator]`); for each, `matchVerbatimBlock([block], key)` (the SAME matching semantics as the done path — the exact locator, or the identity fallback when the list holds exactly one block of that identity); REPLACES ONLY the matched nodes with `buildVerbatimPanel(block, opts)`; the UNMATCHED pending nodes STAY PENDING (settling them is the done/failure path's job — a mid-turn pass must never burn a placeholder for a not-yet-shown fence); returns the number matched (0 → the caller appends). Never throws (the house contract — a missing `root` → 0).
- `frontend/assets/app.js`: the new `ev.type === "verbatim_block"` branch beside the `tool_result` branch (L4612): `const n = hydrateVerbatimNow(bubble, ev, VERBATIM_LINK_OPTS); if (n === 0) appendUnlocatedVerbatim(wrap, [ev], VERBATIM_LINK_OPTS);` — a fenced block lands IN PLACE at its fence (the model said "here it is" → the panel is there); an unfenced block lands at the `answer_image` position (the `appendUnlocatedVerbatim` contract — below the current text, above the sources) AT THE MOMENT the call succeeds, not at the end of the answer. `appendUnlocatedVerbatim`'s state-free idempotence (same canonical locator already placed → skip, L463–472) makes the done pass's re-run a no-op for the same block. No client-side kill-switch gate: the server's gate is the only one (a `False` turn sends no frame — the branch never fires).
- **A3 — the done/restore dedupe (task 03).** The one remaining duplicate: a mid-turn APPENDED panel (locator A) + the model later fences the SAME locator A → the fence placeholder hydrates at done into a second panel. `hydrateVerbatim` (the done AND restore path — L423) gains the placed-scan: before building a matched panel for a `key`, if a PLACED resolved panel (``.verbatim[data-bor-locator]`, NOT `.verbatim-unresolved` — the `appendUnlocatedVerbatim` placed-scan convention) carrying the same canonical `locator` exists in the message wrap (`root.closest(".msg")`), that placed panel is REMOVED first — the in-text panel wins (the fence is where the model wanted it; the appended panel has served its mid-turn purpose). TWO DIFFERENT ranges of the same file are LEGITIMATE (the `shown_texts` dedupe is per canonical locator, not per identity) and stay — the scan is per-locator, never per-identity. The restore path (L3044–3046) and the shared page ride the SAME functions — no separate change; the parity pin is the test.
- **A4 — the unchanged contract (pinned across tasks).** The `done` frame's `answer_text` list, the saved-record key, the restore path and the shared page are byte-identical to phase 182 for a turn that never calls `show_text`; a `BOR_ANSWER_TEXT=0` turn is byte-identical end to end (no frame, no placeholder, no panel); the `verbatim_blocks=N` per-turn log line counts the done-frame blocks as before.
## Context
- `app/rag/agent.py` — `VerbatimBlock` (imported L343), `holder.shown_texts` (L1848), the show_text branch (L2238–2340), the round loop (L2640–2697), the piece union types (L928/L992); `app/api/chat.py` — the piece dispatch (L1096–1130), the done-time resolution (L1319–1347); `app/schemas.py` — `VerbatimBlockRef` (L185), `ChatToolResultEvent` (L321), the `extra` convention; `app/rag/verbatim.py` — `VerbatimBlock` (L94), `block_for`, `canonical_locator` (D7).
- `frontend/assets/verbatim.js` — `isVerbatimBlock` (L70), `matchVerbatimBlock` (L404), `hydrateVerbatim` (L423), `appendUnlocatedVerbatim` (L447), `settleVerbatimPlaceholders` (L480); `frontend/assets/app.js` — the `tool_result` branch (L4612), the done branch's hydrate/append (L4506–4511), the restore hydrate/append (L3044–3046).
- `tests/e2e/mock_llm.py` (the mock's show_text flow — the phase-182 E2E precedent; verify the exact mock arm), the phase-182 E2E suites (verify the names in `tests/e2e/`), the JS marker-pin unit module (verify the name — the `test_save_chat_ui.py` convention), `tests/integration/` the chat SSE contract module (verify the name).
## Tasks
1. `01_verbatim_block_sse.md` — the `VerbatimBlockPiece`, the `ChatVerbatimBlockEvent`, the chat dispatch branch; the agent mock pins, the SSE contract pins, the kill-switch wire pin.
2. `02_midturn_hydration_and_append.md` — `hydrateVerbatimNow` + the app.js `verbatim_block` branch (in-place fence hydration at mid-turn; unfenced blocks land at the call moment; unmatched placeholders stay pending).
3. `03_done_and_restore_dedupe.md` — the `hydrateVerbatim` placed-scan dedupe (per-locator; two ranges stay; the restore/shared parity pin; the kill-switch and the `verbatim_blocks=N` pins stay green).
4. `04_e2e_and_gates.md` — the dedicated Playwright suite (the mid-turn arm, the fence arm, the no-double arm, the kill-switch wire arm, the restore arm), the full gates, the docs pass, the atomic commit.
## Testing & Quality
- **Unit:** the agent mock pins (a successful `show_text` → exactly one `VerbatimBlockPiece` after the round's `tool_result` and before the next round; the five refusal paths → zero; the `answer_text=False` posture → zero); the `VerbatimBlockPiece`/`ChatVerbatimBlockEvent` shape pins; the JS marker pins (the `hydrateVerbatimNow` matched-only contract, the app.js branch).
- **Integration:** the chat SSE contract — a mock turn with `show_text` → the stream carries the `verbatim_block` frame (the exact `VerbatimBlockRef` fields) before `done`; a `BOR_ANSWER_TEXT=0` app → the captured frame sequence is byte-identical to the pre-phase sequence (the kill-switch wire pin); the done frame's `answer_text` is unchanged (the phase-182 pins stay green).
- **E2E (AGENTS rule 4, task 04):** `tests/e2e/test_verbatim_inline_midturn.py`, run in isolation — the owner-visible arm: the panel is on screen WHILE the answer is still streaming (before `done`), the fence hydrates in place mid-turn, no double panel, the kill-switch wire arm, the restore parity.
- **Regression:** the phase-182 E2E suites green unedited; `tests/e2e/test_chat_history.py` (the restore contract) green.
- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`); the frontend change is marker-pinned per the house JS convention.
- **Lint/types:** `uv run ruff check . && uv run pyright`.
## Documentation & Consistency
- Final pass reviews `README.md` and `AGENTS.md` in their entirety against the post-phase app and fixes every inconsistency found, pre-existing ones included.
- `README.md`: the verbatim section notes the mid-turn landing (a `show_text` panel appears the moment the call succeeds — in place when fenced, at the answer position when not; the done/restore contract is unchanged) (task 04). `AGENTS.md`: the phase-182 clause gains the mid-turn frame clause if it names the done-frame resolution (verify, do not add if absent).
## Completion Criteria
- [ ] a successful `show_text` call yields exactly one `verbatim_block` frame (the unit + integration pins); every refusal yields none.
- [ ] a `BOR_ANSWER_TEXT=0` turn's frame sequence is byte-identical to pre-phase (the wire pin) — no frame, no placeholder, no panel.
- [ ] a pending fence placeholder for a shown block hydrates IN PLACE at mid-turn (before `done`); an unfenced block is on screen at the call moment (the E2E arms).
- [ ] a mid-turn appended panel + a same-locator fence → exactly ONE panel after `done` (the dedupe pin); two different ranges of one file → two panels (the pin).
- [ ] the restore path shows exactly one panel per locator after a reload (the E2E arm); the `done` frame's `answer_text`, the record key and the `verbatim_blocks=N` log line are unchanged (the phase-182 pins green unedited).
- [ ] test suite green, coverage >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] no behavior change in completed phases (a turn without `show_text` is byte-identical end to end).
- [ ] one atomic Conventional Commits commit with `--no-gpg-sign`.
@@ -0,0 +1,60 @@
# Task 01 — The additive `verbatim_block` SSE frame (A1)
**Phase:** `192_verbatim_inline_midturn` · **Source:** `TODO.md:6 — "The verbatim tag appear at the end and only when the llm is done answering …"`
**Story:** n/a
## Objective
The server half: a `show_text` success yields a new `VerbatimBlockPiece` → a new additive `verbatim_block` SSE frame lands AFTER the round's `tool_result` frame and BEFORE the next model round — the block the agent already holds, sent at the moment it exists. Success-only by construction; `BOR_ANSWER_TEXT=0` → zero frames (byte-identical sequence).
## Work
1. `app/rag/agent.py`:
- a new `@dataclass VerbatimBlockPiece` beside `ToolResultPiece` (the same style — a field docstring naming the phase-95/182 lineage: `block: VerbatimBlock` — the class is already imported at L343);
- the round loop: `shown_before = len(holder.shown_texts)` immediately BEFORE `_execute_tool` (L2640, the `trunc_before` pattern); after the round's `ToolResultPiece` yield (L2690–2697), `for block in holder.shown_texts[shown_before:]: yield VerbatimBlockPiece(block=block)` — one line, the same snapshot-then-yield shape (a round can add at most one block — the already-shown dedupe is a refusal — so the loop is a one-iteration shape, but the snapshot keeps a future multi-add correct);
- the generator union types (L928/L992) gain `VerbatimBlockPiece`; the module docstring's piece paragraph (the L215–225 display-tools region) gains the one-line clause (the mid-turn frame, the done-frame count UNCHANGED — `verbatim_blocks=N` still counts the done frame's blocks, never the mid-turn frames);
- the `run()` docstring: the emission-order contract (the frame is after the round's `tool_result`, before the next round — a refused call emits nothing, the `shown_texts.append` at L2322 is the single success gate).
2. `app/schemas.py` — `ChatVerbatimBlockEvent(BaseModel)` (the `ChatToolResultEvent` L321 pattern, directly after it):
- `type: Literal["verbatim_block"] = "verbatim_block"` + the EXACT `VerbatimBlockRef` field set (L185 — `source` ≤ 120, `path` ≤ 1000, `title` ≤ 500, `start_line`/`end_line`/`total_lines` `ge=0`, `text` ≤ 600 000, `truncated: bool = False`, `locator` ≤ 2100) — the docstring: the mid-turn twin of the done frame's `answer_text` item; the client's `isVerbatimBlock` check (verbatim.js L70) accepts the event as-is (the key set matches — a pin); the sibling events' `model_config`/`extra` convention is mirrored EXACTLY (verify `ChatToolResultEvent`'s config and copy it).
3. `app/api/chat.py` — the dispatch branch after the `ToolResultPiece` branch (L1104–1119):
```python
if isinstance(piece, VerbatimBlockPiece):
# Phase 192 (task 01, A1): the mid-turn verbatim frame — the block
# the show_text call just displayed, sent at the moment it exists.
# Additive: a refused call yields no piece (no frame); a
# answer_text=False turn offers no tool (no frame) — the frame
# sequence is byte-identical to pre-phase in both cases.
yield sse_event(
ChatVerbatimBlockEvent(
source=piece.block.source,
path=piece.block.path,
title=piece.block.title,
start_line=piece.block.start_line,
end_line=piece.block.end_line,
total_lines=piece.block.total_lines,
text=piece.block.text,
truncated=piece.block.truncated,
locator=piece.block.locator,
).model_dump()
)
continue
```
(import `VerbatimBlockPiece` beside `ToolResultPiece` at L290; import `ChatVerbatimBlockEvent` beside the event imports at L310–322).
4. `tests/unit/test_agent.py` (verify the module name — the agent mock suite):
- `test_show_text_success_yields_one_verbatim_block_piece` — a mock turn whose answer calls `show_text` (the phase-182 mock precedent — verify the helper in the module) → exactly ONE `VerbatimBlockPiece`, positioned AFTER the round's `ToolResultPiece` (if a truncation piece exists) and BEFORE the next model round's first `StreamPiece`; the piece's `block` is the `holder.shown_texts` entry (same object);
- `test_show_text_refusals_yield_no_verbatim_block_piece` — the five refusal paths (the missing-arg `MISSING_SHOW_TEXT_ARGS`, the `answer_text=False` `UNKNOWN_TOOL`, the no-document refusal, the `is_image` refusal, the block-cap refusal, the already-shown dedupe — six pins, the module's refusal fixtures) → zero `VerbatimBlockPiece`s in the stream;
- `test_kill_switch_turn_yields_no_verbatim_block_piece` — the `answer_text=False` app posture → the tool is absent from the offered tools (the phase-182 D8 pin stays green) AND zero `VerbatimBlockPiece`s.
5. `tests/integration/test_chat_api.py` (verify the module name — the SSE contract suite):
- `test_verbatim_block_frame_lands_before_done` — a mock-LLM turn with a `show_text` call → the captured SSE stream carries a `type="verbatim_block"` frame with the EXACT `VerbatimBlockRef` field set (the pin asserts every key of `model_dump()`) positioned after the matching `tool_result`/`tool` frames and before `done`; the `done` frame's `answer_text` list is the same list the pre-phase resolution produced (the phase-182 pin, extended — the done-time resolution is untouched);
- `test_kill_switch_stream_is_byte_identical` — the `BOR_ANSWER_TEXT=0` module app: the captured frame sequence (types + payloads) equals the pre-phase expected sequence (the documented baseline in the test — a turn with the mock's `show_text` attempt gets the phase-182 refusal path; NO `verbatim_block` frame anywhere).
6. Verify: `uv run pytest tests/unit/test_agent.py tests/integration/test_chat_api.py -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the success-position pin, the six refusal pins, the kill-switch pin.
- Integration: the frame contract pin, the kill-switch wire pin, the done-frame parity pin.
- Coverage: **>90%** on `app/`.
## Completion Criteria
- [ ] a successful `show_text` yields exactly one `verbatim_block` frame (after `tool_result`, before the next round) — unit + integration pins.
- [ ] every refusal path yields zero frames; the `answer_text=False` stream is byte-identical to pre-phase (the wire pin).
- [ ] the done frame's `answer_text` and the `verbatim_blocks=N` log line are unchanged (the phase-182 pins green unedited).
- [ ] full test suite **100% passing**.
- [ ] no behavior change in completed work (a turn without `show_text` is byte-identical — the pre-existing SSE pins green).
@@ -0,0 +1,66 @@
# Task 02 — Mid-turn landing: the fence hydrates in place, the unfenced block lands at the call moment (A2)
**Phase:** `192_verbatim_inline_midturn` · **Source:** `TODO.md:6 — "… which makes the chat read strangely."`
**Story:** n/a
## Objective
The client half: a `verbatim_block` frame settles the pending fence placeholder IN PLACE (the matching semantics of the done path) or, when nothing matches, lands the panel at the answer position AT THE MOMENT the call succeeds — so the reading order is the model's order: "…here it is:" → panel → "…as you can see…". Unmatched placeholders STAY pending (settling is the done/failure path's job).
## Work
1. `frontend/assets/verbatim.js` — a NEW `hydrateVerbatimNow(root, block, opts)` (after `hydrateVerbatim`, L446):
```js
/* hydrateVerbatimNow(root, block, opts) — the MID-TURN pass (phase 192,
* A2): settle the PENDING placeholders THIS ONE block matches, leave
* every other placeholder pending.
*
* The matching is the done path's semantics (matchVerbatimBlock with a
* ONE-block list): the exact canonical locator, or the identity
* fallback when the block is the only one of that identity — a fence
* for range B of a shown file A hydrates with A's server extent (the
* caption shows the SERVER's range, the phase-182 contract), so the
* mid-turn pass and the done pass can never disagree.
*
* ONLY matched nodes are replaced (buildVerbatimPanel); the unmatched
* pending nodes STAY .verbatim-pending — settling them is the
* done/failure path's job (settleVerbatimPlaceholders), and a mid-turn
* pass must never burn a placeholder for a not-yet-shown fence.
*
* Returns the number of placeholders settled (the caller appends the
* block — appendUnlocatedVerbatim — only when it is 0). Never throws:
* a missing root → 0. */
```
- implementation: `const list = [block]` guarded by `isVerbatimBlock(block)` (a malformed frame → 0, the house never-throw contract); the same `querySelectorAll(".verbatim-pending[data-bor-locator]")` walk as `hydrateVerbatim`; `matchVerbatimBlock(list, key)`; replace only when matched; count the replacements.
2. `frontend/assets/app.js` — the new branch beside the `tool_result` branch (L4612):
```js
} else if (ev.type === "verbatim_block") {
// Phase 192 (task 02, A2): the show_text panel lands at the moment
// the call succeeds — the fence it names hydrates IN PLACE; an
// unfenced block lands at the answer position now, not at done.
// The done pass re-runs the same functions: hydrateVerbatim is
// idempotent (only .verbatim-pending is touched) and
// appendUnlocatedVerbatim skips an already-placed locator, so the
// done frame and this frame can never double a panel.
const n = hydrateVerbatimNow(
wrap.querySelector(".bubble"), ev, VERBATIM_LINK_OPTS
);
if (n === 0) appendUnlocatedVerbatim(wrap, [ev], VERBATIM_LINK_OPTS);
}
```
- the import line: `hydrateVerbatimNow` joins `hydrateVerbatim` / `appendUnlocatedVerbatim` in the existing verbatim.js import (verify the import site);
- the `answer_text=False` gate is SERVER-side only (the frame never arrives) — NO client-side settings check (a client gate would be a second, drift-prone gate — the docstring says so).
3. `tests/unit/` — the JS marker pins (the verbatim JS pin module — verify the name; the `test_save_chat_ui.py` convention):
- `hydrateVerbatimNow` exists with the matched-only contract (the marker names the `matchVerbatimBlock` reuse + the "unmatched stay pending" clause + the never-throw guard);
- the app.js handler references `verbatim_block` + `hydrateVerbatimNow` + `appendUnlocatedVerbatim` in the `n === 0` shape (the source marker — the append only on zero matches);
- the `isVerbatimBlock` guard marker (a malformed frame settles nothing).
4. Verify: `uv run pytest tests/unit/<js-pin-module> -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the marker pins above (the matched-only contract is the task's heart — the E2E proves the behavior, the markers prove the shape).
- E2E: task 04 (the owner-visible arm).
- Coverage: **>90%** on `app/` (unchanged — the task is frontend; the app-side gate stays green).
## Completion Criteria
- [ ] `hydrateVerbatimNow` settles ONLY the pending placeholders the single block matches; the rest stay `.verbatim-pending`; returns the match count; never throws (marker pins).
- [ ] the app.js `verbatim_block` branch hydrates in place and appends ONLY on zero matches (marker pins); no client-side kill-switch gate (the marker asserts its absence).
- [ ] the existing `hydrateVerbatim` / `appendUnlocatedVerbatim` / `settleVerbatimPlaceholders` functions are UNTOUCHED (the phase-182 pins green unedited).
- [ ] no behavior change in completed work (a turn without the frame type never enters the branch — the pre-existing E2E suites green).
@@ -0,0 +1,36 @@
# Task 03 — The done/restore dedupe and the unchanged-contract pins (A3/A4)
**Phase:** `192_verbatim_inline_midturn` · **Source:** `TODO.md:6 — "… makes the chat read strangely."`
**Story:** n/a
## Objective
Close the one remaining duplicate: a mid-turn APPENDED panel and a later fence for the SAME canonical locator must end as ONE panel (the in-text one). The dedupe lives in `hydrateVerbatim` (the done AND restore path — one function, so the shared page rides it for free); per-locator, never per-identity — two DIFFERENT ranges of one file are legitimate and stay.
## Work
1. `frontend/assets/verbatim.js` — `hydrateVerbatim` (L423–445) gains the placed-scan, inside the per-node loop:
- before `node.replaceWith(buildVerbatimPanel(block, opts))` for a MATCHED key: `const wrap = root.closest(".msg") || root;` (the message wrap — the same host `appendUnlocatedVerbatim` scans); remove every PLACED resolved panel (`.verbatim[data-bor-locator]`, NOT `.verbatim-unresolved` — the `appendUnlocatedVerbatim` placed-scan convention, L463–467) whose `data-bor-locator` equals the key (the canonical-locator comparison — the exact `===`, the phase-182 D7 key), then build the in-text panel in the node's place;
- UNMATCHED nodes: UNCHANGED (still settle to `buildVerbatimUnresolved`);
- the docstring gains the A3 paragraph: the mid-turn appended panel is superseded by the in-text fence panel (the fence is where the model wanted it); the scan is PER LOCATOR — two different ranges of one file (the `shown_texts` per-locator dedupe admits both) are both legitimate placements and BOTH survive (the scan never compares identities);
- `appendUnlocatedVerbatim` is UNTOUCHED (its state-free idempotence — the `placed` set skip, L468–472 — already makes the done pass a no-op for an appended block).
2. `tests/unit/` — the JS marker pins (the verbatim JS pin module — the task-02 module):
- the `hydrateVerbatim` placed-scan marker: the scan targets `.verbatim[data-bor-locator]` (the class + attribute pair), EXCLUDES `.verbatim-unresolved`, removes on the exact `data-bor-locator` key (the three markers — the per-locator contract), and the "two ranges stay" clause is named in the docstring (the marker asserts the docstring contains the per-locator-not-per-identity sentence — the house docstring-pinning convention);
- the `root.closest(".msg")` wrap marker.
3. The unchanged-contract pins (A4) — the existing suites stay green UNEDITED, and three NEW integration/E2E-preparation pins:
- `tests/integration/test_chat_api.py` — `test_done_frame_and_log_line_unchanged_by_midturn_frames` — a mock turn WITH a `show_text` call: the `done` frame's `answer_text` list equals the phase-182 resolution (the task-01 pin, now extended with the per-turn log line assertion: `verbatim_blocks=N` where `N` = the done-frame block count, the mid-turn frames NOT counted — the captured log line pin, the house log-pin convention).
- `tests/unit/test_schemas.py` (or the phase-182 pin module — verify): `ChatVerbatimBlockEvent`'s `model_dump()` key set == `VerbatimBlockRef`'s key set + `type` (the `isVerbatimBlock` compatibility pin — the client check's required keys are present).
- the restore-parity marker (the app.js restore path L3044–3046 UNTOUCHED — a marker asserting the restore path still calls `hydrateVerbatim` + `appendUnlocatedVerbatim` with `m.answer_text` — the shared page's hydration call site likewise untouched; the BEHAVIOR pin is the E2E arm).
4. Verify: `uv run pytest tests/unit/<js-pin-module> tests/integration/test_chat_api.py -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the placed-scan markers (per-locator, unresolved-excluded, wrap derivation, the two-ranges-stay clause).
- Integration: the done-frame + log-line parity pin; the schema compatibility pin.
- E2E: task 04 (the no-double arm, the two-ranges arm, the restore arm).
- Coverage: **>90%** on `app/` (unchanged).
## Completion Criteria
- [ ] a mid-turn appended panel + a same-locator fence → exactly ONE panel after `done` (the marker pins now; the E2E proves it) — the in-text panel wins, the placed panel removed.
- [ ] two different ranges of one file → two panels after `done` (the marker clause + the E2E arm).
- [ ] the restore path and the shared page's call sites are untouched (marker pins); the `done` frame's `answer_text` and the `verbatim_blocks=N` log line are unchanged (integration pins).
- [ ] `appendUnlocatedVerbatim` / `settleVerbatimPlaceholders` / the task-02 functions are untouched (the phase-182 pins green unedited).
- [ ] full test suite **100% passing**.
- [ ] no behavior change in completed work.
@@ -0,0 +1,36 @@
# Task 04 — The phase E2E suite, the full gates, the docs pass, the commit
**Phase:** `192_verbatim_inline_midturn` · **Source:** `TODO.md:6`
**Story:** n/a
## Objective
The dedicated Playwright suite proves the owner-visible contract — the panel is on screen WHILE the answer is still streaming, the fence lands in place, nothing doubles, the kill switch is a byte-identical wire — run in isolation per AGENTS rule 4; the full gates pass; the docs are consistent; one atomic commit.
## Work
1. `tests/e2e/test_verbatim_inline_midturn.py` (NEW — the module docstring names the TODO item and the A4 unchanged-contract clause) — the module-app fixture pattern (the phase-182 E2E convention — verify in `tests/e2e/conftest.py` / the sibling verbatim suites; the mock LLM from `tests/e2e/mock_llm.py` — verify the phase-182 mock's `show_text` arm; if the mock drives `show_text` via a fence-only turn, extend the mock with a direct `show_text` tool call arm in `tests/e2e/mock_llm.py` — the mock is a test fixture, the extension is additive and the phase-182 pins must stay green):
- the fixture: a small KB with one document whose content the mock's `show_text` call targets (the phase-182 fixture convention).
- **(a) the mid-turn arm (the TODO regression):** a mock turn: an intro delta ("…here is the exact text:") → the `show_text` call (the mock arm) → the continuation deltas → `done`. Assert: the panel (`.verbatim[data-bor-locator]` with the call's locator — caption + gutter + body) is in the DOM BEFORE the `done` frame is consumed (the Playwright shape: `page.wait_for_selector` on the panel resolves while the stream is still running — a sentinel delta after the call proves the stream is live; the panel's DOM position is BELOW the intro text and ABOVE the final sentinel delta — the inline position assertion, the `answer_image`-position contract of `appendUnlocatedVerbatim`); the `done` pass adds nothing (the final panel count == 1).
- **(b) the fence-in-place arm:** a mock turn: an intro delta → the `show_text` call for `source/path#12-20` → the final answer containing the ` ```bor ` fence for the SAME locator → `done`. Assert: the fence placeholder is hydrated IN PLACE (the panel is at the fence position — between the intro and the final sentinel delta, NOT at the answer tail) before `done` (the mid-turn `hydrateVerbatimNow` pass); after `done`, the total panel count for that locator == 1 (the dedupe — the task-03 arm).
- **(c) the two-ranges arm:** a mock turn with TWO `show_text` calls — range 1–10 and range 11–20 of the same document (the `shown_texts` per-locator dedupe admits both) — no fences. Assert: exactly TWO panels after `done`, both at the answer position in call order, both with their own captions (the per-locator-not-per-identity pin, owner-visible).
- **(d) the kill-switch wire arm:** the `BOR_ANSWER_TEXT=0` module app — the same mock turn (the mock's `show_text` attempt hits the phase-182 refusal) → the captured SSE frame sequence (types + payloads via the house SSE-capture pattern) equals the pre-phase baseline (documented in the test); NO `verbatim_block` frame, NO panel, NO placeholder in the DOM.
- **(e) the restore arm:** run arm (a) to completion (a saved conversation — the phase-158 auto-save), reload the page, restore → exactly ONE panel for the locator (the record's `answer_text` through the same dedupe functions — the A4 parity pin, owner-visible).
2. Regression, run in isolation: the phase-182 E2E suites (verify the exact names in `tests/e2e/` — the verbatim display suite, the fence suite), `tests/e2e/test_chat_history.py` (the restore contract) — green unedited.
3. The full gates, in order:
- `uv run pytest` (unit + integration) — 100% passing;
- `uv run pytest --cov=app --cov-report=term-missing` — **>90%** on `app/`;
- `uv run pytest tests/e2e/test_verbatim_inline_midturn.py -v --no-cov` — green in isolation;
- `uv run ruff check . && uv run pyright` — clean.
4. Documentation pass (the phase file's Documentation section): `README.md` + `AGENTS.md` reviewed in their entirety against the post-phase app; the verbatim section notes the mid-turn landing (a `show_text` panel appears the moment the call succeeds — in place when fenced, at the answer position when not; the `done`/restore/shared contract is unchanged, the kill switch byte-identical); `AGENTS.md` corrected only where it names the done-frame resolution as the ONLY landing point.
5. Commit: one atomic, professional commit (Conventional Commits, `--no-gpg-sign`), e.g.
`feat(verbatim): show_text panels land mid-turn, at the moment the call succeeds (phase 192)` — the body names the additive `verbatim_block` frame, the `hydrateVerbatimNow` matched-only pass, the per-locator dedupe, and the A4 unchanged contract. `.agents/phases/todo/192_verbatim_inline_midturn/` moves to `.agents/phases/complete/` in the SAME commit (AGENTS rule 8).
## Testing & Quality
- E2E: the suite above, isolated; the regression suites green.
- Coverage: **>90%** on `app/`.
- Lint/types: clean.
## Completion Criteria
- [ ] `tests/e2e/test_verbatim_inline_midturn.py` green in isolation — (a) the panel is on screen before `done`, at the inline position; (b) the same-locator fence hydrates in place mid-turn and never doubles; (c) two ranges of one file → two panels; (d) the `answer_text=False` wire is byte-identical with no panel/placeholder; (e) the restore shows one panel per locator.
- [ ] `uv run pytest` 100% passing; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] `README.md` / `AGENTS.md` consistent (the mid-turn landing clause).
- [ ] the phase directory is in `.agents/phases/complete/`; one atomic commit with `--no-gpg-sign`.
@@ -0,0 +1,55 @@
# Phase 193 — The sync label counts DOWN: "N remaining"
**Source:** `TODO.md` L6 — "The sync status should show how many files are left to sync, not how many files out of the total."
**Story:** n/a (owner directive)
**Depends on:** none functionally (builds on the completed phase-64/98/128/129/152 sync-label lineage).
## Evidence (diagnosed from the code, read this not the chat)
1. **The running label counts UP.** `frontend/assets/sources.js` `fmtSyncLabel` (L1053–1058): the import phase and the upload job render `count = total > 0 ? `(${done}/${total})` : ""` — `Syncing… notes/a.md (1234/27000)`. The owner wants the distance to the end, not the position in the whole: `25766 remaining`.
2. **The data already exists.** `SyncStatus.files_done` / `files_total` (the phase-64 payload — the running label's only inputs) carry everything: `N = total − done` is a subtraction, no server change, no new field, no new poll. The `#sync-progress` line (the phase-152 A4: the untruncated full label rides the button title + `#sync-progress` via `setSyncLabel`) gets the new part through the SAME builder — nothing to change there.
3. **The other count parts are different animals — they stay.** `· chunk n/m` (phase 129 — one file's embed batch), `· image n/m` (phase 152 — one PDF's vision span), and the summaries phase's `(n/m)` (phase 98 — the FOLDER position, always emitted, `fmtSyncPhaseLabel` L1089+) are per-file / per-folder position parts, not "files of the total". The terminal result line (`counts_line` — "N added · M updated …") is a different surface and is untouched.
## Design (shared by all tasks — the executor reads this, not the chat)
- **D1 — the part (task 01).** `fmtSyncLabel`'s `count` becomes `total > 0 && total - done > 0 ? `${total - done} remaining` : ""` — the owner-confirmed shape (ASSUMPTION A1, confirmed 2026-10-18):
- `N remaining` shows while `N ≥ 1`; at the final file (`N == 0`) the part is HIDDEN — the main part names the file in flight, and a "0 remaining" next to the last file's name is the stale-looking edge the owner's complaint is about;
- plain integer (no thousands separator — the existing count part is plain; the house label style);
- the POSITION is unchanged (main → count → chunk → image — the phase-128 parts rework order);
- the UPLOAD job shares the builder and the format (owner-confirmed A2 — `Importing <file> N remaining`; the upload's `files_total` is the pre-walk count, the phase-98 contract — the same subtraction semantics);
- `total === 0` (the pre-walk null total, the phase-64 contract) → no count part (unchanged);
- the chunk / image parts and the summaries-phase folder position are UNTOUCHED (byte-identical);
- the docstring (L1044–1058 region) is updated to the new part (the phase-64/98 lineage paragraph keeps its history, the `(done/total)` description becomes the `N remaining` one).
- **D2 — the pinned suites move (task 02).** The running-label count pins retarget to the new part; the terminal-line pins stay unedited. The definitive list is found by `grep -rn "Syncing…\|Importing" tests/e2e/*.py | grep -E "\\\\d|/\{N|[0-9]+\)"` at execution time (the known pins: `test_sync_label_mobile.py` `IMPORT_FULL_RE` L226 + the docstring L93/L213; `test_sync_upload_progress.py` L653/L667/L850).
## ASSUMPTIONS (owner-confirmed 2026-10-18)
- **A1:** `N remaining` while `N ≥ 1`, hidden at `N == 0` (the final file — the main part names it).
- **A2:** the upload job shares the format (one builder, one shape).
## Context
- `frontend/assets/sources.js` — `fmtSyncLabel` (L1053–1058), `fmtSyncPhaseLabel` (L1089+, the phase-aware wrapper: overview / summaries / import), `setSyncLabel` (the A4 full-label placement — the button title + `#sync-progress`), the `counts_line` helper (the terminal result line — UNTOUCHED); `app/schemas.py` `SyncStatus` (`files_done` / `files_total` — the payload, UNTOUCHED); `app/api/sync.py` (the status endpoint — UNTOUCHED).
- `tests/e2e/test_sync_label_mobile.py`, `tests/e2e/test_sync_upload_progress.py`, `tests/e2e/test_sync_chunk_progress.py`, `tests/e2e/test_sync_summary_visibility.py` (the phase-64/98/128/129/152 label suites — the count-part pins; the chunk/image/summaries pins stay), `tests/e2e/test_sync_git_freeze.py` + `test_sync_session_reset.py` + `test_fast_path_deferred_content.py` (the `counts_line` terminal pins — unedited).
## Tasks
1. `01_remaining_label.md` — the `fmtSyncLabel` count part becomes `N remaining` (the D1 spec), the docstring update, the JS marker pin.
2. `02_update_pinned_e2e.md` — the running-label count pins retarget to the new part (the D2 list); the full suite green.
3. `03_e2e_and_gates.md` — the phase's own suite (the count-down through the real UI), the full gates, the docs pass, the atomic commit.
## Testing & Quality
- **Unit (JS marker pin):** the new count expression (`total - done > 0` guard, the plain-integer shape, the `total > 0` guard kept, the upload kind sharing the builder — the marker names the expression).
- **E2E (regression, task 02):** the retargeted count pins + the chunk / image / summaries pins green; the terminal `counts_line` pins green UNEDITED.
- **E2E (phase, AGENTS rule 4, task 03):** `tests/e2e/test_sync_remaining_count.py`, run in isolation — the count-down arm (the part decreases across polls; hidden at the final file; the chunk part coexists), the upload arm (the shared format).
- **Coverage:** **>90%** on `app/` (unchanged — the task is frontend; the app-side gate stays green).
- **Lint/types:** `uv run ruff check . && uv run pyright`.
## Documentation & Consistency
- Final pass reviews `README.md` and `AGENTS.md` in their entirety against the post-phase app and fixes every inconsistency found, pre-existing ones included.
- `README.md`: the sync section's running-label description (the `(done/total)` example, if present) becomes the `N remaining` one; `AGENTS.md` — the phase-64 sync-label clause (if it names the count part) gains the new shape (task 03 — verify, do not add if absent).
## Completion Criteria
- [ ] the running file-count part is `N remaining` (`N = total − done`, plain, shown while `N ≥ 1`, hidden at `N == 0`, `total === 0` → no part) — the marker pin + the E2E count-down arm.
- [ ] the upload label uses the same shape (the E2E upload arm); the chunk / image / summaries parts are byte-identical (the unedited pins green).
- [ ] the terminal result line is byte-identical (the `counts_line` pins green unedited); the `#sync-progress` line carries the new part through the same builder (the A4 pin).
- [ ] test suite green, coverage >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] no behavior change in completed phases (the server, the payload, the phases of the run all untouched — the change is the one label part).
- [ ] one atomic Conventional Commits commit with `--no-gpg-sign`.
@@ -0,0 +1,31 @@
# Task 01 — The `N remaining` count part (D1)
**Phase:** `193_sync_remaining_count` · **Source:** `TODO.md:7 — "The sync status should show how many files are left to sync, not how many files out of the total."`
**Story:** n/a
## Objective
The one-line change the owner asked for: `fmtSyncLabel`'s running file-count part renders the distance to the end (`N remaining`, `N = total − done`) instead of the position in the whole (`(done/total)`) — sync and upload alike (one builder), hidden at the final file, the pre-walk `total === 0` behavior unchanged, the chunk / image / summaries parts untouched.
## Work
1. `frontend/assets/sources.js` — `fmtSyncLabel` (L1053–1058):
- the `count` expression: `const count = total > 0 && total - done > 0 ? `${total - done} remaining` : "";` (the `total > 0` guard kept — the phase-64 pre-walk contract: no count part before the walk knows the total; the plain integer — no thousands separator, the existing part's style; the POSITION in the returned object is unchanged — the phase-128 parts order main → count → chunk → image is preserved by `fmtSyncPhaseLabel` which consumes `count` unchanged);
- the docstring block above the function (L1044–1052) + the `fmtSyncPhaseLabel` docstring (L1060–1088): the `(done/total)` description becomes the `N remaining` one — the phase-64/98 lineage KEPT (the "count part" history), the new clause: `N = total − done`, shown while `N ≥ 1`, hidden at `N == 0` (the final file — the main part names it; a "0 remaining" next to the last file's name is the stale-looking edge), the upload job shares the builder and the shape (the phase-98 D2 "upload keeps the bare fmtSyncLabel parts" clause updated: still bare, now with the `N remaining` part);
- `setSyncLabel` / `#sync-progress` placement: UNTOUCHED (the A4 full label rides the same builder — the new part flows through automatically);
- the terminal `counts_line` helper: UNTOUCHED.
2. `tests/unit/` — the JS marker pin (the sources-JS pin module — verify the name; the `test_save_chat_ui.py` convention):
- the `fmtSyncLabel` count expression marker (the `total > 0 && total - done > 0` guard + the `` `${total - done} remaining` `` shape — the source marker asserts the exact expression);
- the `total === 0` guard marker (no count part before the walk);
- the "hidden at N == 0" clause marker (the docstring names the final-file edge);
- the chunk / image / `fmtSyncPhaseLabel` parts UNTOUCHED marker (the three part builders' source is unchanged — the `· chunk n/m` / `· image n/m` / summaries `(n/m)` strings appear only in their own builders, not in `fmtSyncLabel`).
3. Verify: `uv run pytest tests/unit/<sources-pin-module> -v` green; `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the marker pins above (the expression is the whole change — the markers pin the shape).
- E2E: task 02 (the retargeted pins) + task 03 (the phase arm).
- Coverage: **>90%** on `app/` (unchanged — the task is frontend).
## Completion Criteria
- [ ] `fmtSyncLabel`'s count part is `` `${total - done} remaining` `` with the two guards (the marker pins).
- [ ] the docstrings name the new part + the hidden-at-final-file edge + the upload sharing (the marker pins).
- [ ] the chunk / image / summaries / terminal-line code is byte-identical (the unedited pins green).
- [ ] no behavior change in completed work (the server and the payload are untouched).
@@ -0,0 +1,25 @@
# Task 02 — The pinned running-label count pins move to `N remaining` (D2)
**Phase:** `193_sync_remaining_count` · **Source:** `TODO.md:7`
**Story:** n/a
## Objective
The full suite stays green: every E2E pin that asserts the running file-count part retargets to the new shape; the chunk / image / summaries pins and the terminal `counts_line` pins stay UNEDITED.
## Work
1. The definitive pin list — run at execution time: `grep -rn "Syncing…\|Importing" tests/e2e/*.py | grep -E "\\\\d|/\{N|[0-9]+\)"` — the known pins (verify each against the list):
- `tests/e2e/test_sync_label_mobile.py` — `IMPORT_FULL_RE` (L226): `^Syncing… {FILE_RE} \([1-4]/4\)(?: · chunk \d+/\d+)?$` → `^Syncing… {FILE_RE} [1-4] remaining(?: · chunk \d+/\d+)?$` (a 4-file fixture: while file `i` is in flight `done = i − 1`, so `N = 4 − done ∈ {1, 2, 3, 4}` — `N == 0` never renders, the part is `[1-4] remaining`; the docstring lines L93/L213 updated to the new shape — the `IMPORT_MAIN_RE` (L217, the no-count mobile clip) is UNTOUCHED);
- `tests/e2e/test_sync_upload_progress.py` — L653, L667, L850: `\(\d+/{N_FILES}\)` → `[0-9]+ remaining` (the same module's `N_FILES` fixture; the L667 arm keeps the ` · chunk \d+/\d+` tail; the L653/L850 arms are the bare label).
2. Any ADDITIONAL pin the grep surfaces (a label regex with a count part in another suite — the phase-128/129/152 lineage: `test_sync_chunk_progress.py`, `test_sync_summary_visibility.py`, `test_sync_git_freeze.py`, `test_sync_session_reset.py` are the candidates — verify each: a pin on the RUNNING label's count part → retargeted; a pin on the CHUNK/IMAGE part, the summaries folder position, or the `counts_line` terminal line → UNTOUCHED).
3. The module docstrings of the retargeted suites: the shape description updated (the `(n/m)` running-count clause → the `N remaining` clause, the phase-64/98 lineage KEPT, one sentence: phase 193 moved the part to the distance-to-end per the owner's directive).
4. Verify: `uv run pytest tests/e2e/test_sync_label_mobile.py tests/e2e/test_sync_upload_progress.py -v --no-cov` green (plus every additional retargeted suite); `uv run pytest tests/e2e/test_sync_chunk_progress.py tests/e2e/test_sync_summary_visibility.py tests/e2e/test_sync_git_freeze.py tests/e2e/test_sync_session_reset.py -v --no-cov` green (the unedited chunk/image/terminal pins); `uv run pytest` (unit + integration) green.
## Testing & Quality
- E2E (regression): the retargeted count pins green; the chunk / image / summaries / terminal pins green unedited.
- Coverage: **>90%** on `app/` (unchanged).
## Completion Criteria
- [ ] every running-label count pin in `tests/e2e/` asserts the `N remaining` shape (the grep list is closed — every hit accounted for: retargeted or documented-untouched).
- [ ] the terminal `counts_line` pins and the chunk / image / summaries pins are green UNEDITED.
- [ ] `uv run pytest` 100% passing (the full suite — no pin left asserting the old shape).
- [ ] no behavior change in completed work (test-only + the task-01 label part).
@@ -0,0 +1,34 @@
# Task 03 — The phase E2E suite, the full gates, the docs pass, the commit
**Phase:** `193_sync_remaining_count` · **Source:** `TODO.md:7`
**Story:** n/a
## Objective
The dedicated Playwright suite proves the owner-visible contract — the label counts DOWN as the run progresses and goes quiet at the final file — run in isolation per AGENTS rule 4; the full gates pass; the docs are consistent; one atomic commit.
## Work
1. `tests/e2e/test_sync_remaining_count.py` (NEW — the module docstring names the TODO item, the A1/A2 assumptions and the "server untouched" clause) — the module-app fixture pattern (the phase-64/98 E2E convention — verify in `tests/e2e/conftest.py` / the sibling sync suites):
- the fixture: a small git source with N files (N ≥ 4, the phase-98 fixture convention — the `N_FILES` shape of `test_sync_upload_progress.py`).
- **(a) the count-down arm:** start the sync through the RAG page's sync flow; capture `#sync-progress` (and the button title — the A4 full-label placement) across the run: while files are in flight the label matches `^Syncing… <file> [1-N] remaining` and the displayed `N` is NON-INCREASING across polls (the `total − done` shape — a strict count-down; the pin: the sequence of captured `N`s, when a new value appears, is smaller); the `· chunk n/m` part coexists when a batch is in flight (the phase-129 part, unbroken); when the walk reaches the final file the part is ABSENT (the main part names the file — `^Syncing… <last-file>$`, no count part — the A1 hidden edge); the run's terminal result line is the phase-64 shape (the `counts_line` helper — byte-identical, the terminal-line pin).
- **(b) the upload arm (A2):** the upload job (the same fixture as an archive, the `test_sync_upload_progress.py` convention) → the label shows `Importing <file> [1-N] remaining` (the shared builder, the shared shape — the pin) and the chunk part coexists (the pin).
- **(c) the pre-walk edge:** the first poll before the walk knows the total → NO count part (`^Syncing…$`, the phase-64 contract — the `total === 0` guard, owner-visible).
2. Regression, run in isolation: `tests/e2e/test_sync_label_mobile.py`, `tests/e2e/test_sync_upload_progress.py` (the task-02 retargets), `tests/e2e/test_sync_chunk_progress.py`, `tests/e2e/test_sync_summary_visibility.py` (the chunk/image/summaries pins) — green.
3. The full gates, in order:
- `uv run pytest` (unit + integration) — 100% passing;
- `uv run pytest --cov=app --cov-report=term-missing` — **>90%** on `app/`;
- `uv run pytest tests/e2e/test_sync_remaining_count.py -v --no-cov` — green in isolation;
- `uv run ruff check . && uv run pyright` — clean.
4. Documentation pass (the phase file's Documentation section): `README.md` + `AGENTS.md` reviewed in their entirety against the post-phase app; the sync section's running-label example (the `(done/total)` shape, if present in `README.md`) becomes the `N remaining` one; `AGENTS.md` corrected only where a phase-64/98 clause names the running count part.
5. Commit: one atomic, professional commit (Conventional Commits, `--no-gpg-sign`), e.g.
`feat(sync): the running label counts down — "N remaining" (phase 193)` — the body names the one-line `fmtSyncLabel` change (sync + upload through the shared builder), the hidden-at-final-file edge, and the untouched chunk/image/summaries/terminal parts. `.agents/phases/todo/193_sync_remaining_count/` moves to `.agents/phases/complete/` in the SAME commit (AGENTS rule 8).
## Testing & Quality
- E2E: the suite above, isolated; the regression suites green.
- Coverage: **>90%** on `app/`.
- Lint/types: clean.
## Completion Criteria
- [ ] `tests/e2e/test_sync_remaining_count.py` green in isolation — (a) the count-down (non-increasing `N` across polls, the chunk part coexisting, the part absent at the final file, the terminal line byte-identical); (b) the upload arm (`Importing <file> N remaining`); (c) the pre-walk edge (no count part before the total is known).
- [ ] `uv run pytest` 100% passing; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] `README.md` / `AGENTS.md` consistent (the running-label shape).
- [ ] the phase directory is in `.agents/phases/complete/`; one atomic commit with `--no-gpg-sign`.
-2
View File
@@ -1,3 +1 @@
# TODO
- [ ] Changing the App name in settings broke, it doesn't update the name now.