Compare commits

..
2 Commits
Author SHA1 Message Date
ducoterra 17dd3bfac1 docs(plan): remove hardcoded phase/migration references to prevent staleness
Build and Push Containers / build-and-push-app (push) Successful in 14s
Build and Push Containers / build-and-push-db (push) Successful in 16s
Refactor §12 roadmap to point to ls commands and 00_phase.md files
instead of listing specific phase numbers and descriptions.

Remove ~20 phase-number references from anchors, revision notes,
tool surface, feedback, and sync sections — keep only the one
stable reference (phase 03 convention).

Replace migration ranges with 'ls alembic/versions/'.
Replace hardcoded retry values with 'configurable (defaults: ...)'.
Status header now points to §12 instead of listing counts.
2026-09-12 10:22:45 -04:00
ducoterra 800bea769a docs(phases): author todo queue phases 98-102
98 sync summary visibility (status phases + pending markers), 99 catalog
one-line clamp + breadcrumb back-nav, 100 72rem page-width consistency,
101 tokens page overhaul, 102 extensionless filename import.
2026-09-12 10:15:22 -04:00
24 changed files with 953 additions and 50 deletions
+41 -50
View File
@@ -1,7 +1,7 @@
# Brain of Reese — Master Plan (minimal)
> **Status:** Minimal re-land (2026-09-10). Phases 01–92 shipped;
> `todo/` holds 93–95 (see §12). Section numbers and anchor IDs match the
> **Status:** Minimal re-land (2026-09-10). See §12 for current state.
> Section numbers and anchor IDs match the
> previous full plan (in git history, commit `dac4a3e`) so the codebase's
> `PLAN §…` / anchor comments stay valid — consult that revision or the
> phase records in `.agents/phases/complete/` for full detail and the
@@ -43,17 +43,17 @@ accounts (one admin + hand-out tokens is the model).
| A2 | Web framework | FastAPI + Pydantic v2 + Uvicorn (async, SSE-friendly) |
| A3 | Database | **PostgreSQL 17 + pgvector** (local build of official `postgres:17`, `db/Containerfile`), cosine (`<=>`) search; one system for relational + vectors |
| A4 | Orchestration | `compose.yaml`, `podman compose up -d` (dev: `db` only; `--profile prod` adds the app container) |
| A5 | LLM backend | OpenAI-compatible self-hosted endpoint `https://aipi.reeseapps.com/v1` via the `openai` async client: **`turbo`** (chat, streams `reasoning_content` thinking), **`embed`** (embeddings), **`lite`** (one-shot: document summaries, KB overview; phase 94 adds sync-time folder summaries) |
| A5 | LLM backend | OpenAI-compatible self-hosted endpoint `https://aipi.reeseapps.com/v1` via the `openai` async client: **`turbo`** (chat, streams `reasoning_content` thinking), **`embed`** (embeddings), **`lite`** (one-shot: document summaries, KB overview, folder summaries) |
| A6 | Embedding dim | **768** (verified against the live endpoint); `chunks.embedding` is fixed at table creation — a dim mismatch must **fail loudly**, never silently re-embed |
| A7 | Retrieval→context | **Hybrid:** cosine top-100 ∪ Postgres FTS top-30 (OR tsquery, `ts_rank`), fused with **RRF (k=60)** → parent docs ranked by best fused chunk → **full text of top-N=2 documents, never truncated** on the retrieval path (owner 2026-08-24: "this should never happen"; the agent `read`-tool cap is a separate owner-permitted path — phase 95) |
| A7 | Retrieval→context | **Hybrid:** cosine top-100 ∪ Postgres FTS top-30 (OR tsquery, `ts_rank`), fused with **RRF (k=60)** → parent docs ranked by best fused chunk → **full text of top-N=2 documents, never truncated** on the retrieval path (owner 2026-08-24: "this should never happen"; the agent `read`-tool cap is a separate owner-permitted path) |
| A8 | Honesty gate | Deflect (LOW mode) **only when** best cosine < `BOR_RELEVANCE_THRESHOLD` (0.62) **and** zero FTS hits; LOW prompt carries weak-hit *titles only* + the `DEFLECT_MODE` marker (the E2E mock keys on its presence) + the plain-text no-tools line |
| A9 | Content scope | Default `md, markdown, txt, yaml, yml, json, py` + Podman quadlet family + `j2`; `BOR_IMPORT_EXTENSIONS` may name **any** well-formed extension or narrow the set; hidden (dot) paths + exclusion list + per-source `ignore_paths` (raw prefixes, no globs) always apply |
| A10 | State & auth | `POST /api/chat` is **stateless** (client-provided `history` only, budget-trimmed — nothing stored per conversation). Auth = single admin, signed `bor_session` cookie (Starlette `SessionMiddleware` + itsdangerous; no server-side session store); admin-issued SHA-256-hashed access tokens are the only other identity; **only** shared chats stay anonymous. Fail-loud at boot while `BOR_ADMIN_PASSWORD`/`BOR_SESSION_SECRET` are empty |
| A11 | Frontend | Vanilla HTML/CSS/JS in git, **no CDN** — everything served by FastAPI `StaticFiles`; system font stack; the navbar views are views of ONE shell document (`frontend/index.html` + `router.js` deep-links, phase 76) |
| A11 | Frontend | Vanilla HTML/CSS/JS in git, **no CDN** — everything served by FastAPI `StaticFiles`; system font stack; the navbar views are views of ONE shell document (`frontend/index.html` + `router.js` deep-links) |
| A12 | Aux services | **None** (no Valkey/queue/SeaweedFS): sync runs in-process, the login rate limit is in-memory per-process, sessions are the signed cookie. Restart clearing in-memory state is accepted |
| A13 | Migrations | Alembic + SQLAlchemy 2.0 (sync) + psycopg 3; **every migration ships a tested downgrade** (A13 — reversible) |
| A14 | Debugging | `debugpy` imported **only when `DEBUGPY=1`** (`app/core/debugging.py`); never imported otherwise (unit-tested) |
| A15 | Chat transport | **SSE** from `POST /api/chat`; event types `thinking`, `delta`, `tool`, `retry`, `done`, `error` (+ `tool_result`, owner-permitted extension — phase 95, not yet landed); no proxy buffering, no client caching on the stream |
| A15 | Chat transport | **SSE** from `POST /api/chat`; event types `thinking`, `delta`, `tool`, `retry`, `done`, `error`, `tool_result`; no proxy buffering, no client caching on the stream |
| A16 | Testing | Per phase: unit + integration (pytest) with **>90% coverage on `app/`** + one dedicated **Playwright E2E file**, run **in isolation** (`--no-cov`); E2E uses a deterministic mock LLM by default (`E2E_REAL_LLM=1` opts into live aipi) |
| A17 | Git | Conventional Commits, **always `--no-gpg-sign`**, one atomic commit per completed phase |
| A18 | Docs push | Save-a-answer-as-doc: server-side draft (`doc_drafts`, long body never in a URL — unguessable `uuid4` token is the URL credential) pushed to a **generic git remote** (`BOR_DOCS_REPO`, URL or local path; no `gh`) on a dedicated branch, `--ff-only`, re-cut per push; **no PR tooling**; inert (hidden + 409) while the repo is unset |
@@ -65,17 +65,17 @@ accounts (one admin + hand-out tokens is the model).
| # | Decision |
|---|----------|
| B1 | Theme `ui_settings` columns: **NULL/empty = "use the default"** — env value for the 3 strings, built-in palette for colors (colors have no env fallback) |
| B3 | **Revised (owner 2026-09-10, phase 93):** the semantic state families (`--ok-*`, `--err-*`, `--accent-*`) **join** the storable palette — 17 variables total, all `NULL = built-in` (the built-in theme stays byte-identical). The 2026-09-09 lock that they are "not identity" is lifted |
| B3 | The semantic state families (`--ok-*`, `--err-*`, `--accent-*`) **join** the storable palette — 17 variables total, all `NULL = built-in` (the built-in theme stays byte-identical). The 2026-09-09 lock that they are "not identity" is lifted |
| B4 | **Byte-identical contract:** with no `ui_settings` row (or a fully default theme) the served HTML is byte-identical to the built-in default — no `#bor-theme` tag, caching rewrites are rewrite-only |
| B5 | Admin-only nav views (Sources, Git sources, Tuning, History, Tokens, Theme) are hidden from non-admins; a monochrome theme must keep every state **text** label ("text + color, never color alone") |
**Key recent revisions (full log in the previous plan revision / phase
records):** A10 — single-admin cookie auth (16), saved chats + shares +
staleness (50/51/53), client history budgets (74), access tokens (79).
A15 — `thinking` (17), `tool` (37), unlimited tool rounds under
`BOR_AGENT_MAX_ROUNDS` (45), `retry` (67), harness-aligned
`ls`/`read`/`grep` surface (70). A7 — the **retrieval** path's
never-truncated contract is unchanged; the phase-95 `read`-tool cap is
records):** A10 — single-admin cookie auth, saved chats + shares +
staleness, client history budgets, access tokens.
A15 — `thinking`, `tool`, unlimited tool rounds under
`BOR_AGENT_MAX_ROUNDS`, `retry`, harness-aligned
`ls`/`read`/`grep` surface. A7 — the **retrieval** path's
never-truncated contract is unchanged; the `read`-tool cap is
the only exception, owner-permitted 2026-09-10.
---
@@ -111,7 +111,7 @@ BOR_GIT_SOURCES is the empty-table git-only fallback):
| API routers | `app/api/` |
| Import / sync tooling | `scripts/import_docs.py`, `app/api/sync.py`, `scripts/git_sync.py`, `scripts/eval_retrieval.py`, `scripts/llm_probe.py` |
| Frontend (shell + standalone pages) | `frontend/` (`index.html` shell; `document.html`, `shared.html`, `login.html`, `doc-edit.html`) |
| Migrations (0001–0015, all reversible) | `alembic/versions/` |
| Migrations (all reversible) | `alembic/versions/` |
---
@@ -119,7 +119,7 @@ BOR_GIT_SOURCES is the empty-table git-only fallback):
```
POST /api/chat {message, history?} (auth: require_user)
→ embed(question) [retried, phase 67 / A15 ext.]
→ embed(question) [retried, A15 ext.]
→ cosine top-100 ∪ FTS top-30 → RRF fuse (k=60) [A7]
├─ HIGH (cosine ≥ 0.62 OR fts_hits > 0):
│ persona + <knowledge_base> + <tuning> + full top-2 <documents>
@@ -136,9 +136,9 @@ restarts) → `delta` (answer tokens) → `done` `{deflected, sources[],
suggestions[]}` (terminal). Failure: `error` (terminal — no `done`, no
`query_log` row; a pre-stream DB outage is a plain 503 JSON).
`BOR_STREAM_THINKING=0` suppresses `thinking` frames server-side (chars
still counted). LLM retries (phase 67, A15 extension): `BOR_LLM_RETRIES`=3 × flat
`BOR_LLM_RETRY_DELAY`=5 s, **only before a request has streamed its
first output frame**.
still counted). LLM retries (A15 extension): retry count and delay are
configurable (defaults: 3 retries, 5 s flat delay), **only before a
request has streamed its first output frame**.
Retrieval details (A7/A8) and the persona/`<tools>` prompt contract: see
the previous plan revision §6 or `app/rag/retriever.py` /
@@ -153,7 +153,7 @@ the full contracts. **Persona text changes through the plan, not in code
## 5. Data Model & Chunking (§5 in older comments)
Tables (full column detail: `app/models.py` — it is the living doc;
migrations 0001–0015, each with a tested downgrade):
each migration has a tested downgrade):
| Table | Purpose |
|-------|---------|
@@ -162,12 +162,12 @@ migrations 0001–0015, each with a tested downgrade):
| `query_log` | every question: top score, FTS hits, deflection, sources, latency (threshold-tuning record) |
| `steering_notes` | owner tuning notes → `<tuning>` section of every turn (char-budgeted) |
| `kb_overview` | single row `id=1`: `lite`-generated KB outline → `<knowledge_base>` section (regenerated on KB change) |
| `git_sources` | source registry: `kind` `git`\|`local`, `url`/`path`, `ignore_paths` JSONB (phase 89) |
| `git_sources` | source registry: `kind` `git`\|`local`, `url`/`path`, `ignore_paths` JSONB |
| `saved_chats` | owner-saved conversations; `messages` = the raw `bor.chat.v1` JSONB; `share_token` (NULL = private, `uuid4` → `/shared/<token>`); `sources_version` (stale when < current) |
| `sources_meta` | single row `id=1`: the KB generation counter — bumped once per KB-changing sync |
| `doc_drafts` | save-as-doc drafts; `token` (uuid4) is the URL credential; `draft` → `pushed` (branch + sha) |
| `api_tokens` | access tokens; only the SHA-256 of the full `bor_…` string is stored; `revoked_at` = dead |
| `ui_settings` | single row `id=1`: Theme tab persistence (3 strings + 9 identity colors; NULL = default, B1 — grows to 17 vars in phase 93) |
| `ui_settings` | single row `id=1`: Theme tab persistence (3 strings + 17 color variables; NULL = default, B1) |
Single-row tables use `id = 1` (the `kb_overview` precedent).
@@ -195,13 +195,13 @@ teaching refusals, and scaffolding filter in `app/rag/agent.py` /
`app/rag/scaffolding.py`. Empty prompt sections omit themselves — a
no-notes/no-overview prompt is byte-identical to the pre-steering text.)
Tool surface (harness-aligned, phase 70): **`ls`** (lists indexed docs
`source: X | path: Y | title: Z`; `path` = a source name — becomes a
drill-down tree in phase 94), **`read`** (combined `source/path`
including the source name; appends the full document — gets a
truncation cap in phase 95), **`grep`** (case-insensitive **fixed
substring**, ≤20 `source/path:line: text` matches, optional one-doc
scope; a locator that adds no source/context — never a regex).
Tool surface (harness-aligned): **`ls`** (lists indexed docs
`source: X | path: Y | title: Z`; `path` = a source name — drill-down
tree); **`read`** (combined `source/path` including the source name;
appends the full document — with a truncation cap); **`grep`** (case-
insensitive **fixed substring**, ≤20 `source/path:line: text` matches,
optional one-doc scope; a locator that adds no source/context — never a
regex).
Rejected calls get deterministic teaching refusals and consume a round;
`holder.tool_calls` counts executed calls only.
@@ -235,8 +235,8 @@ Rejected calls get deterministic teaching refusals and consume a round;
deflected) / error / KB-offline / stopped — has a defined UI, every
failure path re-enables its controls, a 300 s pre-token guard
(`TURN_TIMEOUT_MS`) turns a hung stream into the error state (counts
only visible time — phase 73), and **no auto-follow during a turn**
(viewport moves only on user intent, phase 42). Pinned by unit tests
only visible time), and **no auto-follow during a turn**
(viewport moves only on user intent). Pinned by unit tests
on the `app.js` state machine + the story E2E suites.
- Frontend house rules: `app.js` and siblings **never build HTML
strings** (createElement + textContent); asset paths carry
@@ -307,30 +307,21 @@ uv run python -m scripts.llm_probe # models + embedding dim
The loop for git sources is *commit → re-run*. Uploads:
`POST /api/git-sources/upload` unpacks + registers (202); the scan is
deferred to Sync (phase 90). Source removal prunes on the next sync
(phase 69).
deferred to Sync. Source removal prunes on the next sync.
---
## 12. Current State & Roadmap
- **Shipped: phases 01–92** (`.agents/phases/complete/` — read-only
history; the shipped-features recap of the previous plan revision and
the README cover it). Migrations through 0015.
- **Next up (`todo/`, in order — already authored via
`phase-authoring`):**
1. `93_theme_semantic_completion` — 8 semantic state colors join the
Theme-tab palette (17 total, B3 revised) + surface panels behind
every page head.
2. `94_ls_tree_drilldown` — `ls` becomes a filesystem-style drill-down
tree with sync-time `lite` folder summaries (`folder_summaries`
table, migration 0017) + real-model battery.
3. `95_read_truncation_cap` — capped `read` (default 128 000 chars
≈ 32k tokens, `BOR_READ_MAX_CHARS`), honest truncation notice to
the LLM + visible "(truncated — showing N of M chars)" marker
(new optional `tool_result` SSE frame — A15 extended to seven
event types).
- **Next free phase number: 96.**
- **Shipped phases:** `ls .agents/phases/complete/` — read-only
history; each phase's `00_phase.md` has the full detail. The shipped-
features recap of the previous plan revision and the README cover it.
- **Migrations:** `ls alembic/versions/` — all reversible.
- **Next up (`todo/`):** Already authored via `phase-authoring`.
Read the directory listing in `.agents/phases/todo/` for the ordered
task list; each file's `00_phase.md` has the full description.
- **Next free phase number:** `max(completed phase numbers) + 1` —
computed from `.agents/phases/complete/`.
- **Post-v1 hooks (deliberately not built):** HNSW index at scale;
inotify auto-import; more providers (the OpenAI-compatible client is
the seam); multi-user accounts (the `require_user` split is the seam);
@@ -357,7 +348,7 @@ phase dir moved to `complete/`.
(`app/rag/retriever.py`) is the only overflow marker; char budgets are
the pattern for anything sent to `lite`.
- **Byte-identical contracts are load-bearing:** deflected turns
(A8), empty prompt sections, the B4 theme no-op, and the phase-33/54
(A8), empty prompt sections, the B4 theme no-op, and the
rewrite-only caching all have tests that assert byte-identity.
- **git is invoked only in `scripts/git_sync.py`** (A11 — stdlib
subprocess); everything else talks Postgres.
@@ -0,0 +1,36 @@
# Phase 100 — Every page matches the RAG page's width (the 72rem container, at every viewport)
**Source:** Owner request (chat, 2026-09-12) — "The theme, tuning, and chat pages are still pretty narrow, I want you to match the width of the RAG page for all other pages to keep things consistent."
**Story:** n/a (owner request — supersedes the PLAN §7.1 "centered 46rem chat column (2× = 92rem at ≥1500px)" rule and the phase-58/27/91 form-column caps; the width lineage: `58_wide_desktop_column`, `86_history_page_width`).
**Context:** `frontend/assets/styles.css`: `.container` is the shared 72rem centered column (`width: 100%; max-width: 72rem; padding-inline: 1.25rem;`). The RAG view's `.sources-shell` (and `.git-sources-shell` / `.history-shell` / `.tokens-shell`) carry NO max-width — their content is the full 72rem container: **that is the width the owner wants everywhere**. The narrow pages: `:root { --chat-column: 46rem; }` caps `.chat-shell` (the chat view — the div IS `.container.chat-shell`, the later rule overriding the 72rem cap), `.shared-shell` (shared.html), `.doc-md` (the document viewer's reading column) and `.doc-summary:has(+ .doc-md)` (exactly four selectors use the token); `@media (min-width: 1500px) { :root { --chat-column: 92rem; } .doc-shell { max-width: calc(var(--chat-column) + 2 * 1.25rem); } }` (phase 58 — the wide-desktop doubling + the standalone document page's cap lift); and three hard-coded `max-width: 46rem` form columns: `.tuning-shell`, `.theme-shell` (phase 91), `.doc-edit-shell` (phase 59). Width pins live in: `tests/unit/test_wide_column_css.py` (the phase-58 contract file — 46rem base, the 92rem media block, the four-token count, the three-hard-coded-46rem negative pin, the provenance comments), `tests/unit/test_markdown_tables.py`, `test_shared_page.py`, `test_pinned_composer.py`, `test_doc_edit_screen.py`, `test_remove_confirm_modal.py`, `test_source_ignore_paths.py` (grep `46rem|92rem|chat-column` under `tests/` for the authoritative list — some hits are fixture text, judge per pin), and E2E measured-width suites: `tests/e2e/test_wide_desktop_column.py` (the measured 1472px/736px contract), `test_document_viewer.py`, `test_save_share_ux.py`, `test_sticky_navbar.py`, `test_markdown_tables.py`, `test_responsive_polish.py` (mobile no-overflow).
## Objective
Every view and page renders its content at the RAG page's width — the full 72rem container at every viewport (centered, with the container's 1.25rem gutters): the chat column, the Tuning and Theme form columns, the doc-edit screen, the shared page, and the document viewer all widen to 72rem; the phase-58 ≥1500px 92rem doubling and the 46rem form caps are retired; the mobile (≤640px) and tablet layouts are unchanged (everything was already full-width below the caps).
## Dependencies
- `86_history_page_width` / `58_wide_desktop_column` (complete) — the width lineage being revised (their suites are updated IN THIS PHASE, the phase-97 task-07/08 precedent — asserted behavior preserved, only the width contract changes).
- No code dependencies beyond `frontend/assets/styles.css` (CSS-only — `app/`, `frontend/*.html`, and every JS module are untouched).
## Decisions recorded here (owner instruction 2026-09-12 — PLAN.md is being redone by the owner)
- **D1 — one width for everything:** the reading/form columns ALL ride the 72rem container. `:root { --chat-column: 72rem; }` (the token stays — the four reading-column selectors keep `max-width: var(--chat-column)`, so `.doc-md`/`.shared-shell`/`.chat-shell`/the doc-summary panel follow automatically); the `@media (min-width: 1500px)` block is DELETED entirely (the 92rem doubling AND the `.doc-shell` cap lift — with the token at 72rem the standalone document page's `.container.doc-shell` simply keeps the 72rem cap); the three hard-coded `max-width: 46rem` caps (`.tuning-shell`, `.theme-shell`, `.doc-edit-shell`) are REMOVED (plus their `margin-inline: auto` centering — the `.container` ancestor centers; the shells keep their flex-column + gap rules, matching `.sources-shell` exactly). Net effect: ZERO literal `max-width: 46rem` rules remain in the file (the negative pin flips). This SUPERSEDES PLAN §7.1's 46rem/92rem rule and the phase-58 owner instruction of 2026-08-31 (recorded here per the phase-94/96/97 convention — the owner's new instruction wins; the PLAN.md anchor table gets the owner's sign-off separately).
- **D2 — the document viewer's modal is untouched:** the same-page doc modal's 1100px panel stays its effective ceiling (`.doc-md` at 72rem is wider than the panel's inner width, so the panel binds — no panel resize, no modal change). The standalone `document.html` page rides the 72rem container like every other page.
- **D3 — CSS-only, contracts intact:** no HTML change (the B4 byte-identical theme contract is unaffected — the served HTML is unchanged; only `styles.css` bytes change), no JS change, no endpoint change; A19's `?v=<deploy-token>` versioning busts the old CSS automatically. The `≤640px` mobile block is UNCHANGED (at 360px the container is already 100% — the caps never bound there; the mobile squeeze rules for chat/shared/tokens/theme keep working).
- **D4 — the comment sweep:** every stale "46rem base, 92rem at >=1500px" / "46rem column contract" / phase-58-provenance comment in `styles.css` is updated to the new contract (the owner instruction 2026-09-12 as provenance at the token + the note that the wide-desktop doubling was retired) — the house rule that comments carry the contract (and `test_wide_column_css.py` pins the comment states).
## Tasks
1. `01_width_css.md` — the CSS change: the token to 72rem, the media block deleted, the three form caps removed, the comment sweep
2. `02_width_unit_pins.md` — the unit pin updates: `test_wide_column_css.py` rewritten to the new contract + the grep-sweep of the other unit files
3. `03_e2e_width_sweep.md` — `test_wide_desktop_column.py` rewritten to the measured 72rem contract + the other E2E width pins + the regression sweep + the atomic commit
## Testing & Quality
- Unit: `tests/unit/test_wide_column_css.py` REWRITTEN to the new contract (the pins below); every other unit file whose pins reference the old widths updated (the grep sweep — `46rem|92rem|chat-column` under `tests/unit/`, judged per pin: a width assertion updates, fixture text does not).
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_wide_desktop_column.py -v --no-cov` in isolation — the phase's dedicated width suite, REWRITTEN to the measured 72rem contract (the phase-97 task-07/08 precedent for updating a completed phase's suite when its contract changes); the other measured-width suites updated in place.
- Coverage: **>90%** on `app/` (untouched — the full-suite gate still runs).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] at any viewport ≥ ~1200px, the chat, shared, document-viewer, Tuning, Theme, and doc-edit content columns measure the 72rem container width (E2E-measured, ±4 px) — identical to the RAG page's; below that, everything is full-width as today; 360px has no horizontal overflow
- [ ] `styles.css` has ZERO literal `max-width: 46rem` rules and NO `@media (min-width: 1500px)` block; the four reading-column selectors still cap with `var(--chat-column)` (now 72rem)
- [ ] the B4 byte-identical theme contract holds (no HTML change — the theme no-op test suite green); the mobile squeeze layouts green
- [ ] `uv run pytest` green; coverage >90%; ruff + pyright clean
- [ ] one atomic Conventional Commit, `--no-gpg-sign` (e.g. `feat(ui): match every page to the RAG page's 72rem width`)
@@ -0,0 +1,27 @@
# Task 01 — The CSS: one 72rem column everywhere (token, media-block deletion, form caps, comment sweep)
**Phase:** `100_page_width_consistency` · **Story:** n/a (owner request)
## Objective
`styles.css` carries the new width contract (D1–D4 of `00_phase.md`): `--chat-column` is 72rem, the phase-58 ≥1500px block is gone, the three 46rem form caps are removed, and every stale width comment tells the new story.
## Work
1. `frontend/assets/styles.css`:
- `:root` — `--chat-column: 72rem;` (was 46rem). The token's provenance comment updates: the owner instruction 2026-09-12 ("match the width of the RAG page for all other pages") + the note that it now EQUALS the `.container`'s 72rem cap (the reading columns ride the same width as every other page; the phase-58 2026-08-31 instruction is superseded).
- DELETE the entire `@media (min-width: 1500px) { … }` block at the bottom of the file (the `:root { --chat-column: 92rem; }` doubling + the `.doc-shell { max-width: calc(var(--chat-column) + 2 * 1.25rem); }` lift — D1). Its phase-58 block comment goes with it; the deletion is total (the `prefers-reduced-motion` block that follows it stays).
- `.tuning-shell` — remove `max-width: 46rem;` and `margin-inline: auto;` (keep `display: flex; flex-direction: column; gap: 1.25rem; flex: 1;` — the shell now matches `.sources-shell` exactly); update its phase-27 comment (the "Same width discipline as the chat column — a centered, capped column" wording) to the full-container-width contract (owner instruction 2026-09-12).
- `.theme-shell` — the same removal (phase-91's "form column" rationale is superseded — the owner wants it wide); comment updated.
- `.doc-edit-shell` — the same removal (phase-59's form-column cap is superseded); comment updated.
- The comment sweep (D4): every remaining reference to "46rem base", "92rem at >=1500px", "the 46rem chat-column width", "the >=1500px 92rem override", and the stale `≤640rem`/`≤640px`-era "46rem column contract" wording in the reading-column regions (`.chat-shell`'s block comment, `.shared-shell`'s inline comment + the shared-page region comment, `.doc-md`'s comment, the doc-summary panel comment, the `≤640px` shared-page note "the shell keeps its base 46rem column (the >=1500px 92rem override never applies here…)", and any others the grep `46rem|92rem|1500px` finds) — rewrite each to the new contract (72rem at every width; the wide-desktop doubling retired 2026-09-12). The `.shared-shell` comment must keep the "the PLAN §7 centered chat column" lineage sentence only if it still reads true — it does NOT (PLAN §7.1's 46rem rule is superseded): reword to "the shared page reads exactly like the chat page — both at the 72rem container width".
- Do NOT touch: the `.container` rule itself, the `≤640px` mobile block (the squeeze rules), the doc modal's 1100px panel, any non-width CSS, any `app/` or HTML/JS file (D3 — CSS-only).
2. Verify with a grep: `grep -n "46rem\|92rem\|1500px" frontend/assets/styles.css` returns NOTHING (or only the new provenance comment's mention of the retired 2026-08-31 instruction, if phrased without the literal values — prefer phrasing it without the literals so the grep is clean).
## Testing & Quality
- Unit: the pin updates land in task 02 — this task's own check is the grep verification above + a visual sanity pass (`uv run uvicorn app.main:app --reload`, compare the chat/tuning/theme pages to the RAG page at 1280px and 1920px — the content columns match; the doc modal looks unchanged).
- Coverage: **>90%** on `app/` (untouched — the full-suite gate runs at the phase's end).
## Completion Criteria
- [ ] `styles.css`: `--chat-column: 72rem` in `:root`; no `@media (min-width: 1500px)` block; zero literal `max-width: 46rem` rules; the three shells (tuning/theme/doc-edit) structurally match `.sources-shell` (flex column, no cap)
- [ ] the width grep is clean (task 02's pins will enforce the comment states — do not leave a stale "46rem base / 92rem" claim anywhere)
- [ ] no HTML/JS/app change (D3)
- [ ] the full unit suite is green EXCEPT the width pins task 02 rewrites (run it to enumerate the breakage — the list must be exactly the files named in task 02)
@@ -0,0 +1,27 @@
# Task 02 — The unit pins: `test_wide_column_css.py` rewritten + the grep-sweep of the other width pins
**Phase:** `100_page_width_consistency` · **Story:** n/a (owner request)
## Objective
The unit layer pins the NEW width contract — `test_wide_column_css.py` becomes the phase-100 contract file, and every other unit pin that asserted the old widths is updated (fixture text that merely mentions a width is left alone).
## Work
1. `tests/unit/test_wide_column_css.py` — REWRITE (keep the file's balanced-brace `_rule_block` helpers + its module purpose — "pin the styles.css markers the width contract depends on"):
- **The token:** `:root` declares `--chat-column: 72rem` with the provenance comment citing the owner instruction 2026-09-12 (replaces the 2026-08-31 pin).
- **The retired doubling:** NO `@media (min-width: 1500px)` block anywhere in the file (`css.count("@media (min-width: 1500px)") == 0`), and no `--chat-column: 92rem` assignment (`css.count("--chat-column: 92rem") == 0`).
- **The four reading columns:** `.chat-shell`, `.shared-shell`, `.doc-md`, `.doc-summary:has(+ .doc-md)` each cap with `max-width: var(--chat-column)` — and exactly those four rules use the token (`css.count("max-width: var(--chat-column)") == 4` — the token count pin survives, the VALUE changes).
- **The flipped negative pin:** ZERO literal `max-width: 46rem` rules remain (`css.count("max-width: 46rem") == 0`) — the three former form columns (`.tuning-shell`, `.theme-shell`, `.doc-edit-shell`) must NOT carry a max-width at all (each `_rule_block` free of `max-width`), and they carry no `margin-inline: auto` (the `.container` centers).
- **The comment pins:** the stale claims are gone (`"≤46rem" not in css`, `"92rem at >=1500px" not in css`, `"46rem base" not in css`); the provenance appears at the token (owner instruction 2026-09-12); the chat-shell / shared-shell comments name the 72rem-everywhere contract (pin the key phrases, not whole paragraphs — the house "pin the contract words" style).
- Update the module docstring to describe the new pins.
2. The grep sweep — `grep -rln "46rem\|92rem\|chat-column" tests/unit/` (the authoritative list; `test_wide_column_css.py` is done above). For each hit, judge per pin: a pin that ASSERTS the old width (a literal in an `assert`, a `_rule_block` expectation, a counted occurrence) is updated to the new contract (72rem / no cap / the new comment wording); a hit that is fixture text, a docstring example, or an unrelated measurement (e.g. a fixture document mentioning "46rem", or a pin about an UNRELATED rule's max-width) is left untouched. Expected candidates (from the phase-100 `00_phase.md` context): `test_markdown_tables.py`, `test_shared_page.py`, `test_pinned_composer.py`, `test_doc_edit_screen.py`, `test_remove_confirm_modal.py`, `test_source_ignore_paths.py` — the actual set is whatever the grep + the run reveal.
3. Run `uv run pytest tests/unit/ -q` — green.
## Testing & Quality
- Unit: the rewritten `test_wide_column_css.py` + the swept pins ARE this task's test layer (they guard the CSS bytes — the house source-pin pattern).
- Coverage: **>90%** on `app/` (untouched — the full-suite gate runs at the phase's end).
## Completion Criteria
- [ ] `test_wide_column_css.py` pins the phase-100 contract (72rem token + 2026-09-12 provenance, no 1500px block, no 92rem, the four token selectors, ZERO literal 46rem rules, the three shells cap-free, the comment states)
- [ ] the grep sweep is complete — every remaining `46rem|92rem|chat-column` reference under `tests/unit/` is either the new contract's pin or non-width fixture text (list the judgment in the commit's test-run notes / the final summary)
- [ ] `uv run pytest tests/unit/ -q` green
- [ ] no behavior change in completed work (the non-width pins in the swept files still pass unchanged)
@@ -0,0 +1,31 @@
# Task 03 — The E2E width sweep: measured 72rem everywhere + updated suites + regressions + commit
**Phase:** `100_page_width_consistency` · **Story:** n/a (owner request)
## Objective
The browser proves the owner's ask: at every desktop viewport the chat, shared, document-viewer, Tuning, Theme, and doc-edit content columns measure the SAME width as the RAG page (the 72rem container) — the phase's dedicated width suite is rewritten to that contract and the other measured-width suites are updated in place.
## Work
1. `tests/e2e/test_wide_desktop_column.py` — REWRITE (the phase-58 measured-width suite becomes the phase-100 one — the phase-97 task-07/08 precedent: a completed phase's suite updates in place when its contract changes; keep its seeding helpers — the fixture markdown doc for the viewer, the shared-chat helper):
- **`test_all_columns_match_the_rag_page`** (the core pin): viewport 1280×800 — measure the `.chat-shell` bounding-box width on `/`, the `.tuning-shell` width on `/tuning.html` (admin), the `.theme-shell` width on `/theme.html` (admin), and the RAG page's `.sources-shell` width on `/sources.html` (admin): all four EQUAL (± 4 px) — the owner's "match the width of the RAG page" as one assertion. Then viewport 1920×1080: the same four still EQUAL each other (± 4 px) and each ≈ 1152 px (72rem at the 16px root, ± 4 px — the box includes the container's 2×1.25rem padding, border-box).
- **`test_reader_columns_wide`** (the token's consumers): viewport 1920×1080 — `.doc-md` in the opened fixture doc ≈ 1058 px (the 1100px border-box modal panel minus its 1px borders and `.doc-modal-content`'s 2×1.25rem padding is the binding ceiling there — 1100 − 2 − 40; D2: pin that the modal is UNCHANGED by asserting the modal panel's own width is still ~1100 px, not that `.doc-md` is 1152); `/shared/<token>`'s `.shared-shell` ≈ 1152 px; the standalone `document.html?source=…&path=…` page's `.doc-md` ≈ 1112 px (its `.container.doc-shell` is the 72rem container → inner content = 1152 − 40 px padding).
- **`test_narrow_unchanged`** (the no-regression leg): viewports 360×800 and 900×600 — `.chat-shell` width == viewport width (100%, no overflow: `document.documentElement.scrollWidth <= viewport width` — the `test_responsive_polish.py` assertion style), and at 900px the chat/tuning/theme shells are all 900px wide (equal to each other — the cap never bound below 72rem anyway).
2. The other measured-width E2E suites — update in place (each in isolation, `--no-cov`, DB up): `test_document_viewer.py` (its `≤ 736 px` standalone `.doc-md` pin → the ≈ 1112 px contract; keep its non-width assertions byte-identical), `test_save_share_ux.py` (any 92rem/46rem measured or source-pinned width references), `test_sticky_navbar.py` (its width-related pins only), `test_markdown_tables.py` (the e2e table-width pins against the reading column), `test_responsive_polish.py` (the mobile pins — expected UNCHANGED; run to confirm). Judge per assertion: a width measurement updates, a layout/behavior assertion never does.
3. The theme no-op contract (D3): run `test_ui_customization.py` + `test_admin_theme_tab.py` in isolation — no HTML changed, so the B4 byte-identical pins must pass UNCHANGED (if one fails, the CSS change touched something it should not — fix the cause, not the pin).
4. Regression sweep (each in isolation, `--no-cov`): the five suites from step 2 (updated) + `test_chat_rag.py` (the chat layout smoke) + `test_theme_semantic_completion.py` (the phase-93 page-head panels at the new width).
5. Full gates + commit: `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean; move `100_page_width_consistency` → `.agents/phases/complete/`; one atomic commit:
```bash
git add -A .agents/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(ui): match every page to the RAG page's 72rem width"
```
## Testing & Quality
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_wide_desktop_column.py -v --no-cov` green in isolation — the phase's dedicated suite.
- Coverage: **>90%** on `app/` (the full-suite gate).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] the rewritten width suite passes in isolation: chat == tuning == theme == RAG at 1280 AND 1920 (± 4 px, ≈ 1152 px at 1920); the doc modal unchanged (~1100 px); shared + standalone document pages ≈ 1152/1112 px; 360px overflow-free
- [ ] the other measured-width suites pass in isolation (updated in place); the theme no-op suites pass UNCHANGED
- [ ] the regression sweep passes in isolation
- [ ] full suite green, coverage >90%, ruff + pyright clean
- [ ] phase dir moved to `complete/`, one atomic `--no-gpg-sign` Conventional Commit
@@ -0,0 +1,78 @@
# Phase 101 — Tokens page: active/revoked table split, per-table search, required name, one-click regenerate
**Source:** Owner request (chat, 2026-09-12) — "The tokens page should move revoked tokens to a separate table below the active ones. Generating a token should not be possible without giving it a name. Both active and revoked token tables should be searchable. I should be able to regenerate active tokens with the click of a button."
**Story:** n/a (owner request — extends `79_api_tokens`: the model/service/API on phase 79 tasks 01–03, the single-table admin view on task 06, the once-block + two-step revoke UI, the E2E `test_api_tokens.py`).
**Context:** `api_tokens` (migration 0012): `id` UUID, `label` (1–120, NOT unique), `token_hash` (sha256 of the full `bor_…` string, unique), `created_at`, `last_used_at`, `revoked_at` (NULL = active). `app/core/tokens.py` service: `generate_token`, `hash_token`, `create_token(db, label) -> (row, plaintext)`, `find_active_by_token`, `mark_used`, `revoke(db, token_id) -> bool` (stamps `revoked_at` only when unset; False when the row is missing). Admin API (`app/api/tokens.py`, router-wide `require_admin`): `POST /api/tokens` (201 `TokenCreated{id, label, token, created_at}` — the ONLY response carrying the plaintext, owner-locked A4), `GET /api/tokens` (200 `TokenList{tokens: [TokenListItem{id, label, created_at, last_used_at, revoked: bool}]}` newest-first — `revoked_at` is NOT returned today), `POST /api/tokens/{id}/revoke` (204 idempotent, 404 unknown). The view (`frontend/index.html` `#view-tokens` + `frontend/assets/tokens.js`, phase-76 fold): the create row (`#token-label` + `#token-generate` — a BLANK label currently sends the fallback `"token"`), the shown-once block (`#token-once` — plaintext in the read-only field's value ONLY, hidden + wiped on the next `loadTokens()` / re-show), ONE full-width table (`#tokens-table` / `#tokens-tbody`, columns Label | Created | Last used | Status (active em-dash vs the rose `.stale-pill` Revoked) | Actions (the two-step inline Revoke — the `history-confirm-*` pattern; revoked rows carry no action)), `#tokens-empty-row`, the `#tokens-status` live region, the `bor:view-refresh` re-show contract (re-entrant `loadTokens()`), the whoami gate (anonymous → the gate, NO fetch). The phase-79 E2E `tests/e2e/test_api_tokens.py` pins the single-table layout — including REVOKED rows living in `#tokens-tbody` (e.g. `test_revocation_closes_the_door` asserts the `.stale-pill` there) — and `auth_helpers.login_with_token(page, app_url, token)` drives the real gate (the regenerate proof reuses it).
## Objective
The Tokens view splits its list into an ACTIVE table and a REVOKED table below it (each independently searchable), a token can no longer be generated without a name, and every active token carries a one-click **Regenerate** (a house two-step confirm, then the new plaintext shown once) that rotates the credential — the old token is revoked and lands in the revoked table, the new one (same label) takes its place in the active table.
## Dependencies
- `79_api_tokens` (complete) — the model, the service, the admin API, the A4 plaintext-once contract, the view module, the E2E helper (`login_with_token`).
- Queue order only: `98_sync_summary_visibility`, `99_kb_tree_table_and_back_nav`, `100_page_width_consistency` (todo) — no code dependency (different views; the tokens table sits in the full-width `.container` already — phase 100 changes nothing here).
## Decisions recorded here (owner review — PLAN.md is being redone by the owner)
- **D1 — the split + the column contracts:** the active table keeps the phase-79 shape MINUS the Status column (a table that is all-active needs no status) — **Label | Created | Last used | Actions** (Actions visually-hidden header, as today). The revoked table (below, its own `.table-wrap` + a visible `Revoked tokens` heading + its own search input) is **Label | Created | Last used | Revoked** (the `revoked_at` date — locale date+time, full ISO on hover, the house `tokens-date-cell` language). The table IS the status — the em-dash / `.stale-pill` column disappears from BOTH tables (the pill's meaning is now the table's position). The revoked section (heading + search + table) is HIDDEN when there are no revoked tokens (an empty table is noise); the active table keeps its empty-state row. Newest-first in both (the server order).
- **D2 — regenerate = rotation, atomic:** new endpoint `POST /api/tokens/{token_id}/regenerate` (admin, 201 `TokenCreated`): in ONE transaction — stamp `revoked_at` on the old row (the `revoke` service primitive) and `create_token(db, old.label)` (same label — the hand-out name persists) — returning the NEW row + the new plaintext exactly once (A4 preserved: the plaintext is per-token, shown once, never re-shown; the old plaintext was already gone). Unknown id → 404 `token not found`; the row is ALREADY revoked → 409 `token already revoked` (a dead token cannot be rotated — the button never offers it, the API stays honest). UI: the active row's Actions cell gains **Regenerate** (before Revoke) — the house two-step inline confirm (first click swaps to `Regenerate? The current token is revoked. [Yes] [No]`, focus to Yes — a destructive rotation deserves the same confirm weight as Revoke; ONE button starts it, per the owner's "click of a button"). On 201: the re-entrant `loadTokens()` runs (the old row moves to the revoked table, the new row lands in the active one), the once-block reveals with the new plaintext + Copy, the live region reads `Regenerated "<label>" — copy the new token now; it won't be shown again.` Failure (409 → the row re-renders in the revoked table + a line; other → neutral retry copy, the button restores — the phase-55 convention).
- **D3 — a name is required:** `generateToken()` trims the label and, when empty, does NOT send — the live region reads `Give the token a name first.`, the label input re-focuses, the Generate button's §7.4 lifecycle is untouched (it was never disabled — the request simply doesn't happen). The `|| "token"` fallback is DELETED (the create row's comment + placeholder update: the name is the hand-out identity, not an optional decoration). The server's 422 on blank/over-long labels stands unchanged (defense in depth — the UI just stops volunteering the fallback).
- **D4 — search = client-side, per table, label-scoped:** one `type="search"` input per table (`#token-search-active` above the active table, `#token-search-revoked` above the revoked table — inside the revoked section, so it hides with it), placeholder `Search active tokens…` / `Search revoked tokens…` (labeled via `aria-label` — the house visually-hidden-label language for single-purpose inputs). Filtering: case-insensitive SUBSTRING over the label, applied on the `input` event to the CURRENTLY rendered rows (no fetch — toggle each row's `hidden`), and RE-APPLIED after every `loadTokens()` (the query state survives re-renders / re-shows — the phase-77 contract). Zero visible matches with a non-empty query → a per-table no-match row (`No tokens match "<query>".` — the query text is textContent-filled; distinct from the active table's true empty-state row `No tokens yet — …`). Empty query → all rows visible, no-match row hidden.
- **D5 — the list API gains `revoked_at`:** `TokenListItem` adds `revoked_at: datetime | None = None` (wire-additive — null for active tokens; the revoked table's Revoked column needs the actual timestamp, and nothing else changes on the wire). The existing `revoked: bool` stays (the client's table split key).
## Design (shared by all tasks — the executor reads this, not the chat)
### The API (task 01)
- `app/core/tokens.py` — `regenerate_token(db, token_id) -> tuple[ApiToken, str] | None` (the service surface, unit-testable without FastAPI): fetch the row (`db.get`) — missing → return `None`; `revoked_at` set → raise `TokenAlreadyRevoked` (a new module-level exception, one sentence docstring: a dead token cannot be rotated); else `revoke(db, row.id)` (stamp) + `create_token(db, row.label)` (new row + plaintext) + `db.flush()` (the caller commits — the `create_token` convention: the service flushes, the endpoint owns the commit). ONE commit covers both writes (the rotation is atomic — a create failure rolls the revoke back with it).
- `app/api/tokens.py` — `POST /{token_id}/regenerate` (201, `response_model=TokenCreated`, the router-wide `require_admin` covers it): `None` → 404 `token not found` (the revoke endpoint's exact message); `TokenAlreadyRevoked` → 409 `token already revoked`; success → commit + refresh + `TokenCreated(id, label, token, created_at)` of the NEW row. The list endpoint passes `revoked_at=row.revoked_at` into `TokenListItem` (D5). Docstrings: the rotate-once semantics (the docstring-carries-the-contract rule).
- `app/schemas.py` — `TokenListItem.revoked_at: datetime | None = None` (the docstring notes D5).
- Unit (`tests/unit/test_tokens.py`): the `regenerate_token` matrix — active row → (new row, new plaintext) with the SAME label, the old row stamped (its original `revoked_at` semantics: first stamp), the new plaintext well-formed (`bor_` + 32 hex, ≠ the old hash); missing id → None; revoked id → raises; the new row's hash is the sha256 of the new plaintext (the lookup round-trips via `find_active_by_token`); the old token no longer authenticates (`find_active_by_token(old_plaintext)` → None).
- Integration (`tests/integration/test_tokens_api.py`): the endpoint matrix — 201 (the 201 body carries the NEW plaintext exactly once; the old row `revoked: true` + `revoked_at` non-null in the next list; the new row active, same label, newer `created_at`); 404 unknown id; 409 already-revoked; 403 anonymous + 403 token-user (the router-wide gate — the existing pattern); the list shape now carries `revoked_at` (null active / timestamp revoked, ISO-8601).
### The view (tasks 02 + 03)
`frontend/index.html` `#view-tokens` (the static skeleton — tokens.js fills the tbodys):
- The create row: the input's `aria-label` → `Token name`, the placeholder → `e.g. alice — required` (D3); the comment updated (no fallback — the name is required).
- A search row above the active table: `<input type="search" id="token-search-active" aria-label="Search active tokens" placeholder="Search active tokens…">` (the house input language — a new `.token-search` class in the CSS, full width, the `#tune-note`/archive-upload input styling family).
- The active table: thead drops the Status `<th>` (four columns now: Label | Created | Last used | the visually-hidden Actions); `#tokens-empty-row` → `colspan="4"`, copy unchanged; a NEW hidden `#tokens-no-match-row` (`colspan="4"`, its text JS-filled — ships empty).
- A NEW revoked section BELOW the active table's wrap (ships `hidden`): `<h2 id="tokens-revoked-heading" class="tokens-revoked-heading">Revoked tokens</h2>` (a visible sub-heading — house typography: `font-family: var(--mono); font-size: 1rem; color: var(--brand-ink);` on the surface-less page background, AA pair, matching the phase-97 `.kb-level h2` voice) + the search input `#token-search-revoked` + `<div class="table-wrap tokens-table-wrap" id="tokens-revoked-wrap" role="region" aria-label="Revoked tokens" tabindex="0">` + `<table class="tokens-table" id="tokens-revoked-table">` (visually-hidden caption `Revoked tokens — newest first`; thead Label | Created | Last used | Revoked; `tbody#tokens-revoked-tbody` + the hidden `#tokens-revoked-no-match-row` `colspan="4"`).
- `frontend/assets/tokens.js` (task 02 — the split + search + required name):
- `loadTokens()` — the fetched list splits: `revoked === false` → `#tokens-tbody`, `true` → `#tokens-revoked-tbody` (the server's newest-first order kept per table); the revoked SECTION (`#tokens-revoked-heading` + search input + wrap) is shown iff the revoked table has ≥ 1 row; the active empty-state row logic stands (0 active rows → the empty row — a 0-row fetch with revoked tokens shows the empty active table + the populated revoked section: honest states); AFTER rendering, both search filters re-apply (D4 — the queries persist in module state `activeQuery` / `revokedQuery`, initialized `""`, never reset by a load).
- `makeRow(tok, { table })` — the active variant: Label | Created | Last used | Actions (Regenerate — task 03 — + Revoke, the existing two-step, unchanged); the revoked variant: Label | Created | Last used | Revoked (the `revoked_at` locale date+time, full ISO in `title` — the `tokens-date-cell` class) + NO actions. The `title` hover attributes carry over (label, created, last-used, revoked).
- The search: per-table `applyFilter(tbody, noMatchRow, query, tableLabel)` — case-insensitive `label.toLowerCase().includes(query.toLowerCase())` over the data rows (the no-match/empty rows excluded); rows toggle `hidden`; `noMatchRow` visible ⟺ query non-empty AND zero data rows visible (its `<td>` textContent = `No tokens match "<query>".` — the query inside the quotes is textContent, never HTML). The `input` listeners set the module query + apply (no fetch). The active search input is revealed with the create row (admin branch); the revoked one with the revoked section.
- `generateToken()` (D3): `const label = (labelInput ? labelInput.value : "").trim();` — `if (!label) { announce("Give the token a name first."); if (labelInput) labelInput.focus(); return; }` — the `|| "token"` fallback deleted; the rest of the function (the 201 once-block reveal, the list re-load FIRST, the label clear, the error lines) unchanged.
- `frontend/assets/tokens.js` (task 03 — regenerate): `makeRegenerateControl(tok, row)` mirroring `makeRevokeControl` (the `history-confirm-*` swap, focus to Yes) — Yes → `POST /api/tokens/<id>/regenerate` → on 201: `loadTokens()` (the re-entrant load — the old row relocates, the new row lands) → the once-block reveal (`onceValue.value = created.token`, `onceBlock.hidden = false` — the same once-block, the A4 value-only contract) → `announce("Regenerated \"<label>\" — copy the new token now; it won't be shown again.")`; 404 → the row re-renders revoked-in-place + `That token was already revoked.` (the revoke control's existing 404 handling, mirrored); 409 → the row relocates via `loadTokens()` + the same line; other failure/network → neutral retry copy + the confirm restores (the `restoreRegenerate` pattern — focus returns). The active Actions cell order: **Regenerate, then Revoke** (the rotation is the primary lifecycle action).
- `frontend/assets/styles.css` — `.token-search` (the input row: `width: 100%;` + the house input surface — reuse the `#tune-note` font/padding/border family, `:focus-visible` via the global rule, ≥ 44px target height), `.tokens-revoked-heading` (the sub-heading, the AA pair noted above), the no-match rows' `<td>` (the `tokens-empty-row` styling family — `color: var(--ink-soft)`), the Regenerate button (the `.token-revoke` family's neutral sibling — or a new `.token-regenerate` class styled identically to `.token-revoke` minus the hover-error: keep it simple — REUSE `.token-revoke`'s classes for the confirm machinery, one extra class for the label if needed; NO new hue — the phase-92 monochrome invariant). The `≤640px` block: the search inputs + the two tables squeeze as today (verify the `tokens-table-wrap` mobile rules cover the second table — they are class-based, so they do).
- Source pins: extend the existing tokens.js full-file source-pin unit file (the phase-79 task-06 pin — locate it under `tests/unit/`, likely `test_tokens_ui.py` or the `test_api_tokens_model.py` sibling; if no dedicated file exists, CREATE `tests/unit/test_tokens_ui.py`): the two tbodys + the section show/hide, the split in `loadTokens`, the filter logic (the case-insensitivity, the re-apply-after-load, the no-match row text), the required-name block (the exact announce copy + NO `|| "token"` anywhere in the file), the regenerate wiring (the POST path, the once-block reveal, the 404/409 branches, the confirm copy) + the styles.css class pins.
### The E2E (task 04)
`tests/e2e/test_tokens_page.py` (new; `app_server` + `mock_llm` + `db_ready`; admin via `auth_helpers.login`; a token-user context via a FRESH browser context + `auth_helpers.login_with_token` — the phase-79 pattern; the e2e-`-labeled` token cleanup autouse fixture from `test_api_tokens.py`):
1. **`test_revoked_tokens_move_to_their_own_table`** — create two tokens (the UI create row, labels `e2e-act` / `e2e-rev`); Revoke `e2e-rev` (the two-step): the active table shows ONLY `e2e-act` (the active thead has NO Status column — four `<th>`); the revoked section is visible BELOW the active table: its heading `Revoked tokens`, the `e2e-rev` row with a Revoked date cell (non-empty, ≠ created) and NO action buttons.
2. **`test_both_tables_are_searchable`** — with ≥ 2 active tokens (`e2e-a1` / `e2e-a2` — labels chosen so one is a substring of the other's neighbor: type `e2e-a1` → only that row visible; type `zzz` → the no-match row reads `No tokens match "zzz".`; clear → both rows back). Same on the revoked search (with the revoked row from test 1's flow or a fresh revoke).
3. **`test_a_token_cannot_be_generated_without_a_name`** — clear the label, click Generate: NO once-block, the live region reads `Give the token a name first.`, the active table's row count is unchanged (and `GET /api/tokens` via the admin cookie shows no new row — the request never happened).
4. **`test_regenerate_rotates_the_token`** — create `e2e-rot`; Regenerate → Yes: the once-block shows a NEW plaintext (≠ the original, `bor_` + 32 hex); the active table has the `e2e-rot` row (the NEW one — its created timestamp is newer); the revoked table now holds the ORIGINAL `e2e-rot` (same label, Revoked date set); the live region carries the D2 line; a fresh token-user context signs in with the NEW token (the gate closes → the app opens — `login_with_token`), and the ORIGINAL token is refused at the gate (the `#auth-gate-error` alert — it was revoked by the rotation).
5. Update `tests/e2e/test_api_tokens.py` in place (the phase-97 task-07/08 precedent): the revoked-row assertions relocate to `#tokens-revoked-tbody` (`test_admin_generates_token_in_ui`'s post-revoke state, `test_revocation_closes_the_door`'s `.stale-pill` → the revoked table's row + its Revoked date cell); the Status-column assertions (if any) drop; every non-layout assertion (the auth flows, the cached token, the 403 walls, the wrong-token error) stays byte-identical.
6. Regression sweep (each in isolation, `--no-cov`, DB up): the updated `test_api_tokens.py` + `test_admin_auth.py` (the gate contract) + `test_shared_header.py` (the header/whoami plumbing — expected unchanged).
7. Full gates + commit: `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean; move `101_tokens_page_overhaul` → `.agents/phases/complete/`; one atomic commit:
```bash
git add -A .agents/ app/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(tokens): split active and revoked tables with search, require a name, and add one-click regenerate"
```
## Tasks
1. `01_regenerate_api.md` — the service `regenerate_token` + `POST /api/tokens/{id}/regenerate` (rotation, atomic) + `TokenListItem.revoked_at` + unit/integration
2. `02_two_tables_search_required_name.md` — the view skeleton (two tables + the revoked section + the search inputs) + the client split/filter + the required-name block
3. `03_regenerate_ui.md` — the Regenerate control (two-step confirm, the once-block reveal, the 404/409 branches) + the CSS + the source pins
4. `04_e2e_tokens_page.md` — `tests/e2e/test_tokens_page.py` + the `test_api_tokens.py` update + the regression sweep + the atomic commit
## Testing & Quality
- Unit: the `regenerate_token` matrix (`tests/unit/test_tokens.py`); the source pins (task 03's file) for the view logic.
- Integration: the endpoint matrix + the list shape (`tests/integration/test_tokens_api.py`); the existing phase-79 auth-enforcement pins stay green (the new route rides the router-wide gate).
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_tokens_page.py -v --no-cov` in isolation; the updated `test_api_tokens.py` + `test_admin_auth.py` + `test_shared_header.py` green in isolation.
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] the Tokens view (admin) shows the active tokens in the top table (Label | Created | Last used | Actions) and the revoked ones in a separate table below (Label | Created | Last used | Revoked) — the section hidden while no token is revoked; both tables filter live by label with a per-table no-match row that survives re-renders
- [ ] a blank/whitespace name generates NOTHING (the live-region line, no request, no row); the `token` fallback is gone
- [ ] Regenerate (one button + the house confirm) rotates: the old token is revoked (lands in the revoked table, refuses new sign-ins immediately) and the new token (same label) is active with its plaintext shown exactly once; 404/409 are pinned at the API level
- [ ] the A4 contract holds end to end (the list never carries plaintext/hashes; the once-block re-shows nothing — the phase-79 pins green)
- [ ] `uv run pytest` green; coverage >90%; ruff + pyright clean
- [ ] `uv run pytest tests/e2e/test_tokens_page.py -v --no-cov` green in isolation (DB up: `podman compose up -d db`); the regression suites green in isolation
- [ ] one atomic Conventional Commit, `--no-gpg-sign`
@@ -0,0 +1,35 @@
# Task 01 — `POST /api/tokens/{id}/regenerate`: the atomic rotation + `TokenListItem.revoked_at`
**Phase:** `101_tokens_page_overhaul` · **Story:** n/a (owner request)
## Objective
The admin can rotate an active token server-side: one atomic call revokes the old row and creates its successor (same label, new plaintext returned exactly once) — plus the list API exposes `revoked_at` so the revoked table can show the revocation date.
## Work
1. `app/core/tokens.py`:
- `class TokenAlreadyRevoked(Exception)` — module-level, one-sentence docstring (a dead token cannot be rotated — the 409's reason).
- `regenerate_token(db, token_id: uuid.UUID) -> tuple[ApiToken, str] | None`:
- `row = db.get(ApiToken, token_id)` — `None` → return `None` (the endpoint maps it to the 404).
- `row.revoked_at is not None` → `raise TokenAlreadyRevoked`.
- `revoke(db, row.id)` (stamps the original revocation time — the existing primitive; it returns True here by construction) + `new_row, plaintext = create_token(db, row.label)` (the SAME label — D2).
- `db.flush()` and return `(new_row, plaintext)` — the CALLER commits (the `create_token` convention: the service flushes, the endpoint owns the commit). ONE commit covers stamp + create (atomicity: a create failure rolls the revoke back with it — note this in the docstring).
- Module docstring: the rotation's contract (the old plaintext was already one-shot; the new one is too — A4 never weakens).
2. `app/api/tokens.py` — `@router.post("/{token_id}/regenerate", response_model=TokenCreated, status_code=201)`:
- `result = token_service.regenerate_token(db, token_id)`; `None` → 404 `token not found` (the revoke endpoint's exact message); `except TokenAlreadyRevoked` → 409 `token already revoked`; success → `db.commit()` + `db.refresh(new_row)` (the server-default `created_at`) + `TokenCreated(id, label, token, created_at)`.
- `list_tokens` — pass `revoked_at=row.revoked_at` into each `TokenListItem` (D5).
- The new route's docstring: rotate-once semantics + the A4 note (this 201 is the new token's ONLY plaintext moment).
3. `app/schemas.py` — `TokenListItem` gains `revoked_at: datetime | None = None` (the docstring: null while active; the ISO-8601 timestamp the revoked table renders — the D5 note).
4. Unit (`tests/unit/test_tokens.py` extensions) — the `regenerate_token` matrix (the existing file's DB-session pattern): active row → a NEW row (different id) with the SAME label + a well-formed new plaintext (`bor_` + 32 hex) whose sha256 is the stored hash; the OLD row stamped with a `revoked_at`; the new plaintext round-trips through `find_active_by_token`; the old plaintext no longer authenticates (`find_active_by_token(old)` → `None`); missing id → `None`; an already-revoked id → `TokenAlreadyRevoked`; the rotation is atomic (a create failure — e.g. simulate by exhausting… if the existing harness cannot fail `create_token`, pin the flush-only/commit-caller contract by asserting the session is NOT committed by the service: the caller's rollback undoes both writes).
5. Integration (`tests/integration/test_tokens_api.py` extensions): 201 (body carries the new plaintext — `bor_`-prefixed, ≠ the original; the follow-up list shows the old row `revoked: true` + non-null `revoked_at` and the new row active, same label, newer `created_at`); 404 unknown id (one message); 409 an already-revoked id (revoked via the endpoint first); 403 anonymous + 403 a token-user session (the router-wide gate — the existing 403 pattern in the file); the list response shape now includes `revoked_at` (null for active, timestamp for revoked — the phase-79 list pins updated in place).
## Testing & Quality
- Unit: the matrix above (`tests/unit/test_tokens.py`).
- Integration: the endpoint matrix + the list shape (`tests/integration/test_tokens_api.py`).
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
## Completion Criteria
- [ ] `POST /api/tokens/{id}/regenerate` → 201 with the new token's plaintext (same label); the old row is revoked in the SAME transaction; 404 / 409 pinned; the admin gate pins hold (403 anonymous + token-user)
- [ ] `GET /api/tokens` items carry `revoked_at` (null active / timestamp revoked); the existing phase-79 list assertions (no plaintext, no hashes, newest-first) still pass
- [ ] the A4 contract: the list never gains a credential field (unit + integration pinned)
- [ ] full test suite green, coverage >90%
- [ ] no behavior change in completed work (the phase-79 auth-enforcement + token-gate suites green)
@@ -0,0 +1,58 @@
# Task 02 — The view: active/revoked tables, per-table search, the required name
**Phase:** `101_tokens_page_overhaul` · **Story:** n/a (owner request)
## Objective
The Tokens view splits its list into an active table and a revoked table below it (each with its own live label search), and a blank name can no longer generate a token — the `token` fallback is gone.
## Work
1. `frontend/index.html` `#view-tokens` (the static skeleton — the D1/D4 shapes from `00_phase.md`; keep every existing id that survives):
- The create row: the input's `aria-label` → `Token name`, the `placeholder` → `e.g. alice — required`; the row's comment updated (a blank name is REFUSED client-side — D3 — the server's 422 is defense in depth).
- A search row between the create row/once-block area and the active table's wrap: `<input type="search" id="token-search-active" aria-label="Search active tokens" placeholder="Search active tokens…" class="token-search">` (ships hidden — revealed in the admin branch with the create row).
- The active table (`#tokens-table`): thead loses the Status `<th>` — four columns now: Label | Created | Last used | `<th scope="col"><span class="visually-hidden">Actions</span></th>` (unchanged); `#tokens-empty-row` → `colspan="4"` (copy unchanged); a NEW `<tr class="tokens-empty-row" id="tokens-no-match-row" hidden><td colspan="4"></td></tr>` (its text JS-filled — the no-match copy, D4).
- The NEW revoked section AFTER the active table's wrap (ships `hidden` — tokens.js shows it when ≥ 1 revoked token):
```html
<h2 id="tokens-revoked-heading" class="tokens-revoked-heading">Revoked tokens</h2>
<input type="search" id="token-search-revoked" aria-label="Search revoked tokens" placeholder="Search revoked tokens…" class="token-search">
<div class="table-wrap tokens-table-wrap" id="tokens-revoked-wrap" role="region" aria-label="Revoked tokens" tabindex="0">
<table class="tokens-table" id="tokens-revoked-table">
<caption class="visually-hidden">Revoked tokens — newest first</caption>
<thead><tr>
<th scope="col">Label</th><th scope="col">Created</th>
<th scope="col">Last used</th><th scope="col">Revoked</th>
</tr></thead>
<tbody id="tokens-revoked-tbody">
<tr class="tokens-empty-row" id="tokens-revoked-no-match-row" hidden><td colspan="4"></td></tr>
</tbody>
</table>
</div>
```
(the view's HTML comments carry the D1/D2/D4 provenance — the house comment style; the section's ids are all new, nothing collides).
2. `frontend/assets/tokens.js` (the split + search + required name):
- Module lookups: the revoked section's heading / search input / wrap / tbody / no-match row + the active no-match row (scoped to `root`, the phase-76 contract).
- Module state: `activeQuery = ""` / `revokedQuery = ""` (the D4 persistent queries — initialized once, NEVER reset by a load).
- `makeRow(tok, table)` — `table` is `"active"` | `"revoked"`: shared Label/Created/Last-used cells (the `title` hovers carry over); the active variant's Actions cell (Regenerate — task 03 — + the existing two-step Revoke, UNCHANGED in this task); the revoked variant's last cell = the `revoked_at` date (locale date+time via the existing `fmtDate`, full ISO in `title`, class `tokens-date-cell`) and NO actions.
- `loadTokens()` — the re-entrant core is preserved (once-block hidden + field wiped, data rows dropped, the fetch, the error lines, the return value — all UNCHANGED); the render step now SPLITS the fetched list by `tok.revoked` (server order kept per table: active rows → `#tokens-tbody`, revoked rows → `#tokens-revoked-tbody`); the active empty-state row shows iff there are zero ACTIVE rows (a 0-active fetch with revoked rows shows the empty-state row AND the populated revoked section — both honest); the revoked SECTION (heading + search input + wrap — a small `setRevokedSectionVisible(n)` helper) shows iff n ≥ 1; FINALLY both filters re-apply (`applyFilter` below — D4: a re-render never loses the queries).
- `applyFilter(tbody, noMatchRow, query, )` — pure DOM: trim + lowercase the query; over the tbody's data rows (everything EXCEPT the no-match row), `row.hidden = query && !label.toLowerCase().includes(query.toLowerCase())` (read the label from the row's label cell textContent — the rows are the data source of truth after a render); `noMatchRow` hidden unless `query && zeroDataRowsVisible` — its `<td>` textContent = `No tokens match "${query}".` (the ORIGINAL, untrimmed user query inside the quotes — textContent only).
- The `input` listeners (armed in the admin branch, after the whoami gate — the anonymous branch arms nothing): set the module query + `applyFilter` (NO fetch — D4).
- `generateToken()` (D3): `const label = (labelInput ? labelInput.value : "").trim();` — `if (!label) { announce("Give the token a name first."); if (labelInput) labelInput.focus(); return; }` — the `|| "token"` DELETED; the placeholder/comment (step 1) and the function's comment updated; the rest of the flow (the disabled/"Generating…" lifecycle, the 201 reveal, the list re-load FIRST, the label clear, the error lines) byte-identical in behavior.
- Module docstring: the phase-101 section (the split, the search contract, the required name, the revoked column).
- The `bor:view-refresh` listener is UNCHANGED (it calls `loadTokens()`, which now splits + re-applies the filters — the queries survive a re-show for free).
3. `frontend/assets/styles.css` (the token region):
- `.token-search { width: 100%; min-height: 44px; padding: 0.55rem 0.8rem; border: 1px solid var(--line); border-radius: var(--radius-sm); background: var(--surface); color: var(--ink); font: inherit; font-size: 0.9rem; }` + `:focus-visible` via the global 3px outline rule (no custom focus rule — the house pattern) + a comment (the search inputs' surface — the `#tune-note`/archive-upload input family, AA pairs, no new hue).
- `.tokens-revoked-heading { margin: 1.5rem 0 0.5rem; font-family: var(--mono); font-size: 1rem; color: var(--brand-ink); }` (the sub-heading — the phase-97 `.kb-level h2` voice, AA on `--bg`).
- The no-match rows reuse the existing `.tokens-empty-row` styling (the class is shared — verify it covers both tables; if the empty-row rule is `#tokens-empty-row`-specific, generalize it to `.tokens-empty-row` — the styling is already class-based, so likely a no-op).
- The `≤640px` block: verify the existing `.token-create` / `.tokens-table-wrap` squeeze rules cover the new inputs + the second table (class-based — expected no change; add a search-input width rule only if the mobile layout needs it).
4. Source pins: locate the phase-79 tokens.js full-file source-pin unit file under `tests/unit/` (grep `tokens.js` in `tests/unit/` — if a dedicated file exists, EXTEND it; if the pins live inside `test_api_tokens_model.py` or similar, extend there; if no dedicated pin file exists, CREATE `tests/unit/test_tokens_ui.py`): the two-tbody split in `loadTokens`, the section show/hide helper, `applyFilter` (case-insensitivity, the re-apply-after-load call, the no-match text with the original query, the data-row exclusion), the required-name block (the exact announce copy `Give the token a name first.`, the early return BEFORE any fetch, and a negative pin: the string `"token"` fallback pattern `|| "token"` is GONE from the file), the search inputs' armed-in-admin-branch wiring + the styles.css class pins.
## Testing & Quality
- Unit: the source pins above are this task's test layer for the JS (`app/` untouched this task — task 01's API pins already cover the server side).
- Coverage: **>90%** on this task's new/modified code (frontend pins cover the JS; the browser proof is task 04's E2E).
- Note: do NOT add an E2E file here — the split/search/name proofs land in task 04.
## Completion Criteria
- [ ] the view skeleton carries the four-column active table, the revoked section (heading + search + the four-column table) shipping hidden, and both search inputs (ids per the D1/D4 contract)
- [ ] `loadTokens` splits by `revoked`, hides the section when empty, keeps the once-block / error / return-value contracts byte-identical, and re-applies BOTH persistent queries after every render (source-pinned)
- [ ] a blank/whitespace name announces `Give the token a name first.`, re-focuses, and sends NO request (source-pinned; the `|| "token"` pattern is gone from the file)
- [ ] full test suite green, coverage >90%
- [ ] no behavior change in completed work (the phase-79 gate / whoami / once-block pins green; the Regenerate button is NOT built yet — task 03 adds it)
@@ -0,0 +1,32 @@
# Task 03 — The Regenerate control: one button, the house confirm, the once-block reveal
**Phase:** `101_tokens_page_overhaul` · **Story:** n/a (owner request)
## Objective
Every active token row carries a **Regenerate** button (before its Revoke): one click opens the house two-step confirm, Yes rotates the token server-side, and the new plaintext appears in the shown-once block — the old token relocates to the revoked table.
## Work
1. `frontend/assets/tokens.js`:
- `makeRegenerateControl(tok, row)` — a structural mirror of `makeRevokeControl` (the D2 contract from `00_phase.md`): a `.token-regenerate` button labeled `Regenerate` (`aria-label: "Regenerate token: <label>"`); first click swaps the cell's contents to the confirm pair — `span.history-confirm-text` `Regenerate? The current token is revoked.` + Yes (`.history-confirm-yes`) + No (`.history-confirm-no`), focus to Yes; No / failure → `restoreRegenerate()` (the button back, focus restored — the revoke control's restore pattern, copied).
- `confirmRegenerate(tok, row, yesBtn, restoreRegenerate)`: `yesBtn.disabled = true`; `POST /api/tokens/<id>/regenerate` (JSON — no body):
- **201** → parse `{ id, label, token, created_at }` → `await loadTokens()` FIRST (the re-entrant load: the old row moves to the revoked table, the new row lands in the active one — the D2 relocation) → the once-block reveal (`if (onceValue) onceValue.value = data.token;` + `if (onceBlock) onceBlock.hidden = false;` — the SAME once-block, the A4 value-only contract: the plaintext lives in the field's value, never a data attribute) → `announce(`Regenerated "${tok.label}" — copy the new token now; it won't be shown again.`)`.
- **404** → the row is gone (revoked elsewhere / regenerated elsewhere): `row.replaceWith(makeRow({ ...tok, revoked: true }, "revoked"))` — wait: the row is an ACTIVE-table row; a 404 means the id no longer exists at all. Mirror the revoke control's 404 handling: re-render the row in the REVOKED table's state is wrong if the row is gone — instead `row.remove()` + `announce("That token no longer exists.")`… **Decision (pinned here):** a 404 on regenerate means the row vanished (revoked AND deleted by another admin, or a stale render) — `row.remove()` + `await loadTokens()` (the re-fetch reconciles both tables) + `announce("That token was already revoked.")` (the revoke control's existing 404 line — one house message for the one common case).
- **409** → the row was revoked between render and click: `await loadTokens()` (reconciles) + `announce("That token was already revoked.")`.
- **other non-2xx / network** → `announce("Couldn't regenerate \"<label>\" — try again.")` (network: the `is the app reachable?` variant — the house two-line convention) + `restoreRegenerate()` (retryable, the button back).
- The active `makeRow`'s Actions cell: `actionsTd.append(makeRegenerateControl(tok, tr), makeRevokeControl(tok, tr))` — Regenerate FIRST (the primary lifecycle action, D2); the two controls each own their own `<span class="tokens-actions">` cell-content swap (verify the two swap-scopes don't interfere — the revoke control swaps `cell`'s children; give each control its OWN wrapper span (the regenerate control gets its own, the revoke control keeps its existing one) so a confirm in one never clobbers the other).
- The `#tokens-status` live-region contract is unchanged (every outcome lands a line — D2's copy is pinned).
- Module docstring: the regenerate section (the rotation semantics + the 404/409 copy).
2. `frontend/assets/styles.css` (the token region): `.token-regenerate` — the `.token-revoke` button's visual language (same size/border/radius/focus; color `var(--ink-soft)` on transparent, hover `var(--brand-soft)` / `var(--brand-ink)` — the neutral action's hover, NOT the revoke's error hover; a `.token-regenerate:hover:disabled` state consistent with the revoke button's disabled rule) + the comment (the rotation button — no new hue, phase-92 invariant; the confirm pair reuses the `history-confirm-*` classes unchanged).
3. Source pins (the file established in task 02): `makeRegenerateControl` (the button label + aria-label, the confirm text EXACTLY `Regenerate? The current token is revoked.`, the focus-to-Yes, the restore path), `confirmRegenerate` (the POST path `/api/tokens/` + `regenerate`, the 201 sequence — load FIRST then once-block reveal then announce (the D2 line, exact), the 404/409 copy, the neutral retry copy, the disabled-while-in-flight), the Actions cell order (Regenerate before Revoke), the per-control wrapper spans, + the `.token-regenerate` CSS pin.
## Testing & Quality
- Unit: the source pins above are this task's test layer (`app/` untouched — task 01's API pins cover the endpoint).
- Coverage: **>90%** on this task's new/modified code (frontend pins cover the JS; the browser proof is task 04's E2E).
- Note: do NOT add an E2E file here — the rotation proof lands in task 04.
## Completion Criteria
- [ ] an active row shows Regenerate (before Revoke); the confirm copy/focus/restore are pinned; Yes → 201 → the old row is in the revoked table, the new row active (same label), the once-block shows the new plaintext, the D2 line is announced (source-pinned; task 04 proves it in a browser)
- [ ] the 404 / 409 / failure branches are pinned (the house copy, the retryable restore, the load-based reconciliation)
- [ ] the revoke control's behavior is byte-identical (its pins green — the shared cell hosts two independent confirm scopes)
- [ ] full test suite green, coverage >90%
- [ ] no behavior change in completed work (the phase-79 once-block / copy / gate pins green)
@@ -0,0 +1,34 @@
# Task 04 — The dedicated E2E: the split tables, the search, the required name, the rotation + regressions + commit
**Phase:** `101_tokens_page_overhaul` · **Story:** n/a (owner request)
## Objective
Pin the owner-visible contract in a browser: revoked tokens live in their own table below the active ones, both tables search, a nameless token cannot be created, and Regenerate rotates the credential end to end (the new token signs in, the old one is refused) — then the phase-79 suite moves its revoked-row pins to the new table.
## Work
1. `tests/e2e/test_tokens_page.py` (new — the phase's dedicated A16 suite, run in isolation). Fixtures: `app_server` + `mock_llm` + `db_ready` (the `conftest` pattern); admin via `tests/e2e/auth_helpers.login(page, app_url, next="/tokens.html")`; a token-user context via a FRESH browser context + `auth_helpers.login_with_token(page, app_url, token)` (the phase-79 helper — drives the real gate). An autouse fixture deletes `e2e-`-labeled tokens (the `test_api_tokens.py` cleanup pattern — `DELETE FROM api_tokens WHERE label LIKE 'e2e-%'`).
- **`test_revoked_tokens_move_to_their_own_table`** — create two tokens through the UI create row (`e2e-act`, `e2e-rev` — label → Generate → the once-block appears each time); Revoke `e2e-rev` (the existing two-step: click Revoke → click Yes): `#tokens-tbody` shows ONLY `e2e-act`; the active table's `<thead>` has exactly four `<th>` (NO Status column); `#tokens-revoked-heading` reads `Revoked tokens` and is visible; `#tokens-revoked-tbody` shows the `e2e-rev` row with a non-empty Revoked date cell (locale format, ≠ the created date) and NO buttons in its row; `#tokens-status` reads `Revoked "e2e-rev".`.
- **`test_both_tables_are_searchable`** — with `e2e-a1` + `e2e-a2` active and one revoked token present: type `e2e-a1` in `#token-search-active` → only the `e2e-a1` row visible (`e2e-a2` hidden); type `zzz` → the no-match row visible reading `No tokens match "zzz".`; clear the input → both rows back, no-match hidden. The same three-beat on `#token-search-revoked` (filter the revoked row in/out). The queries SURVIVE a re-show: type a query, leave via the Tuning nav link, come back (the phase-77 refresh re-renders) → the filter is still applied (the visible set unchanged).
- **`test_a_token_cannot_be_generated_without_a_name`** — with the label empty, click Generate: `#token-once` stays hidden, `#tokens-status` reads `Give the token a name first.`, the active table's data-row count is unchanged, and `GET /api/tokens` (admin cookie via `page.request.get`) shows no new row (the request never happened). Whitespace-only label → the same refusal.
- **`test_regenerate_rotates_the_token`** — create `e2e-rot` (capture its plaintext from `#token-once-value`); on its row click Regenerate → the confirm pair appears (`Regenerate? The current token is revoked.`, focus on Yes) → click Yes: `#token-once` re-appears with a NEW plaintext (≠ the original, matches `bor_[0-9a-f]{32}`); `#tokens-status` reads `Regenerated "e2e-rot" — copy the new token now; it won't be shown again.`; the active table has exactly ONE `e2e-rot` row (the new one — its created cell is newer than the old row's); the revoked table now holds an `e2e-rot` row (the original — its Revoked date is set); a fresh token-user context signs in with the NEW token (the gate closes, the chat view usable — `login_with_token` + an assertion on the signed-in state) and the ORIGINAL token is refused in another fresh context (the `#auth-gate-error` role=alert visible after submitting it — the rotation killed it, the phase-79 revocation semantics).
2. `tests/e2e/test_api_tokens.py` — UPDATE IN PLACE (the phase-97 task-07/08 precedent — the layout changed, the auth contracts did not):
- The revoked-row assertions relocate to `#tokens-revoked-tbody`: `test_admin_generates_token_in_ui` (its post-revoke expectations — the row's NEW home; the `.stale-pill` assertion drops with the Status column — assert the revoked row + its Revoked date cell instead) and `test_revocation_closes_the_door` (the two-step revoke now lands the row in the revoked table — update the locator + the pill assertion the same way; the `Revoked "e2e-revoke".` live-region line is UNCHANGED).
- Any Status-column / 5-`<th>` / `colspan="5"` references update to the four-column contract; `#tokens-empty-row`'s colspan is 4.
- EVERYTHING else — the anonymous lockout, the shared-chats-stay-open, the token-user app usage, the cached-token reload, the 403 walls, the sign-out clears, the wrong-token generic error — stays byte-identical (this suite is the phase-79 auth contract; only its layout locators move).
3. Regression sweep (each in isolation, `--no-cov`, DB up): the updated `test_api_tokens.py` + `test_admin_auth.py` (the gate contract — expected untouched) + `test_shared_header.py` (the whoami/header plumbing — expected untouched).
4. Full gates + commit: `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean; move `101_tokens_page_overhaul` → `.agents/phases/complete/`; one atomic commit:
```bash
git add -A .agents/ app/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(tokens): split active and revoked tables with search, require a name, and add one-click regenerate"
```
## Testing & Quality
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_tokens_page.py -v --no-cov` green in isolation.
- Coverage: **>90%** on `app/` (the full-suite gate).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] the four E2E tests pass in isolation (the split + the column contract, both searches incl. the re-show persistence, the nameless refusal, the full rotation with the new-token-signs-in / old-token-refused proof)
- [ ] the updated `test_api_tokens.py` passes in isolation (the phase-79 auth contract intact, the layout pins relocated)
- [ ] the regression suites pass in isolation
- [ ] full suite green, coverage >90%, ruff + pyright clean
- [ ] phase dir moved to `complete/`, one atomic `--no-gpg-sign` Conventional Commit
@@ -0,0 +1,61 @@
# Phase 102 — Extensionless filenames: sync `Dockerfile` / `Containerfile` when their name is in `BOR_IMPORT_EXTENSIONS`
**Source:** owner report (chat): "files without extensions never get synced, so 'Dockerfile' and 'Containerfile' get skipped even if I put them in BOR_IMPORT_EXTENSIONS."
**Story:** n/a (defect — the import pipeline of `38_local_directory_sources` / `56_import_extensions_env`; the badge display rides on the phase-10 viewer).
**Context:** the importer's format filter is `path.suffix.lower() in extensions` (`app/rag/importer.py`, `iter_importable_files`) and `Settings.import_extension_set` (the dotted set, `app/config.py`) feeds it. `Path("Dockerfile").suffix` is `""`, so **extensionless files never match — no value of `BOR_IMPORT_EXTENSIONS` can import them**. The validator already accepts `dockerfile` as a well-formed token (lowercase alphanumerics, 1-16 chars, phase 56 — "any extension"), so today the user can *type* the format and it still silently imports nothing. The viewer's format badge (`doc_format`, `app/api/docs.py`, rendered by `frontend/assets/document.js`) falls back to `text` for any suffix-less path; the import `formats=` counter (`ImportSummary.formats`) keys extensionless files under `unknown`.
## Objective
An extensionless file is imported **iff its lowercased full filename is a token in `BOR_IMPORT_EXTENSIONS`** — exact name, case-insensitive. With `dockerfile,containerfile` in the env (the owner's current config), `Dockerfile` and `Containerfile` sync like any other A9 file: chunks, `lite` summary, Sources tree, viewer with a truthful format badge.
## Dependencies
- `56_import_extensions_env` (the env-driven extension scope this phase extends) — complete
- `38_local_directory_sources` / `97_kb_tree_catalog` (import pipeline + drill-down tree the E2E asserts on) — complete
## Design (shared by all tasks — the executor reads this, not the chat)
### D1 — the matching rule (task 01)
A file under a source root is in scope **iff**:
1. its lowercased dotted suffix is a member of `import_extension_set` (the existing rule — `kubernetes.md` → `.md`), **or**
2. it has **no suffix** and its lowercased **full filename** equals a bare token of `import_extension_set` (`Dockerfile` → `dockerfile`).
Consequences (pinned by tests): `DOCKERFILE` matches (case-insensitive name); `mydockerfile` and `dockerfile`-suffixed lookalikes do **not** (exact name only — the list is user-controlled and exact is the only predictable rule); `Dockerfile.dev` is governed by its `.dev` suffix as today; dot-prefixed files (`.dockerfile`) stay skipped by the pre-existing hidden-component rule; the rule applies at any depth (`services/api/Dockerfile` matches).
### D2 — one choke point (task 01)
`app/rag/importer.py` gains `match_extension(path: Path, extensions: frozenset[str]) -> str | None` — returns the matched **bare lowercased token** (``md``, ``dockerfile``) or `None` when out of scope; `extensions` keeps the existing dotted form (``{'.md'}``), bare tokens are derived by stripping the leading dot. `iter_importable_files` filters with `match_extension(path, extensions) is not None` (the walk and the phase-64 progress pre-walk both call this function — one change covers both, `total` stays honest). The `summary.formats` counter in `import_sources` uses `match_extension(path, …) or "unknown"`, so an imported `Dockerfile` counts under `dockerfile:1` in the PLAN §9 line, not `unknown:1`.
### D3 — the badge (task 02)
`doc_format(path, extensions: frozenset[str] = frozenset())` in `app/api/docs.py`: a **non-empty suffix still returns the suffix token unconditionally** (display must never depend on the import list — an out-of-scope `readme.rst` still badges `rst`); a suffix-less path returns the name token when it matches per D1, else `text` (today's fallback, byte-identical for every existing row). The content endpoint (`GET /api/documents/content`) passes `get_settings().import_extension_set` (`app.config`, lru-cached). `frontend/assets/document.js` needs no change: only `md`/`markdown` trigger markdown rendering, so a `dockerfile` badge renders the raw `<pre>` view — correct for a build file.
### D4 — no changes (pinned)
Title stays `full_path.stem` (`Dockerfile` — already correct); the phase-30 `lite` summary applies (non-markdown); prune semantics unchanged — a file that stops matching (file renamed, or token removed from the env) leaves `seen` and is deleted by the next `prune=True` run (the A9 junk precedent).
### D5 — docs
`app/config.py` `import_extensions` docstring + the `.env.example` "Import scope" comment gain the extensionless rule, with `dockerfile,containerfile` as the example (no default-list change — the built-in A9 default stays exactly as phase 56 locked it).
## Tasks
1. `01_name_token_matching.md` — `match_extension` in the importer + the walk + the `formats` counter (unit-pinned).
2. `02_format_badge_and_docs.md` — the viewer badge via `doc_format` + config/`.env.example` docs.
3. `03_integration_and_e2e.md` — the dedicated integration suite + the Playwright story suite + regressions + atomic commit.
## Testing & Quality
- Unit: `tests/unit/test_importer.py` — the matching matrix (D1), walk inclusion/exclusion, the `formats` counter; `tests/unit/test_document_viewer.py` — `doc_format` (with/without the token set, suffix precedence, `text` fallback).
- Integration: `tests/integration/test_import_extensionless.py` (new) — in-process `import_sources` against a story-dedicated fixture dir with the deterministic mock LLM (the phase-02/56 seeding pattern): positive import (counts, per-format `dockerfile:1,containerfile:1,md:1`, lite summary generated for the extensionless files), negative import (no token → not walked), prune-on-token-removal; `tests/integration/test_document_content.py` — the `format` field carries `dockerfile` for an extensionless row and `text` for an unknown-name row.
- E2E (mandatory, house rule 4): `tests/e2e/test_extensionless_import.py`, run in isolation (DB up) — admin Signs in, the drill-down tree lists `Dockerfile`/`Containerfile`, the viewer modal shows badge `dockerfile` + title + summary, `Makefile` (no token) is absent, rows cleaned up in `finally`.
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`) — the phase is "Complete" only when the full suite passes and the gate holds.
## Completion Criteria
- [ ] `BOR_IMPORT_EXTENSIONS` containing `dockerfile,containerfile` (plus the usual formats) makes a `Dockerfile` and a `Containerfile` under a source sync end-to-end: imported, chunked, `lite`-summarized, listed in the Sources drill-down tree, openable in the viewer.
- [ ] Case-insensitivity: `DOCKERFILE` imports under the `dockerfile` token; `mydockerfile` does **not**; `Dockerfile.dev` follows its `.dev` suffix; `.dockerfile` stays hidden-skipped.
- [ ] The import `formats=` counter reports `dockerfile:1` (not `unknown:1`) for an imported `Dockerfile`.
- [ ] The viewer format badge shows `dockerfile`/`containerfile`; a suffix-less file whose name is not in the env still badges `text`; every suffixed row's badge is unchanged.
- [ ] Removing a token from the env prunes the previously-imported extensionless documents on the next `prune=True` run.
- [ ] The existing suites stay green in isolation: `test_import_documents.py`, `test_import_extensions_env.py`, `test_quadlet_jinja_import.py`, `test_document_viewer.py`, `test_kb_tree.py`.
- [ ] `uv run pytest` green; `app/` coverage >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] `uv run pytest tests/e2e/test_extensionless_import.py -v --no-cov` green in isolation (DB up).
- [ ] No behavior change to completed phases for any suffixed path (the existing walk rule is untouched — only the suffix-less branch is new).
- [ ] One atomic Conventional Commit, `--no-gpg-sign` (e.g. `fix(import): sync extensionless files (Dockerfile, Containerfile) named in BOR_IMPORT_EXTENSIONS`), `.agents/` phase files moved to `complete/` with the phase.
## Commit
```bash
git add app/ tests/ .env.example .agents/phases/ && git commit --no-gpg-sign -m "fix(import): sync extensionless files (Dockerfile, Containerfile) named in BOR_IMPORT_EXTENSIONS"
```
@@ -0,0 +1,42 @@
# Task 01 — The name-token match: `match_extension`, the walk, the `formats` counter
**Phase:** `102_extensionless_filenames` · **Source:** owner report — "files without extensions never get synced … 'Dockerfile' and 'Containerfile' get skipped even if I put them in BOR_IMPORT_EXTENSIONS."
**Story:** n/a (defect).
## Objective
One pure helper decides import scope; the walk and the per-format counter use it; extensionless files whose lowercased full name is a `BOR_IMPORT_EXTENSIONS` token are finally imported.
## Work
1. `app/rag/importer.py` — new module-level pure function, placed near `iter_importable_files`:
```python
def match_extension(path: Path, extensions: frozenset[str]) -> str | None:
"""The bare lowercased token *path* imports under, or ``None``.
1. Non-empty lowercased dotted suffix in *extensions* (the A9 rule —
``kubernetes.md`` → ``md``).
2. No suffix: the lowercased FULL filename equals a bare token of
*extensions* (``Dockerfile`` → ``dockerfile``) — the phase-102
extensionless rule. Exact name only: ``mydockerfile`` never
matches the ``dockerfile`` token.
"""
```
*extensions* keeps the existing dotted form (``{'.md', '.dockerfile'}``); derive the bare token set by stripping the leading dot. Rule 1 first; rule 2 only when ``path.suffix`` is empty.
2. `app/rag/importer.py` — `iter_importable_files`: replace the `path.suffix.lower() not in extensions` skip with `match_extension(path, extensions) is None`. The hidden-component check and the phase-89 ignore-prefix check stay before it, byte-identical.
3. `app/rag/importer.py` — the `summary.formats` line in `import_sources` (`ext = path.suffix.lower().lstrip(".") or "unknown"`): use `match_extension(path, llm.settings.import_extension_set) or "unknown"` so an imported `Dockerfile` counts under `dockerfile`, never `unknown`.
4. `app/rag/importer.py` — module docstring: one sentence for the extensionless rule in the scope paragraph (tokens may also name extensionless files by exact lowercased name — phase 102).
ASSUMPTIONS (pinned in `00_phase.md` D1/D2): exact-name, case-insensitive matching; no partial name matching; dotted set kept as the parameter form so no `app/config.py` change is needed in this task.
## Testing & Quality
- Unit (`tests/unit/test_importer.py`, new tests next to the existing `iter_importable_files` ones):
- `match_extension` matrix: `kubernetes.md` + `{'.md'}` → `md`; `Kubernetes.MD` → `md` (case-insensitive suffix, as today); `Dockerfile` + `{'.md', '.dockerfile'}` → `dockerfile`; `DOCKERFILE` → `dockerfile`; `Dockerfile` without the token → `None`; `mydockerfile` with the token → `None`; `Dockerfile.dev` with only `dockerfile` → `None` (with `.dev` it → `dev`); `readme.rst` + `{'.md'}` → `None`.
- walk: a fixture tree with `Dockerfile`, `Containerfile`, `mydockerfile`, `Dockerfile.dev`, `.dockerfile`, `notes.md` — with tokens `md,dockerfile,containerfile` the walk yields exactly `Dockerfile`, `Containerfile`, `notes.md` (sorted); the hidden `.dockerfile` stays skipped.
- counter: in-process `import_sources` (the file's existing `db` + fake-LLM pattern) over a tree with one `Dockerfile` + one `.md` → `summary.formats == {"dockerfile": 1, "md": 1}` and the PLAN §9 log line carries `dockerfile:1` (not `unknown`).
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
## Completion Criteria
- [ ] `match_extension` exists, pure, dotted-set parameter, rule order pinned by the matrix above
- [ ] `iter_importable_files` + the progress pre-walk import extensionless name-token files (both go through the helper)
- [ ] the `formats` counter reports the matched token, never `unknown`, for in-scope files
- [ ] full test suite green, coverage >90%, `uv run ruff check . && uv run pyright` clean
- [ ] no behavior change for any suffixed path (existing `test_importer.py` cases byte-identical)
@@ -0,0 +1,32 @@
# Task 02 — The truthful badge: `doc_format` learns the name-token rule + config/env docs
**Phase:** `102_extensionless_filenames` · **Story:** n/a (defect, display half).
## Objective
The viewer's format badge shows `dockerfile`/`containerfile` for extensionless documents that matched by name (instead of the generic `text`), and the config/`.env.example` docs tell the user the extensionless rule exists.
## Work
1. `app/api/docs.py` — `doc_format(path: str, extensions: frozenset[str] = frozenset()) -> str`:
- non-empty suffix → the suffix token, **unconditionally** (display never depends on the import list — an out-of-scope `readme.rst` still badges `rst`);
- no suffix → reuse the task-01 helper semantics from `app/rag/importer.py` (import `match_extension` — `app.api.docs` already imports from `app.rag`): the lowercased full name returns the token when it is one of `extensions` (dotted form, as passed), else `text`.
- Default `extensions=frozenset()` ⇒ today's exact behavior for every caller that passes nothing (byte-identical).
- Docstring: update the fallback line — ``text`` when the path has no suffix **and its name is not a configured token (phase 102)**.
2. `app/api/docs.py` — the content endpoint (`GET /api/documents/content`, `format=doc_format(doc.path)` at ~L143): pass `get_settings().import_extension_set` (`from app.config import get_settings` — the lru-cached accessor; no per-request Settings construction).
3. `app/config.py` — `import_extensions` docstring: one sentence — a token also matches **extensionless files whose lowercased full filename equals it exactly** (`dockerfile` → `Dockerfile`), case-insensitive, no partial names (phase 102).
4. `.env.example` — "Import scope" comment: add the same rule with the example `dockerfile,containerfile` (e.g. `add your own (e.g. md,sh,toml,dockerfile,containerfile)`); the default `BOR_IMPORT_EXTENSIONS` line is unchanged.
5. No frontend change: `frontend/assets/document.js` renders `doc.format` into the `format-badge` span and only `md`/`markdown` get markdown rendering — a `dockerfile` badge already lands in the raw `<pre>` view, which is the correct treatment for a build file. Verify, don't edit.
## Testing & Quality
- Integration (`tests/integration/test_document_content.py`, extend the `test_content_format_from_suffix` neighborhood — seed via the file's existing `_seed_doc` + TRUNCATE-in-`finally` pattern):
- seeded `path="services/api/Dockerfile"` (source `Homelab`) → `body["format"] == "dockerfile"` under the default settings (the endpoint reads the real `get_settings()` — if the test needs the token, set `BOR_IMPORT_EXTENSIONS` for the client/session via the house env-override pattern used by other API tests; the default list does **not** contain `dockerfile`, so the assertion must run with a settings carrying it);
- seeded `path="README"` → `format == "text"` (unchanged — `readme` is not a token; the existing line at ~L554 stays green);
- seeded `path="notes/README.dev"` → `format == "dev"` (suffix precedence — display ignores the import list);
- `doc_format` directly: default-args call on an extensionless path → `text` (the no-arg contract pinned).
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
## Completion Criteria
- [ ] the content endpoint's `format` field carries the name token for an extensionless document whose name is in `BOR_IMPORT_EXTENSIONS`, `text` otherwise, and every suffixed value is unchanged
- [ ] `doc_format` with no second argument behaves byte-identically to today
- [ ] `app/config.py` docstring + `.env.example` document the extensionless rule with the `dockerfile,containerfile` example
- [ ] full test suite green, coverage >90%, `uv run ruff check . && uv run pyright` clean
- [ ] `frontend/assets/document.js` verified untouched (raw `<pre>` rendering is correct for the new badge values)
@@ -0,0 +1,39 @@
# Task 03 — The proof: dedicated integration suite, the Playwright story suite, regressions, commit
**Phase:** `102_extensionless_filenames` · **Story:** n/a (defect — phase-level E2E per house rule 4).
## Objective
End-to-end proof that `Dockerfile`/`Containerfile` sync when their names are in `BOR_IMPORT_EXTENSIONS` — import → chunks → `lite` summary → drill-down tree → viewer badge — plus the negative and prune proofs, regressions, and the atomic commit.
## Work
1. Fixture `tests/fixtures/extensionless_kb/` (new, story-dedicated, distinctive source name = the dir name, the phase-56 pattern):
- `Dockerfile` — a few lines of plausible compose/build content carrying a unique sentinel (e.g. `DOCKERFILE-PROBE-SENTINEL-7a3e`);
- `Containerfile` — same shape, its own sentinel;
- `Makefile` — a few lines, **no token in the env under test** (the negative control);
- `notes.md` — one short markdown doc (the control row + the `md` count).
2. Integration `tests/integration/test_import_extensionless.py` (new; the `tests/integration/test_import_extensions_env.py` pattern — in-process `import_sources` against a fixture/tmp tree, `Settings(_env_file=None, import_extensions=…)` + the house mock LLM, `db` fixture, TRUNCATE/delete rows in `finally`):
- positive: extensions `md,dockerfile,containerfile` over the fixture → `added == 3`, `errors == 0`, `formats == {"dockerfile": 1, "containerfile": 1, "md": 1}` (NO `unknown` key), the `Dockerfile` row has chunks and a `lite` summary (non-markdown → phase-30 path);
- case: a `DOCKERFILE` file + token `dockerfile` imports (its row's path is `DOCKERFILE`);
- negative: extensions `md` only → `notes.md` imported, `Dockerfile`/`Containerfile`/`Makefile` absent from `documents`;
- prune: import with the tokens, then re-run `prune=True` with extensions `md` → the two extensionless rows are deleted, the `md` row survives (mirrors `test_prune_removes_files_now_excluded_by_format_filter`).
3. E2E `tests/e2e/test_extensionless_import.py` (new; **copy the skeleton of `tests/e2e/test_import_extensions_env.py`** — the module-docstring conventions, the in-thread seeding pattern, `_drill`/`_go_top`, the distinctive-source row cleanup in `finally`):
- boot/app with the house `mock_llm` + `db_ready` fixtures; seed via `import_sources([FIXTURES], LLMClient(settings))` with `import_extensions="md,dockerfile,containerfile"` (`Settings(_env_file=None, llm_base_url=…, import_extensions=…)`);
- test 1 (admin): summary assertions as in the integration positive case, then `login(page, app_url)`, `_drill(page, SOURCE)` — rows for `Dockerfile`, `Containerfile`, `notes.md` each count 1; `Makefile` count 0 (negative control); click the `Dockerfile` path link → same-page modal: `#doc-modal-meta .doc-source-badge` = source, `.format-badge` = `dockerfile` (the D3 badge), title `Dockerfile`, `pre.doc-raw` contains the sentinel, the phase-30 summary line renders (house assertion style — `test_summary_in_viewer.py`); still on `/sources.html`;
- test 2 (anonymous): the sign-in gate renders and `GET /api/docs` 403s (the phase-16 contract, same shape as the phase-56 suite's second test).
4. Regressions (isolated runs, per AGENTS.md rule 9): `tests/e2e/test_import_documents.py`, `test_import_extensions_env.py`, `test_quadlet_jinja_import.py`, `test_document_viewer.py`, `test_kb_tree.py`.
5. Commit: full suite + `--cov=app` >90% + `uv run ruff check . && uv run pyright` clean, then move this phase dir `102_extensionless_filenames/` to `.agents/phases/complete/` and commit **everything together** (AGENTS.md rule 8 — `.agents/` is tracked), one atomic commit:
```bash
git add app/ tests/ .env.example .agents/phases/ && git commit --no-gpg-sign -m "fix(import): sync extensionless files (Dockerfile, Containerfile) named in BOR_IMPORT_EXTENSIONS"
```
## Testing & Quality
- The suites above ARE this task's tests; coverage gate **>90%** on `app/` (the fixture files and the E2E file need no coverage, `app/` is the measured package).
- E2E runs in isolation: `uv run pytest tests/e2e/test_extensionless_import.py -v --no-cov` (DB up: `podman compose up -d db`).
## Completion Criteria
- [ ] `tests/fixtures/extensionless_kb/` exists with `Dockerfile`, `Containerfile`, `Makefile`, `notes.md` (sentinels unique to this suite)
- [ ] integration suite green: positive (counts + `dockerfile:1,containerfile:1,md:1` + summary), case, negative, prune
- [ ] E2E green in isolation: tree rows (incl. `Makefile` absent), the `dockerfile` format badge in the modal, the raw-content sentinel, the anonymous gate
- [ ] all five regression suites green in isolation
- [ ] `uv run pytest` green, `app/` coverage >90%, `uv run ruff check . && uv run pyright` clean
- [ ] one atomic Conventional Commit with `--no-gpg-sign` (message above); phase dir moved to `.agents/phases/complete/` in the same commit
@@ -0,0 +1,85 @@
# Phase 98 — Sync makes summary generation visible: status phases + "summary pending" in the catalog
**Source:** Owner request (chat, 2026-09-12) — "When syncing sources, the user has no idea when summaries (document or directory) are happening, they just see the number pause for a really long time. Also before the summaries generate it looks like those directories/files were missed, the UI should tell the user those summaries are waiting to generate."
**Story:** n/a (owner request — the sync progress surface on `64_sync_upload_progress`; the folder-summary generator + gap-fill on `94_ls_tree_drilldown` / `96_oneshot_resilience`; the drill-down catalog tree on `97_kb_tree_catalog`).
**Context:** `POST /api/sync` runs in-process as one background task (`app/api/sync.py` `_run_sync`): model check → clone/pull → `import_sources` (the ONLY phase with per-file progress — the phase-64 hook feeds `SyncStatus.current_file` / `files_done` / `files_total`, which the RAG-page sync button polls every 2 s and renders as "Syncing… \<file\> (n/m)") → `regenerate_overview` (ONE `lite` call, change-gated) → `generate_folder_summaries` (one `lite` call PER candidate folder — up to hundreds; the phase-94 generator, per-folder fail-soft, `only_missing` gap-fill since phase 96) → the `sources_meta` bump. After the import finishes the count sits at its final value for the ENTIRE overview + folder-summary span (minutes on a large KB) — the user's "number pauses for a really long time": nothing on the wire says what is happening. `GET /api/sync/status` today returns `state` / `started_at` / `finished_at` / `detail` / `error` / `current_file` / `files_done` / `files_total` (null/0 idle; terminal states clear `current_file` but keep the final counts). The catalog tree (`GET /api/docs/tree`, the pure `build_kb_tree` in `app/api/docs.py` + the `KbTree*` schemas in `app/schemas.py`) renders a Description cell per source/folder from the stored `folder_summaries` rows — a folder with NO stored row shows an EMPTY cell (the agent's `ls` shows the count only, the phase-94 rule) — after a fail-soft miss or a cleared manual description it looks like the folder was missed, when in fact the next sync's gap-fill (`missing_folder_summaries`, phase 96) will generate it. `generate_folder_summaries(db, llm, *, skip=False, only_missing=False)` iterates `sorted(candidates)` (candidate = recursive subtree ≥ `MIN_DOCS_PER_FOLDER = 2` docs, `app/rag/folder_summaries.py`), skipping manual rows (`kept_manual`) and failing soft per folder (`failed`). The RAG view's sync machinery lives in `frontend/assets/sources.js` (`fmtSyncLabel`, `enterSyncRunningState`, the two-job poll decision tree — the upload job's status has no summary phases and stays bare "Importing…"). E2E conventions: the phase-64 `tests/e2e/test_sync_upload_progress.py` tight-poll recorder (a daemon thread polling the status endpoint at ~100 ms while the UI's 2 s poll drives the label) + the `tests/e2e/slow_llm.py` proxy (per-request delay, `SLOW_DELAY_S`) that stretches a run past the poll cadence; the phase-96 `tests/e2e/test_oneshot_llm_retry.py` direct-DB row deletion + re-sync pattern; the mock LLM's deterministic `FOLDER_SUMMARY_MODE` one-liner (`Fixture folder summary for <folder>.`).
## Objective
While a sync runs, the status endpoint and the sync button tell the user exactly which phase the run is in — the import phase keeps its byte-identical file label, the KB-overview phase is named, and the (long) folder-summary phase reports the folder being summarized plus a done/total count — and after a sync, every source/folder whose summary is due but missing shows an explicit "Summary pending" marker in the catalog (row Description cell AND the level block) instead of an empty cell that reads as "missed". The pending set is exactly the candidate set the phase-96 gap-fill regenerates on the next sync — the marker's copy says so.
## Dependencies
- `97_kb_tree_catalog` (complete) — the tree endpoint / `build_kb_tree` / `KbTree*` schemas / the RAG-view tree UI (`makeDescCell`, the level block, `renderLevel`).
- `96_oneshot_resilience` (complete) — `missing_folder_summaries` + the `only_missing` gap-fill (the pending marker's honest "waiting" semantics) + the E2E row-deletion pattern.
- `94_ls_tree_drilldown` (complete) — the folder-summary generator, `MIN_DOCS_PER_FOLDER`, the per-folder fail-soft loop this phase instruments.
- `64_sync_upload_progress` (complete) — `SyncStatus`'s progress fields, the 2 s poll, the live-label contract this phase extends.
## Decisions recorded here (owner review — PLAN.md is being redone by the owner)
- **D1 — additive status fields, existing contract byte-identical:** `SyncStatus` / the status JSON gain exactly four keys — `phase` (`"import"` | `"overview"` | `"summaries"` | null), `current_summary` (the `source` / `source/folder` being summarized, or null), `summaries_done`, `summaries_total` (ints, 0 idle/terminal-reset). The model-check and clone/pull prelude keeps `phase: null` (the bare "Syncing…" label stays — the phase-64 pins hold); terminal states clear `phase` + `current_summary` and KEEP the final `summaries_done` / `summaries_total` (the phase-64 keep-final-counts convention). `current_file` / `files_done` / `files_total` / `detail` are untouched.
- **D2 — the labels (UI):** the sync button's running label for the sync job becomes: `phase "overview"` → `Writing KB overview…`; `phase "summaries"` → `Summarizing folders… <current_summary> (n/m)` (the folder part omitted when `current_summary` is null — the first poll of the phase); anything else → today's `Syncing… <file> (n/m)` logic, byte-identical (import + prelude). The UPLOAD job's label is untouched (its status endpoint has no phase). The untruncated label still rides the button `title` + `#sync-result` (the aria-live announcer); CSS ellipsizes the label span only (phase-64 A4 contract).
- **D3 — the pending rule (ONE concept):** a source or folder node is `summary_pending` ⟺ its recursive document count ≥ `MIN_DOCS_PER_FOLDER` (2) AND it has NO stored `folder_summaries` row (AI or manual — the builder sees stored rows only). That is exactly `missing_folder_summaries`'s candidate set (phase 96) — the marker is honest: the next sync's gap-fill (or the changed-KB regeneration) will generate it. A < 2-document folder is NEVER pending (it never gets a summary — its one file line IS its description). FILE nodes carry no pending flag — the file table has no description column and the pending concept is the folder-summary one (the owner's "directories/files" is served by the source + folder rows, which are the catalog's description-bearing rows).
- **D4 — the marker's surfaces:** the row Description cell shows the muted text `Summary pending` (a `title` carries "No stored description yet — the next sync will generate one.") with the ALWAYS-present Edit button kept (a manual save creates the row and clears the marker in place); the level block (`#kb-level`) shows when the current level has a stored description (as today) **or** is pending (title + a pending note line + the Edit button — write one manually right now); neither → hidden (today's behavior). State is text + color, never color alone (B5); the muted ink-soft pair is AA on the surface (no new hue — the phase-92 monochrome invariant).
- **D5 — the generator gets an optional progress hook:** `generate_folder_summaries` gains `on_progress: Callable[[int, int, str, str], None] | None` (done, total, source, folder_path) — called once per sorted candidate BEFORE its attempt (manual skips advance the counter — they are instant). `None` (the `scripts/import_docs.py` CLI path) is a no-op: the CLI's log-only stats contract is unchanged. No new endpoint, no new env var, no SSE event, no migration.
## Design (shared by all tasks — the executor reads this, not the chat)
### The sync status phases (task 01)
- `app/api/sync.py` — `SyncStatus` dataclass + the `/status` response gain the four D1 fields (defaults null/0; reset with the run at the top of `_run_sync`, exactly where `current_file` resets). `_run_sync` sets `_status.phase = "import"` immediately before `import_sources`; wraps `regenerate_overview` with `phase = "overview"` (and back — to `"summaries"` when the folder branch runs, else the run proceeds to the bump/terminal); the folder-summary branch (changed-KB full regeneration **or** the gap-fill) sets `phase = "summaries"` and passes the hook:
```python
def _summary_hook(done: int, total: int, source: str, folder_path: str) -> None:
_status.current_summary = source if folder_path == "" else f"{source}/{folder_path}"
_status.summaries_done = done
_status.summaries_total = total
```
(the closure-captures-`_status` convention the file's `_hook` already uses). The no-gap skip branch sets no phase (stays `"import"`). Terminal paths (success AND the except block) set `phase = None`, `current_summary = None` — keep `summaries_done` / `summaries_total` (final counts, D1).
- `app/rag/folder_summaries.py` — `generate_folder_summaries(..., on_progress: Callable[[int, int, str, str], None] | None = None)`: inside the `for` loop over `keys`, index `i` (enumerate), call `on_progress(i + 1, len(keys), source, folder_path)` before the manual-skip check (the counter advances for instant skips, D5). The stats dict, the fail-soft behavior, the logger line, and the flush-only transaction contract are untouched. Docstrings updated (the module rule).
- `app/api/sync.py`'s module docstring: the status paragraph names the four new fields + the phase machine (the house "docstrings carry the contracts" rule).
### The sync button (task 02)
- `frontend/assets/sources.js` — the sync-job running label: replace the single `fmtSyncLabel(kind, currentFile, done, total)` call sites for the SYNC job with a phase-aware builder (the upload job keeps the bare label — its status has no phase):
- `phase === "overview"` → `Writing KB overview…`
- `phase === "summaries"` → `Summarizing folders… ` + (current_summary ? `${current_summary} ` : ``) + `(${summaries_done}/${summaries_total})`
- otherwise → today's `Syncing… [current_file] [(done/total)]` (byte-identical — the phase-64 pins).
`enterSyncRunningState` (and the poll's `enterSyncRunningState("sync", …)` call + `initSyncButton`'s re-attach) thread the phase fields through; the button `title` + `#sync-result` mirror the full untruncated label (A4). The load-time re-attach (`initSyncButton`) re-enters a RUNNING run with whatever phase the status reports (a mid-summaries reload shows the summaries label — the never-stale contract).
- `tests/unit/test_frontend_sync_upload.py` — the label-builder pins gain the three phase cases (+ the upload-unchanged negative case).
### The tree pending flag (task 03)
- `app/schemas.py` — `KbTreeFolder` and `KbTreeSource` gain `summary_pending: bool = False` (wire-additive; `KbTreeFile` untouched).
- `app/api/docs.py` — `build_kb_tree`: import `MIN_DOCS_PER_FOLDER` from `app.rag.folder_summaries` (the builder already imports `folder_of` from there — no new dependency edge). A source node: pending ⟺ its recursive document count (the builder already computes per-source rows) ≥ `MIN_DOCS_PER_FOLDER` AND `(source, "")` not in `summaries`. A folder node in `_level_children`: pending ⟺ `counts[sub] >= MIN_DOCS_PER_FOLDER` AND `(source, sub)` not in `summaries`. The endpoint's fetches are unchanged (the `summaries` mapping already carries ALL stored rows).
- Unit (`tests/unit/test_kb_tree_builder.py`): the pending matrix — a ≥2-doc folder with no row → true; with a stored row (any) → false; a 1-doc folder with no row → false (never pending); the source root with ≥2 docs and no `(source, "")` row → true on the source node; a registered 0-document source → false; multi-source independence.
- Integration (`tests/integration/test_docs_api.py`): the tree endpoint returns `summary_pending` — and the CROSS-CHECK (D3, one concept): for a seeded dataset with a partial summary table, the set of `(source, folder_path)` flagged pending in the tree (root = `""`) equals `missing_folder_summaries(db)` (phase 96's public function) — the marker can never drift from the gap-fill.
### The pending UI (task 04)
- `frontend/assets/sources.js`:
- `makeDescCell` — when `node.summary` is empty (or absent) AND `node.summary_pending` → the text span carries the marker: class `kb-summary-pending`, text `Summary pending`, `title` = `No stored description yet — the next sync will generate one.` (textContent only — the house rule). The Edit button is unaffected (always present — a manual save CREATES the row). The editor's success path (`node.summary = data.summary` in `wireDescriptionEdit`) additionally clears the flag in place: `node.summary_pending = false` (a created description is no longer pending — no re-fetch).
- `renderLevel` — the level block shows when `node.summary` (today) OR `node.summary_pending` (new): title as today (the full source-relative path); the summary `<p>` shows the stored text, or the pending note `No description stored yet — the next sync will generate one. (You can write one yourself.)` when pending; the block stays hidden for a non-pending level with no stored description (the ls rule, unchanged).
- `frontend/assets/styles.css` — `.kb-summary-pending { color: var(--ink-soft); }` (5.1:1 on `--surface`, AA; no italic, no new hue) — the row-cell font-size already applies (the class sits on the existing text span).
- Source pins (`tests/unit/test_kb_tree_ui.py` + the styles.css pins there): the marker copy + title, the pending branch in `makeDescCell`/`renderLevel`, the in-place flag clear on save, the class name in the CSS.
### The E2E (task 05)
`tests/e2e/test_sync_summary_visibility.py` (new; `app_server` + `mock_llm` + `db_ready` fixtures; admin login via `tests/e2e/auth_helpers.py`; the temp-local-source seeding + `POST /api/sync` pattern from `test_ls_tree_drilldown.py` / `test_oneshot_llm_retry.py`; the sync leg of the summary-phase test runs against the `slow_llm` proxy — the `test_sync_upload_progress.py` fixture, `SLOW_DELAY_S` sized so the overview + 3 folder calls outlive the recorder's 100 ms cadence by ~15×):
1. **Endpoint — the phase machine:** seed a temp local source with TWO ≥ 2-doc folders (candidates: the source root + 2 folders = 3); start a KB-changing sync; the tight-poll recorder asserts: some running tick has `phase == "overview"`; some running ticks have `phase == "summaries"` with `summaries_total == 3`, `current_summary` non-null (starts with the source name; the root call is the bare source name), `summaries_done` monotonically increasing up to 3; every `phase == "summaries"` tick keeps `files_done == files_total` (the import finished — the pause the user reported); the terminal tick has `phase` null, `current_summary` null, and `summaries_done == summaries_total == 3`.
2. **UI — the label:** a fresh admin page on `/sources.html` starts the sync (button click) and the page's own 2 s poll renders a label matching `/Summarizing folders/ ` with `(n/3)` at some point (generous timeout, the phase-64 UI pattern); after settle the button reads the terminal label (the existing `Synced HH:MM` contract) and `#sync-result` carries the counts.
3. **Tree — the pending marker:** after a successful (mock-LLM) sync, DELETE one folder's `folder_summaries` row directly (the phase-96 pattern) + DELETE the source-root row; reload the RAG view (nav re-show → the phase-77 refresh re-fetch) → the two affected rows' Description cells show `Summary pending` (with the title), the intact folder's cell shows its stored `Fixture folder summary for …` line and NO marker; clicking an affected folder shows the level block with the pending note; saving a manual description from the row's Edit button clears the marker in place (the cell shows the text) — and a second unchanged sync's gap-fill regenerates the OTHER deleted row (its marker goes away, its cell carries the deterministic mock line).
## Tasks
1. `01_summary_phase_status.md` — `SyncStatus` + the status JSON gain `phase` / `current_summary` / `summaries_done` / `summaries_total`; the generator's `on_progress` hook; `_run_sync` sets the phases
2. `02_sync_label_summary_phases.md` — the sync button's phase-aware labels (overview / summaries) + the label-builder unit pins
3. `03_tree_pending_flag.md` — `summary_pending` on the tree schemas + the `build_kb_tree` rule + the `missing_folder_summaries` cross-check
4. `04_tree_pending_ui.md` — the row-cell + level-block pending markers + the in-place clear + the CSS class + source pins
5. `05_e2e_summary_visibility.md` — `tests/e2e/test_sync_summary_visibility.py` + the regression sweep + the atomic commit
## Testing & Quality
- Unit: the `on_progress` call matrix in the generator's suite (order/values, manual-skip advance, `None` no-op, `skip=True` unchanged); the status shape in `tests/unit/test_sync_button.py` (idle nulls/zeros; the four new keys); the label builders in `tests/unit/test_frontend_sync_upload.py` (D2's three cases + upload negative); the builder's pending matrix (`tests/unit/test_kb_tree_builder.py`); the source pins (`tests/unit/test_kb_tree_ui.py`, the styles.css class).
- Integration: `tests/integration/test_sync_api.py` (the status field contract across the state machine); `tests/integration/test_sync_folder_summaries.py` (the hook fires on the changed-KB AND gap-fill branches, never on the skip branch); `tests/integration/test_docs_api.py` (the pending shape + the D3 cross-check).
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_sync_summary_visibility.py -v --no-cov` in isolation; regressions green in isolation: `test_kb_tree.py`, `test_ls_tree_drilldown.py`, `test_sync_button.py`, `test_sync_upload_progress.py`, `test_oneshot_llm_retry.py`, `test_local_directory_sources.py`.
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] while a sync runs, `GET /api/sync/status` reports the phase: the import keeps `current_file`/counts with `phase "import"`, the overview shows `phase "overview"`, the folder span shows `phase "summaries"` with `current_summary` + a done/total that climbs to the candidate count; terminal states clear the phase + folder but keep the final summary counts
- [ ] the RAG-page sync button reads `Writing KB overview…` / `Summarizing folders… <folder> (n/m)` in those phases (aria-live + button title carry the untruncated text) and is byte-identical to today in the prelude/import/upload cases
- [ ] a source/folder with ≥ 2 docs and no stored summary row shows `Summary pending` in its tree row AND its level block (the marker set equals `missing_folder_summaries` — integration-pinned); a manual save or the next sync's gap-fill makes the marker go away; < 2-doc folders and file rows never show it
- [ ] the CLI import path and its log lines are byte-identical (the hook is optional, the stats contract untouched); the agent's `ls` is byte-identical (the tree reads the same rows)
- [ ] `uv run pytest` green; coverage >90%; ruff + pyright clean
- [ ] `uv run pytest tests/e2e/test_sync_summary_visibility.py -v --no-cov` green in isolation (DB up: `podman compose up -d db`); the regression suites green in isolation
- [ ] one atomic Conventional Commit, `--no-gpg-sign` (e.g. `feat(sync): surface the summary phases of a sync and mark folders whose summaries are pending`)
@@ -0,0 +1,29 @@
# Task 01 — Sync status phases: `phase` / `current_summary` / `summaries_done` / `summaries_total` + the generator progress hook
**Phase:** `98_sync_summary_visibility` · **Story:** n/a (owner request)
## Objective
`GET /api/sync/status` reports which phase a running sync is in (import / overview / folder summaries) with per-folder summary progress, so the UI (task 02) can tell the user exactly what is happening while the file count sits still.
## Work
1. `app/api/sync.py` — `SyncStatus` gains four fields (D1, `00_phase.md`): `phase: Literal["import", "overview", "summaries"] | None = None`, `current_summary: str | None = None`, `summaries_done: int = 0`, `summaries_total: int = 0`. The `/status` response dict returns them (null/0 idle — the dataclass defaults cover the never-run state). `_run_sync` resets all four at run start (next to the existing `current_file` / `files_done` / `files_total` reset) and drives them:
- `phase = "import"` immediately before `import_sources` (the model-check + clone/pull prelude stays `None` — D1).
- `phase = "overview"` before `await regenerate_overview(llm)` (inside the existing `added + updated > 0` gate).
- the folder-summary branch: when it CALLS the generator (changed-KB full regeneration OR the `only_missing` gap-fill) set `phase = "summaries"` first and pass `on_progress=_summary_hook`; the no-gap skip branch sets no phase (stays `"import"`).
- the `_summary_hook(done, total, source, folder_path)` closure (D5's shape — mirrors the existing `_hook` convention): `current_summary = source if folder_path == "" else f"{source}/{folder_path}"`, plus the done/total.
- BOTH terminal paths (the success block and the `except` block) set `phase = None` and `current_summary = None` — and keep `summaries_done` / `summaries_total` (the run's final counts, the phase-64 keep-final-counts convention).
- The module docstring's status paragraph names the four fields + the phase machine (the docstring-carries-the-contract rule).
2. `app/rag/folder_summaries.py` — `generate_folder_summaries(db, llm, *, skip=False, only_missing=False, on_progress: Callable[[int, int, str, str], None] | None = None)`: `enumerate` the `keys` loop and call `on_progress(i + 1, len(keys), source, folder_path)` BEFORE the manual-skip check (instant skips advance the counter — D5). Guard with `if on_progress is not None` at the call site (or an early `noop` default — either, but `None` from the CLI must be a zero-cost no-op). Docstrings: the generator's docstring notes the hook (done counts processed keys including instant manual skips; `total = len(keys)` at loop start) and the module docstring's generation paragraph mentions it. No change to the stats dict, the fail-soft per-folder behavior, the flush-only contract, or the logger line.
3. `scripts/import_docs.py` — UNCHANGED (it calls the generator without the hook — the log-only stats contract is untouched; verify by reading the call sites: no keyword to add).
## Testing & Quality
- Unit (`tests/unit/test_folder_summaries.py` extensions): the hook fires once per candidate in `keys` order with the right `(done, total, source, folder_path)` (done climbs 1..total, total = the candidate count); manual-skip keys still advance the counter; a failed (LLMError) key still advances; `only_missing=True` → total = the missing count; `skip=True` → zero calls; `on_progress=None` → the generator behaves byte-identically to today (the fake LLM call log unchanged).
- Unit (`tests/unit/test_sync_button.py` — the status-shape pins): the `/status` dict carries the four new keys — null/0 in the idle state; extend any running-state shape assertions.
- Integration (`tests/integration/test_sync_api.py` + `tests/integration/test_sync_folder_summaries.py`): the status contract across the state machine (idle → running carries `phase "import"` with the file hook as today; terminal success/failed clear `phase` + `current_summary` and keep the final summary counts); the hook fires on BOTH generation branches (changed-KB regeneration, unchanged-KB gap-fill) and NEVER on the no-gap skip branch (a zero-`FOLDER_SUMMARY_MODE`-call unchanged re-sync leaves `phase` at `"import"` through to the terminal).
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
## Completion Criteria
- [ ] `GET /api/sync/status` (idle) returns `phase: null`, `current_summary: null`, `summaries_done: 0`, `summaries_total: 0` alongside the unchanged existing keys
- [ ] the generator's `on_progress` call matrix is unit-pinned (order, values, skip/fail advance, `None` no-op); the CLI import path is untouched
- [ ] full test suite green, coverage >90%
- [ ] no behavior change in completed work (the phase-64/94/96 sync + folder-summary suites green)
@@ -0,0 +1,25 @@
# Task 02 — The sync button names the summary phases (label builder + pins)
**Phase:** `98_sync_summary_visibility` · **Story:** n/a (owner request)
## Objective
The RAG-page sync button (and its aria-live line) shows the user what the running sync is doing in the post-import span: `Writing KB overview…` and `Summarizing folders… <folder> (n/m)` — while the prelude, import, and upload labels stay byte-identical (the phase-64 contract).
## Work
1. `frontend/assets/sources.js` — the sync-job running label (D2, `00_phase.md`):
- Replace/augment the label path: the SYNC job's running label is now phase-aware — `status.phase === "overview"` → `Writing KB overview…`; `status.phase === "summaries"` → `Summarizing folders… ` + (status.current_summary ? `${status.current_summary} ` : ``) + `(${status.summaries_done}/${status.summaries_total})`; anything else (null phase prelude, `"import"`, and every field the old label used) → today's `Syncing… [current_file] [(done/total)]` logic, byte-identical.
- Thread the fields: `enterSyncRunningState` gains the phase fields for the sync job (extend its signature or pass the status object — keep the upload call sites on the bare label: the upload status has no phase, D2). Update the poll's `enterSyncRunningState("sync", …)` call and `initSyncButton`'s running re-attach (a reload mid-summaries must re-enter with the summaries label — the never-stale contract).
- The A4 contract carries over: the untruncated label rides the button `title` (set for every running label, not just file labels — remove/adjust the "title only when there is a current file" rule for the phase labels) and `#sync-result` (the aria-live announcer); CSS ellipsizes the label span only (no CSS change expected — verify the existing `.sync-label` ellipsis handles the longer text; the mobile `max-width: none` override already lifts the cap).
- The module docstring's sync section: update the decision-tree comments (branch 1 now names the three label forms).
2. `tests/unit/test_frontend_sync_upload.py` — label-builder pins: the overview label (exact string, no counts); the summaries label WITH `current_summary` (exact: `Summarizing folders… <src>/<folder> (1/3)` shape) and without (bare `Summarizing folders… (1/3)`); the fall-through — a null-phase and an `"import"`-phase status produce today's byte-identical labels (the existing pins keep passing); the upload job's label is untouched (negative case).
## Testing & Quality
- Unit: the label-builder matrix above (this task's core — the `00_phase.md` D2 strings are the contract, pinned to the exact copy).
- Coverage: **>90%** on this task's new/modified code (frontend source pins are the house pattern for JS — the unit file above IS the coverage for this task; `app/` unchanged).
- Note: the E2E proof of the live label lands in task 05 (the UI leg) — do NOT add an E2E file here.
## Completion Criteria
- [ ] the three sync-job label forms are pinned to the exact D2 strings (overview / summaries±folder / today's fall-through byte-identical); the upload label is pinned untouched
- [ ] a mid-summaries page reload re-enters the running state with the summaries label (the `initSyncButton` path, source-pinned)
- [ ] full test suite green, coverage >90%
- [ ] no behavior change in completed work (the phase-64 sync-label pins green)
@@ -0,0 +1,26 @@
# Task 03 — `summary_pending` on the tree: the schemas + `build_kb_tree` + the gap-fill cross-check
**Phase:** `98_sync_summary_visibility` · **Story:** n/a (owner request)
## Objective
`GET /api/docs/tree` flags every source/folder whose summary is due but missing (`summary_pending`) — exactly the candidate set the phase-96 gap-fill regenerates — so the UI (task 04) can say "waiting to generate" instead of showing an empty cell.
## Work
1. `app/schemas.py` — `KbTreeFolder` and `KbTreeSource` gain `summary_pending: bool = False` (wire-additive; the `KbTree` docstring + the two node docstrings note the D3 rule from `00_phase.md`: pending ⟺ recursive count ≥ `MIN_DOCS_PER_FOLDER` AND no stored row; `KbTreeFile` stays untouched).
2. `app/api/docs.py` — `build_kb_tree` (and `_level_children` where folder nodes are built):
- Import `MIN_DOCS_PER_FOLDER` from `app.rag.folder_summaries` (next to the existing `folder_of` import — no new dependency edge; `app.rag.folder_summaries` already depends on nothing in `app.api`).
- Folder node in `_level_children`: `summary_pending = counts[sub] >= MIN_DOCS_PER_FOLDER and (source, sub) not in summaries`.
- Source node in `build_kb_tree`: pending ⟺ the source's recursive document count (the builder already groups `doc_rows` per source) ≥ `MIN_DOCS_PER_FOLDER` AND `(source, "") not in summaries`.
- `build_kb_tree`'s docstring: the pending rule (D3) + the pointer that it is `missing_folder_summaries`'s candidate set (one concept — the cross-check below pins it).
3. Endpoint: NO fetch change (the `summaries` mapping the endpoint already builds carries ALL stored rows — AI and manual).
## Testing & Quality
- Unit (`tests/unit/test_kb_tree_builder.py` extensions) — the pending matrix: a ≥ 2-doc folder with no stored row → `summary_pending` true; the same folder WITH a stored row (the builder cannot tell AI from manual — any row) → false; a 1-doc folder with no row → false (a single-document folder never gets a summary — never pending); the name-sharing edge (the phase-94 count rule): documents `one/a` AND `one/a/b` → folder `one/a` exists and its recursive count is 2 (the document whose path EQUALS the folder name counts) → pending true with no stored row — pending follows the RECURSIVE count, not the number of direct children; the source root: a source with ≥ 2 docs and no `(source, "")` row → the SOURCE node is pending; a registered 0-document source → never pending; two sources pending independently (one with a root row, one without).
- Integration (`tests/integration/test_docs_api.py` extensions): the endpoint returns `summary_pending` on source + folder nodes (both values, a nested case); **the D3 cross-check (one concept end to end):** seed a multi-folder dataset, DELETE some `folder_summaries` rows (the phase-96 pattern), then assert `set of (source, folder_path) flagged pending in the fetched tree (root = "")` == `set(missing_folder_summaries(db))` — the marker can never drift from the gap-fill.
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
## Completion Criteria
- [ ] the tree JSON carries `summary_pending` on source + folder nodes only, computed by the D3 rule (unit matrix green)
- [ ] the integration cross-check: pending set == `missing_folder_summaries(db)` on a dataset with partial summaries
- [ ] full test suite green, coverage >90%
- [ ] no behavior change in completed work (the existing tree shape assertions in `test_kb_tree_builder.py` / `test_docs_api.py` still pass — the field is additive)
@@ -0,0 +1,27 @@
# Task 04 — The "Summary pending" markers: row cell + level block + the in-place clear
**Phase:** `98_sync_summary_visibility` · **Story:** n/a (owner request)
## Objective
The catalog tells the owner a summary is waiting: a pending source/folder row shows `Summary pending` in its Description cell (Edit button kept) and the level block shows a pending note when drilled into — writing a manual description clears the marker in place.
## Work
1. `frontend/assets/sources.js`:
- `makeDescCell` (D4, `00_phase.md`): when `node.summary` is empty/absent AND `node.summary_pending` → the text span carries the marker — class `kb-summary-pending`, textContent `Summary pending`, `title` = `No stored description yet — the next sync will generate one.` (textContent only — the house rule; the span already exists, this only changes its content/class/attribute in the pending case). The Edit button is UNCHANGED (always present — a manual save creates the row).
- `wireDescriptionEdit`'s success path: after `node.summary = data.summary`, also set `node.summary_pending = false` (a created/updated description is no longer pending — the in-place clear, no re-fetch).
- `renderLevel`'s level-block branch: show `#kb-level` when `node.summary` (as today) OR `node.summary_pending` — title as today (the full source-relative path); `#kb-level-summary` shows the stored text, or — when pending — the note `No description stored yet — the next sync will generate one. (You can write one yourself.)`; a level that is neither stored nor pending stays hidden (the ls rule, unchanged). The block's Edit button (write one manually now) already ships in the static markup.
- Module docstring: the phase-98 note (the marker's surfaces + the in-place clear).
2. `frontend/assets/styles.css` — `.kb-summary-pending { color: var(--ink-soft); }` (5.1:1 on `--surface` — AA; text + color, never color alone (B5); no italic, no new hue — the phase-92 monochrome invariant). The class sits on the existing text span, so the row cell's existing font-size/line-height apply — the marker must not change row height.
3. Source pins (`tests/unit/test_kb_tree_ui.py` + its styles.css pins): the marker copy + `title` (exact strings from D4), the pending branch condition in `makeDescCell` (pending + no summary → marker; a stored summary → the text, never the marker; non-pending + no summary → the empty cell as today), the in-place `summary_pending = false` on save, the `renderLevel` pending-note branch (exact note string), and the `.kb-summary-pending` class in the CSS.
## Testing & Quality
- Unit: the source pins above are this task's test layer (the house pattern for frontend logic — `app/` untouched).
- Coverage: **>90%** on this task's new/modified code (frontend pins cover the JS; `app/` unchanged).
- Note: the E2E proof of the markers (incl. the gap-fill recovery) lands in task 05.
## Completion Criteria
- [ ] a pending row's Description cell shows `Summary pending` (with the D4 title) + the always-present Edit button; a stored-description row and a non-pending empty row are unchanged
- [ ] drilling into a pending level shows the level block with the pending note + Edit; a stored level and a neither-stored-nor-pending level behave exactly as today
- [ ] saving a manual description clears the marker in place (source-pinned; no re-fetch)
- [ ] full test suite green, coverage >90%
- [ ] no behavior change in completed work (the phase-97 tree/editor pins green)
@@ -0,0 +1,29 @@
# Task 05 — The dedicated E2E: phase machine live, labels live, pending markers + gap-fill recovery
**Phase:** `98_sync_summary_visibility` · **Story:** n/a (owner request)
## Objective
Pin the owner-visible contract in a browser: the sync's summary phases are visible live (endpoint + button label), and missing folder summaries read "waiting to generate" — then the next sync's gap-fill makes the marker go away.
## Work
1. `tests/e2e/test_sync_summary_visibility.py` (new — the phase's dedicated A16 suite, run in isolation). Fixtures: `app_server` + `mock_llm` + `db_ready` (the `conftest` pattern); admin login via `tests/e2e/auth_helpers.py`; the temp-local-source seeding + `POST /api/sync` pattern from `test_ls_tree_drilldown.py` / `test_oneshot_llm_retry.py`; for test 1–2 the `slow_llm` proxy fixture from `tests/e2e/test_sync_upload_progress.py` (`SLOW_DELAY_S` sized so ONE LLM call outlives the recorder's ~100 ms cadence by ~15× — the phase-64 sizing rationale; the mock LLM stays on its port for the fast tests). An autouse cleanup deletes the temp source's KB rows + `folder_summaries` rows (the phase-96 cleanup pattern).
- **`test_sync_status_reports_the_summary_phases`** (endpoint — the slow-LLM leg): seed a temp local source with TWO ≥ 2-doc folders (candidate set: the source root + 2 folders = 3 — the root is a candidate too, `folder_path ""`); make the KB change-gate fire (first sync of the source); a daemon-thread recorder tight-polls `GET /api/sync/status` (~100 ms) from the 202 until the terminal state (the phase-64 recorder pattern). Assert: some running tick has `phase == "overview"`; some running ticks have `phase == "summaries"` with `summaries_total == 3`, non-null `current_summary` (starts with the source name; one tick is the bare source name — the root call), `summaries_done` strictly increasing across the summaries ticks up to 3; every `phase == "summaries"` tick has `files_done == files_total` (the import finished — the reported pause is now labeled); the terminal tick: `phase` null, `current_summary` null, `summaries_done == summaries_total == 3`, `state "success"`.
- **`test_sync_button_names_the_summary_phase`** (UI — the slow-LLM leg): a fresh admin page on `/sources.html`; click the sync button; within a generous timeout (the phase-64 UI pattern) the page's own 2 s poll renders `#sync-label` matching `/Summarizing folders/` with a `(n/3)` suffix (the label is built by the page JS — this test asserts the RENDERED text, not the endpoint); on settle the button shows the terminal `Synced …` label (the phase-32 contract) and `#sync-result` carries the counts line.
- **`test_missing_folder_summaries_read_as_pending_and_self_heal`** (tree — the fast mock LLM): seed a source with two ≥ 2-doc folders + a root, sync (mock `FOLDER_SUMMARY_MODE` → every candidate gets the deterministic `Fixture folder summary for <folder>.` line); open the RAG view: no `Summary pending` anywhere. DELETE one folder's row AND the source-root row directly (`db_ready` — the phase-96 pattern); trigger the phase-77 re-fetch (re-click the active RAG nav link, or `page.reload()`): the source row (top level) + the affected folder's row show `Summary pending` (+ the D4 title attribute), the intact folder's cell shows its stored line and NO marker; click the affected folder → the level block is visible with the pending note (exact D4 string); use the row's Edit → Save a manual description → the marker is gone in place (the cell shows the saved text — no reload); then run a second (unchanged-KB) sync → the OTHER deleted row (the source root) is gap-filled: its marker is gone, its cell/level shows the deterministic mock line.
2. `tests/e2e/mock_llm.py` — NO change expected (the `FOLDER_SUMMARY_MODE` branch + the phase-96 failure injection already cover this suite's needs); if a test needs a deterministically SLOWED folder call the slow proxy is the seam (it delays every request it proxies) — do not add new mock branches unless a test genuinely requires one.
3. Regression sweep (each in isolation, `--no-cov`, DB up): `test_kb_tree.py`, `test_ls_tree_drilldown.py`, `test_sync_button.py`, `test_sync_upload_progress.py`, `test_oneshot_llm_retry.py`, `test_local_directory_sources.py`. Fix ONLY pins broken by the additive changes (new status keys, the new schema field, the marker) — asserted behavior changes nowhere else.
4. Full gates + commit: `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean; move `98_sync_summary_visibility` → `.agents/phases/complete/`; one atomic commit:
```bash
git add -A .agents/ app/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(sync): surface the summary phases of a sync and mark folders whose summaries are pending"
```
## Testing & Quality
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_sync_summary_visibility.py -v --no-cov` green in isolation.
- Coverage: **>90%** on `app/` (the full suite gate).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] the three E2E tests above pass in isolation (the phase machine + the live label + the pending markers + the gap-fill recovery)
- [ ] the regression suites pass in isolation
- [ ] full suite green, coverage >90%, ruff + pyright clean
- [ ] phase dir moved to `complete/`, one atomic `--no-gpg-sign` Conventional Commit
@@ -0,0 +1,70 @@
# Phase 99 — Catalog table polish: one-line descriptions + the browser back button walks the breadcrumb
**Source:** Owner request (chat, 2026-09-12) — "The description columns of the table cause the rows to grow way too much in height. Just provide the first line that fits and cut it off with '...'. The user can click the folder and see the full description at the top anyway, so that's not a huge deal. Leave the edit button in the column, that's a nice touch. Clicking 'back' in the browser should take you back up the directory breadcrumbs."
**Story:** n/a (owner request — the drill-down catalog on `97_kb_tree_catalog`; the shell router on `76_spa_nav_shell` / `77_navbar_refresh`).
**Context:** The RAG view's ONE folders/sources table (`#folders-table`, `#folders-tbody`) renders `makeDescCell` rows: a Description `<td>` holding a text `<span>` (the stored description — a 1–3 sentence `lite` paragraph, or the phase-98 `Summary pending` marker) + the ALWAYS-present `Edit` button (`wireDescriptionEdit` swaps the cell's children for the inline textarea on edit). CSS: `.kb-folders-table td:nth-child(3) { white-space: normal; min-width: 18rem; max-width: 44rem; }` — the paragraph WRAPS, so a long description grows the row to many lines (the owner's "rows grow way too much in height"). The level block (`#kb-level`) shows the current directory's FULL description when drilled in (the owner's "see the full description at the top" — it stays). Navigation is purely client-side: `goTo(target)` sets the module `current = {source, folder}` and re-renders — source-row clicks, folder-link clicks, and breadcrumb-segment clicks (`crumbSegment`, `href="#"` + preventDefault) all call `goTo` with NO history entry, so the browser back button leaves the RAG view entirely instead of popping one breadcrumb level. The shell router (`frontend/assets/router.js`) pushes `{ view: name }` state on navbar switches and its `popstate` handler reads ONLY `window.location.pathname` (a state-only push that keeps the URL therefore cannot disturb it); the phase-77 `bor:view-refresh` CustomEvent re-runs a view's load on re-show (a switch back, an active-link re-click — which pushes NOTHING, or back/forward onto the view).
## Objective
Long folder descriptions stop stretching the catalog rows — the Description cell shows the first line that fits, ellipsized, with the Edit button kept in the cell (full text on hover; the full description remains at the top of the level when you click in) — and the browser back/forward buttons walk the directory breadcrumb: every drill (source, folder, or breadcrumb jump) is a history entry, so Back pops one level up, Forward re-descends, and Back at the top level leaves the RAG view exactly as today.
## Dependencies
- `97_kb_tree_catalog` (complete) — the tree UI (`makeDescCell`, `wireDescriptionEdit`, `renderLevel`, the breadcrumb, `goTo`), the level block.
- `96_oneshot_resilience` / `98_sync_summary_visibility` (todo/complete) — queue order only; if 98 lands first, the `Summary pending` marker text must remain clamped/consistent (a one-line marker — no behavior coupling, the cell restructure in task 01 simply wraps whatever the span contains).
- `76_spa_nav_shell` / `77_navbar_refresh` (complete) — the router's pushState/popstate/refresh contracts this phase integrates with (untouched).
## Decisions recorded here (owner review — PLAN.md is being redone by the owner)
- **D1 — the clamp is visual only:** the Description cell clamps to ONE line — `white-space: nowrap` + `overflow: hidden` + `text-overflow: ellipsis` on the text span, the cell a fixed one-line height (the row height becomes independent of description length). The full text stays in the DOM (the accessible name is unchanged — the ellipsis is CSS-only) AND on hover via the span's `title` (the house pattern — the tokens label cell and the folder-link `title`s do the same). The Edit button stays in the cell (owner: "that's a nice touch") — the cell is a flex row: text flexes + ellipsizes, the button is `flex: 0 0 auto`. The level block's `.kb-level p` keeps the FULL unclamped description (owner: the full text at the top is the escape hatch).
- **D2 — the drill state becomes the history state:** every `goTo` (source-row click, folder-link click, breadcrumb-segment click) does `history.pushState({ view: "rag", kb: target }, "")` — a STATE-ONLY entry (the URL stays the shell's current path; no new route, no URL contract change — the phase-76 deep-link surface is untouched). `popstate`: the entry's `state.kb` (when present) is adopted WITHOUT pushing; an entry without `kb` (the boot entry, a router view entry, any foreign state) resets the drill to the TOP level. The phase-77 `bor:view-refresh` (re-show) aligns `current` with `history.state` the same way — so a fresh nav visit to RAG (the router's entry carries no `kb`) starts at the top level, while an active-link re-click (no pushState — the current drilled entry's state is still on top) keeps the drill. Back at the top level = leave the RAG view (the router's existing pathname-based switch — UNCHANGED). The router module is not modified: its popstate reads the pathname only, and a state-only push never changes it.
- **D3 — the `renderLevel`/`loadTree` machinery is otherwise untouched:** the never-stale `resetVanishedLocation` (post-refetch), the `loadSeq` race token, the re-entrant render, and the anonymous gate all stand; the history integration lives in `goTo` + two listeners (popstate, and the refresh listener's alignment step).
## Design (shared by all tasks — the executor reads this, not the chat)
### The one-line clamp (task 01)
- `frontend/assets/sources.js` — `makeDescCell`: the `<td>`'s content becomes ONE flex wrapper `div.kb-desc-cell` holding the text `<span>` (class `kb-desc-text`) and the Edit button (today's two children, re-homed). `wireDescriptionEdit`'s row `container` becomes the WRAPPER (the editor swap then fills the wrapper: `replaceChildren(editor, actions, status)` inside it — the textarea's existing `width: 100%` keeps filling the cell; verify the open-editor layout in the phase-97 E2E regression, task 03). The text span always carries `title` = the full description text (set whenever non-empty — the hover escape hatch, D1). The marker case (phase 98, if landed) is unaffected: the marker span content/class is set as today inside the wrapper.
- `frontend/assets/styles.css`:
- `.kb-folders-table td:nth-child(3)` — `white-space: nowrap` (was `normal`); keep `min-width: 18rem; max-width: 44rem;` (the column width is the clamp's budget).
- `.kb-desc-cell { display: flex; align-items: center; gap: 0.4rem; min-width: 0; }` (the gap replaces the button's `margin-left: 0.4rem` — remove that rule's effect by the wrapper's gap; keep the level-block button rule untouched).
- `.kb-desc-text { flex: 1 1 auto; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }` (the `min-width: 0` is what lets the flex item shrink — the house comment pattern, cf. `.sync-label`).
- `.kb-summary-edit` stays `flex: 0 0 auto` (already) — the button never wraps to a second line.
- The open-editor case: `.kb-desc-cell .kb-summary-editor` must keep `width: 100%` behavior as a flex item (add `flex: 1 1 auto; min-width: 0;` if needed) — the editor fills the cell, Save/Cancel below it (the existing `.kb-summary-actions` margin).
- The level block (`.kb-level p`) is UNTOUCHED (full text, D1).
### The back button (task 02)
- `frontend/assets/sources.js`:
- `goTo(target)` → the two-step `applyTarget(target, push)`: when `push` is true, `history.pushState({ view: "rag", kb: target }, "")` FIRST (a no-op if `target` equals `current` — never push duplicate entries: a re-click of the current row/segment is skipped), then set `current` + `renderLevel()`. Source-row / folder-link / breadcrumb-segment click handlers call `applyTarget(target, true)`.
- `window.addEventListener("popstate", …)` (registered once at mount, in the admin branch next to the refresh listener — the anonymous gate never installs it): `const kb = event.state && event.state.kb;` → `kb` present → `applyTarget(kb, false)`; absent → `applyTarget({ source: null, folder: null }, false)`. Rendering a currently-hidden view is harmless (the router's own popstate listener, registered earlier, has already switched the visible view for foreign entries — the pathname did not change for rag-internal entries, so no switch happens and this listener's render IS the visible one).
- The `bor:view-refresh` listener (re-show): BEFORE `loadTree()`, align `current` with `history.state` — `history.state && history.state.kb` present → adopt it (the active-link re-click case: the drilled entry is still on top → the drill survives); absent → reset to the top level (a fresh nav entry carries no `kb`). Then the existing `loadTree()` render.
- Boot: `current` starts at the top level; NO pushState on boot (the browser's own entry is the base — the phase-76 boot contract).
- `resetVanishedLocation` (the post-refetch never-stale reset) stays as the DOM-side backstop (a re-fetch after a gap where the current level vanished still resets — it does not push, which is correct: it follows the history state).
- Module docstring: the phase-99 history-integration contract (D2) — the push/adopt/reset rules.
- `frontend/assets/router.js` — UNCHANGED (its popstate reads the pathname; the state-only pushes are invisible to it). No new route, no `_shell_routes` change.
### The E2E (task 03)
`tests/e2e/test_kb_tree_nav.py` (new; the temp-local-source seeding + sync pattern from `test_ls_tree_drilldown.py` — seed a source with a NESTED folder `one/two` + ≥ 2 docs per relevant folder so the mock stores descriptions; admin login via `auth_helpers`; the long manual description for the clamp assertion is created through `PATCH /api/folders/summary` with the admin cookie — the phase-97 pattern):
1. **`test_back_button_walks_the_breadcrumb`** — full navigation chain: load `/` (chat), click the RAG nav link (the router's pushState entry), click the source row (push #1), click folder `one` (push #2), click folder `two` (push #3) — the breadcrumb shows `Knowledge base / <src> / one / two` (last segment `aria-current`). `page.goBack()` → the level is `one` (level title `<src>/one`, `two` listed as a subfolder, the breadcrumb's current segment = `one`); `page.goBack()` → the source root (`<src>`, breadcrumb current = the source); `page.goBack()` → the TOP level (source rows, breadcrumb hidden); `page.goBack()` → the CHAT view is visible (the router contract — Back at the top level leaves the view) and the RAG view is hidden; `page.goForward()` → back to the top level; `page.goForward(); page.goForward(); page.goForward()` → `one` → then the folder level `two` again (the whole chain replays).
2. **`test_breadcrumb_jump_then_back`** — at `one/two`, click the `Knowledge base` breadcrumb segment (a jump: pushes a top-level entry) → the top level renders; `page.goBack()` → returns to `one/two` (the jump's origin — standard history semantics, pinned).
3. **`test_fresh_nav_visit_starts_at_the_top`** — drill to `one/two`, leave via the Tuning nav link, click the RAG nav link again (a fresh router entry, no `kb` state) → the RAG view shows the TOP level (the D2 alignment); drill to `one`, re-click the ACTIVE RAG link (no pushState) → the drill SURVIVES (still at `one` after the refresh re-render).
4. **`test_description_cell_clamps_to_one_line`** — via the admin cookie `PATCH /api/folders/summary` set a long manual description (≥ 4 wrapped lines worth, e.g. 300 chars) on folder `one`; at the source level: the `one` row's Description cell — the cell's `<td>` bounding-box height equals a short-description row's height (± 4 px — both one line), the text span's computed `text-overflow` is `ellipsis` and `white-space` is `nowrap`, the span's `title` carries the FULL text, and the Edit button is visible IN the same cell (not wrapped below). Click `one` → the level block's `#kb-level-summary` textContent is the FULL long text (unclamped at the top — the owner's escape hatch).
5. Regression sweep (each in isolation, `--no-cov`): `test_kb_tree.py` (the EDITOR flow now lives in the restructured cell — the critical one), `test_ls_tree_drilldown.py`, `test_navbar_refresh.py` (the refresh contract the alignment step hooks). Fix ONLY pins broken by the additive changes — no asserted behavior changes elsewhere.
6. Full gates + commit: `uv run pytest` green; coverage >90%; `uv run ruff check . && uv run pyright` clean; move `99_kb_tree_table_and_back_nav` → `complete/`; one atomic commit:
```bash
git add -A .agents/ frontend/ tests/ && git commit --no-gpg-sign -m "fix(ui): clamp catalog descriptions to one line and walk the breadcrumb with the browser back button"
```
## Tasks
1. `01_description_line_clamp.md` — the Description cell's one-line clamp (flex wrapper + ellipsis + hover title) with the Edit button kept
2. `02_breadcrumb_history_nav.md` — the drill state becomes the history state (pushState per drill, popstate adopt/reset, refresh alignment)
3. `03_e2e_tree_nav.md` — `tests/e2e/test_kb_tree_nav.py` + the regression sweep + the atomic commit
## Testing & Quality
- Unit: `tests/unit/test_kb_tree_ui.py` extensions — the clamp structure (the `.kb-desc-cell` wrapper, the `kb-desc-text` span + its `title`, the button inside the wrapper, the editor container = the wrapper) and the history wiring (`pushState` on drill, the popstate adopt/reset handler, the refresh alignment, the no-duplicate-push rule) + `styles.css` pins (`.kb-desc-cell` / `.kb-desc-text` declarations, `white-space: nowrap` on the cell, the editor's flex fill).
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_kb_tree_nav.py -v --no-cov` in isolation; `test_kb_tree.py` + `test_ls_tree_drilldown.py` + `test_navbar_refresh.py` green in isolation.
- Coverage: **>90%** on `app/` (untouched this phase — the full-suite gate still runs; the frontend pins + E2E are the test layer).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] a long stored description renders ONE line in the catalog row (row height independent of description length, source-pinned + E2E-measured), ellipsized, with the full text on hover and in the DOM, and the Edit button in the same cell; the level block shows the full text
- [ ] Back pops one breadcrumb level at a time (drill → … → top), Forward re-descends, Back at the top leaves the RAG view (router contract intact), a breadcrumb jump + Back returns to the jump's origin, a fresh nav visit starts at the top, and an active-link re-click keeps the drill
- [ ] `uv run pytest` green; coverage >90%; ruff + pyright clean
- [ ] `uv run pytest tests/e2e/test_kb_tree_nav.py -v --no-cov` green in isolation (DB up: `podman compose up -d db`); the regression suites green in isolation
- [ ] one atomic Conventional Commit, `--no-gpg-sign`
@@ -0,0 +1,31 @@
# Task 01 — The Description cell clamps to one line (Edit button kept, full text on hover)
**Phase:** `99_kb_tree_table_and_back_nav` · **Story:** n/a (owner request)
## Objective
A long stored description no longer grows its catalog row: the Description cell shows the first line that fits, ellipsized, with the Edit button in the same cell and the full text on hover (and in the DOM).
## Work
1. `frontend/assets/sources.js` — `makeDescCell` (D1, `00_phase.md`):
- The `<td>`'s content becomes ONE wrapper `div.kb-desc-cell` holding the text `<span class="kb-desc-text">` and the Edit `<button>` (today's two children, re-homed; the td itself gets no new class).
- The text span ALWAYS carries `title` = the full description text when non-empty (the hover escape hatch; the marker case — phase 98's `Summary pending` span, if landed — keeps its own marker title, never overridden by this rule: set the hover title only for real description text).
- `wireDescriptionEdit`'s ROW `container` becomes the WRAPPER (the level-block editor is unaffected — its container is still `.kb-level-body`): the open/close swaps happen inside the wrapper, so the `<td>` always keeps the wrapper and the cell layout survives the editor swap.
- Module docstring note: the cell's flex structure (text flexes + ellipsizes, button fixed).
2. `frontend/assets/styles.css` (the `.kb-folders-table` / phase-97 tree region):
- `.kb-folders-table td:nth-child(3)` — `white-space: nowrap` (was `normal`); keep `min-width: 18rem; max-width: 44rem;` (the column's width is the clamp's budget) and the font-size.
- `.kb-desc-cell { display: flex; align-items: center; gap: 0.4rem; min-width: 0; }` — and the row-cell button's spacing comes from this gap: adjust the existing `.kb-folders-table .kb-summary-edit { margin-left: 0.4rem; }` (drop the margin — the gap replaces it; the LEVEL-block rule `.kb-level-body .kb-summary-edit { margin-top: 0.5rem; }` is untouched).
- `.kb-desc-text { flex: 1 1 auto; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }` (the `min-width: 0` is what lets the flex item shrink — comment it, the `.sync-label` house pattern).
- The open-editor case: the `.kb-summary-editor` inside the flex wrapper must keep filling the cell — add `.kb-desc-cell .kb-summary-editor { flex: 1 1 auto; min-width: 0; }` if the existing `width: 100%` does not (verify against the phase-97 editor E2E, task 03); `.kb-summary-actions` / `.kb-summary-status` stack below it as today.
- NO change to `.kb-level p` (the level block's full description — D1), no new hue (phase-92 invariant), `:focus-visible` via the global rule (the button is unchanged).
3. Source pins (`tests/unit/test_kb_tree_ui.py` + its styles.css pins): the wrapper + span classes, the `title` on non-empty text (and NOT on the marker/empty case), the editor container = the wrapper, the CSS declarations (`.kb-desc-cell` flex + gap, `.kb-desc-text` ellipsis triad + `min-width: 0`, `white-space: nowrap` on `td:nth-child(3)`, the margin removal).
## Testing & Quality
- Unit: the source pins above are this task's test layer (`app/` untouched).
- Coverage: **>90%** on this task's new/modified code (frontend pins cover the JS; the measured-height proof is task 03's E2E).
- Note: do NOT add an E2E file here — the clamp's browser proof lands in task 03.
## Completion Criteria
- [ ] the cell renders text + Edit button in one flex row; long text ellipsizes instead of wrapping (CSS-pinned); the full text is in the DOM + the span's `title`
- [ ] the inline editor still opens/saves/cancels from the restructured cell (source-pinned; the phase-97 E2E regression in task 03 proves it in a browser)
- [ ] full test suite green, coverage >90%
- [ ] no behavior change in completed work (the level block's full description, the marker cell — if phase 98 landed — untouched)
@@ -0,0 +1,29 @@
# Task 02 — The browser back button walks the breadcrumb (drill state = history state)
**Phase:** `99_kb_tree_table_and_back_nav` · **Story:** n/a (owner request)
## Objective
Every catalog drill (source, folder, or breadcrumb jump) becomes a history entry — Back pops one level up, Forward re-descends, Back at the top level leaves the RAG view (the router contract), a fresh nav visit starts at the top, and an active-link re-click keeps the drill.
## Work
1. `frontend/assets/sources.js` (D2/D3, `00_phase.md`):
- `applyTarget(target, push)`: set `current = { source: target.source, folder: target.folder }` + `renderLevel()`. When `push` is true, FIRST `history.pushState({ view: "rag", kb: target }, "")` — but SKIP the push when `target` is deep-equal to `current` (a re-click of the current row/segment pushes no duplicate entry). `goTo(target)` becomes `applyTarget(target, true)` — the existing call sites (source-row click, folder-link click, `crumbSegment` click) keep calling `goTo`.
- A `window` `popstate` listener (registered ONCE at mount, in the ADMIN branch next to the refresh listener — the anonymous gate installs neither): read `event.state && event.state.kb` → present: `applyTarget(kb, false)`; absent: `applyTarget({ source: null, folder: null }, false)`. (Rendering a currently-hidden view is harmless — the router's earlier-registered popstate listener owns view switches for foreign entries; for rag-internal entries the pathname never changed, so this listener's render is the visible one.)
- The `bor:view-refresh` listener (the phase-77 re-show): BEFORE `loadTree()`, align with `history.state` — `history.state && history.state.kb` present → `applyTarget(kb, false)` (the active-link re-click case: no pushState happened, the drilled entry is still on top → the drill survives the refresh); absent → `applyTarget({ source: null, folder: null }, false)` (a fresh nav entry carries no `kb` → top level).
- Boot: `current` stays the top-level initial value; NO pushState on boot (the browser's own entry is the base).
- `resetVanishedLocation` (the post-refetch never-stale reset) is UNCHANGED — it still resets `current` without pushing when a re-fetch finds the drilled level gone (it follows whatever the history state says; no new entry).
- Module docstring: the phase-99 history contract — the push/adopt/reset rules + the URL-stays-put note (state-only entries; the phase-76 deep-link surface untouched).
2. `frontend/assets/router.js` — UNCHANGED (its popstate reads the pathname only; state-only pushes are invisible to it). Verify by running its unit suite (`tests/unit/test_frontend_router.py`) untouched.
3. Source pins (`tests/unit/test_kb_tree_ui.py`): the `pushState` call with `{ view: "rag", kb: target }` (and the no-URL-change — the second arg is `""` / no third arg), the duplicate-push skip, the popstate handler's adopt/reset branch (the `event.state.kb` read), the refresh alignment branch (`history.state` read), and that the anonymous branch installs no popstate listener.
## Testing & Quality
- Unit: the source pins above are this task's test layer (`app/` untouched).
- Coverage: **>90%** on this task's new/modified code (frontend pins cover the JS; the browser proof is task 03's E2E).
- Note: do NOT add an E2E file here — the navigation proof lands in task 03.
## Completion Criteria
- [ ] drill clicks push state-only history entries (no URL change, no duplicate on re-click); Back/Forward adopt the entry's `kb` without pushing; a `kb`-less entry resets to the top (source-pinned)
- [ ] a fresh nav visit to the RAG view aligns to the top level; an active-link re-click keeps the current drill (source-pinned)
- [ ] `router.js` and its unit suite are untouched; the anonymous gate still fetches nothing and installs no listeners
- [ ] full test suite green, coverage >90%
- [ ] no behavior change in completed work (the phase-76/77 router contracts green — `test_frontend_router.py`)
@@ -0,0 +1,29 @@
# Task 03 — The dedicated E2E: back-walks-the-breadcrumb + the measured clamp + regressions + commit
**Phase:** `99_kb_tree_table_and_back_nav` · **Story:** n/a (owner request)
## Objective
Pin both owner-visible contracts in a browser: the browser Back/Forward buttons walk the directory breadcrumb level by level, and a long description renders exactly one row-line (Edit button in-cell, full text on hover + at the top).
## Work
1. `tests/e2e/test_kb_tree_nav.py` (new — the phase's dedicated A16 suite, run in isolation). Fixtures: `app_server` + `mock_llm` + `db_ready` (the `conftest` pattern); admin login via `tests/e2e/auth_helpers.py`; the temp-local-source seeding + `POST /api/sync` pattern from `test_ls_tree_drilldown.py` — seed a source with a NESTED folder `one/two` and ≥ 2 docs per folder (so the mock `FOLDER_SUMMARY_MODE` stores deterministic descriptions at every level). An autouse cleanup removes the temp source's KB rows + `folder_summaries` rows (the phase-96 cleanup pattern).
- **`test_back_button_walks_the_breadcrumb`** — load `/` (chat), click the RAG nav link (the router's pushState entry), click the source row (push #1), click folder `one` (push #2), click folder `two` (push #3): the breadcrumb renders `Knowledge base / <src> / one / two` (last segment `aria-current="page"`). Then, exactly one level per press: `page.goBack()` → level `one` (level title `<src>/one`, `two` listed as a subfolder, breadcrumb current = `one`); `page.goBack()` → the source root (title `<src>`, breadcrumb current = the source); `page.goBack()` → the TOP level (the source rows, breadcrumb hidden); `page.goBack()` → the CHAT view visible + the RAG view hidden (the router contract — Back at the top leaves the view); `page.goForward()` → the top level again; three more `page.goForward()`s → the source root → `one` → `two` (the whole chain replays, current segment correct at each step).
- **`test_breadcrumb_jump_then_back`** — drill to `one/two`; click the `Knowledge base` breadcrumb segment (a jump — pushes a top-level entry) → the top level renders; `page.goBack()` → back at `one/two` (the jump's origin — standard history semantics, pinned).
- **`test_fresh_nav_visit_starts_at_the_top`** — drill to `one/two`; navigate to Tuning (nav link); click the RAG nav link again (a fresh router entry — no `kb` state) → the RAG view shows the TOP level (the D2 alignment, after the refresh re-render); then drill to `one` and re-click the ACTIVE RAG nav link (no pushState) → the refresh re-render KEEPS the drill (still at `one` — the active re-click contract).
- **`test_description_cell_clamps_to_one_line`** — with the admin cookie, `PATCH /api/folders/summary` sets a LONG manual description (≥ 4 wrapped lines worth — e.g. 300 chars) on folder `one` (the phase-97 endpoint); at the source level: the `one` row's Description `<td>` bounding-box height equals a short-description row's height (± 4 px — both one line); the `.kb-desc-text` span's computed style: `text-overflow: ellipsis`, `white-space: nowrap`; the span's `title` attribute carries the FULL long text; the Edit button is visible IN the same cell (its bounding box shares the cell's row — not wrapped below). Click `one` → the level block's `#kb-level-summary` textContent is the FULL long text (unclamped at the top — the owner's escape hatch).
2. Regression sweep (each in isolation, `--no-cov`, DB up): `test_kb_tree.py` (the EDITOR flow now lives in the restructured cell — the critical regression), `test_ls_tree_drilldown.py`, `test_navbar_refresh.py` (the refresh contract the alignment step hooks). Fix ONLY pins broken by the additive changes — no asserted behavior changes elsewhere.
3. Full gates + commit: `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean; move `99_kb_tree_table_and_back_nav` → `.agents/phases/complete/`; one atomic commit:
```bash
git add -A .agents/ frontend/ tests/ && git commit --no-gpg-sign -m "fix(ui): clamp catalog descriptions to one line and walk the breadcrumb with the browser back button"
```
## Testing & Quality
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_kb_tree_nav.py -v --no-cov` green in isolation.
- Coverage: **>90%** on `app/` (the full-suite gate; `app/` is untouched this phase).
- Lint/types: `uv run ruff check . && uv run pyright`.
## Completion Criteria
- [ ] the four E2E tests pass in isolation (the back/forward walk, the jump, the fresh-visit/re-click alignment, the measured one-line clamp)
- [ ] the regression suites pass in isolation (the phase-97 editor flow green in the restructured cell)
- [ ] full suite green, coverage >90%, ruff + pyright clean
- [ ] phase dir moved to `complete/`, one atomic `--no-gpg-sign` Conventional Commit