phase: 97_kb_tree_catalog
All completion criteria verified — everything is green, no defects found. Final report: ## Phase 97 final verification pass — ALL GREEN **Verified (no code changes needed):** - `GET /api/docs/tree` (admin), `build_kb_tree` pure builder, `PATCH /api/folders/summary`, migration 0018 (`manually_edited`, head confirmed), generator skip/keep + `kept_manual` stat, RAG tree UI + edit affordance in `sources.js`/`index.html`/`styles.css` - `tests/e2e/test_kb_tree.py`: 8 passed — top level, drill source/folder, edit round-trip, clear, manual-desc-survives-sync, reload fallback, anonymous gate - Integration: tree shape/order/403/empty/indexed-only + PATCH update/create/root/clear/404/403/no-LLM + stat-walk equivalence (in `test_docs_api.py`); 3-field `folder_summaries=` import token preserved **Gates (exact commands):** - `uv run pytest --cov=app --cov-report=term-missing` → **2053 passed**, TOTAL coverage **99%** (>90% ✓) - `uv run ruff check . && uv run pyright` → **All checks passed / 0 errors** - `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` → **8 passed** in isolation - 30 story/RAG-view E2E suites run **one per process**: all passed, incl. `test_ls_tree_drilldown` (agent `ls` byte-identical ✓), `test_import_documents`, `test_edit_summaries`, `test_admin_auth`, `test_kb_overview` **Completion criteria:** tree view ✓ · edit round-trip + clear ✓ · manual persists/clear resets ✓ · `ls` unchanged ✓ · pytest/coverage/lint ✓ · E2E isolation ✓ · commit — left to harness per protocol (working tree untouched, `git add/commit` not run) **Deviations:** none. **Next pending phase:** none — `todo/` contains only 97 (96 already committed).
This commit is contained in:
@@ -0,0 +1,104 @@
|
||||
# Phase 97 — The RAG view shows the KB as the drill-down tree the agent sees, with editable folder descriptions
|
||||
|
||||
**Source:** Owner request (chat, 2026-09-11) — "The RAG page shows a list of all files uploaded by the user, but the agent sees a tree structure after the last phase [94]. I want the UI to show what the agent sees — a tree that the user can click through so the user can see descriptions of the directories. The user should be able to edit the descriptions of the directories just like they can edit the summaries of any file."
|
||||
**Story:** n/a (owner request — the RAG view on the `02_story_import_documents` catalog; the folder descriptions + tree concept on `94_ls_tree_drilldown`; the edit affordance pattern on `57_edit_document_summaries`).
|
||||
**Context:** The RAG (Knowledge base) view — `frontend/index.html` `#view-rag` + `frontend/assets/sources.js` (shell view module, phase 76) — renders the flat all-documents table from `GET /api/docs` (`app/api/docs.py`: source | path | title | chunks | indexed, `#docs-tbody`), with KB-wide stat cards, the sync button/poll machinery, and the phase-16 anonymous gate. Phase 94 made the agent's `ls` a drill-down tree over stored sync-time folder descriptions: `folder_summaries` (migration 0017, PK `(source, folder_path)`, `folder_path ""` = source root, rows only for ≥ 2-doc folders), the shared folder concept (`app/rag/folder_summaries.folder_of`; existence rule: a folder exists ⟺ some indexed path starts with `folder + "/"`; counts are the recursive subtree), and the pure level functions `app/rag/agent.py` `ls_top` / `group_folder_listing` / `ls_folder`. Phase 57's edit affordance (admin `PATCH /api/documents/summary` + the `.doc-summary` Edit → inline textarea → Save/Cancel → live-region status in `frontend/assets/document.js`) is the interaction model to mirror. Migration head is `0017_folder_summaries`. The deterministic E2E mock already returns the canned `FOLDER_SUMMARY_MODE` one-liner naming the folder (phase 94).
|
||||
|
||||
## Objective
|
||||
The admin's RAG (Knowledge base) view shows the catalog as the same drill-down tree the agent's `ls` walks — sources at the top, then per level the subfolders with their stored descriptions and the level's files — so the owner can click through the directories, read the descriptions the agent reads, and edit (or clear) any directory's description with the phase-57 inline edit affordance. A manually-edited description persists: the sync-time generator never overwrites and never prunes it.
|
||||
|
||||
## Dependencies
|
||||
- `94_ls_tree_drilldown` (complete) — the `folder_summaries` table, the stored per-folder descriptions, and the shared folder concept (`folder_of`, the existence rule, the recursive counts, `group_folder_listing` for the cross-check).
|
||||
- `57_edit_document_summaries` (complete) — the edit affordance pattern (Edit → inline textarea → Save/Cancel → live-region status) and the admin-gated summary-PATCH precedent.
|
||||
- `96_oneshot_resilience` (todo) — queue order only (numeric); no code dependency (different subsystem).
|
||||
|
||||
## Decisions recorded here (owner review — PLAN.md is being redone by the owner)
|
||||
- **The tree replaces the flat all-documents table in the RAG view** (derived from the request: "I want the UI to show what the agent sees"). KB-wide stat cards stay; the per-level file table keeps the existing 5-column contract (`#docs-table` / `#docs-tbody`) so file rows are unchanged in shape; the flat `GET /api/docs` endpoint is untouched (API surface unchanged). Existing story suites whose catalog assertions assume the flat layout are updated IN THIS PHASE (tasks 07/08) — the asserted behavior (a document is listed, linkable, viewable) is preserved; only navigation gains a drill step.
|
||||
- **Manual edits persist across syncs** (derived from "just like they can edit the summaries of any file" — an edited file summary is never silently rewritten by a re-import): `folder_summaries` gains `manually_edited`; the phase-94 generator SKIPS (never overwrites) and never prunes a manual row. Clearing a description deletes the row — the next KB-changing sync regenerates an AI description for that folder (that IS the reset path; no separate regenerate button in v1).
|
||||
- **The UI tree is a superset of the `ls` top level:** sources list as registered sources in registry order (the `ls()` order, including a registered 0-document source — the phase-70/72 invariant) FOLLOWED by indexed sources that are not registered (path order) — ad-hoc `scripts/import_docs --source ~/X` imports and removed-but-not-yet-pruned sources. The catalog has never hidden an indexed document (the phase-02 contract); the agent's `ls` keeps showing registry sources only (phase 94) — unchanged.
|
||||
- **No re-embedding on edit** (contrast with phase 57): a folder description is never embedded (no chunk, no retrieval role beyond the `ls` line) — the PATCH is a pure DB write, no LLM call.
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
|
||||
### The tree endpoint (task 02)
|
||||
`GET /api/docs/tree` (admin, like `/api/docs`) returns the FULL recursive tree in one fetch — the UI drills client-side, zero per-level fetches:
|
||||
|
||||
```json
|
||||
{
|
||||
"sources": [
|
||||
{
|
||||
"name": "alpha", "documents": 5, "summary": "… or null",
|
||||
"children": [
|
||||
{ "kind": "folder", "path": "one", "documents": 2, "summary": "… or null",
|
||||
"children": [
|
||||
{ "kind": "file", "path": "one/a.md", "title": "A", "chunks": 3,
|
||||
"indexed_at": "2026-09-11T08:00:00+00:00" }
|
||||
]
|
||||
},
|
||||
{ "kind": "file", "path": "root-note.md", "title": "Root note", "chunks": 1,
|
||||
"indexed_at": "…" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- **Sources:** registered sources first in `list_source_names` (registry) order, then indexed-only sources (alphabetical) — the superset rule above. `documents` = the source's whole recursive count; `summary` = the stored `(source, "")` row or null. A registered 0-document source still lists (`0`, no children).
|
||||
- **Folder nodes:** `path` source-relative (never `""` — the source node IS the root); direct subfolders only, in path order (the phase-94 existence rule); `documents` = the recursive subtree count; `summary` = the stored row (AI OR manual) or null; `children` = its own subfolders + direct files, same shape.
|
||||
- **File nodes:** direct files only (`folder_of(path) == parent`), in path order (catalog order — the same order `GET /api/docs` serves); `path` source-relative; `title` / `chunks` (content + `is_summary` chunks — the same count `/api/docs` returns) / `indexed_at` verbatim.
|
||||
- **Pure builder** `build_kb_tree(names, doc_rows, summaries)` — module-level in `app/api/docs.py` (unit-testable without a DB): `names` = the registry source names in order; `doc_rows` = `(source, path, title, chunks, indexed_at)` tuples in the `/api/docs` query order; `summaries` = `{(source, folder_path): summary}` over ALL stored rows. It reuses `app.rag.folder_summaries.folder_of` and the phase-94 existence rule — ONE concept end to end: the UI tree is the `ls` tree plus file metadata. **Cross-check property (unit-pinned):** for a single-source dataset the builder's level equals `app.rag.agent.group_folder_listing`'s output (same subfolder paths/counts/summaries, same file paths/titles in order) — "the UI shows what the agent sees" as a test, at the root and one nested level.
|
||||
- `GET /api/docs` stays untouched.
|
||||
|
||||
### Edit persistence (task 01)
|
||||
- Migration `0018_folder_summary_manual_flag` (down `0017_folder_summaries` — confirm with `alembic heads`): `folder_summaries.manually_edited` Boolean NOT NULL server_default `false`; tested downgrade drops the column (A13, the `test_migration_0017.py` pattern).
|
||||
- `generate_folder_summaries` (`app/rag/folder_summaries.py`): fetch the existing rows ONCE before the upsert loop (they are already fetched for the prune step — restructure into one `{(source, folder_path): row}` dict). Upsert loop: an existing row with `manually_edited` is SKIPPED (no `lite` call — no burn on owner text) and counted `stats["kept_manual"] += 1`. Prune loop: deletes only NON-manual rows (a manual row for a folder that dropped below 2 documents is kept — owner content persists until cleared). The generator's logger line gains `kept_manual=%d`. **The import summary-line token STAYS 3 fields** (`scripts/import_docs.py` `folder_summaries=<generated>/<failed>/<pruned>` — `tests/integration/test_import_docs_overview.py` pins it).
|
||||
|
||||
### The edit endpoint (task 03)
|
||||
`PATCH /api/folders/summary` (admin) — body `{"source": str, "folder_path": str, "summary": str}`; `folder_path = ""` = the source root. Lives in `app/api/docs.py` next to the phase-57 document-summary PATCH (same `kb` router family).
|
||||
- **Source check:** registered (registry) OR has indexed documents → else 404 `{"detail": "source not found"}`.
|
||||
- **Folder check:** `""` is valid for an allowed source; otherwise the phase-94 existence rule over the source's indexed paths (some path starts with `folder_path + "/"`) → else 404 `{"detail": "folder not found"}`. DB-only (the `/documents/content` rule — traversal strings such as `../../etc` are simply not prefixes of any indexed path; no filesystem).
|
||||
- **Non-empty after strip** → upsert the row with `manually_edited = True` + fresh UTC `updated_at`. **Empty/whitespace** → delete the row if present (clear — the phase-57 analog; the reset path). Commit; respond `{"source", "folder_path", "summary"}` (`summary` null after a clear).
|
||||
- No LLM/embedding call (the decision above); the docstring says so.
|
||||
|
||||
### The RAG view (tasks 04/05)
|
||||
`frontend/index.html` `#view-rag` + `frontend/assets/sources.js` (the house createElement/textContent contract — never innerHTML with document-derived data):
|
||||
- **New static skeleton** (in `index.html`, after the stat cards, before the existing file `.table-wrap`):
|
||||
- `nav#kb-crumb[aria-label="Catalog location"][hidden]` — JS fills the segments.
|
||||
- `section#kb-level[hidden][aria-labelledby="kb-level-title"]` — `h2#kb-level-title` (the level's full source-relative path, e.g. `alpha/two`), `p#kb-level-summary`. (The level Edit button lands in task 05.)
|
||||
- `div#folders-wrap.table-wrap[hidden][role="region"][aria-label="Folders"][tabindex="0"]` + `table#folders-table` (visually-hidden caption, the `.docs-table` styling) — `thead`: **Folder | Documents | Description**, `tbody#folders-tbody` (JS-filled rows). ONE table for every level: at the top level the rows are the SOURCES themselves (the `ls()` equivalence — sources list exactly like folders: name, count, description).
|
||||
- **File table:** the existing `.table-wrap` / `#docs-table` / `#docs-tbody` is UNCHANGED in columns (Source | Path | Title | Chunks | Indexed) and in row rendering (`makeRow` — the path link still opens `openDocumentModal`, the no-JS href escape hatch intact). It now holds the CURRENT LEVEL's direct files; hidden when the level has none (at the top level it is always hidden — files are seen per source, as with `ls(source)`).
|
||||
- **Navigation state:** module-level `current = {source: string | null, folder: string | null}` (`{null, null}` = top level; `folder ""` = the source root). Source row click → `{source, ""}`; folder link click → `{source, folder}`; breadcrumb segment click → that ancestor. Breadcrumb: hidden at the top level; one link per ancestor (source, then folders), the last segment a `span` with `aria-current="page"`.
|
||||
- **Level rendering:** the level block shows the CURRENT level's stored description (source: the `(source, "")` row; folder: its row) and is hidden when none is stored (the `ls` rule — count only, no placeholder); the folders table = the direct subfolders (count + stored description + row Edit button in task 05); the file table = the direct files. `#folders-wrap` hidden when no subfolders.
|
||||
- **Stat cards:** unchanged KB-wide values — computed by walking the whole in-memory tree (documents, chunks sum, max `indexed_at`) — identical to today's flat walk.
|
||||
- **Empty state:** `#sources-empty` shows ONLY when the tree has zero sources (nothing registered, nothing indexed); a registered 0-document source renders its row (`0 documents`) instead.
|
||||
- **Never-stale reload (PLAN §7.4):** every re-fetch (boot, `bor:view-refresh`, sync-success settle, upload-success settle) re-renders the current level; if the current position no longer exists in the new tree (source unregistered/pruned, folder gone), the view RESETS to the top level — no stale breadcrumb, no stale block. The `loadSeq` race token (phase 79) carries over to the tree load.
|
||||
- **Unchanged:** the sync button/poll/label/banner/error-modal machinery, the anonymous gate (no `/api/docs*` fetch for anonymous — the gate branch never fetches the tree), the shell router wiring, the stat-card markup.
|
||||
- **Edit (task 05):** Edit buttons on `#kb-level` AND on every source/folder row's Description cell (a description can be CREATED where none is stored — a < 2-document folder or the generator's fail-soft miss; the button is always present — the view is admin-only already, the endpoint's `require_admin` is the API-level gate). The phase-57 interaction: swap the description text node for `textarea.kb-summary-editor` (prefilled via `value` — the XSS contract), Save / Cancel, `p.kb-summary-status[role="status"][aria-live="polite"]`. Save → `PATCH /api/folders/summary` `{"source", "folder_path", "summary"}` (folder_path `""` for the source root). 200 → re-render the text (textContent only) + status "Description updated." (empty save → the text goes away — level block hidden / cell emptied — + "Description cleared."); the in-memory tree node's summary updates in place (no re-fetch). Failure → neutral retry copy (the phase-55 convention), the editor stays open with the user's text. Cancel → restore the text node.
|
||||
- **`styles.css`:** `.kb-crumb` (+ link / current segment), `.kb-level` (+ title / summary typography), `.folder-link`, `.kb-summary-edit`, `.kb-summary-editor` (min-height 8rem — the phase-57 spec), `.kb-summary-save`, `.kb-summary-cancel`, `.kb-summary-status` — house dark-tech palette, AA contrast (text + color, never color alone), no new hue (phase-92 invariant — monochrome-safe), `:focus-visible` via the global outline rule, ≥ 24px targets, no CDN.
|
||||
|
||||
## Tasks
|
||||
1. `01_manual_flag.md` — migration 0018 + `FolderSummary.manually_edited` + the generator's skip/keep logic + stats/log
|
||||
2. `02_tree_api.md` — `GET /api/docs/tree` (pure builder + schemas + endpoint)
|
||||
3. `03_folder_summary_api.md` — `PATCH /api/folders/summary` (update / create / clear)
|
||||
4. `04_tree_ui.md` — the RAG view tree: level rendering, drill navigation, breadcrumb, level block, folder table, reload fallback
|
||||
5. `05_folder_edit_ui.md` — the edit affordance (level block + rows) + PATCH wiring + editor + status + styles + source pins
|
||||
6. `06_e2e_kb_tree.md` — the dedicated Playwright suite `tests/e2e/test_kb_tree.py`
|
||||
7. `07_catalog_suites_import_sync.md` — the existing import/sync/upload story suites onto the drill-down catalog
|
||||
8. `08_catalog_suites_viewer_nav.md` — the existing viewer/nav/UX story suites + the regression run
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `build_kb_tree` (multi-source order, indexed-only sources, a 0-document registered source, nested counts, the existence rule, a file/folder name collision, ordering, summaries present/absent, file metadata verbatim) + the `group_folder_listing` cross-check; the generator's keep/keep-out (a manual row survives regeneration AND prune, stats, the logger line); migration 0018 up/down (A13).
|
||||
- Integration: the tree endpoint (shape, ordering, 403 anonymous, empty registry, indexed-only + orphan visibility per the superset rule, the stat-walk equivalence); the PATCH endpoint (update, create where no row exists, source root, clear + double-clear, 404 unknown source/folder/traversal, 403 non-admin) — the `tests/integration/test_docs_api.py` pattern; the `import_docs` line keeps its 3-field `folder_summaries=` token with a manual row present.
|
||||
- Frontend source pins (house style, the `tests/unit/test_save_chat_ui.py` pattern): `sources.js` (the `/api/docs/tree` fetch, the drill-state transitions, the PATCH path + body, the textContent contract, the status copies, the reset-to-top fallback, the three refresh wirings), `styles.css` (the new class names).
|
||||
- E2E (mandatory, A16): `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` in isolation; the updated story suites green in isolation (the task 07/08 lists), including `test_ls_tree_drilldown.py`, `test_edit_summaries.py`, `test_import_documents.py`.
|
||||
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
|
||||
- Lint/types: `uv run ruff check . && uv run pyright`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The RAG view (admin) lists the sources with counts + descriptions; clicking through shows, per level, the subfolders (count + description) and the level's files (existing 5-column table, the document-modal link unchanged)
|
||||
- [ ] The level block shows the current directory's description; Edit → inline textarea → Save round-trips to `PATCH /api/folders/summary` (the new text renders; an empty save clears it — the row is deleted)
|
||||
- [ ] A manual description survives a KB-changing sync (the generator's `kept_manual`), and clearing it resets the folder to AI generation on the next sync
|
||||
- [ ] The agent's `ls` output is byte-identical for the same data (the phase-94 suites green) — the tree reads the same rows; the only schema change is the additive flag column
|
||||
- [ ] `uv run pytest` green; coverage >90%; ruff + pyright clean
|
||||
- [ ] `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` green in isolation (DB up: `podman compose up -d db`); the updated story suites green in isolation
|
||||
- [ ] One atomic Conventional Commit, `--no-gpg-sign` (e.g. `feat(kb): show the catalog as the drill-down tree the agent sees, with editable folder descriptions`)
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 01 — Manual-edit flag: migration 0018 + the generator keeps the owner's descriptions
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
A stored folder description the owner edited can never be overwritten (or pruned) by the sync-time generator — `folder_summaries.manually_edited` (migration 0018) plus the phase-94 generator's skip/keep logic.
|
||||
|
||||
## Work
|
||||
1. `alembic/versions/0018_folder_summary_manual_flag.py` (down revision `0017_folder_summaries` — confirm with `uv run alembic heads`) — add `folder_summaries.manually_edited` Boolean NOT NULL server_default `false`; the **tested downgrade** drops the column (A13 — the `tests/integration/test_migration_0017.py` pattern: a new `tests/integration/test_migration_0018.py`).
|
||||
2. `app/models.py` — `FolderSummary.manually_edited: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default=text("false"))` + the docstring: set ONLY by `PATCH /api/folders/summary` (phase 97, task 03); the generator (below) skips a manual row on regeneration and never prunes it — an owner correction is never silently rewritten (the phase-97 `00_phase.md` decision).
|
||||
3. `app/rag/folder_summaries.py` — `generate_folder_summaries`:
|
||||
- Fetch the existing rows ONCE before the upsert loop (they are already fetched for the prune step — restructure into a single `{(source, folder_path): FolderSummary}` dict used by both loops).
|
||||
- Upsert loop: an existing row with `manually_edited` → SKIP (no `llm.chat` call — no `lite` burn on owner text) and `stats["kept_manual"] += 1`.
|
||||
- Prune loop: delete only rows whose `manually_edited` is False (a manual row for a folder that dropped below 2 documents is KEPT — owner content persists until cleared).
|
||||
- The stats dict gains `"kept_manual": 0`; the logger line becomes `folder_summaries: generated=%d failed=%d pruned=%d kept_manual=%d`.
|
||||
- **The import summary-line token STAYS 3 fields** — `scripts/import_docs.py`'s `folder_summaries=<generated>/<failed>/<pruned>` is unchanged (`tests/integration/test_import_docs_overview.py` pins it; `app/api/sync.py` just logs the dict).
|
||||
4. Docstrings: the module docstring's storage paragraph + the `generate_folder_summaries` docstring note the flag's two rules (skip on regenerate, keep on prune) — the house "module docstrings carry the contracts" rule.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit (`tests/unit/test_folder_summaries.py` extensions): a manual row survives a regeneration pass (skipped — the fake LLM is never called for it — `kept_manual` right, the row's text/stamp untouched); a manual row survives the prune (its folder drops below 2 documents); a non-manual row is still pruned and still regenerated; `skip=True` no-op unchanged; the logger line format (caplog, 4 fields).
|
||||
- Integration: `test_migration_0018.py` (up: column present, default false, existing rows backfilled false; down: column dropped); an `test_import_docs_overview.py`-pattern case with a manually-edited row inserted directly — the gate fires, the token stays `folder_summaries=<g>/<f>/<p>`, and the manual row's text is unchanged afterward.
|
||||
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] migration up/down green; the flag round-trips through the model
|
||||
- [ ] the generator never overwrites or prunes a manual row; stats + log carry `kept_manual`; the import token is unchanged
|
||||
- [ ] full test suite green, coverage >90%
|
||||
- [ ] no behavior change in completed work (the phase-94 generator + sync suites green)
|
||||
@@ -0,0 +1,26 @@
|
||||
# Task 02 — `GET /api/docs/tree`: the full recursive tree the UI renders
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
One admin endpoint that returns the catalog as the recursive tree the agent's `ls` walks (with the file metadata the rows and stat cards need) — the RAG view's single fetch (the shape contract is in `00_phase.md`, section "The tree endpoint").
|
||||
|
||||
## Work
|
||||
1. `app/schemas.py` — next to `DocList`: `KbTreeFile` (`path`, `title`, `chunks: int (ge=0)`, `indexed_at: str`), `KbTreeFolder` (`path`, `documents: int (ge=0)`, `summary: str | None`, `children` — recursive union of folder/file nodes), `KbTreeSource` (`name`, `documents: int (ge=0)`, `summary: str | None`, `children`), `KbTree` (`sources: list[KbTreeSource]`). The `00_phase.md` JSON shape is the contract (Pydantic v2 handles the recursive union; `kind` discriminators optional — the JSON keys `kind: "folder" | "file"` must appear as spec'd).
|
||||
2. `app/api/docs.py`:
|
||||
- `build_kb_tree(names, doc_rows, summaries) -> list[KbTreeSource]` — PURE, module-level (unit-testable without a DB). Inputs: `names` = registry source names in order; `doc_rows` = `(source, path, title, chunks, indexed_at)` tuples in the `/api/docs` query order (source, path); `summaries` = `{(source, folder_path): str}` over ALL stored rows. Behavior per `00_phase.md`: sources = `names` first (each always present, even 0 documents), then the distinct indexed sources not in `names` (alphabetical — the superset rule); a folder node exists iff some indexed path starts with `folder + "/"` (the phase-94 existence rule — reuse `app.rag.folder_summaries.folder_of`, never re-derive "folder"); `documents` = recursive subtree count; subfolders in path order, direct files (`folder_of(path) == parent`) in input (catalog) order; file nodes carry `title` / `chunks` / `indexed_at` verbatim; `summary` = the stored row (any row — AI or manual) or null.
|
||||
- `GET /docs/tree` endpoint (`Depends(require_admin)`, `response_model=KbTree`) — compose: `names = list_source_names(db)` (import from `app.rag.agent` — `app/api/chat.py` already imports from that module, no cycle), `doc_rows` = the same outerjoin/grouped query `GET /docs` uses (id excluded is fine — the tree has no document ids), `summaries` = all `folder_summaries` rows for the listed sources; call the builder.
|
||||
- Module docstring: add the new endpoint's contract line (the house docstring convention — one line per endpoint).
|
||||
3. `GET /api/docs` is UNCHANGED (same query, same response — its suites stay green untouched).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit (`tests/unit/` — new `test_kb_tree_builder.py` or extend the existing docs unit file): multi-source — registry order preserved + an indexed-only source appended alphabetically; a registered 0-document source (`documents: 0`, no children); a nested document counts into the source, every ancestor folder, and its own folder; the existence rule (a folder node only with a true descendant — a file path is never a folder); a file/folder NAME COLLISION (a document sharing a directory's name — both appear: the folder node via its descendants + the file node); subfolder path order + file catalog order; `summary` present/absent; file `chunks` / `indexed_at` / `title` verbatim; an indexed document under an UNLISTED source is impossible (every doc source is listed by construction — the superset rule) while the registry order still leads.
|
||||
- **The cross-check property (the "UI shows what the agent sees" pin):** for a single-source dataset, the builder's root level equals `app.rag.agent.group_folder_listing(source, "", (path, title) rows, {folder: summary})` — same subfolder `(path, count, summary)` triples in order AND the same file `(path, title)` pairs in order (uncapped — pass the full file list; the builder has no 50-line cap, the UI is for humans); repeat for one nested level.
|
||||
- Integration (`tests/integration/test_docs_api.py` extensions, the house pattern — real app + real DB): seed documents across two sources (one with nested folders, one flat) + `folder_summaries` rows → the shape/ordering/counts/summaries; 403 anonymous; empty registry + no documents → `{"sources": []}`; an indexed-only source (documents with a source absent from `git_sources`) lists after the registered ones; the stat-walk equivalence — summing the file nodes' `chunks` + their count equals what `GET /api/docs` returns for the same data (the stat-card values are unchanged by the redesign).
|
||||
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/docs/tree` returns the `00_phase.md` shape (admin; 403 otherwise); `GET /api/docs` byte-identical in behavior
|
||||
- [ ] the pure builder is unit-pinned, including the `group_folder_listing` cross-check at two levels
|
||||
- [ ] full test suite green, coverage >90%, ruff + pyright clean
|
||||
- [ ] no behavior change in completed work (the phase-94 ls suites green)
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 03 — `PATCH /api/folders/summary`: the admin folder-description editor endpoint
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The endpoint behind the view's edit affordance: update / create / clear a stored folder description, marking every non-empty save `manually_edited` (task 01's keep/keep-out rules apply from this point on).
|
||||
|
||||
## Work
|
||||
1. `app/schemas.py` — `FolderSummaryUpdate` (`source: str`, `folder_path: str` — `""` = the source root, `summary: str`) + `FolderSummaryResult` (`source`, `folder_path`, `summary: str | None`), next to the phase-57 `SummaryUpdate` / `SummaryResult`.
|
||||
2. `app/api/docs.py` — `PATCH /folders/summary` on the same `kb` router, next to the phase-57 `PATCH /documents/summary` (the house docstring line for it):
|
||||
- `Depends(require_admin)` — the catalog is admin-only (phase 16); the endpoint gate is the API-level defense in depth (the RAG view never renders for anonymous).
|
||||
- **Source check:** the source is registered (`list_source_names(db)`) OR has indexed documents (`SELECT 1 FROM documents WHERE source = … LIMIT 1`) → else 404 `{"detail": "source not found"}`.
|
||||
- **Folder check:** `folder_path == ""` is valid for an allowed source; otherwise the phase-94 existence rule over the source's indexed paths (some `Document.path` starts with `folder_path + "/"`) → else 404 `{"detail": "folder not found"}`. DB-only (the `/documents/content` rule — traversal strings such as `../../etc` are simply not prefixes of any indexed path; no filesystem access).
|
||||
- **Non-empty after `strip()`** → upsert the `folder_summaries` row with `summary = stripped text`, `manually_edited = True`, fresh UTC `updated_at` (insert or update — a manual description can be created where no row exists: a < 2-document folder, or the generator's fail-soft miss).
|
||||
- **Empty/whitespace** → `db.delete(row)` when a row exists (clear — the phase-57 analog; the row may be AI-written or manual, either way it is gone; the next KB-changing sync regenerates an AI row — the reset path). A clear with no row is a 200 no-op.
|
||||
- `db.commit()`; respond `{"source", "folder_path", "summary"}` (`summary` null after a clear).
|
||||
- **No LLM/embedding call** — a folder description is never embedded (no chunk, no retrieval role beyond the `ls` line); the docstring records the deliberate contrast with phase 57's `is_summary` re-embed.
|
||||
3. Router registration needs no change (same router object as `/docs` / `/documents/summary`).
|
||||
|
||||
## Testing & Quality
|
||||
- Integration (`tests/integration/test_docs_api.py` extensions): update an existing AI-written row (text replaced, `manually_edited` true, `updated_at` advanced — a second save with a different text updates in place); create where no row exists (< 2-document folder); the source root (`folder_path: ""`) round-trips; clear (the row is deleted, the response `summary` null, a second clear is a 200 no-op); 404 unknown source / unknown folder / a traversal folder path / a folder under an unknown source; 403 anonymous AND a live access-token user who is not the admin (`require_admin` — the phase-79 token users exist); the round-trip through the task-02 endpoint — `GET /api/docs/tree` shows the new text after the save and null after the clear.
|
||||
- Coverage: **>90%** on this task's new/modified code (full gate: `app/`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the endpoint contract is pinned (update / create / source-root / clear / double-clear / 404s / 403s)
|
||||
- [ ] every non-empty save sets `manually_edited` — the task-01 generator keep/keep-out applies from this point on
|
||||
- [ ] no LLM call on the path — source pin: the PATCH handler never constructs an `LLMClient` (grep pin, the house source-pin pattern)
|
||||
- [ ] full test suite green, coverage >90%, ruff + pyright clean
|
||||
@@ -0,0 +1,37 @@
|
||||
# Task 04 — The RAG view renders the tree: drill navigation, breadcrumb, level block, folder table
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The Knowledge base view (admin) shows the catalog as the drill-down tree — sources at the top, then per level the subfolders (count + description) and the level's files — replacing the flat all-documents table; descriptions are read-only here (task 05 adds the edit affordance).
|
||||
|
||||
## Work
|
||||
1. `frontend/index.html` `#view-rag` — static skeleton additions (the house no-JS-safe skeleton convention), after `#stat-cards`, before the existing file `.table-wrap`:
|
||||
- `nav#kb-crumb[aria-label="Catalog location"][hidden]` — empty; the JS fills the segments (createElement).
|
||||
- `section#kb-level[hidden][aria-labelledby="kb-level-title"]` — `h2#kb-level-title`, `p#kb-level-summary` (the level Edit button lands in task 05 — do NOT add it here).
|
||||
- `div#folders-wrap.table-wrap[hidden][role="region"][aria-label="Folders"][tabindex="0"]` + `table#folders-table.kb-folders-table` (a `.visually-hidden` caption, the `.docs-table` styling) — `thead` **Folder | Documents | Description**, `tbody#folders-tbody` (JS-filled). ONE table for every level — at the top level the rows are the SOURCES themselves (the `ls()` equivalence: sources list exactly like folders — name, count, description).
|
||||
- The existing `.table-wrap` / `#docs-table` / `#docs-tbody` block, `#sources-empty`, `#stat-cards`, the sync button/result/banner/gate: UNCHANGED.
|
||||
2. `frontend/assets/sources.js` — the catalog load becomes tree-based (the module header docstring gains the phase-97 section, the house convention):
|
||||
- `loadDocs()` → `loadTree()`: `fetch("/api/docs/tree")` in the admin branch only (the anonymous gate branch NEVER fetches — the phase-16 soft rule, unchanged); keep the `loadSeq` race token (phase 79) — only the newest load may touch the DOM; store the parsed tree in a module-level `kbTree`; render the current level.
|
||||
- `current = {source: string | null, folder: string | null}` — module state, initial `{null, null}` = top level (`folder ""` = the source root).
|
||||
- `renderLevel()` from `current` + `kbTree`:
|
||||
- **Top level:** breadcrumb hidden; level block hidden; folders table = the source rows (name in the Folder cell, recursive count, the `(source, "")` description text when stored); the file table HIDDEN (files are seen per source — `ls()` shows no files at the top).
|
||||
- **Inside a source/folder:** breadcrumb segments (the source name, then the folder chain — each an `<a class="kb-crumb-link">`, the last segment a `<span aria-current="page">`); level block = the current level's stored description (title = the full source-relative path, e.g. `alpha/two`) — HIDDEN when none is stored (the `ls` rule: count only, no placeholder); folders table = the direct subfolders (`.folder-link` name, count, stored description text); file table = the direct files — `makeRow` UNCHANGED (the path link still `preventDefault`s + `openDocumentModal`; the no-JS href escape hatch intact); `#folders-wrap` hidden when the level has no subfolders, the file wrap hidden when it has no direct files.
|
||||
- **Stat cards:** walk the WHOLE tree (document count, chunks sum, max `indexed_at`) — the values identical to today's flat walk (`fmtDate` reuse).
|
||||
- **Empty state:** `#sources-empty` visible ONLY when `kbTree.sources` is empty (both wraps hidden); a registered 0-document source renders its row (`0`) instead — the semantic change is deliberate (the `ls` invariant), noted in the module docstring.
|
||||
- **Navigation (client-side, no fetch, no URL change):** source/folder link click → set `current`, re-render; breadcrumb link click → that ancestor; the top-level breadcrumb (if ever rendered) → `{null, null}`.
|
||||
- **Refresh wirings — the `loadDocs` call sites become `loadTree` (same points, same semantics):** the `bor:view-refresh` listener (admin branch), `applySyncSuccess` (the KB just changed), the upload-success branch of `startSyncPolling`. **Never-stale fallback (PLAN §7.4):** after a re-fetch, if `current.source` is no longer in the tree's sources, or `current.folder` no longer exists under it, reset `current = {null, null}` BEFORE rendering — no stale breadcrumb, no stale block.
|
||||
- The sync button / poll / label / banner / error-modal / gate code is otherwise UNTOUCHED.
|
||||
3. `frontend/assets/styles.css` — `.kb-crumb` (+ `.kb-crumb-link`, the `aria-current` segment styling), `.kb-level` (+ title/summary typography), `.kb-folders-table` (reuse the `.docs-table` rules), `.folder-link` (row drill link — hover/focus affordance, ≥ 24px target) — house dark-tech palette, AA contrast, no new hue (phase-92 invariant — monochrome-safe), `:focus-visible` via the global outline rule, no CDN.
|
||||
|
||||
## Testing & Quality
|
||||
- Frontend source pins (house pattern — new `tests/unit/test_kb_tree_ui.py`, mirroring `tests/unit/test_sync_button.py`'s sources.js pins): the exact `/api/docs/tree` fetch path; the `#kb-crumb` / `#kb-level` / `#folders-tbody` / `#kb-level-title` ids referenced; the drill-state transitions (source click, folder click, breadcrumb up, top reset); the textContent contract (no `innerHTML` with document-derived data — the module's standing rule); the empty-state semantic (`sources` empty → `#sources-empty`); the reset-to-top fallback on a vanished location; `loadTree` wired at exactly the three refresh points (view-refresh, sync success, upload success) — and the anonymous branch still never fetches.
|
||||
- The existing sources.js / sync-button / router pins stay green (the untouched machinery).
|
||||
- Coverage: the pins are the unit gate; the `app/` gate is untouched (no backend in this task).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] an admin RAG view lists the sources (count + description when stored); clicking through drills (the breadcrumb goes up); the level block shows the current directory's description; the level's files render as the existing 5-column rows with the modal link
|
||||
- [ ] the stat cards / empty state / sync machinery behave as today (the untouched pins green); anonymous still gets the gate with zero catalog fetches
|
||||
- [ ] a re-fetch after the current location vanished resets to the top level (pinned)
|
||||
- [ ] `uv run pytest` green (the pins); `uv run ruff check . && uv run pyright` clean
|
||||
- [ ] no backend change; the phase-94 + phase-57 suites green
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 05 — Folder description editor: the phase-57 affordance on the level block + the rows
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The owner edits (or clears) any directory's description with the exact interaction of the file-summary editor — Edit button → inline textarea (prefilled) → Save/Cancel → live-region status — wired to `PATCH /api/folders/summary` (task 03).
|
||||
|
||||
## Work
|
||||
1. `frontend/index.html` — inside `#kb-level`: `button.kb-summary-edit#kb-level-edit` (type=button, the phase-57 label "Edit", ≥ 24px target) after `p#kb-level-summary`.
|
||||
2. `frontend/assets/sources.js` — ONE shared editor function (mirrors the phase-57 wiring in `frontend/assets/document.js`; this module never builds HTML from document-derived data — the editor parts are static createElement, the description text is textContent only):
|
||||
- **Edit buttons:** on `#kb-level` (the static button) AND on every source/folder row's Description cell (created with the row in task 04's row builder — `.kb-summary-edit`). Always present — a description can be CREATED where none is stored (a < 2-document folder, or the generator's fail-soft miss; the editor opens prefilled with the empty string). No whoami gate in the view (the RAG view is admin-only already — the phase-16 gate; the endpoint's `require_admin` is the API-level gate).
|
||||
- **On Edit:** swap the description text node (level block `p` content / row cell content) for: `textarea.kb-summary-editor` (current value via `.value` — the XSS contract), `button.kb-summary-save` "Save", `button.kb-summary-cancel` "Cancel", `p.kb-summary-status[role="status"][aria-live="polite"]`; focus the textarea.
|
||||
- **Save** → `PATCH /api/folders/summary` with `{"source", "folder_path", "summary"}` — `folder_path` is `""` for the source root, the source-relative folder path otherwise (both known from `current` / the row data).
|
||||
- 200 → re-render the description text (textContent only) — the level block AND/OR the row cell where the edit happened — + status "Description updated."; an empty save → the text goes away (level block hidden when empty / row cell emptied) + status "Description cleared."; update the in-memory `kbTree` node's `summary` in place (no re-fetch — the tree state stays coherent; the reload fallback is the safety net).
|
||||
- Failure (non-2xx / network) → neutral retry copy (the phase-55 convention — e.g. "Couldn't save the description — try again."), the editor stays open with the user's text, the stored text is untouched.
|
||||
- **Cancel** → restore the text node (and drop the editor + status).
|
||||
3. `frontend/assets/styles.css` — `.kb-summary-edit`, `.kb-summary-editor` (min-height 8rem, the phase-57 spec), `.kb-summary-save`, `.kb-summary-cancel`, `.kb-summary-status` — house palette, AA contrast, `:focus-visible` via the global rule, no CDN.
|
||||
4. Module header docstring: the phase-97 edit section (the house per-phase note convention).
|
||||
|
||||
## Testing & Quality
|
||||
- Source pins (`tests/unit/test_kb_tree_ui.py` extensions): the exact PATCH path + body shape (`folder_path ""` for the source root); the editor element construction (textarea `.value` prefill, Save/Cancel, the live-region `role="status"`); the status copies "Description updated." / "Description cleared."; the in-place `kbTree` update on success; the failure path keeps the editor (pinned via the status-copy + open-editor assertions, the phase-55 pin style); textContent re-render (no `innerHTML` on the description); the `styles.css` class names (the five `.kb-summary-*` + `.kb-summary-edit` if distinct).
|
||||
- Coverage: the pins are the unit gate; no backend change.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Edit → Save round-trips: the new text renders in the row and/or level block, the server row is `manually_edited` (the task-03 integration pins cover the flag)
|
||||
- [ ] an empty save clears (the text is gone, "Description cleared."); Cancel restores; a failure keeps the editor + shows neutral copy
|
||||
- [ ] a description can be created where none was stored (the editor opens prefilled empty; the save creates the row — task-03 pin)
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean
|
||||
@@ -0,0 +1,32 @@
|
||||
# Task 06 — The dedicated story suite: `tests/e2e/test_kb_tree.py`
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The phase's Playwright gate (A16 — one dedicated E2E file, run in isolation): the drill-down tree, the directory descriptions, and the edit/clear round-trip pinned against the real app + the deterministic mock.
|
||||
|
||||
## Work
|
||||
`tests/e2e/test_kb_tree.py` — module docstring per the house pattern (story mapping, the run command, the fixture description, the mock notes):
|
||||
- **Fixture** (the phase-94 suite's pattern — a host temp-dir tree via `tmp_path_factory`, TWO local sources registered through the authenticated API, the real in-process `POST /api/sync` pipeline; the mock's existing `FOLDER_SUMMARY_MODE` branch stores the canned one-liner naming the folder, so every stored description is deterministic):
|
||||
- `alpha/` — `root-note.md` at the source root, `one/` (2 docs), `two/` (2 docs);
|
||||
- `beta/` — `gamma/` (2 docs).
|
||||
- Total: 7 documents; alpha counts 5 (root + 2 + 2), beta counts 2.
|
||||
- **Tests** (Playwright Mapping Rule — each test is one distinct observable):
|
||||
1. `test_top_level_lists_sources_with_descriptions` — the source rows: `alpha` (5) + `beta` (2) in registry order with the canned source-root descriptions; the top-level file table is HIDDEN (files are per-source — the `ls()` equivalence); the stat cards read 7 documents + the computed chunks total.
|
||||
2. `test_drill_into_source` — click the `alpha` row → breadcrumb `alpha` (aria-current); the level block shows alpha's root description (the canned text); the folder rows `one` / `two` each with count 2 + their canned summaries; the file row `root-note.md` (title + chunks column) in the file table.
|
||||
3. `test_drill_into_folder` — drill to `alpha` → `two` → breadcrumb `alpha` → `two`; the level block shows `two`'s description; the file rows `two-a` / `two-b` (titles, the Source column `alpha`); `#folders-wrap` hidden (no subfolders); the breadcrumb link on `alpha` goes back up to the source level.
|
||||
4. `test_edit_folder_description` — on the `alpha` level, click Edit on the `one/` row (or the level block for `alpha` — the executor's pick, pinned in the test) → the textarea prefilled with the canned text → set the new text → Save → the new text renders in the row/block + status "Description updated."; a SQL assert (the `SessionLocal` house pattern) on `folder_summaries`: the row's `summary` is the new text AND `manually_edited` is true.
|
||||
5. `test_clear_folder_description` — an empty save → the text is gone (row cell emptied / level block hidden) + status "Description cleared."; the SQL assert: no row for the folder.
|
||||
6. `test_manual_description_survives_a_changed_sync` — edit a description (test-4 style); add a new file to the fixture dir; `POST /api/sync` (the mock regenerates the OTHER folders' summaries) → the edited description is UNCHANGED (the catalog re-fetch renders it; the SQL row keeps the manual text + the flag) while the untouched folders show the canned regenerated text (the `kept_manual` path, E2E-pinned).
|
||||
7. `test_reload_falls_back_to_top_level` — drill to `alpha/two`; delete `two/`'s documents + chunks directly (the house DB pattern); re-show the RAG view (the nav link — the `bor:view-refresh` trigger) → the breadcrumb is hidden and the top level renders (the never-stale contract, PLAN §7.4).
|
||||
8. `test_anonymous_sees_the_gate` — the RAG view for an anonymous visitor: the sign-in gate visible, no folders/file table, no Edit affordance, and no `/api/docs/tree` request (the phase-16 soft rule — assert via the response/locator state, the existing suites' pattern).
|
||||
- **MOCK note:** no mock changes needed (the `FOLDER_SUMMARY_MODE` branch is phase 94's). The suite is mock-only in the standard sense — `E2E_REAL_LLM=1` keeps the canned summaries unavailable, so the suite follows the phase-94 convention (its assertions key on the canned text; the module docstring states the requirement).
|
||||
|
||||
## Testing & Quality
|
||||
- Run in isolation (DB up): `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov`.
|
||||
- The suite must not modify app/frontend source to pass — it asserts on the shipped behavior of tasks 01–05.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] the suite green in isolation; the test → observable mapping documented in the module docstring
|
||||
- [ ] the edit/clear/survive-sync/fallback observables all pinned (tests 4–7)
|
||||
- [ ] no changes to completed-phase code beyond what tasks 01–05 shipped
|
||||
@@ -0,0 +1,35 @@
|
||||
# Task 07 — The existing import / sync / upload story suites onto the drill-down catalog
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The story suites that assert the catalog through the old flat table keep asserting the SAME behavior (a document is listed, linkable, viewable; the stat cards hold) — with a drill step where the asserted path lives under a folder or source. No product behavior changes; navigation only.
|
||||
|
||||
## Work
|
||||
- **The recipe** (apply per suite — mechanical, assertion intent unchanged):
|
||||
1. **Nested-path row assertions** (`#docs-tbody tr` has_text a path with `/`): before asserting, drill — click the source row (top level), then each folder link (`#folders-tbody .folder-link`) one per path segment, until the level containing the asserted file renders. A small local drill helper per suite (a few lines: `for seg in path_segments: page.locator(f"#folders-tbody .folder-link:has-text('{seg}')").click()`) is fine — the house style is per-suite helpers, no new shared module unless two suites need the identical one.
|
||||
2. **Total row counts** (`#docs-tbody tr` to_have_count N over a whole KB): the flat total no longer exists in one tbody — re-assert per level (drill to each folder, assert that level's count; the sum is implied) or assert the stat cards (`#stat-docs` still holds the KB total — unchanged).
|
||||
3. **`.first` visible waits** (the catalog-rendered signal): the top-level file table is now always hidden (files are per-source — the `ls()` equivalence) — drill into a source (and folder, if all its files are nested) BEFORE the wait; the source/folder row itself is a valid "the catalog rendered" signal where the intent is just that.
|
||||
4. **Empty-state assertions:** unchanged semantics where the registry is empty (`#sources-empty`); where a source IS registered but the KB is cleared, `#docs-tbody` is still 0 rows (the file table hides; the source row renders `0` — adjust any `#sources-empty` expectation in that state if present).
|
||||
5. Suites that seed by direct `import_sources` (no registry row) work unchanged in principle — their source appears as an indexed-only source row (the `00_phase.md` superset rule); the drill just clicks that row.
|
||||
- **Suites (the known set from a source scan — the executor runs each in isolation and fixes whatever is red; the list is a starting point, not a guarantee):**
|
||||
- `test_import_documents.py` — EXPECTED_ROWS all nested under `homelab/…` + `deployments/…`: drill per folder for each row; the 13-stat + chunk/last-indexed cards; the hidden-junk absence (scoped after a drill); the empty-state case (registry empty — should already pass).
|
||||
- `test_admin_auth.py` — the 13 row count (per-level re-assert or the stat cards).
|
||||
- `test_sticky_navbar.py` — the TOTAL_DOCS count (same).
|
||||
- `test_sync_upload_progress.py` — the N_FILES counts + the `docs/*.md` rows (drill `docs`).
|
||||
- `test_upload_no_scan.py` — the `notes/…` rows + counts (drill `notes`; the `skipme.md` absence).
|
||||
- `test_sync_button.py`, `test_sync_model_down.py` — the FIXTURE_DOC row (drill as needed).
|
||||
- `test_import_extensions_env.py` — the SH_REL / MD_REL rows (drill).
|
||||
- `test_quadlet_jinja_import.py` — the `homelab/quadlet/…` rows (drill).
|
||||
- `test_retrieval_quality.py` — the `.hidden` absence (scoped after a drill).
|
||||
- `test_archive_upload_sources.py` — the cleared-state row count (the empty-state semantic, point 4).
|
||||
- anything else that turns red in the regression run (the full E2E list — the AGENTS.md rule-9 gate).
|
||||
- **Do NOT change the asserted product behavior.** If a suite's assertion cannot be preserved through the drill (a genuine behavior change, not a navigation one), stop and flag it in the phase record — no silent deviation (AGENTS.md rule 3 spirit).
|
||||
|
||||
## Testing & Quality
|
||||
- Each updated suite green in isolation: `uv run pytest tests/e2e/test_<suite>.py -v --no-cov` (DB up).
|
||||
- Coverage: n/a (tests-only task) — the `app/` gate is untouched.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] every suite in the list green in isolation; the asserted document behavior preserved (the drill is the only change)
|
||||
- [ ] no app/frontend source changes from this task (tests only)
|
||||
@@ -0,0 +1,33 @@
|
||||
# Task 08 — The existing viewer / nav / UX story suites + the regression run
|
||||
|
||||
**Phase:** `97_kb_tree_catalog` · **Story:** n/a (owner request)
|
||||
|
||||
## Objective
|
||||
The remaining story suites that touch the catalog table (document viewer, navigation, UX) are green on the tree view, and the phase's full validation gate is run end to end.
|
||||
|
||||
## Work
|
||||
- **The same recipe as task 07** (drill before asserting nested rows; per-level counts; drill before `.first` waits; the empty-state semantic) for:
|
||||
- `test_document_viewer.py` — the `gitlab-compose.yaml` / `xss-fixture.md` rows (drill, then the row link opens the modal — the viewer flow itself is unchanged).
|
||||
- `test_document_back_navigation.py` — the `kubernetes.md` row (drill).
|
||||
- `test_edit_summaries.py` — the DOC_PATH row (drill — the phase-57 summary-edit flow on the VIEWER is untouched by this phase; only the catalog navigation to reach the doc changes).
|
||||
- `test_summary_in_viewer.py` — the YAML/MD rows (drill).
|
||||
- `test_markdown_tables.py` — the TABLES_PATH row (drill).
|
||||
- `test_responsive_polish.py` — the `backup_rotation` row + the desktop/mobile `.first` waits (drill).
|
||||
- the `.first`-visible waits against all-nested KBs: `test_chat_persistence.py`, `test_navbar_refresh.py`, `test_nav_consistency.py`, `test_nav_switch_keeps_stream.py`, `test_shared_header.py`, `test_sources_midstream_bug.py` — drill to a level that has a file before the wait (or assert the source/folder row where the intent is just "the catalog rendered").
|
||||
- anything else that turns red in the regression run.
|
||||
- **The regression run (the phase gate, the AGENTS.md rule-9 commands):**
|
||||
1. `uv run pytest --cov=app --cov-report=term-missing` — unit + integration green, TOTAL > 90%.
|
||||
2. `uv run ruff check . && uv run pyright` — clean.
|
||||
3. The catalog-touching E2E list in isolation — at minimum the task 07/08 suites above PLUS the completed phases' suites this phase's area of change can touch: `test_ls_tree_drilldown.py` (the agent's `ls` — must be byte-identical for the same data), `test_kb_overview.py`, `test_edit_summaries.py`, `test_import_documents.py`, `test_sync_button.py`, `test_admin_auth.py`.
|
||||
4. `uv run pytest tests/e2e/test_kb_tree.py -v --no-cov` (task 06's suite) green in isolation.
|
||||
5. The full `tests/e2e/` sweep in isolation for any suite the run above did not cover and that references the RAG view — the final pass (the `00_phase.md` completion criteria) is the authority; this task leaves nothing red.
|
||||
|
||||
## Testing & Quality
|
||||
- Each updated suite green in isolation; the gates above green.
|
||||
- Coverage: the `app/` TOTAL > 90% (this task is tests-only — the coverage is the tasks 01–03 code, already gated).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] every suite in the list green in isolation; the asserted behavior preserved (drill = the only change)
|
||||
- [ ] unit + integration green, TOTAL coverage > 90%, ruff + pyright clean
|
||||
- [ ] `test_ls_tree_drilldown.py` green — the agent's tree is byte-identical (the only schema change is the additive flag column)
|
||||
- [ ] no app/frontend source changes from this task (tests only)
|
||||
Reference in New Issue
Block a user