refactor(agents): migrate .agent/ planning tree to .agents/
Standardize on the .agents/ directory (shared with project skills): phases/, user_stories/, reports/, screenshots/, validate.sh, and phase-sessions/ + pipeline.log all move to .agents/ (git mv preserves history; runtime artifacts move alongside). Updates every reference in AGENTS.md, README.md, .gitignore, app docstrings, and test story headers. Historical KB content in data/ and the runtime pipeline.log transcript are left untouched.
This commit is contained in:
@@ -0,0 +1,45 @@
|
||||
# Phase 57 — Edit + Re-embed Document Summaries
|
||||
|
||||
**Source:** `TODO.md` L4 — "Be able to edit the summaries for documents in the RAG. Click an edit button in summary box and change the summary that the AI created. re-embed that document after changing the summary."
|
||||
**Story:** n/a (TODO-derived — owner roadmap confirmation 2026-08-31)
|
||||
**Context:** The summary box is the `.doc-summary` section rendered by the shared core in `frontend/assets/document.js` (page + modal surfaces) — it renders only when `doc.summary` is non-empty, as a labeled `h2` + text-node `<p>` (XSS contract: `textContent` only). Phase 30 stores a non-markdown document's `lite` digest on `documents.summary` **and** indexes it as one extra embedded chunk (`is_summary=True`, position −1 — `app/rag/importer.py:313–336`). The document viewer is deliberately PUBLIC (phase 16 owner decision — only the catalog is admin-gated), so the new edit affordance and endpoint must be admin-gated (`require_admin`, `app/core/auth.py`); `GET /api/whoami` (`app/api/auth.py:56`) already drives admin reveals on static pages. Async endpoints exist in `app/api/` (`app/api/chat.py:223`, `app/api/git_sources.py:231`), so the re-embed (`await llm.embed([...])`) fits a standard `async def` handler.
|
||||
|
||||
## Objective
|
||||
An admin can edit (or clear) the AI-generated summary directly in the summary box; on save the stored summary and its `is_summary` chunk are updated and the chunk is **re-embedded** — anonymous visitors see the unchanged public viewer with no edit affordance.
|
||||
|
||||
## Dependencies
|
||||
- `56_import_extensions_env` (todo, preceding)
|
||||
|
||||
## Tasks
|
||||
1. `01_update_summary_api.md` — `PATCH /api/documents/summary` (admin): update/clear `documents.summary`, replace the `is_summary` chunk, re-embed it (one `embed` call, DB untouched if the LLM fails).
|
||||
2. `02_summary_edit_ui.md` — Edit button in the `.doc-summary` panel (admin-only) → inline textarea + Save/Cancel → live-region status on both viewer surfaces.
|
||||
3. `03_e2e_edit_summaries.md` — story Playwright suite + regressions + the atomic commit.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: the existing document-content API tests (phase 10) gain the PATCH cases — update + re-embed (chunk content + non-NULL vector + unchanged chunk count), clear (summary NULL + chunk deleted), markdown doc with no prior `is_summary` chunk (one created), 404 unknown pair, 403 anonymous, LLM-failure error mapping (502/503, DB unchanged).
|
||||
- Frontend source pins (house style, `tests/unit/test_save_chat_ui.py` pattern): `document.js` (whoami gate, editor wiring, PATCH path, `textContent` render contract), `styles.css` (new `.doc-summary-*` classes).
|
||||
- E2E (mandatory, A16): `tests/e2e/test_edit_summaries.py`, run in isolation (mock `SUMMARY_MODE` fixture KB).
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Admin: `PATCH /api/documents/summary` with new text updates `documents.summary`, replaces the `is_summary` chunk's content, and stores a fresh embedding (content chunks untouched — count unchanged).
|
||||
- [ ] Admin: PATCH with an empty/whitespace summary clears `documents.summary` (NULL) and deletes the `is_summary` chunk.
|
||||
- [ ] Anonymous: 403 on PATCH; no Edit button in the viewer; the viewer otherwise renders byte-for-byte as today.
|
||||
- [ ] The Edit button appears in the summary box (document page **and** modal), opens a textarea prefilled with the current summary, and Save reflects the change in the panel with a live-region confirmation; Cancel restores.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%.
|
||||
- [ ] `uv run pytest tests/e2e/test_edit_summaries.py -v --no-cov` green in isolation (DB up).
|
||||
- [ ] Regression E2E suites green in isolation: `test_document_summaries.py`, `test_document_viewer.py`, `test_cache_busting.py`.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agents/phases/complete/` (`.agents/` stays untracked — owner instruction, commit 281f355).
|
||||
|
||||
## Locked decisions
|
||||
- **Owner-locked (2026-08-31, roadmap confirmation, D4):**
|
||||
1. Summary editing is **admin-only** (endpoint + UI affordance). The document viewer stays public.
|
||||
2. "Re-embed that document" = replace the `is_summary` chunk with a fresh embedding (one `embed` call). The document's content chunks are **not** re-embedded — the summary is the only text that changed.
|
||||
3. Clearing the summary (empty save) sets `documents.summary = NULL` and deletes the `is_summary` chunk.
|
||||
- **Fail-before-write:** the embedding happens before any DB mutation; an LLM failure returns an error (502/503, `app/api/git_sources.py` `ModelUnavailableError`-style mapping) and leaves the row and chunk untouched.
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add app/ frontend/ tests/ && git commit --no-gpg-sign -m "feat(kb): edit + re-embed document summaries from the viewer (admin)"
|
||||
```
|
||||
@@ -0,0 +1,40 @@
|
||||
# Task 01 — PATCH /api/documents/summary (admin, re-embed)
|
||||
|
||||
**Phase:** `57_edit_document_summaries` · **Source:** `TODO.md:4` — "Be able to edit the summaries for documents in the RAG. … re-embed that document after changing the summary."
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
A single admin endpoint that updates (or clears) a document's stored summary and re-embeds its `is_summary` chunk — embed first, mutate second, so a failed LLM call never leaves a half-updated row.
|
||||
|
||||
## Work
|
||||
1. `app/schemas.py`:
|
||||
- `SummaryUpdate { source: str, path: str, summary: str }` (request).
|
||||
- `SummaryResult { source: str, path: str, summary: str | None, chunks: int }` (response — `summary` NULL after a clear).
|
||||
2. `app/api/docs.py` — new route (extend the module docstring's route list):
|
||||
```python
|
||||
@router.patch("/documents/summary", response_model=SummaryResult)
|
||||
async def update_document_summary(
|
||||
payload: SummaryUpdate,
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
_admin: None = Depends(require_admin), # noqa: B008
|
||||
) -> SummaryResult: ...
|
||||
```
|
||||
- Look up `Document` by `(source, path)` → 404 `{"detail": "document not found"}` (same shape as `GET /api/documents/content`).
|
||||
- `text = payload.summary.strip()`.
|
||||
- **text == ""** → delete the existing `is_summary` chunk (if any) and set `doc.summary = None`.
|
||||
- **text != ""** → `llm = LLMClient()`; `vector = (await llm.embed([text]))[0]` **before** any mutation (embed failure → 502/503 with a `detail` naming the failure, following the `ModelUnavailableError` handling in `app/api/git_sources.py`; the DB is untouched). Then upsert the `is_summary` chunk: an existing one (position −1) gets `content`/`embedding` replaced; a missing one (markdown doc, or a phase-30 fail-soft import) is created as `Chunk(document_id=doc.id, position=-1, is_summary=True)`; set `chunk.embedding = vector`; `doc.summary = text`.
|
||||
- Commit; return `SummaryResult` with the post-change total chunk count (the `func.count(Chunk.id)` join from `get_document_content`).
|
||||
3. Note in the route docstring: admin-only (the viewer stays public — phase 16), re-embed scope is the summary chunk only (D4).
|
||||
|
||||
## Testing & Quality
|
||||
- Integration (extend the phase-10 document-content API test file):
|
||||
- Update: seeded non-md doc (mock `SUMMARY_MODE` digest) → PATCH new text → `documents.summary` == new text; the `is_summary` chunk's `content` == new text and `embedding` is non-NULL; total chunk count unchanged.
|
||||
- Clear: PATCH `""`/`" "` → `documents.summary` NULL; `is_summary` chunk row deleted; count −1.
|
||||
- Markdown doc (no prior `is_summary` chunk) → PATCH → one `is_summary` chunk exists at position −1 with the new embedding.
|
||||
- 404 unknown `(source, path)` (incl. a traversal string); 403 anonymous on PATCH.
|
||||
- LLM failure: point the mock at a dead port for the embed call (or raise in the fake) → 502/503 with `detail`; row + chunk unchanged.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] PATCH update, clear, and markdown-doc cases behave per the criteria above; 404/403/LLM-failure mappings pinned.
|
||||
- [ ] No content-chunk mutation on any code path (embedding assignment touches only the `is_summary` chunk).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Task 02 — Edit affordance in the summary box (admin-only)
|
||||
|
||||
**Phase:** `57_edit_document_summaries` · **Source:** `TODO.md:4` — "Click an edit button in summary box and change the summary that the AI created."
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
The `.doc-summary` panel gains an Edit button that opens an inline editor (prefilled textarea + Save/Cancel + live-region status) — visible to admins only, on both viewer surfaces through the one shared core.
|
||||
|
||||
## Work
|
||||
1. `frontend/assets/document.js` (the shared core — document page **and** modal render through it):
|
||||
- After the `.doc-summary` section is built (the existing `doc.summary` non-empty branch, ~L116), append an Edit button to the section's header row: `<button type="button" class="doc-summary-edit">Edit</button>` (24px+ target, house AA palette, `:focus-visible`).
|
||||
- Admin gate: a small `docAdminReady()` helper — `GET /api/whoami` (the boot-fetch pattern `brand.js`/`app.js` use). Non-admin (or fetch failure) → **no button, no editor wiring, no admin-only network call** — the public viewer is byte-for-byte unchanged.
|
||||
- Editor: on Edit, swap `.doc-summary-text` for:
|
||||
- `<textarea class="doc-summary-editor">` prefilled with the current summary (value, not innerHTML — XSS contract), min-height 8rem;
|
||||
- Save / Cancel buttons (`.doc-summary-save`, `.doc-summary-cancel`);
|
||||
- `<p class="doc-summary-status" role="status" aria-live="polite"></p>`.
|
||||
- Save → `PATCH /api/documents/summary` `{source, path, summary}` (source/path from the page's query params — the modal core already carries them, `document-modal.js:65`). Success → re-render the text node (`textContent` only) and status "Summary updated."; an empty save that clears → status "Summary cleared." and rebuild the content container so the panel disappears (the renderer only shows it for non-empty summaries). Cancel → restore the text node. Failure → status error with neutral retry copy (phase-55 convention), editor stays open with the user's text.
|
||||
2. `frontend/assets/styles.css`: `.doc-summary-edit`, `.doc-summary-editor`, `.doc-summary-save/-cancel`, `.doc-summary-status` — house dark-tech palette (AA contrast per the phase-08 tokens), system fonts, no CDN, `:focus-visible` via the global outline rule.
|
||||
|
||||
## Testing & Quality
|
||||
- Frontend source pins (house pattern, `tests/unit/test_save_chat_ui.py`):
|
||||
- `document.js`: whoami gate (button absent without admin), editor element construction (textarea value, buttons, live region), the exact PATCH path + body shape, `textContent` re-render (no `innerHTML` on user text).
|
||||
- `styles.css`: the five new class names present.
|
||||
- The existing `document.js` pins (phase 36 summary-panel contract) stay green.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Admin sees the Edit button in the summary box on both surfaces; anonymous never does (and no `/api/whoami`-beyond call leaks).
|
||||
- [ ] Save round-trips to the PATCH endpoint; the panel reflects the new text (or disappears on clear); Cancel restores; failures keep the editor + show neutral copy.
|
||||
- [ ] `uv run pytest` green (unit pins); `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 03 — E2E: edit the summary in the browser + commit
|
||||
|
||||
**Phase:** `57_edit_document_summaries` · **Source:** `TODO.md:4` — "Be able to edit the summaries for documents in the RAG. Click an edit button in summary box … re-embed that document after changing the summary."
|
||||
**Story:** n/a (TODO-derived)
|
||||
|
||||
## Objective
|
||||
Prove the edit → save → re-embed loop in the browser (admin + anonymous views), run the regressions, and commit the phase.
|
||||
|
||||
## Work
|
||||
1. Story-dedicated fixture dir `tests/fixtures/summary_edit_kb/` (house pattern: `summary_kb/` — the shared `docs/` fixtures stay pinned): one non-md A9 doc, e.g. `quadlet/llamacpp.container`, token-diluted so the mock `SUMMARY_MODE` digest is the stored summary (follow `tests/e2e/test_document_summaries.py`'s header docstring for the digest/sentinel mechanics).
|
||||
2. `tests/e2e/test_edit_summaries.py` (Playwright, DB up, mock LLM; admin login via `e2e.auth_helpers.login`):
|
||||
- `test_admin_edits_summary` — seed in-process (the `test_document_summaries.py` seeding pattern); open `/document.html?source=…&path=…` (admin) → the `.doc-summary` panel shows the digest **and** a visible Edit button → click → textarea prefilled with the digest + Save/Cancel + live region → replace the text (a distinctive new sentence) → Save → the panel text updates; `httpx GET /api/documents/content` (no cookie needed — public read) returns `summary` == the new text; DB check (direct `SessionLocal`): the `is_summary` chunk's `content` == new text, its `embedding` is non-NULL, and the total chunk count is unchanged.
|
||||
- `test_admin_clears_summary` — Edit → select-all + delete → Save → the panel disappears from the DOM; API `summary` is null; the `is_summary` chunk row is gone (count −1).
|
||||
- `test_anonymous_cannot` — fresh context (no login): the panel renders the digest but **no** Edit button; `httpx PATCH /api/documents/summary` without the cookie → 403.
|
||||
- DB isolation: the fixture's `source` name is distinctive — never assert on absolute row counts; delete the rows it creates in a `finally` (admin cookie).
|
||||
3. Regression pass (isolation runs): `test_document_summaries.py` (the retrieval story — the summary chunk must still be retrievable and resolve to the parent doc), `test_document_viewer.py`, `test_cache_busting.py` (no new page — `styles.css`/`document.js` are rewritten assets, `?v=` picks them up automatically).
|
||||
4. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
5. Commit (Conventional Commits, `--no-gpg-sign`) — the message from the phase overview's Commit section; move `.agents/phases/todo/57_edit_document_summaries/` → `.agents/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: `uv run pytest tests/e2e/test_edit_summaries.py -v --no-cov` green in isolation.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Edit + Save + clear loops pass in isolation (admin); anonymous pins pass.
|
||||
- [ ] The re-embedded chunk is verifiable in the DB (new content, non-NULL vector, count discipline).
|
||||
- [ ] Regression suites pass in isolation.
|
||||
- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agents/phases/complete/`.
|
||||
Reference in New Issue
Block a user