chore(agent): track .agent/ planning tree in git

Remove the blanket .agent/ gitignore so the phase roadmap, user stories, reports, and PLAN.md are versioned with the code. Only runtime artifacts (.agent/phase-sessions/, .agent/pipeline.log) remain ignored. Update AGENTS.md git protocol rule to match.
2026-09-01 10:18:22 -04:00
parent 5fa620fde5
commit 4971e2859d
818 changed files with 23964 additions and 4 deletions
@@ -0,0 +1,39 @@
+# Phase 44 — Markdown tables (chat, viewer, thinking)
+
+**Source:** `TODO.md` L6 — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
+**Story:** `.agent/user_stories/markdown-tables.md`
+**Context:** `frontend/assets/markdown.js` is the shared escape-first renderer (no libs, A11): fence protection → escape → inline transforms (`code`, `**bold**`, `*em*`, h1–h3, lists) → paragraph pass → fence restore. It has **no table support** — GFM pipe tables render as one raw `|`-littered paragraph. The renderer serves the chat answer, the document viewer/modal, and the thinking block, so one change covers all three.
+
+## Objective
+GFM pipe tables render as semantic, styled, XSS-safe `<table>` elements everywhere the shared renderer runs, with a horizontal-overflow guard for wide tables.
+
+## Dependencies
+- `43_thinking_scroll_back` (todo) — sequential only (the thinking block also renders markdown; no shared-file conflict beyond the renderer itself).
+- `08_story_dark_tech_theme` (complete) — the palette tokens `.md-table` must use.
+- `26_document_modal_viewer` / `10_story_document_viewer` (complete) — the second renderer consumer (viewer/modal).
+
+## Tasks
+1. `01_table_renderer_and_styles.md` — table pass in `markdown.js` + `.md-table` CSS.
+2. `02_mock_table_trigger.md` — deterministic table answer (incl. a wide table) in `mock_llm.py`.
+3. `03_tables_e2e_and_commit.md` — unit pins + story E2E suite + regressions + commit.
+
+## Testing & Quality
+- Unit: new `tests/unit/test_markdown_tables.py` — source pins in the house style (regex over `markdown.js` / `styles.css`): the table-protection pass exists and runs **after** the fence pass and **before** the escape pass; cells are escaped + inline-transformed; output carries `class="md-table"`, `<thead>`, `th scope="col"`, and the `.md-table-wrap` wrapper; `styles.css` has the wrapper overflow rule + table borders + reduced-motion-relevant rules. (Behavior is browser-proven by the E2E; unit pins catch silent regressions without a browser — the established frontend pattern.)
+- Coverage: frontend-only — `app/` TOTAL unchanged, >90%.
+- E2E (mandatory, A16): `tests/e2e/test_markdown_tables.py`, run in isolation.
+
+## Completion Criteria
+- [ ] A pipe table in a chat answer renders `<div class="md-table-wrap"><table class="md-table">` with `<thead>`/`<tbody>`, `<th scope="col">` headers, correct cell texts; no raw `|---|` in the bubble.
+- [ ] A wide table scrolls inside its wrapper; the 46rem column does not overflow the page.
+- [ ] XSS-safe (escaped cells), fences win over tables, lone pipes stay text.
+- [ ] The document viewer/modal renders the same table for a fixture document containing one.
+- [ ] `uv run pytest` green; coverage TOTAL unchanged.
+- [ ] `uv run pytest tests/e2e/test_markdown_tables.py -v --no-cov` green in isolation (DB up).
+- [ ] Regression E2E suites green in isolation: `test_chat_rag.py`, `test_document_viewer.py`, `test_document_summaries.py`, `test_smoke.py`.
+- [ ] `uv run ruff check . && uv run pyright` clean.
+- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.
+
+## Locked decisions
+- **A11 untouched** — still the local ~90-line renderer, no library, no CDN.
+- **Owner-locked (2026-08-27, roadmap A3):** scope = GFM pipe tables (header + separator + body); links/blockquotes/hr out of scope; alignment colons parsed but rendered left; wide tables get the `overflow-x: auto` wrapper.
+- **A16/A17 honoured** — one story E2E suite, one atomic commit.
@@ -0,0 +1,40 @@
+# Task 01 — Table pass in the shared renderer + styles
+
+**Phase:** `44_markdown_tables` · **Source:** `TODO.md:6` — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
+**Story:** `.agent/user_stories/markdown-tables.md`
+
+## Objective
+`renderMarkdown` turns GFM pipe-table blocks into semantic tables (XSS-safe, inline markdown in cells), wrapped in a horizontal-overflow container, styled in the dark-tech palette.
+
+## Work
+1. `frontend/assets/markdown.js` — in `renderMarkdown(md)`, between step 1 (fence protection) and step 2 (escape + inline transforms), add a **table protection pass** using the same placeholder mechanism as the fences:
+   - **Detection** (line-oriented over the fence-protected text): a **table block** starts at a line containing `|` whose **next** line is a **separator** — the separator line consists of ≥1 pipe-separated cells, each matching `^\s*:?-+:?\s*$` (allowing a leading/trailing pipe and inter-cell whitespace). The block then extends over every following line that still contains `|` (the body; zero body rows is a valid table — header only). A maximal such block is one table. Anything else (a single `|` in prose, a separator with no `|`-header line above it, a 1-line "table") is left untouched.
+   - **Extraction:** for each table run, split each line on `|`, drop the leading/trailing empty entries produced by leading/trailing pipes, `trim()` each cell.
+   - **Cell rendering:** each cell goes through the same inline pipeline as the rest of the text — `escapeHtml(cell)` first (XSS-safe, invariant of the renderer), then the inline transforms (`` `code` ``, `**bold**`, `*em*` — the exact same `.replace` chain step 2 uses; factor the inline chain into a small local helper if it makes the cell path cleaner, keeping the whole-text path byte-identical in output).
+   - **Assembly:**
+     ```html
+     <div class="md-table-wrap">
+     <table class="md-table">
+     <thead><tr><th scope="col">h1</th>…</tr></thead>
+     <tbody><tr><td>…</td>…</tr>…</tbody>
+     </table>
+     </div>
+     ```
+     Rows with fewer cells than the header are padded with empty `<td>`; rows with more are truncated to the header width (defensive — the mock and real answers are well-formed). Alignment colons in the separator are **parsed but ignored** (all cells left — owner decision).
+   - **Placeholders:** reuse the `\u0000CODEn\u0000` array pattern — e.g. push the table HTML into a second array and emit `\u0000TABLEn\u0000`, restored alongside the code blocks in step 4 (update the restore step accordingly; tables inside the protected span are already final HTML — they must not re-enter the paragraph pass, which the placeholder guarantees).
+   - Update the file's header comment (the ~60-line no-CDN renderer now also does tables — 2026-08-27, `TODO.md` L6).
+2. `frontend/assets/styles.css` — near the markdown/content styling (the chat bubble content rules):
+   - `.md-table-wrap { overflow-x: auto; }` — the wrapper is the scroller;
+   - `.md-table { border-collapse: collapse; width: 100%; font-size: 0.9rem; }`;
+   - `.md-table th, .md-table td { border: 1px solid var(--line); padding: 0.4rem 0.6rem; text-align: left; vertical-align: top; }`;
+   - `.md-table thead th { background: <surface-darker token>; color: var(--ink); }` — pick the existing token that keeps ≥4.5:1 (PLAN §7.2: ink `#e8ebf4` on surface `#121a2e` is 14.5:1 — use the plain surface family, not brand);
+   - ensure the rule set is inside or consistent with the reduced-motion constraints (no animation involved — nothing to still).
+3. Sanity: run an existing markdown-consuming E2E (e.g. `test_chat_rag.py`) to confirm byte-identical output for non-table content (the inline-chain factor, if done, must not change any existing rendering).
+
+## Testing & Quality
+- Unit: `tests/unit/test_markdown_tables.py` (new) source pins per the phase overview (pass ordering, escape-first for cells, output markers, CSS rules). Full suite green.
+- Coverage: **>90%** on `app/` (unchanged — frontend-only).
+
+## Completion Criteria
+- [ ] `renderMarkdown` handles the shapes in the story's acceptance criteria 1–4 (table, XSS cell, fence-wins, non-tables stay text) — verifiable via the unit pins now and the E2E in task 03.
+- [ ] No existing rendering changes for non-table markdown (regression suite from step 3 green).
@@ -0,0 +1,40 @@
+# Task 02 — Deterministic table answer in the mock
+
+**Phase:** `44_markdown_tables` · **Source:** `TODO.md:6` — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
+**Story:** `.agent/user_stories/markdown-tables.md`
+
+## Objective
+The E2E mock serves a byte-stable table answer (plus a deliberately wide table and an XSS cell) on demand, following the existing trigger convention.
+
+## Work
+1. `tests/e2e/mock_llm.py` —
+   - add `TABLE_TRIGGER = "show me a table"` (same case-insensitive-substring convention as `LONG_ANSWER_TRIGGER` / `THINKING_TRIGGER` / `TOOLS_TRIGGER`);
+   - in `compose_answer(body)`, **before** the default tail-echo branch (and before `DEFLECT_MODE` — a deflection prompt never carries the marker, same reasoning as `SUMMARY_MODE`): when the trigger is in the lowercased user message, return the fixed table answer:
+     ```
+     Here's the shape, in a table:
+
+     | Service | Port | Host |
+     |---|---|---|
+     | Caddy | 80 | homelab-gw |
+     | GitLab | 8929 | homelab-git |
+     | ntfy | 2087 | homelab-ntfy |
+
+     <img src=x onerror=alert(1)>
+
+     And the wide one:
+
+     | A very long column header to force overflow | Second column with some padding text | Third column | Fourth | Fifth |
+     |---|---|---|---|---|
+     | value-one | value-two | value-three | value-four | value-five |
+     ```
+     (The `<img onerror>` line is the XSS assertion's payload — it must survive the mock byte-for-byte so the E2E can prove the renderer neutralizes it; the wide table guarantees `scrollWidth > clientWidth` inside the 46rem column.)
+   - keep the answer a plain grounded response (no `DEFLECT_MODE` interplay): the trigger question is asked against an on-topic fixture so the honesty gate is HIGH in the E2E (the suite asserts non-deflection as part of the table test).
+2. Update the module docstring's marker list (the file documents every trigger — add the table row).
+3. `uv run pytest tests/e2e/mock_llm.py-related unit tests` — run `uv run pytest tests/unit -k "mock" tests/integration -x` (or the mock's existing test file, if any — check `tests/` for mock-specific tests) to prove the new branch breaks no existing flow; the full suite is green (the new branch only fires on the marker).
+
+## Testing & Quality
+- Unit/integration: full suite green; the new branch is covered by the E2E (task 03) — if a mock-level unit test file exists, add the table case there so the branch is unit-covered too.
+- Coverage: **>90%** on `app/` (mock lives in `tests/` — the gate is unchanged).
+
+## Completion Criteria
+- [ ] `TABLE_TRIGGER` returns the fixed table answer (byte-stable), including the XSS line and the wide table; no existing mock behavior changes for marker-less requests.
@@ -0,0 +1,28 @@
+# Task 03 — Tables E2E + regressions + commit
+
+**Phase:** `44_markdown_tables` · **Source:** `TODO.md:6` — "Certain markdown formatting isn't working - tables for example don't get rendered as tables in the chat response."
+**Story:** `.agent/user_stories/markdown-tables.md`
+
+## Objective
+Prove the table contract in the browser — chat, overflow, XSS, viewer, and the two "not a table" regressions — then commit the phase.
+
+## Work
+1. `tests/e2e/test_markdown_tables.py` (new) — mock-only, DB up, per the story's Playwright Mapping Rule:
+   - `test_chat_table_renders` — ask an on-topic question containing `TABLE_TRIGGER` (pick a fixture topic that retrieves HIGH — reuse a question pattern from `test_chat_rag.py`); the brain bubble contains `<div class="md-table-wrap"><table class="md-table">`, a `<thead>` with three `<th scope="col">` (Service/Port/Host), the body cell texts ("Caddy", "8929", …), and **no** `|---|` separator text in the bubble;
+   - `test_wide_table_scrolls` — in the same answer, the wide table's wrapper has `scrollWidth > clientWidth`; horizontal scrolling (wheel/`scrollLeft`) moves it; the page itself has no horizontal overflow (`document.documentElement.scrollWidth <= clientWidth`);
+   - `test_table_xss_safe` — the `<img src=x onerror=…>` line renders as visible text (no `<img>` element inside the bubble; `onerror` can never fire — assert `page.evaluate` found zero injected img nodes and the tag text is present);
+   - `test_viewer_table_renders` — add a fixture document (extend `tests/fixtures/docs/homelab/` with a small `.md` file containing a pipe table — e.g. `tables.md` with a 3×3 table; re-import per the `test_document_documents.py`/`test_import_documents.py` fixture pattern), open it from the Sources table (admin) in the modal; the modal content renders `<table class="md-table">`;
+   - `test_fence_not_a_table` — a question/fixture whose content puts `|`-heavy lines inside a ``` fence (existing fixtures have fenced blocks — pick/extend one) renders `<pre><code>` with no `<table>`;
+   - `test_plain_pipe_stays_text` — an off-trigger grounded answer containing a single `|` in prose (assert via an existing deterministic answer or a minimal new fixture) renders as text, no `<table>`.
+   - Assert non-deflection (`.is-deflected` absent) in the table tests — the honesty gate interplay is part of the contract.
+2. Regression pass (isolation runs): `test_chat_rag.py`, `test_document_viewer.py`, `test_document_summaries.py` (the renderer is shared — summaries render through it too), `test_smoke.py`.
+3. `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL unchanged; `uv run ruff check . && uv run pyright` clean.
+4. Commit (Conventional Commits, `--no-gpg-sign`), e.g. `feat(chat): render markdown tables in answers, viewer, and thinking`, staging this phase's files; move `.agent/phases/todo/44_markdown_tables/` → `.agent/phases/complete/`.
+
+## Testing & Quality
+- E2E: `uv run pytest tests/e2e/test_markdown_tables.py -v --no-cov` green in isolation.
+- Coverage: **>90%** on `app/` (unchanged — frontend-only phase).
+
+## Completion Criteria
+- [ ] The story E2E suite passes in isolation (all six tests); the four regression suites pass in isolation.
+- [ ] One atomic `--no-gpg-sign` commit; phase dir moved to `.agent/phases/complete/`.