Commit Graph
18 Commits
Author SHA1 Message Date
ducoterra 0f6b9ff7e6 phase: 86_history_page_width
All criteria verified. Final report:

**Phase 86 final verification pass — all green, no defects found.**

- **Verified (previously implemented):** `position: relative` on shared `.table-wrap` (the one-rule A3 fix, +12 lines in `frontend/assets/styles.css`); unit pin `tests/unit/test_table_wrap_stacking.py` (3 tests); E2E story `tests/e2e/test_history_page_width.py` (5 tests: History 626→≤375, Tokens 618→≤375, SPA switch with window-sentinel, RAG regression, desktop 1280×800)
- **Tests/lint (this pass):** `uv run pytest` → 1720 passed · `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%** (>90% floor) · E2E in isolation: `test_history_page_width.py` 5 passed, `test_chat_history.py` 5 passed, `test_api_tokens.py` 9 passed, `test_smoke.py` 3 passed · `uv run ruff check . && uv run pyright` → clean, 0 errors
- **Completion criteria:** ① 375px direct+SPA `scrollWidth ≤ innerWidth` on History/Tokens — PASS (E2E pins) ② in-card scroll preserved + full-width + a11y spans in DOM — PASS ③ Sources + desktop regression — PASS ④ full suite/coverage/E2E/lint gates — PASS ⑤ `git diff --stat` limited to `styles.css` (+12) + new test files + phase files, no `app/`/markup/JS — PASS ⑥ commit/move — left to harness per executor rules (working tree intact, `todo/` dir removal already reflected)
- **Notable:** red→green (CSS reverted → 626px failure) and manual live check already recorded in `.agents/reports/86_history_page_width/`; pre-existing untracked `.agents/remediation_plan.md` (Sep 7 security audit) untouched
- **Next pending phase:** `87_big_read_progress`
2026-09-08 02:38:45 -04:00
ducoterra 412a560348 phase: 85_mobile_menu_gate_overlap
All verification complete. Final report:

**Phase 85 final verification pass — all green** (tasks 01–03 already complete; independently re-verified from scratch)
- Verified fix in tree: `styles.css` `.auth-gate` z-index 500→15 + rewritten stacking comment (no `z-index: 500` left); `index.html`/`document.html` comment-only; `token-gate.js` docstring-only (logic byte-identical); `test_mobile_hamburger_nav.py` real-click conversion + new `test_anonymous_toggle_tappable_with_gate_up`; new `tests/unit/test_gate_header_stacking.py` (3 pins); `test_api_tokens.py` untouched
- `uv run pytest` → 1717 passed, 1 warning (exit 0)
- `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%** (>90%)
- E2E in isolation: `test_mobile_hamburger_nav.py` **8 passed**; `test_api_tokens.py` **9 passed** (unchanged); `test_smoke.py` **3 passed**; `test_document_viewer.py` **7 passed**
- `uv run ruff check . && uv run pyright` → clean / 0 errors
- Live probe (375×812, anonymous, fresh server): on `/` and `/history.html` `elementFromPoint` at toggle → SVG `path`, never `#auth-gate`; real click opens menu (`aria-expanded=true`); exactly one visible nav link (Chat) + sign-in copy; Chat link topmost over gate; `#main` inert — criteria 1, 2, 3 confirmed directly
- Criteria: (1) real-click menu on / + /history ✅ (2) dropdown above gate ✅ (3) anonymous contents + #main inert ✅ (4) admin byte-identical (phase-46 tests green) ✅ (5) doc-viewer gate under bar (CSS pin + doc-viewer suite) ✅ (6) full gate green ✅ (7) diff scoped to the 5 code files + new unit test + phase files, nothing in `app/` ✅ (8) commit + todo→complete move left to harness per executor rules (task files already in `complete/`)
- Note: `.agents/remediation_plan.md` (untracked) is a pre-existing artifact of the earlier security audit — left untouched; a missing "N passed" line seen mid-pass was my own `-q`+addopts `-q` = `-qq` quirk, not a defect
- Next pending phase: `86_history_page_width`
2026-09-08 01:56:37 -04:00
ducoterra f4150421bb phase: 84_docs_push_error_sanitization
**Phase 84 — final verification pass: all green, no defects found**

- Verified implementation: `app/core/errors.py` (verbatim lift of sync masker), `app/api/sync.py` alias import, docs-push 502 `detail=sanitize_error(str(exc))`, all five `llm.py` error sites sanitized; new/extended test pins in place
- Tests: `uv run pytest` → **1714 passed, 0 failed**; targeted pins (new unit ×2 + integration ×1, existing 502 pin) → 13 passed; sync/git-sources regression → 67 passed
- Coverage: `uv run pytest --cov=app --cov-report=term-missing` → **99%** (`app/core/errors.py` 100%, `app/rag/llm.py` 100%) — >90% met
- E2E isolation: `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → **3 passed**
- Lint/types: `uv run ruff check .` → clean; `uv run pyright` → **0 errors**
- Criteria: 502 masks `*****@`/never token + row untouched ✅; LLM base-URL masked, credential-free strings byte-identical ✅; `_CREDS_RE` only in `app/core/errors.py` (working-tree grep) ✅; full gate green ✅; `git diff --stat` limited to the 4 app files + 2 modified test files + 3 phase task files (untracked: new module, new unit test, complete/ dir, reports, audit plan) ✅
- Commit/phase move left to the harness per instructions (task files already in `complete/`)
- No deviations; nothing to fix
- Next pending phase: **85_mobile_menu_gate_overlap**
2026-09-08 01:11:45 -04:00
ducoterra fa189dede7 phase: 83_chat_save_payload_limits
All verification criteria are confirmed green. Final report:

**Phase 83 — final verification pass (all tasks already complete; verified, nothing to fix)**

- Verified `app/schemas.py` carries every A1 cap (text/thinking 32 000, sources 20, suggestions 50×200, tools 50, SourceRef 120/1000/500, ToolCall 100/2000, messages ≤200) — boundary-only, `app/api/chats.py` untouched
- Verified all test pins exist and pass: 32 unit boundary tests (both edges of every cap + round-trip), 4 oversized-422 integration pins (text/201-msgs/21-sources/PUT, each with nothing-stored/row-unchanged assertions), A3 SSE done-event pin at column maxima, dedicated E2E (anon 40 000-char POST → 422 + small save → 201)
- `uv run pytest tests/unit/test_schemas.py -v` — 32 passed
- `uv run pytest tests/integration/test_chats_api.py` — exit 0; A3 pin — passed
- `uv run pytest tests/e2e/test_chat_save_payload_limits.py -v --no-cov` — 2 passed (isolation); `uv run pytest tests/e2e/test_chat_history.py -v --no-cov` — 5 passed
- `uv run pytest` — exit 0 (~1 704 tests, 0 fail/skip); `uv run pytest --cov=app` — TOTAL 99%, `app/schemas.py` 100% (>90% ✓)
- `uv run ruff check . && uv run pyright` — clean (0 errors)
- `git diff --stat` — only `app/schemas.py`, 3 test files (+`test_chat_api.py` A3 pin, sanctioned by task 02), phase files; no `app/api/chats.py`/`alembic`/`frontend`/`pyproject`/`uv.lock` diff ✓
- All completion criteria met; commit + phase move left to the harness per pipeline rules (changes stay in working tree)
- Deviation note: list caps use `max_length` instead of `max_items` — identical in pydantic 2.13 (`max_items` is a deprecated alias); both boundaries behaviorally pinned
- Next pending phase: `84_docs_push_error_sanitization`
2026-09-08 00:33:21 -04:00
ducoterra e29d68d9f0 phase: 82_security_headers
All completion criteria verified green — no defects found, nothing to fix. Final report:

**Phase 82 (security headers) — final verification pass: all green**

- Verified prior-run implementation: `app/core/security_headers.py` (pure-ASGI, header-only, exact A1 CSP), registration in `app/main.py` after `configure_caching` (outermost), unit/integration/E2E suites.
- Deviation confirmed sound: `data:`-URI favicon (blocked by locked CSP) → static `frontend/assets/favicon.svg` in 5 templates + Containerfile `cp`; SVG element byte-identical to the old data-URI (verified programmatically); serves 200 with all three headers.
- Curl check (server booted like e2e conftest, log: `/tmp/curl_security_headers_final.log`): `/`, `/api/health`, `/assets/styles.css`, `/nope` (404) → all three headers, CSP exactly `default-src 'self'; base-uri 'none'; frame-ancestors 'none'`.
- `uv run pytest tests/unit/test_security_headers.py tests/integration/test_security_headers.py -v --no-cov` → 13 passed (incl. SSE byte-identity pin).
- `uv run pytest tests/e2e/test_security_headers.py -v --no-cov` (isolated) → 2 passed (headers + zero CSP violations + painted page).
- SSE tripwire `uv run pytest tests/e2e/test_chat_rag.py -v --no-cov` → 3 passed.
- `uv run pytest --cov=app --cov-report=term-missing` → 1665 passed, app/ 99% (>90%); `uv run ruff check . && uv run pyright` → clean (0 errors).
- `git diff --stat` limited to phase-82 files + the two documented deviations (favicon set, `tests/unit/__init__.py`); no `pyproject.toml`/`uv.lock`/JS diffs.
- Commit + phase-dir move left to the harness per pipeline rules (not executed by me).

Next pending phase: `83_chat_save_payload_limits`.
2026-09-07 23:54:41 -04:00
ducoterra 42a4222949 phase: 81_login_rate_limit
All gates verified. The phase is complete — everything was already implemented by prior task runs; this pass verified each completion criterion end-to-end and found no defects.

**Phase 81 final verification report** (tasks 01–03 all present in `complete/`; no code changes needed this pass)

- Verified: `app/core/rate_limit.py` (stdlib sliding window, fail-open) + both login routes' 429 pre-check/record/reset wiring + all unit/integration pins present (11th-429, blocked-success-still-429, clean-counter reset, shared counter, autouse `clean_rate_limit` fixture documented).
- Live check (task 03): dev server + 11 rapid wrong logins → `401 ×10, 429`; 429 body carries `retry-after: 900` + generic detail; server restarted (per-process counter cleared by design).
- `uv run pytest tests/unit/test_rate_limit.py -v --no-cov` → 10 passed
- `uv run pytest --cov=app --cov-report=term-missing` → 1652 passed, **TOTAL 99%** (>90%; rate_limit.py 100%, auth.py 100%)
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` (isolation) → 3 passed
- `uv run ruff check . && uv run pyright` → All checks passed / 0 errors, 0 warnings
- Completion criteria: all met, except commit + phase-dir move — per harness rules I left all changes uncommitted in the working tree (harness commits atomically and moves the phase).
- Diff scope: exactly `app/core/rate_limit.py`, `app/api/auth.py`, `tests/unit/test_rate_limit.py`, `tests/integration/test_auth_api.py` + phase files; `pyproject.toml` / `uv.lock` / `frontend/` untouched.
- Deviations: none in code; commit/move deferred to harness as instructed.
- Next pending phase: `82_security_headers`.
2026-09-07 23:08:46 -04:00
ducoterra e2bed52751 chore(agent): phase roadmap from TODO.md, 3 phases (85-87)
TODO.md L3-L5 converted to executable phases (protocol B - append):
- 85_mobile_menu_gate_overlap (L3): the phase-79 token gate (z 500, fixed
  full-viewport) sits above the sticky header (z 20), so an
  unauthenticated visitor's tap on the mobile hamburger hits the gate
  overlay and the menu is unreachable until login. Reproduced:
  elementFromPoint at the toggle resolves to #auth-gate on every shell
  view; real clicks are intercepted. Fix: gate at z 15 (below the
  header + its mobile dropdown), #main stays inert-locked; the
  phase-46 E2E's programmatic-click workaround becomes a real click +
  a new TODO-regression pin.
- 86_history_page_width (L4): at 375px the History page panned ~250px
  into a blank region (document scrollWidth 626) although the table
  scrolled fine inside its card. Root cause: the .visually-hidden
  Actions header span is position:absolute with no positioned ancestor,
  so its 1px box (at the 640px table's right edge) leaks into the
  document's scrollable overflow. Fix: position:relative on the shared
  .table-wrap card (the identical Tokens-view defect, measured 618, is
  fixed by the same rule; RAG is clean and pinned). New E2E story.
- 87_big_read_progress (L5): after a tool read the UI sat on a static
  'Reading <path>' line while the model prefilled the big context -
  the turn looked frozen. Frontend-only: a ticking '(Ns)' suffix on
  the latest tool line after 5s of frame silence (settle on the next
  frame, live-only - restored lines stay timer-free) + the existing
  10s aria-only typing clock promoted to a visible 'Ns' hint. New
  source-level unit pins + slow-proxy E2E story (deterministic >=6s
  gaps via the mock tool flow).

TODO.md cleared (items now live in .agents/phases/todo/). The pre-existing
uncommitted 81-84 phases + remediation_plan.md are a separate workstream
and are NOT part of this commit.
2026-09-07 21:12:03 -04:00
ducoterra 7fce6572d0 feat: phases 77–80 — navbar view refresh, static background, API tokens, history suggestion chips
Build and Push Containers / build-and-push-app (push) Successful in 1m45s
Build and Push Containers / build-and-push-db (push) Successful in 13s
Single consolidated commit for four completed, validated phases (77, 78,
79, 80). The pipeline run left all work uncommitted because the harness
commits only with PHASE_COMMIT=1 while child executors are forbidden from
committing; the phases themselves all passed validation and moved to
.agents/phases/complete/.

Phase 77 — navbar view refresh
- router.js dispatches bor:view-refresh on re-show / active re-click /
  popstate (gated on wasMounted; first show and boot exempt)
- History / RAG / Sources / Tuning re-fetch on refresh (admin branch);
  Chat deliberately excluded (stream survival)
- History "Refresh" button (admin-only, in-flight disable + status line)
- New story suite tests/e2e/test_navbar_refresh.py (7 tests)

Phase 78 — static background
- Removed the animated glow layers; static 44px grid over the flat --bg
  canvas; default and reduced-motion renders byte-identical
- Updated background/theme E2E suites; removed bg-glow test pins

Phase 79 — API tokens
- api_tokens model + migration 0012; hash-only token service
- Admin tokens API + Tokens admin view; POST /api/token-auth;
  live-revoking require_user on chat / suggestions / document content
- Frontend token gate with localStorage cache; anonymous E2E suites
  migrated to token login
- New story suite tests/e2e/test_api_tokens.py (9 tests)

Phase 80 — history suggestion chips
- last_questions() endpoint with SEED fallback; startNewChat() refetch
- Seed-semantics docs (config.py, .env.example, README)
- Integration state matrix + E2E suite rewritten to the 4 chip states

Also included: phase-76 report artifacts and the repo restore-test-db
skill (previously untracked), scripts/* ruff fixes from phase 77.

Final gate state (phase 80 final pass, covers everything above):
- uv run pytest --cov=app → 1637 passed, 0 failed, app/ coverage 99%
- uv run ruff check . && uv run pyright → clean, 0 errors
- Per-phase story E2E suites green in isolation
2026-09-07 12:39:01 -04:00
ducoterra 495d042a98 chore(agent): phase roadmap from TODO.md — 4 phases (77–80)
Build and Push Containers / build-and-push-app (push) Successful in 1m54s
Build and Push Containers / build-and-push-db (push) Successful in 13s
Protocol B append: navbar refresh + History refresh button (77, TODO L3),
static background — glow layers removed (78, TODO L4), admin-issued API
tokens with the in-app gate + browser caching, only shared chats stay
anonymous (79, TODO L5), onboarding chips as the last 3 questions asked
with the env seed only before the first (80, TODO L6).

TODO.md cleared — its items now live in .agents/phases/todo/.
Owner-confirmed assumptions recorded in each phase overview
(A1–A7, chat 2026-09-06).
2026-09-06 23:54:11 -04:00
ducoterra f221b40fce feat(agent): add CSV benchmark recorder + summary/embedding test scripts and skills
New files:
- scripts/model_benchmark.py — shared CSV recorder for all model tests
- scripts/test_summary_model.py — summary model quality benchmark (coherence, coverage, brevity, hallucination)
- scripts/test_embed_model.py — embedding model benchmark (dimension, cosine accuracy, speed)
- .agents/skills/test-summary-model/SKILL.md — skill for testing summary models
- .agents/skills/test-embed-model/SKILL.md — skill for testing embedding models
- benchmarks/README.md — schema documentation

Updated:
- .agents/skills/test-chat-model/SKILL.md — now also records to CSV

All three scripts write to benchmarks/model_benchmarks.csv with one row
per run per check. The CSV accumulates results across runs for comparison.
2026-09-06 21:58:35 -04:00
ducoterra ffa919b8bf fix(chat): keep in-flight answers alive across in-app view switches
Root cause (owner repro, verified in a real browser 2026-09-06): the
five navbar views (Chat, RAG, Sources, Tuning, History) were separate
HTML documents, so a navbar click was a REAL cross-document navigation
— the chat page unloaded, the in-flight SSE fetch was aborted, and the
phase-48 teardown (app/api/chat.py `finally`, "chat: turn cancelled")
stopped the model. Observed: send question -> click RAG mid-stream ->
click Chat -> the answer never finished: no `query_log` row, and on
return a dangling question with no brain record (the pre-token pagehide
partial persist skips because `acc` is empty).

Phase-48 LOCKED-DECISION REFINEMENT (owner-confirmed 2026-09-06,
flagged per AGENTS.md rule 3, not silently deviated): "real navigation
cancels the fetch" now means LEAVING THE APP — tab close,
external/other-document navigation, the Stop button. In-app navbar
switches are client-side view switches and no longer cancel.

Fix — Option A (SPA shell), chosen over B (Service Worker owns the
stream) and C (server-side turn registry + resume):
- frontend/index.html is the shell: ONE `<main id="main">` holds the
  five `<section class="view">` blocks; hidden views carry BOTH
  `hidden` and `inert` (WCAG — no focus/keyboard traversal). The
  shared header, the single `doc-modal-*` skeleton, and the
  `#app-version` footer each exist exactly once; the per-view copies
  from the four folded pages are dropped.
- New frontend/assets/router.js (vanilla module — no framework, no
  bundler, No-CDN rule intact): lazy-imports a view module on FIRST
  show only (mount-once, hide-forever — the chat view's in-flight SSE
  reader persists across switches; that persistence IS the fix);
  intercepts same-shell navbar links with preventDefault +
  history.pushState (never a document load); handles popstate; single
  writer of `.nav-link` active state (is-active + aria-current),
  document.title, and the per-view meta description (values carried
  over from the old pages' heads, brand-resolved at write time).
- Each folded page's JS becomes `export async function mount(root)` —
  root-scoped queries; `initSharedHeader()` dropped (the header boots
  once in the shell via the chat module; the admin flag comes from the
  same cached `fetchIsAdmin()` promise — zero extra requests).
- app/main.py: a small list-driven route factory serves the shell for
  /tuning.html, /sources.html, /git-sources.html, /history.html —
  registered AFTER the API routers and BEFORE the static catch-all
  (routes-first). The phase-33 caching middleware applies no-cache +
  `?v=` rewriting unchanged; app/core/caching.py needed NO change
  (the view paths did not change — pinned by the integration tests).
- The four old view .html files are DELETED (one source of truth);
  deep links to the old URLs keep working (the router picks the view
  from the pathname); `/?chat=<id>` is unaffected; the Containerfile
  bundles router.js (inlining the lazy view modules) and drops the
  folded page files.
- app/schemas.py: HistoryTurn.text cap 4000 -> 32000 — the shell
  keeps long saved answers in the chat, and the old cap (stricter than
  the 24_000-char total history budget) 422-rejected any second turn
  in such a chat (found by the phase-42 E2E suite on the shell).

Boundaries: login.html, shared.html, doc-edit.html, document.html
REMAIN separate documents (flow pages, not navbar tabs); a mid-stream
navigation to doc-edit/document.html still cancels per phase 48
(follow-up candidate, out of scope). The SSE API is unchanged. Real
departures still cancel the turn — phase 48 intact (pinned by
tests/e2e/test_stop_generation.py, unchanged, and by the new suite's
real-departure control).

Tests:
- Phase-20 suite REWRITTEN to the new semantics
  (tests/e2e/test_sources_midstream_bug.py): a navbar switch no longer
  cancels — the stream survives the switch and the FULL answer
  settles; the pagehide partial persist REMAINS for real departures
  (the partial's exact shape — first streamed chunk prefix, no done
  metadata — is still pinned there).
- NEW story suite tests/e2e/test_nav_switch_keeps_stream.py (mock
  LLM): the owner repro (send -> RAG mid-stream -> Chat: window
  sentinel survives = same document, FULL answer, exactly one brain
  turn in bor.chat.v1, exactly one settled query_log row, auto-saved
  row matches) + the same mid-stream switch against the other three
  views + the real-departure-still-cancels control + the no-switch
  baseline.
- tests/unit/test_frontend_router.py: source-level pins of the router
  invariants (click interceptor targets ONLY same-shell view paths,
  pushState-only switches, mount-once guard, hidden+inert pair,
  single-writer active state/title); shell-route integration tests
  (each folded path serves the shell with no-cache + `?v=` body; a
  non-view path still 404s); the file-reading unit pins re-pointed at
  the shell (the four view files are gone — the shell is the source
  of truth).

Verification (this commit): full suite green — 1565 unit+integration
tests, app/ coverage 99% (>90% floor); ruff + pyright clean; the
phase's E2E suites green in isolation (house protocol, AGENTS.md rule
9). Owner repro verified in a real browser against the real LLM
(dev server :8010, headful Chromium): "tell me about everquest" ->
RAG mid-stream -> Chat — the answer completed with one brain bubble
and no error banner, `query_log` gained exactly one settled row
(deflected=True: the dev KB holds no EverQuest docs — the settle, not
the topic, is the proof), zero "chat: turn cancelled" lines for that
turn; the control (real navigation to /shared.html mid-stream) still
cancelled (no settled row, the cancel line logged, the partial
persisted on return). Screenshots: .agents/screenshots/76_manual_*.

Phase 76 (76_spa_nav_shell) complete — moved to
.agents/phases/complete/.
2026-09-06 06:31:31 -04:00
ducoterra 7e567bddf3 prepping for SPA conversion
Build and Push Containers / build-and-push-app (push) Successful in 14s
Build and Push Containers / build-and-push-db (push) Successful in 11s
2026-09-05 21:50:56 -04:00
ducoterra 0e4651c779 feat(docs): save the whole chat session as a doc
Build and Push Containers / build-and-push-app (push) Successful in 1m46s
Build and Push Containers / build-and-push-db (push) Successful in 12s
Phase 75 (TODO.md L4): "Save as doc" now drafts a document from the
ENTIRE chat session — every question and answer up to the click, in
order — instead of only the clicked bubble's answer; the existing
doc-edit screen's free-form body editing is how the user edits out
anything they don't want to keep from previous replies (no new UI
surface).

Task 01 (frontend):
- app.js buildSessionTranscript(): walks the bor.chat.v1 conversation
  record in order — a numbered section per user turn ("## N.
  <question, raw>" + blank line + the raw answer text; more answers
  join under the same heading), sections blank-line separated, all
  trailing whitespace collapsed to one final newline. Only the raw
  persisted text travels (m.who + m.text — no thinking blocks, no
  source chips, no tune metadata); a brain record before the first
  user record is skipped; a heading-only section marks a user turn
  whose answer never landed (A6, owner-confirmed 2026-09-08).
- saveAsDoc(btn): the draft body is buildSessionTranscript(); the
  dead single-bubble markdown parameter is dropped (the button's
  appendSaveAsDocButton signature is unchanged — one button per
  bubble). Title/path/double-click guard/hand-off are unchanged
  (defaultDocTitle: the last question, whitespace-collapsed,
  <=120 chars; docs/<slug>.md).
- Unit: the app.js source pins move to the transcript shape (whole
  session, no thinking, no dead parameter).

Task 02 (E2E):
- tests/e2e/test_save_doc_session.py (bare-repo fixture, the
  phase-59 convention — git as source of truth): three DISTINCT
  on-topic turns in one session (turn 1 carries the phase-17
  "think out loud" trigger so its record has a thinking block the
  transcript must exclude) -> save on the LAST bubble -> the
  prefilled body is ## 1./## 2./## 3. in order, byte-exact against
  the deterministic mock, thinking-free -> edit the whole
  section-2 block out of the body -> push -> git show
  bor-docs:<path> equals the EDITED body byte-for-byte (section 2's
  question and answer provably absent; sections 1 and 3 byte-exact;
  the UI's sha prefix is git rev-parse bor-docs). Second test:
  the button on the FIRST bubble still drafts the whole session
  (A6 — the transcript is the session at click time, title stays
  the last question); canceling leaves the branch tip untouched.
- tests/e2e/test_response_to_docs.py: the phase-59 single-turn body
  expectation moves to the transcript shape ("## 1. <question>" +
  the answer's markdown) — the rest of the suite unchanged.

Also lands the phase-74 file moves (00_phase.md /
03_mock_marker_e2e.md -> complete/) and the phase reports — the
house convention of committing .agents/ with the phase.
2026-09-05 16:59:31 -04:00
ducoterra 055c0b5d85 feat(rag): pass chat history with prior thinking to the LLM
Build and Push Containers / build-and-push-app (push) Successful in 1m39s
Build and Push Containers / build-and-push-db (push) Successful in 11s
Phase 74 (TODO.md L4): a follow-up question now reaches the model WITH
the conversation so far — every prior user/brain turn and the prior
thinking blocks on brain turns (preserve-thinking) — while
POST /api/chat stays stateless (A10): the client provides the history
in the request body and the server stores nothing new.

Server (task 01):
- ChatRequest.history: optional list[HistoryTurn] (who: user|brain,
  text, optional thinking) — absent/empty keeps the request
  byte-identical to pre-phase-74 (the two-message [system, user]
  request; the kill-switch semantics are pinned in the integration
  suite).
- app.rag.prompts.history_to_messages: pure mapper — walks the turns
  newest-first against the settings budgets (history_max_turns=40 /
  history_max_chars=24000, BOR_HISTORY_MAX_TURNS /
  BOR_HISTORY_MAX_CHARS); a capped turn is dropped WHOLE (never cut
  mid-answer); the kept window is returned oldest-first; brain turns
  carry their thinking as reasoning_content (A4) only when
  non-empty.
- Both branches feed it: the deflected path splices it between the
  system prompt and the current user message (the phase-71 recovery
  still rebuilds from messages[1:]), the grounded agent receives
  run_agent(..., history=hist); llm.py's message params widen to
  list[dict[str, Any]] (string-only messages stay byte-identical on
  the wire — the SDK passes message dicts through verbatim).
- The per-turn log line (PLAN §9) gains history_msgs=N after
  kb_chars=N.
- Pins: tests/unit/test_history.py (mapper: mapping, reasoning
  gating, both budgets, drop-whole, ordering, empty default),
  tests/unit/test_config.py (the two settings + env overrides),
  tests/unit/test_agent.py (the history splice + the default),
  tests/integration/test_chat_api.py (deflected AND grounded forward
  the history incl. reasoning_content, no-history byte-identity, 422
  pins, the log field).

Client (task 02):
- runTurn — the single funnel for fresh send / phase-49 retry /
  phase-53 stale-regen — sends history = the conversation record
  minus the current question, with thinking only on brain records
  that streamed one (undefined drops the key from the JSON, the
  record's convention); the question is never duplicated into the
  history.

Wire proof (task 03):
- The mock's echo my history marker (HISTORY_TRIGGER) answers with
  the deterministic history echo — history: N prior messages; last
  answer tail: <last 24 chars>; thinking: yes|no — checked BEFORE
  the DEFLECT_MODE branch (like TABLE_TRIGGER), so it fires on both
  turn branches whatever the gate says; the module docstring records
  the user/assistant-only history invariant that keeps every
  existing (tool-result-classified) marker flow unaffected.
- tests/e2e/test_llm_history.py (isolated): a grounded follow-up and
  a deflected follow-up both receive history: 2 prior messages +
  thinking: yes + the byte-exact tail of turn 1's answer (derived
  from the persisted bor.chat.v1 record — the same array the client
  maps into the body); a cold start receives history: 0 prior
  messages / last answer tail: none / thinking: no.
- Regressions green in isolation: chat_rag, chat_history (phase 50),
  agent_document_tools, harness_aligned_tools, stop_generation,
  retry_answer, response_to_docs.
2026-09-05 16:04:40 -04:00
ducoterra a16130c71d fix(chat): keep generating while the tab is hidden
Root cause (task 01): none of C1-C3 - in Chromium 151 (real mode) a
merely-hidden tab neither stops the stream (frames arrive at full rate;
turn completes) nor fires pagehide on tab switch; C1's double-record
path was proven latent via a synthetic pagehide (trigger is
browser-dependent, e.g. Safari) and C2 (the 120s pre-token guard) was
confirmed to fire while hidden.

- C1: the pagehide partial-persist is correlated with the turn's settle
  (leavePartialIndex) - the done/stop settle REPLACES it in place
  (identity-guarded rememberBrainTurn in-place mode), so bor.chat.v1
  and the auto-saved saved_chats row keep exactly ONE brain turn per
  question; a real navigation never runs a settle, so the leave-save
  is unchanged.
- C2: the visibility re-arm gives the still-armed pre-token guard a
  fresh TURN_TIMEOUT_MS when the tab returns to visible - hidden time
  no longer counts toward the 120s guard.
- Phase-48 teardown contract untouched: Stop / tab close / real
  navigation still cancel the fetch and stop the model.
- Unit pins: tests/unit/test_frontend_hidden_tab.py (the app.js
  mechanisms without a browser).
- E2E pins: tests/e2e/test_hidden_tab_stream.py - synthetic pagehide
  mid-stream completes exactly once with one brain turn (localStorage
  + auto-saved row), reload restores one bubble, no-event baseline,
  and the fake-clock pre-token guard re-arm (discriminating: fails
  with the re-arm disabled).
2026-09-05 14:34:33 -04:00
ducoterra 45c3fa2863 chore(agent): phase roadmap from TODO.md (phases 73-75), clear the file
Convert the two unchecked TODO items into executable phases (Protocol B,
appended after the 72 completed phases):

- 73_hidden_tab_stream (TODO L3): a merely-hidden browser tab must never
  stop a generating answer; repro/root-cause decision tree + the pagehide
  partial-correlation fix + the hidden-tab E2E pin.
- 74_llm_chat_history (TODO L4, history): client-provided history in
  POST /api/chat (stateless, A10) mapped through both the deflected and
  grounded agent paths, prior thinking blocks preserved via
  reasoning_content, capped oldest-first; mock echo marker + E2E.
- 75_save_doc_full_session (TODO L4, save-as-doc): the Save-as-doc draft
  body becomes the full session transcript; edit-out happens in the
  existing doc-edit body; multi-turn git-verified E2E.

Owner-confirmed assumptions A1-A7 are recorded as ASSUMPTION lines in the
task files. TODO.md is cleared (items now live in .agents/phases/todo/).
2026-09-05 12:14:21 -04:00
ducoterra dbf2af26c6 refactor(agents): migrate .agent/ planning tree to .agents/
Standardize on the .agents/ directory (shared with project skills):
phases/, user_stories/, reports/, screenshots/, validate.sh, and
phase-sessions/ + pipeline.log all move to .agents/ (git mv preserves
history; runtime artifacts move alongside).

Updates every reference in AGENTS.md, README.md, .gitignore, app
docstrings, and test story headers. Historical KB content in data/
and the runtime pipeline.log transcript are left untouched.
2026-09-05 10:57:07 -04:00
ducoterra bb2803bebd feat(skills): add test-chat-model skill — add a chat model and run the controlled tool-calling battery
Codifies the 2026-09-05 turbo comparison workflow as a project skill under
.agents/skills/: switch BOR_LLM_CHAT_MODEL in .env, run the fixture gate
(twice, for variance) + the locked derived gate with per-turn wall timing,
interpret the two metrics against the reference model rates (re-read habit:
lite ~100%, turbo ~12%; usage-floor MISS as test artifact; caps as real
regression), record the verdicts byte-exact in TOOL_CALLING_TESTING.md, and
commit the doc. Rules baked in: never touch the battery/thresholds/fixtures,
never edit app code, never commit .env.
2026-09-05 00:13:21 -04:00