All completion criteria verified. Everything is green.
**Phase 105 final verification pass — all criteria verified**
- Verified the full implementation in the working tree: `git_sources.include_hidden` column + alembic `0019` (dev DB at head, column present), `iter_importable_files`/`import_sources` flag support with `str(root)`-keyed map used by both walk and progress pre-walk, `GitSourcePatchIn` rename with optional fields, sync/CLI pipeline wiring (OR-collision), and the per-row "Hidden" checkbox + tag + error line on the Sources page
- Unit + integration: `uv run pytest` → exit 0 (2148 tests collected, all pass; this sandbox occasionally swallows pytest's final status line — exit codes verified)
- Coverage: `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (3879 stmts, 15 miss) — >90% gate ✓
- Dedicated E2E: `uv run pytest tests/e2e/test_hidden_folders_toggle.py -v --no-cov` → **6 passed in 23.20s** (DB up, in isolation)
- Regression E2E in isolation: `test_source_ignore_paths` 6 passed, `test_git_sources_admin` 6 passed, `test_local_directory_sources` 3 passed, `test_sync_button` 3 passed, `test_smoke` 3 passed
- Lint/types: `uv run ruff check .` + `uv run pyright` → clean (0 errors/warnings)
**Completion criteria:** (1) checkbox persists via PATCH 200 → "hidden on" tag + GET round-trips `include_hidden: true`; failure path reverts box + `role="alert"` canned message ✓; (2) flag OFF byte-identical (only `visible.md` indexed), ON indexes `.hidden/note.md` into the KB catalog, `EXCLUDED_DIRS` excluded both states ✓; (3) A2: flag OFF → `detail.pruned==1`, doc gone from catalog ✓; (4) PATCH bool-only/list-only/both/neither no-op, phase-89 fixed 422s unchanged, 404, anonymous 403 (incl. bool-only body) ✓; (5) env-fallback rows render no checkbox, WCAG-clean (aria-label, keyboard focus, visible label, text tag) ✓; (6) full gate green ✓; (7) commit left to the harness per instructions (no `git add`/`commit` run; phase files untouched).
**Deviations:** none — no defects found; no code changes were needed on this pass.
**Next pending phase:** `.agents/phases/todo/98_sync_summary_visibility`.
All verification is complete — this pass needed no code changes. Final report:
**Phase 91 — Admin Theme tab: final verification pass (all 6 tasks already in `complete/`)**
- Verified pre-paint theming end-to-end: `ui_settings` store + resolver, admin `GET/PUT /api/ui-settings`, `CachingMiddleware` inline-`<style id="bor-theme">` injection before `</head>` (incl. `/shared/<token>` prefix branch, unit-pinned), CSP sha256 exemption for the inline tag, Theme tab shell + `theme.js` editor, CSS-file theming fully retired.
- No defects found; zero changes made — working tree left exactly as the task executors left it.
- Tests: `uv run pytest --cov=app` → 1841 passed, 0 failed (TOTAL coverage **99%**; theming/ui_settings/caching all 100%); `uv run pytest tests/e2e/test_admin_theme_tab.py -v --no-cov` → **5 passed** in isolation.
- Lint/types: `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings.
- Criteria: (1) unset deployment byte-identical, no `#bor-theme` anywhere — ✓ (unit no-op test + E2E reset byte-compare); `rg "BOR_THEME|themes/"` → single hit is the permitted doc-history comment in `frontend/index.html`. (2) admin-only gate + 403s for anonymous and token users — ✓ (E2E test 3). (3) saved theme inline before `</head>` on every page incl. `/shared/<token>`, computed `--brand` on first paint for admin + anonymous — ✓ (E2E test 2 + unit). (4) reset → byte-identical; 5 contrast pairs warn <4.5:1, non-blocking — ✓ (E2E tests 4–5). (5) suite green, >90% coverage, lint clean — ✓. (6) commit deferred to harness per rules.
- Notable: `.agents/PLAN.md` is absent from the repo — the phase overview's Design section was used as the binding spec; no deviation resulted.
- Next pending phase: **none** — 91 is the last phase in `todo/`.
All verification is complete and green. Final report:
**Phase 90 (upload_no_scan) — final verification pass: all criteria verified, no fixes needed**
- Verified (not re-implemented): all 3 tasks already complete; working tree carries the phase's uncommitted changes
- Upload pipeline: `_run_upload` stops after row upsert; success = `{"message": "uploaded"}`, null/0/0 progress; `UploadOut` removed from `app/schemas.py`; gates/unpack/swap/failed states intact
- Copy: button "Upload", bare "Processing…", result line "Uploaded \<name\> — press Sync sources to import it."; hint + caption re-pointed at Sync; no "Upload &" remnants in `frontend/`
- Tests: `pytest tests/unit/test_git_sources.py tests/integration/test_git_sources_upload.py tests/unit/test_frontend_sync_upload.py` → 56 passed; E2E phase suite `tests/e2e/test_upload_no_scan.py` → 3 passed (zero docs after upload; ignore edit honored by Sync — 2 added, `notes/skipme.md` excluded; re-upload in-place, still zero docs); affected suites `test_archive_upload_sources.py` + `test_sync_upload_progress.py` + `test_source_removal_cleanup.py` → 15 passed
- Full suite: `uv run pytest --cov=app --cov-report=term-missing` → **1808 passed, 99% coverage** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors
**Completion criteria:**
1. Upload settles "ready for sync", zero docs, row + ignore editor — ✅ (E2E test 1)
2. Button "Upload", no "Upload &" copy — ✅ (`rg` empty in frontend/, app/, tests/)
3. RAG-page "Sync sources" imports upload honoring edited ignores — ✅ (isolated E2E test 2)
4. Suite green, >90% coverage, lint clean — ✅ (99%, ruff+pyright clean)
5. Atomic commit — ✅ left to harness per executor rules (no git add/commit run; tree staged-ready)
**Deviations:** `regenerate_overview` still appears once in `git_sources.py` — inside `delete_git_source` (phase-69 locked total-removal behavior, not the upload leg; upload leg is scan-free). `.agents/PLAN.md` does not exist in this repo; locked decisions A1–A4 were enforced from the phase files.
**Next pending phase:** `91_admin_theme_tab` (in `todo/`).
All verification complete — TODO.md was already cleared in the roadmap commit; the two extra unit-test diffs are necessary fake-signature adaptations for the new keywords. Everything is green, no fixes were needed.
## Phase 89 — final verification pass: ALL GREEN
**Verified (all 6 task files present in `complete/`):**
- `git_sources.ignore_paths` JSONB column + migration 0013; `alembic downgrade -1 && upgrade head` round-trips (head `0013`)
- Importer: `normalize_ignore_path`/`is_ignored`/`_ignore_for_root`, `ignore` in walk + progress pre-walk, `ignore_by_root` in `import_sources`
- API: GET/POST carry list; admin-only `PATCH` (replace, 404/422 fixed details, anonymous 403)
- Pipelines wired: `_run_sync`, `_run_upload` re-upload, `scripts/import_docs.py`
- Sources-page box: dialog, §7.4 save lifecycle, `N ignored` tag, a11y; env rows get no box
**Test/lint results:**
- `uv run pytest` → 1808 passed
- `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%** (>90%)
- `uv run pytest tests/e2e/test_source_ignore_paths.py -v --no-cov` → 6 passed (isolated, DB up)
- Regressions in isolation: `test_git_sources_admin` 6, `test_archive_upload_sources` 5, `test_sync_button` 3, `test_smoke` 3 — all passed
- `uv run ruff check . && uv run pyright` → clean (0 errors)
**Completion criteria:** box→PATCH 200→count+GET round-trip ✅ · sync excludes `ignore/` (no docs/chunks/embeddings/summaries) + prunes newly-ignored (pruned==2) ✅ · no-mid-path rule E2E ✅ · PATCH 404/422/replace/clear/403 ✅ · full gate green ✅ · commit + phase move left to harness per rules.
**Deviations:** none blocking — E2E pins `files == 4` (overview's "5" was an off-by-one vs its own 6-file tree, documented in-test); `tests/unit/test_importer.py` + `test_sync_button.py` test-double fakes extended for the new keywords (needed for the suite to stay green).
**Next pending phase:** none — `todo/` holds only this phase.
Single consolidated commit for four completed, validated phases (77, 78,
79, 80). The pipeline run left all work uncommitted because the harness
commits only with PHASE_COMMIT=1 while child executors are forbidden from
committing; the phases themselves all passed validation and moved to
.agents/phases/complete/.
Phase 77 — navbar view refresh
- router.js dispatches bor:view-refresh on re-show / active re-click /
popstate (gated on wasMounted; first show and boot exempt)
- History / RAG / Sources / Tuning re-fetch on refresh (admin branch);
Chat deliberately excluded (stream survival)
- History "Refresh" button (admin-only, in-flight disable + status line)
- New story suite tests/e2e/test_navbar_refresh.py (7 tests)
Phase 78 — static background
- Removed the animated glow layers; static 44px grid over the flat --bg
canvas; default and reduced-motion renders byte-identical
- Updated background/theme E2E suites; removed bg-glow test pins
Phase 79 — API tokens
- api_tokens model + migration 0012; hash-only token service
- Admin tokens API + Tokens admin view; POST /api/token-auth;
live-revoking require_user on chat / suggestions / document content
- Frontend token gate with localStorage cache; anonymous E2E suites
migrated to token login
- New story suite tests/e2e/test_api_tokens.py (9 tests)
Phase 80 — history suggestion chips
- last_questions() endpoint with SEED fallback; startNewChat() refetch
- Seed-semantics docs (config.py, .env.example, README)
- Integration state matrix + E2E suite rewritten to the 4 chip states
Also included: phase-76 report artifacts and the repo restore-test-db
skill (previously untracked), scripts/* ruff fixes from phase 77.
Final gate state (phase 80 final pass, covers everything above):
- uv run pytest --cov=app → 1637 passed, 0 failed, app/ coverage 99%
- uv run ruff check . && uv run pyright → clean, 0 errors
- Per-phase story E2E suites green in isolation
Root cause (owner repro, verified in a real browser 2026-09-06): the
five navbar views (Chat, RAG, Sources, Tuning, History) were separate
HTML documents, so a navbar click was a REAL cross-document navigation
— the chat page unloaded, the in-flight SSE fetch was aborted, and the
phase-48 teardown (app/api/chat.py `finally`, "chat: turn cancelled")
stopped the model. Observed: send question -> click RAG mid-stream ->
click Chat -> the answer never finished: no `query_log` row, and on
return a dangling question with no brain record (the pre-token pagehide
partial persist skips because `acc` is empty).
Phase-48 LOCKED-DECISION REFINEMENT (owner-confirmed 2026-09-06,
flagged per AGENTS.md rule 3, not silently deviated): "real navigation
cancels the fetch" now means LEAVING THE APP — tab close,
external/other-document navigation, the Stop button. In-app navbar
switches are client-side view switches and no longer cancel.
Fix — Option A (SPA shell), chosen over B (Service Worker owns the
stream) and C (server-side turn registry + resume):
- frontend/index.html is the shell: ONE `<main id="main">` holds the
five `<section class="view">` blocks; hidden views carry BOTH
`hidden` and `inert` (WCAG — no focus/keyboard traversal). The
shared header, the single `doc-modal-*` skeleton, and the
`#app-version` footer each exist exactly once; the per-view copies
from the four folded pages are dropped.
- New frontend/assets/router.js (vanilla module — no framework, no
bundler, No-CDN rule intact): lazy-imports a view module on FIRST
show only (mount-once, hide-forever — the chat view's in-flight SSE
reader persists across switches; that persistence IS the fix);
intercepts same-shell navbar links with preventDefault +
history.pushState (never a document load); handles popstate; single
writer of `.nav-link` active state (is-active + aria-current),
document.title, and the per-view meta description (values carried
over from the old pages' heads, brand-resolved at write time).
- Each folded page's JS becomes `export async function mount(root)` —
root-scoped queries; `initSharedHeader()` dropped (the header boots
once in the shell via the chat module; the admin flag comes from the
same cached `fetchIsAdmin()` promise — zero extra requests).
- app/main.py: a small list-driven route factory serves the shell for
/tuning.html, /sources.html, /git-sources.html, /history.html —
registered AFTER the API routers and BEFORE the static catch-all
(routes-first). The phase-33 caching middleware applies no-cache +
`?v=` rewriting unchanged; app/core/caching.py needed NO change
(the view paths did not change — pinned by the integration tests).
- The four old view .html files are DELETED (one source of truth);
deep links to the old URLs keep working (the router picks the view
from the pathname); `/?chat=<id>` is unaffected; the Containerfile
bundles router.js (inlining the lazy view modules) and drops the
folded page files.
- app/schemas.py: HistoryTurn.text cap 4000 -> 32000 — the shell
keeps long saved answers in the chat, and the old cap (stricter than
the 24_000-char total history budget) 422-rejected any second turn
in such a chat (found by the phase-42 E2E suite on the shell).
Boundaries: login.html, shared.html, doc-edit.html, document.html
REMAIN separate documents (flow pages, not navbar tabs); a mid-stream
navigation to doc-edit/document.html still cancels per phase 48
(follow-up candidate, out of scope). The SSE API is unchanged. Real
departures still cancel the turn — phase 48 intact (pinned by
tests/e2e/test_stop_generation.py, unchanged, and by the new suite's
real-departure control).
Tests:
- Phase-20 suite REWRITTEN to the new semantics
(tests/e2e/test_sources_midstream_bug.py): a navbar switch no longer
cancels — the stream survives the switch and the FULL answer
settles; the pagehide partial persist REMAINS for real departures
(the partial's exact shape — first streamed chunk prefix, no done
metadata — is still pinned there).
- NEW story suite tests/e2e/test_nav_switch_keeps_stream.py (mock
LLM): the owner repro (send -> RAG mid-stream -> Chat: window
sentinel survives = same document, FULL answer, exactly one brain
turn in bor.chat.v1, exactly one settled query_log row, auto-saved
row matches) + the same mid-stream switch against the other three
views + the real-departure-still-cancels control + the no-switch
baseline.
- tests/unit/test_frontend_router.py: source-level pins of the router
invariants (click interceptor targets ONLY same-shell view paths,
pushState-only switches, mount-once guard, hidden+inert pair,
single-writer active state/title); shell-route integration tests
(each folded path serves the shell with no-cache + `?v=` body; a
non-view path still 404s); the file-reading unit pins re-pointed at
the shell (the four view files are gone — the shell is the source
of truth).
Verification (this commit): full suite green — 1565 unit+integration
tests, app/ coverage 99% (>90% floor); ruff + pyright clean; the
phase's E2E suites green in isolation (house protocol, AGENTS.md rule
9). Owner repro verified in a real browser against the real LLM
(dev server :8010, headful Chromium): "tell me about everquest" ->
RAG mid-stream -> Chat — the answer completed with one brain bubble
and no error banner, `query_log` gained exactly one settled row
(deflected=True: the dev KB holds no EverQuest docs — the settle, not
the topic, is the proof), zero "chat: turn cancelled" lines for that
turn; the control (real navigation to /shared.html mid-stream) still
cancelled (no settled row, the cancel line logged, the partial
persisted on return). Screenshots: .agents/screenshots/76_manual_*.
Phase 76 (76_spa_nav_shell) complete — moved to
.agents/phases/complete/.
Phase 74 (TODO.md L4): a follow-up question now reaches the model WITH
the conversation so far — every prior user/brain turn and the prior
thinking blocks on brain turns (preserve-thinking) — while
POST /api/chat stays stateless (A10): the client provides the history
in the request body and the server stores nothing new.
Server (task 01):
- ChatRequest.history: optional list[HistoryTurn] (who: user|brain,
text, optional thinking) — absent/empty keeps the request
byte-identical to pre-phase-74 (the two-message [system, user]
request; the kill-switch semantics are pinned in the integration
suite).
- app.rag.prompts.history_to_messages: pure mapper — walks the turns
newest-first against the settings budgets (history_max_turns=40 /
history_max_chars=24000, BOR_HISTORY_MAX_TURNS /
BOR_HISTORY_MAX_CHARS); a capped turn is dropped WHOLE (never cut
mid-answer); the kept window is returned oldest-first; brain turns
carry their thinking as reasoning_content (A4) only when
non-empty.
- Both branches feed it: the deflected path splices it between the
system prompt and the current user message (the phase-71 recovery
still rebuilds from messages[1:]), the grounded agent receives
run_agent(..., history=hist); llm.py's message params widen to
list[dict[str, Any]] (string-only messages stay byte-identical on
the wire — the SDK passes message dicts through verbatim).
- The per-turn log line (PLAN §9) gains history_msgs=N after
kb_chars=N.
- Pins: tests/unit/test_history.py (mapper: mapping, reasoning
gating, both budgets, drop-whole, ordering, empty default),
tests/unit/test_config.py (the two settings + env overrides),
tests/unit/test_agent.py (the history splice + the default),
tests/integration/test_chat_api.py (deflected AND grounded forward
the history incl. reasoning_content, no-history byte-identity, 422
pins, the log field).
Client (task 02):
- runTurn — the single funnel for fresh send / phase-49 retry /
phase-53 stale-regen — sends history = the conversation record
minus the current question, with thinking only on brain records
that streamed one (undefined drops the key from the JSON, the
record's convention); the question is never duplicated into the
history.
Wire proof (task 03):
- The mock's echo my history marker (HISTORY_TRIGGER) answers with
the deterministic history echo — history: N prior messages; last
answer tail: <last 24 chars>; thinking: yes|no — checked BEFORE
the DEFLECT_MODE branch (like TABLE_TRIGGER), so it fires on both
turn branches whatever the gate says; the module docstring records
the user/assistant-only history invariant that keeps every
existing (tool-result-classified) marker flow unaffected.
- tests/e2e/test_llm_history.py (isolated): a grounded follow-up and
a deflected follow-up both receive history: 2 prior messages +
thinking: yes + the byte-exact tail of turn 1's answer (derived
from the persisted bor.chat.v1 record — the same array the client
maps into the body); a cold start receives history: 0 prior
messages / last answer tail: none / thinking: no.
- Regressions green in isolation: chat_rag, chat_history (phase 50),
agent_document_tools, harness_aligned_tools, stop_generation,
retry_answer, response_to_docs.
Standardize on the .agents/ directory (shared with project skills):
phases/, user_stories/, reports/, screenshots/, validate.sh, and
phase-sessions/ + pipeline.log all move to .agents/ (git mv preserves
history; runtime artifacts move alongside).
Updates every reference in AGENTS.md, README.md, .gitignore, app
docstrings, and test story headers. Historical KB content in data/
and the runtime pipeline.log transcript are left untouched.
Phase 72 (72_teaching_refusals) — completed under the 2026-09-04 controlled
methodology (owner directive: stop clearing/re-importing the homelab KB per
iteration; measure tool-calling accuracy on a controlled fixture KB, target
>90%).
Real-model gate verdicts (live, configured chat model 'lite', fixture KB):
- Controlled fixture battery (the new methodology's pass condition —
contract accuracy >= 90%): PASS, 4 consecutive runs:
gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 8/11 executed (73%) contract 11/11 (100%) 2026-09-04 (wall 43.4s)
gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 8/13 executed (62%) contract 12/13 (92%) 2026-09-04 (wall 50.6s)
gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 7/11 executed (64%) contract 11/11 (100%) 2026-09-04 (wall 46.8s)
gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 9/15 executed (60%) contract 14/15 (93%) 2026-09-04 (wall 54.8s)
- Locked derived battery (phase-72 task 05, executed >= 90% bar, run
unchanged on the same fixture KB):
gate: lite FAIL turns=10 answered=10 caps=0 tool-turns=10 calls 5/15 executed (33%) contract 12/15 (80%) 2026-09-04 (wall 47.7s)
The teaching works — every bare-path trap self-corrects in exactly one
round, zero cap hits, zero repeat loops, 10/10 answered. The locked
executed bar is blocked by ALREADY_IN_CONTEXT dedupe refusals on the
corrected re-reads (the trap question seeds its target, so the correct
combined-form read is refused for redundancy) — a copy-invariant model
behavior (five copy variants, 0/15 re-reads flipped, 2026-09-03 -> 04)
and an app-semantics decision for the owner (TOOL_CALLING_TESTING.md
sections 5 and 7), not a copy lever.
Copy changes this phase owns (unit pins updated to follow):
- app/rag/agent.py: ls teaching refusals (path-like scope -> document-path
line; unknown source -> no-source line with the source-name
parenthetical), read/grep 'did you mean source/path?' teaching
(find_path_candidates: exact or suffix path match, catalog order, cap 3),
ALREADY_IN_CONTEXT naming the correct action (answer from the text
already in the prompt), read tool description front-loaded with the
do-not-read rule (the 2026-09-04 controlled telemetry: the re-read is
the only remaining refusal class; contract accuracy 92-100% across runs)
- app/rag/prompts.py: TOOLS_SECTION states the document-identity contract
up front (ls path = source name; read/grep = combined source/path
including the source name; do-not-read for <documents> documents placed
next to the read teaching; one-call-per-reply and never-repeat rules)
- tests: refusal pins (unit + integration), new dedicated E2E suite
tests/e2e/test_tool_path_teaching.py (mock misuse flow, green in
isolation), regression suites green in isolation (harness_aligned_tools,
agent_document_tools, agent_unlimited_tools, search_tool, chat_rag).
Gates: uv run pytest green (1501); coverage TOTAL 99% (>90%); ruff +
pyright clean. Carries the still-uncommitted phase-71 todo/ -> complete/
move and both phases' .agent/reports/ (AGENTS.md 8).
The model treated the combined 'source/path' string (as printed in
search result lines, read-result headers and refusals) as the
document's identity and passed it as 'source' — e.g.
source='homelab/active/container_caddy/caddy.md' instead of
source='homelab', path='active/container_caddy/caddy.md'.
- Rewrite the read_document description with the split rule (source =
before the FIRST '/', path = after it) and a worked example; share
the source/path parameter descriptions between read_document and
search_documents; map search result lines back onto the split.
- New _resolve_document: on a lookup miss with a '/' in source, retry
at the first slash (source names are directory basenames and can
never contain '/'), plus a continuation candidate for a split at a
later slash; a self-corrected combined form for an already-in-context
document is still rejected as ALREADY_IN_CONTEXT.
- A slash-carrying source that matches nothing gets an educational
refusal naming the corrected arguments instead of the generic line
that repeated the combined form.
Verified live against aipi (lite) + the imported homelab KB: A/B on
the exact failure scenario (5 runs each, right after a
combined-source search result) — old descriptions 5/5 combined, new
descriptions 5/5 clean; two live UI turns (Playwright) produced only
clean split arguments, including a multi-hop read of
install_caddy_deskwork.yaml that landed in done.sources. Full suite:
1376 passed, app coverage 99% (agent.py 100%), ruff + pyright clean,
agent/search E2E green in isolation.
Fixed: index.html meta description, empty-state sub and composer
placeholder (A1); app/config.py default suggestion chips → the four
neutral A2 defaults (BOR_SUGGESTIONS override unchanged); sources.html
KB page-sub → the current source model (git repos + local dirs +
uploaded archives, Sync pulls/imports); git-sources.html example URL
→ your-repo.git (A3); all 9 footers → neutral default in
span.footer-text (the phase-62 hook); E2E/unit conftests force the
code defaults so a local .env cannot leak corpus copy into tests;
new unit text pins + dedicated E2E suite.
Task 02 verification read-through — no change needed:
- sources.html sync result/error copy (matches the real sync behavior)
- tuning.html page-sub (accurate as written)
- history.html page-sub (accurate as written)
- doc-edit.html page-sub (accurate as written)
- git-sources.html page-sub (accurate as written)
- #sources-gate anonymous copy (accurate as written)
Phase 49 (owner request, chat 2026-08-28: "The git sources page should
remove local directory and should instead accept a tarball or zipfile
upload which it will unpack and scan … reuploading the same tarball
should not create a new folder, but should unpack and overwrite the
previously unpacked content" — design confirmed in the same
conversation):
* POST /api/git-sources/upload (admin-only, require_admin): accepts
.tar/.tar.gz/.tgz/.zip, streams it with the BOR_UPLOAD_MAX_MB cap
(bounds BOTH the compressed upload and the total extracted bytes —
zip-bomb guard), safely unpacks (absolute/traversal/symlink/hardlink
escape and device/FIFO members rejected), and atomically swaps the
content in over BOR_UPLOAD_DIR/<name>/ (name = filename minus the
archive suffix — no missing window, a failed upload never touches the
existing folder/row/KB). The git_sources row is upserted by path
(kind='local', no duplicates, added_at preserved), the models are
checked fail-fast (503 sanitized when down — the folder/row stay
committed and the next sync/re-upload retries idempotently), and the
source is scanned synchronously in the request (single-source
import_sources prune=True + change-gated KB overview), answering 200
with the sync-style counts. One upload at a time (409); the request
session is released before the scan so a concurrent TRUNCATE cannot
deadlock against it.
* app/rag/archive_upload.py: ArchiveUploadError, ARCHIVE_SUFFIXES,
archive_source_name (safe-name derivation), unpack_archive (guarded
zip/tar extraction with the extracted-byte cap, no partial state),
swap_in (atomic replace with restore-on-failure) — fully unit-tested.
* app/config.py + .env.example: BOR_UPLOAD_DIR (default
~/bor-sources/uploads, deliberately separate from the git checkouts)
and BOR_UPLOAD_MAX_MB (default 512; a validator fails loud at
startup on <= 0).
* python-multipart added to the dependencies — FastAPI's required
multipart parser (an A2 implementation detail, phase locked decision).
* The Sources page: the phase-38 "Add a local directory" form is
removed; #archive-upload-form takes its place (labeled file input,
"Upload & scan" button, the §7.4 never-stale lifecycle, inline
role=alert error, role=status count line); hint + table caption
updated. The POST /api/git-sources kind=local API contract is
UNCHANGED — a plain directory is still registrable via the API, and
existing Local rows list/remove/sync exactly as before.
* The phase-38 story E2E (test_local_directory_sources.py) is rewritten
API-driven — the form it drove is gone; its acceptance stands.
* The story E2E (test_archive_upload_sources.py): the swap,
upload→scan→list (the deterministic "Uploading…" in-flight state, the
Local row, /api/docs + the RAG catalog), same-filename re-upload
(in-place replace, prune, no duplicate row, v2-only folder), the
422 inline error + recovery (the form is not wedged), and the
anonymous gate + 403.
* README: the archive-upload section (formats, naming rule, in-place
replace, both new settings), the local-directory form removal noted,
config reference rows for BOR_UPLOAD_DIR / BOR_UPLOAD_MAX_MB.
Gates: unit+integration green, app/ coverage 99%, the story E2E green
in isolation, the regression suites (git sources admin, local
directory sources, sync button, import documents, nav rename, smoke,
shared header) green in isolation, ruff + pyright clean.
Note: per this phase's file-level staging, frontend/assets/styles.css
also carries the small same-day in-flight owner rework already in the
working tree (the .sign-in-mobile companion rule for the phase-48
mobile sign-in copy); the phase-49 change is the upload form's block.
Phase 47 (owner permission 2026-08-27, TODO.md L10–11, roadmap R1): the
full Podman quadlet family (.container, .network, .volume, .image,
.pod, .kube, .swap, .os, .endpoint) and .j2 Jinja templates join the
allowed + default A9 import formats, chunked as plain text (owner
decision — no TOML/Jinja-aware splitter). No env configuration needed:
a default import now indexes them.
- app/config.py: _ALLOWED_IMPORT_EXTENSIONS + the default
import_extensions CSV gain the ten names (the original seven first);
the never-widen BOR_IMPORT_EXTENSIONS validator is untouched and
still rejects truly unknown extensions.
- app/rag/chunker.py: ten _FORMAT_CHUNKERS entries -> chunk_text
(HARD_MAX_CHARS 1200 honored, unknown-suffix fallback unchanged);
docstring/comments cite the A9 revision 2026-08-27.
- tests/fixtures/docs/homelab/: quadlet/compose.container (realistic
quadlet TOML, >1500 chars, [Unit]/[Service]/[Container] sections,
RESE-QUADLET-SENTINEL-77aa), quadlet/lan.network,
quadlet/cache.volume, templates/deploy.j2 (for/set/if Jinja
constructs + RESE-JINJA-SENTINEL-33dd). Every suite that seeds the
fixture tree updates its 9 -> 13 document-count constants.
- tests/unit/test_config.py: allowed set carries all seventeen formats,
default CSV + dotted import_extension_set include the ten, the
validator accepts the new names and still rejects unknowns.
- tests/unit/test_chunker.py: dispatch parity with chunk_text for every
new suffix (parametrized), the .container fixture chunks >=2 under
the cap with the sentinel surviving, the .j2 fixture keeps {{ }}
verbatim, the unknown-suffix fallback is unchanged.
- tests/unit/test_importer.py: a default-extensions walk over a temp
tree indexes exactly the ten new files (unknown/hidden/excluded
filtered), the original seven still walk, stem-title fallback holds.
- tests/integration/test_import_quadlet_jinja.py (new): import_sources
over a temp tree with .container/.volume/.j2 -> documents + chunks
rows with stem titles; delta re-import updates only the changed .j2
doc; prune drops the deleted .volume doc with cascade.
- tests/e2e/test_quadlet_jinja_import.py (new, story suite, mock-only,
isolation): GET /api/docs (admin session) lists the four new-format
docs with non-zero chunk counts and stem titles; the Sources table
renders a row + .doc-link per file; the phase-26 modal shows the
.container TOML ([Container] section + sentinel) with stem title and
the container format badge; a RESE-JINJA-SENTINEL-33dd question
FTS-matches the .j2 chunk -> honest-positive (A8: LOW requires zero
FTS hits) — the bubble is not .is-deflected and a source chip names
templates/deploy.j2.
- README.md + .env.example: the extended default format set (A9
revised 2026-08-27, plain-text chunking, narrow-only rule intact).
- .agent/PLAN.md: the A9 revision (owner-locked R1) — A9 row status,
the revision note under the anchors table, and the §5 chunking-policy
+ §11 workflow lines. The only PLAN edit this phase.
Gates: uv run pytest 795 passed; app/ coverage TOTAL 99% (>90%);
ruff check + pyright clean; story E2E 4/4 in isolation (DB up);
regression E2E suites test_import_documents (3) / test_sync_button
(3) / test_git_sources_admin (6) green in isolation.
Also records the 47_quadlet_jinja_import task-file moves (01–03)
todo/ -> complete/.
ducoterra