refactor(agents): migrate .agent/ planning tree to .agents/

Standardize on the .agents/ directory (shared with project skills): phases/, user_stories/, reports/, screenshots/, validate.sh, and phase-sessions/ + pipeline.log all move to .agents/ (git mv preserves history; runtime artifacts move alongside). Updates every reference in AGENTS.md, README.md, .gitignore, app docstrings, and test story headers. Historical KB content in data/ and the runtime pipeline.log transcript are left untouched.
2026-09-05 10:57:07 -04:00
parent 766702c750
commit dbf2af26c6
1118 changed files with 664 additions and 664 deletions
@@ -0,0 +1,10 @@
+All verification complete. Final report:
+
+**Phase 28 — Git-Based Sources (final verification pass) — GREEN**
+
+- **Verified (all tasks 01–04 already implemented):** `BOR_GIT_SOURCES`/`BOR_SOURCES_DIR` settings + `git_source_list` parser; `scripts/git_sync.py::clone_or_pull` (shallow clone / `--ff-only` pull, `GitSyncError` with stderr); `import_docs` source resolution (`--source` > git > legacy defaults, fail-before-import); integration tests; `.env.example` + README (git-sources section, env table, stale "new tab"→modal bullets); story file exists.
+- **Live end-to-end check (real git, real aipi, real DB):** first run cloned `file://` repo into `BOR_SOURCES_DIR/src/` + indexed (added=1); 2nd run pulled the pushed revision + re-indexed (updated=1); `--source` override imported manually with no clone; invalid URL → exit 1, repo named, zero import. Test docs removed from DB afterwards.
+- `uv run pytest` → **390 passed**; `uv run pytest --cov=app` → **TOTAL 99%** (≥ pre-change: only `app/` delta is 100%-covered config lines); `uv run ruff check . && uv run pyright` → **clean**; `tests/e2e/test_smoke.py --no-cov` → **3 passed** (no E2E required for this phase).
+- **Commit:** `3d044f3` `feat(rag): git-based import sources — …` `--no-gpg-sign`, 12 files — phase-28 files only (incl. `-f` complete/28 task files, per repo convention). Phase dir move left to the harness.
+- **Deviations:** (1) concurrent in-tree Phase 29 edits (`frontend/index.html`, `sources.html` tuning-nav links) + owner's new `TODO.md` items appeared mid-session — left uncommitted, not phase-28 work; (2) PLAN §11 still shows the old `~/Homelab` default — phase said record there, but executor rules forbid editing `.agents/PLAN.md`; flagged.
+- **Next pending phase:** `29_tuning_nav_link` (executor appears already active on it).
@@ -0,0 +1,47 @@
+........................................................................ [ 18%]
+........................................................................ [ 36%]
+........................................................................ [ 55%]
+........................................................................ [ 73%]
+........................................................................ [ 92%]
+..............................                                           [100%]
+=============================== warnings summary ===============================
+.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.13.13-final-0 _______________
+
+Name                     Stmts   Miss  Cover
+--------------------------------------------
+app/__init__.py              1      0   100%
+app/api/__init__.py          0      0   100%
+app/api/auth.py             22      0   100%
+app/api/chat.py             95      0   100%
+app/api/docs.py             23      0   100%
+app/api/health.py           10      0   100%
+app/api/steering.py         42      0   100%
+app/api/suggestions.py       8      0   100%
+app/config.py               60      0   100%
+app/core/__init__.py         0      0   100%
+app/core/auth.py            20      0   100%
+app/core/debugging.py       29      2    93%
+app/core/logging.py         13      0   100%
+app/db.py                   21      0   100%
+app/main.py                 36      0   100%
+app/models.py               46      0   100%
+app/rag/__init__.py          0      0   100%
+app/rag/chunker.py         206      4    98%
+app/rag/importer.py        150      5    97%
+app/rag/llm.py             105      0   100%
+app/rag/prompts.py          44      0   100%
+app/rag/retriever.py        93      3    97%
+app/rag/suggestions.py      27      0   100%
+app/schemas.py              68      0   100%
+--------------------------------------------
+TOTAL                     1119     14    99%
+coverage gate: app/ 99% (>90%) OK
+All checks passed!
+0 errors, 0 warnings, 0 informations
+validation OK
@@ -0,0 +1,17 @@
+Task 01 complete. Report:
+
+**Implemented (phase 28, task 01 — `BOR_GIT_SOURCES` + `BOR_SOURCES_DIR` settings)**
+- `app/config.py` (Import scope section): `git_sources: str = ""` (raw comma-separated URLs), `sources_dir: str = "~/bor-sources"` (raw string, expansion in the script), and `git_source_list` property returning stripped non-empty URLs (`[]` when unset/blank).
+- `tests/unit/test_config.py`: 4 new tests — defaults (`""` / `[]` / `~/bor-sources` raw), env override CSV parsing (whitespace trimmed, empties dropped, raw string preserved), whitespace-only value → `[]`, `BOR_SOURCES_DIR` override.
+- `.env.example`: "Import sources (git; phase 28)" block documenting both vars with the task's example values.
+
+**Test / lint / coverage**
+- `uv run pytest tests/unit/test_config.py -v --no-cov` → 15 passed
+- `uv run pytest --cov=app --cov-report=term-missing` → 370 passed, TOTAL **99%** (≥ pre-change, >90% gate)
+- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
+
+**Decisions / deviations**
+- No commit yet: AGENTS.md rule 8 = one atomic commit per *completed phase*; phase 28 is 1/4 tasks done (commit belongs to task 04 / harness handoff). Pre-existing `TODO.md` working-tree modification left untouched.
+- README rewrite left to task 04 per the task file's own Work list (phase overview's mention deferred accordingly).
+
+**Next pending task:** `.agents/phases/todo/28_git_based_sources/02_git_clone_pull_utility.md`
@@ -0,0 +1,47 @@
+........................................................................ [ 19%]
+........................................................................ [ 38%]
+........................................................................ [ 58%]
+........................................................................ [ 77%]
+........................................................................ [ 97%]
+..........                                                               [100%]
+=============================== warnings summary ===============================
+.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.13.13-final-0 _______________
+
+Name                     Stmts   Miss  Cover
+--------------------------------------------
+app/__init__.py              1      0   100%
+app/api/__init__.py          0      0   100%
+app/api/auth.py             22      0   100%
+app/api/chat.py             95      0   100%
+app/api/docs.py             23      0   100%
+app/api/health.py           10      0   100%
+app/api/steering.py         42      0   100%
+app/api/suggestions.py       8      0   100%
+app/config.py               60      0   100%
+app/core/__init__.py         0      0   100%
+app/core/auth.py            20      0   100%
+app/core/debugging.py       29      2    93%
+app/core/logging.py         13      0   100%
+app/db.py                   21      0   100%
+app/main.py                 36      0   100%
+app/models.py               46      0   100%
+app/rag/__init__.py          0      0   100%
+app/rag/chunker.py         206      4    98%
+app/rag/importer.py        150      5    97%
+app/rag/llm.py             105      0   100%
+app/rag/prompts.py          44      0   100%
+app/rag/retriever.py        93      3    97%
+app/rag/suggestions.py      27      0   100%
+app/schemas.py              68      0   100%
+--------------------------------------------
+TOTAL                     1119     14    99%
+coverage gate: app/ 99% (>90%) OK
+All checks passed!
+0 errors, 0 warnings, 0 informations
+validation OK
@@ -0,0 +1,16 @@
+All gates green. Task 02 is complete.
+
+**Implemented**
+- `scripts/git_sync.py` — `clone_or_pull(url, dest)`: shallow `git clone --depth 1` when dest has no `.git` (creates parent dirs first), `git pull --ff-only` when it does; `GitSyncError` carrying git stderr on failure; "git was not found on PATH" when the binary is missing. Stdlib `subprocess` only (A11); auth delegated to URL/SSH config, documented in the module docstring.
+- `tests/unit/test_git_sync.py` — 7 unit tests with mocked `subprocess.run`: clone vs pull dispatch, exact argv + cwd, parent-dir creation, existing-dir-without-`.git` → clone path, non-zero exit (clone + pull) errors with stderr, missing-git error, stdout capture.
+- Verified against a real local repo end-to-end (clone → new upstream commit → ff-only pull → invalid URL fails loudly).
+
+**Test / lint / coverage**
+- `uv run pytest --cov=app --cov-report=term` → 377 passed, TOTAL 99% (>90% gate; unchanged from pre-task)
+- `uv run ruff check .` → clean; `uv run pyright` → 0 errors
+
+**Decisions / deviations**
+- None; followed the task spec's design verbatim (shallow clone, ff-only policy, single git-invocation module).
+
+**Next pending task**
+- `.agents/phases/todo/28_git_based_sources/03_import_docs_script_refactor.md` — wire `BOR_GIT_SOURCES` → `BOR_SOURCES_DIR` resolution into `scripts/import_docs.py`.
@@ -0,0 +1,47 @@
+........................................................................ [ 19%]
+........................................................................ [ 38%]
+........................................................................ [ 57%]
+........................................................................ [ 76%]
+........................................................................ [ 95%]
+.................                                                        [100%]
+=============================== warnings summary ===============================
+.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.13.13-final-0 _______________
+
+Name                     Stmts   Miss  Cover
+--------------------------------------------
+app/__init__.py              1      0   100%
+app/api/__init__.py          0      0   100%
+app/api/auth.py             22      0   100%
+app/api/chat.py             95      0   100%
+app/api/docs.py             23      0   100%
+app/api/health.py           10      0   100%
+app/api/steering.py         42      0   100%
+app/api/suggestions.py       8      0   100%
+app/config.py               60      0   100%
+app/core/__init__.py         0      0   100%
+app/core/auth.py            20      0   100%
+app/core/debugging.py       29      2    93%
+app/core/logging.py         13      0   100%
+app/db.py                   21      0   100%
+app/main.py                 36      0   100%
+app/models.py               46      0   100%
+app/rag/__init__.py          0      0   100%
+app/rag/chunker.py         206      4    98%
+app/rag/importer.py        150      5    97%
+app/rag/llm.py             105      0   100%
+app/rag/prompts.py          44      0   100%
+app/rag/retriever.py        93      3    97%
+app/rag/suggestions.py      27      0   100%
+app/schemas.py              68      0   100%
+--------------------------------------------
+TOTAL                     1119     14    99%
+coverage gate: app/ 99% (>90%) OK
+All checks passed!
+0 errors, 0 warnings, 0 informations
+validation OK
@@ -0,0 +1,17 @@
+All green. Final report:
+
+## Task 03 complete — `import_docs` resolves git sources → local dirs
+
+**Implemented**
+- `scripts/import_docs.py` rewired: new `_resolve_sources()` (precedence: `--source` > `BOR_GIT_SOURCES` > legacy `DEFAULT_SOURCES`) and `repo_name()` (strips `.git`, basename after last `/` or `:`, slug fallback); git sync runs before any import, `GitSyncError` → `import_docs: git sync failed: …` on stderr + exit 1 (no partial import); new INFO log line lists the dirs being imported; `--prune`/`--limit`/summary unchanged.
+- `tests/integration/test_import_docs_git.py` (new, 13 tests): fake `clone_or_pull`/`import_sources` drive `main()` end-to-end — git clone→import, `--source` wins with zero git calls, `DEFAULT_SOURCES` fallback, failing sync → rc 1 + repo named + zero imports, plus `repo_name` cases.
+
+**Verification**
+- `uv run pytest --cov=app --cov-report=term-missing` → 390 passed, TOTAL **99%** on `app/` (≥ pre-change)
+- `uv run ruff check . && uv run pyright` → clean (0 errors)
+- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → 3 passed
+- Real-world check: `file://` bare repo → first run cloned + indexed into `BOR_SOURCES_DIR/bor-test/` (source=`bor-test`), second run pulled with `unchanged=1`; test rows cleaned from DB.
+
+**Decisions** — followed task spec exactly; `get_settings()`/`clone_or_pull`/`import_sources` monkeypatched in tests (no env/DB coupling). No commit made (phase 01/02 changes still uncommitted; commit belongs to the phase's final task per the established pattern).
+
+**Next pending task:** `.agents/phases/todo/28_git_based_sources/04_integration_test_and_docs.md`
@@ -0,0 +1,47 @@
+........................................................................ [ 18%]
+........................................................................ [ 36%]
+........................................................................ [ 55%]
+........................................................................ [ 73%]
+........................................................................ [ 92%]
+..............................                                           [100%]
+=============================== warnings summary ===============================
+.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.13.13-final-0 _______________
+
+Name                     Stmts   Miss  Cover
+--------------------------------------------
+app/__init__.py              1      0   100%
+app/api/__init__.py          0      0   100%
+app/api/auth.py             22      0   100%
+app/api/chat.py             95      0   100%
+app/api/docs.py             23      0   100%
+app/api/health.py           10      0   100%
+app/api/steering.py         42      0   100%
+app/api/suggestions.py       8      0   100%
+app/config.py               60      0   100%
+app/core/__init__.py         0      0   100%
+app/core/auth.py            20      0   100%
+app/core/debugging.py       29      2    93%
+app/core/logging.py         13      0   100%
+app/db.py                   21      0   100%
+app/main.py                 36      0   100%
+app/models.py               46      0   100%
+app/rag/__init__.py          0      0   100%
+app/rag/chunker.py         206      4    98%
+app/rag/importer.py        150      5    97%
+app/rag/llm.py             105      0   100%
+app/rag/prompts.py          44      0   100%
+app/rag/retriever.py        93      3    97%
+app/rag/suggestions.py      27      0   100%
+app/schemas.py              68      0   100%
+--------------------------------------------
+TOTAL                     1119     14    99%
+coverage gate: app/ 99% (>90%) OK
+All checks passed!
+0 errors, 0 warnings, 0 informations
+validation OK
@@ -0,0 +1,16 @@
+Task 04 complete. Report:
+
+**Implemented**
+- `tests/integration/test_import_docs_git.py`: verified it covers all task 04 cases (mocked `clone_or_pull` materializing a fixture `.md` in `BOR_SOURCES_DIR`, `main([])` with `BOR_GIT_SOURCES`, `GitSyncError` → rc 1 naming the repo with zero imports, `--source` override never touching git); added the missing "summary print includes added > 0" assertion (`added=1` via capsys)
+- README: §5 now documents both source modes (git via `BOR_GIT_SOURCES` / manual `--source`) with `~/Homelab + ~/Deployments` kept as the *previous* fallback default; new "Git-based sources" subsection (clone first run / `--ff-only` pull, `--source` overrides, failed sync aborts before import); stale "opens in a new tab" bullets (chip + Sources table) updated to "almost-fullscreen modal" with `/document.html` as full-page/direct-link form; config reference gained `BOR_GIT_SOURCES` + `BOR_SOURCES_DIR` rows
+- `.env.example` comment block verified present/correct (task 01); `.agents/user_stories/git-sources.md` verified complete; fixed stale example comment in `import_docs` docstring
+
+**Results**
+- `uv run pytest tests/integration/test_import_docs_git.py -v --no-cov` → 13 passed
+- `uv run pytest --cov=app --cov-report=term-missing` → 390 passed, TOTAL 99% (gate >90% met)
+- `uv run ruff check . && uv run pyright` → clean (fixed 2 E501 from my docstring edit)
+- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → 3 passed (smoke stays green)
+
+**Deviations:** none — the integration test file pre-existed from task 03's work; I completed rather than rewrote it. No commit made (harness manages phase finalization).
+
+**Next pending task:** phase `29_tuning_nav_link` (task 04 is the last of phase 28).
@@ -0,0 +1,47 @@
+........................................................................ [ 18%]
+........................................................................ [ 36%]
+........................................................................ [ 55%]
+........................................................................ [ 73%]
+........................................................................ [ 92%]
+..............................                                           [100%]
+=============================== warnings summary ===============================
+.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
+  /var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
+    from starlette.testclient import TestClient as TestClient  # noqa
+
+-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
+================================ tests coverage ================================
+_______________ coverage: platform linux, python 3.13.13-final-0 _______________
+
+Name                     Stmts   Miss  Cover
+--------------------------------------------
+app/__init__.py              1      0   100%
+app/api/__init__.py          0      0   100%
+app/api/auth.py             22      0   100%
+app/api/chat.py             95      0   100%
+app/api/docs.py             23      0   100%
+app/api/health.py           10      0   100%
+app/api/steering.py         42      0   100%
+app/api/suggestions.py       8      0   100%
+app/config.py               60      0   100%
+app/core/__init__.py         0      0   100%
+app/core/auth.py            20      0   100%
+app/core/debugging.py       29      2    93%
+app/core/logging.py         13      0   100%
+app/db.py                   21      0   100%
+app/main.py                 36      0   100%
+app/models.py               46      0   100%
+app/rag/__init__.py          0      0   100%
+app/rag/chunker.py         206      4    98%
+app/rag/importer.py        150      5    97%
+app/rag/llm.py             105      0   100%
+app/rag/prompts.py          44      0   100%
+app/rag/retriever.py        93      3    97%
+app/rag/suggestions.py      27      0   100%
+app/schemas.py              68      0   100%
+--------------------------------------------
+TOTAL                     1119     14    99%
+coverage gate: app/ 99% (>90%) OK
+All checks passed!
+0 errors, 0 warnings, 0 informations
+validation OK