refactor(agents): migrate .agent/ planning tree to .agents/
Standardize on the .agents/ directory (shared with project skills): phases/, user_stories/, reports/, screenshots/, validate.sh, and phase-sessions/ + pipeline.log all move to .agents/ (git mv preserves history; runtime artifacts move alongside). Updates every reference in AGENTS.md, README.md, .gitignore, app docstrings, and test story headers. Historical KB content in data/ and the runtime pipeline.log transcript are left untouched.
This commit is contained in:
@@ -0,0 +1,46 @@
|
||||
# Phase 38 — Local directory sources
|
||||
|
||||
**Source:** `TODO.md` L11 — "Also need a way to import from existing directory if it's not a git repo"
|
||||
**Story:** `.agents/user_stories/local-directory-sources.md`
|
||||
**Context:** `35_git_sources_admin` (todo — runs first: the `git_sources` table, the admin API, the sources page, `effective_git_sources()`), `32_admin_sync_button` (the in-process sync pipeline), `28_git_based_sources` (the `import_docs` resolution order + `clone_or_pull`), `16_admin_auth` (the `require_admin` pattern).
|
||||
|
||||
## Objective
|
||||
Make an **existing local directory** (not a git repo) a first-class source: the admin adds it on the sources page (kind `local` + path), and the Sync button and `import_docs` import it alongside the git checkouts — with fail-loud validation at add-time and at sync-time.
|
||||
|
||||
## Dependencies
|
||||
- `35_git_sources_admin` (todo — runs before this phase) — the table/API/page this phase extends.
|
||||
- `32_admin_sync_button` (complete) — the in-process sync pipeline this phase extends.
|
||||
- `28_git_based_sources` (complete) — `import_docs` source-resolution order + `scripts/git_sync.clone_or_pull`.
|
||||
- `16_admin_auth` (complete) — the `require_admin` router pattern.
|
||||
|
||||
## Tasks
|
||||
1. `01_kind_column.md` — migration 0007: `kind` + `path` columns on `git_sources` (reversible) + model update.
|
||||
2. `02_api_local_kind.md` — the admin API accepts/returns the local kind (path validation → 422, per-kind 409); git contract unchanged.
|
||||
3. `03_sync_import_local.md` — the sync pipeline + `import_docs` resolve DB git + local rows together (local = direct walk; missing dir fails loudly before importing).
|
||||
4. `04_admin_page_local.md` — the page: a second add form for local directories + kind badges on list rows + updated sync hint.
|
||||
5. `05_e2e_docs_commit.md` — the story E2E (add / validate / sync-import / prune-remove), README, commit, move the phase dir.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the kind/path validation + the combined source resolution (git + local, origin logging, both-empty fail-loud).
|
||||
- Integration: migration 0007 up/down; the API local-kind contract (403/201/409/422); the sync pipeline with a temp local dir (the KB actually updated); the existing `test_git_sources_api.py` / `test_sync_api.py` / `test_import_docs_git.py` suites stay green through the indirection.
|
||||
- Coverage: **>90%** on `app/`.
|
||||
- E2E (mandatory, A16): `tests/e2e/test_local_directory_sources.py`, run in isolation.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] Migration 0007 applied; `git_sources` has `kind` (default `git`, check `git`|`local`) + `path` (unique, nullable); existing rows read as `kind='git'`.
|
||||
- [ ] `POST /api/git-sources` with `kind=local` + an existing directory → 201; a missing/relative path → 422 naming the path; a duplicate path → 409; anonymous → 403 on all routes (the phase-35 contract extended).
|
||||
- [ ] `POST /api/sync` with mixed git + local rows imports both in one run (prune over the union); a missing local dir → `failed` status with the path named (sanitized, phase-32 convention).
|
||||
- [ ] `import_docs` (no `--source`) resolves DB git + local rows; `--source` still wins; the env fallback stays git-only; both-empty fails loudly ("no sources configured").
|
||||
- [ ] The page: the Local add form + the Git/Local badges + the updated hint; anonymous still gets the sign-in gate.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` >90%; `uv run pytest tests/e2e/test_local_directory_sources.py -v --no-cov` green in isolation; the task-05 regression list green.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] UI Structure Check (AGENTS.md rule 5) + no CDN (rule 6).
|
||||
- [ ] One `--no-gpg-sign` commit; phase directory moved to `.agents/phases/complete/`.
|
||||
|
||||
## Locked decisions
|
||||
- **The phase-35 table is extended, not duplicated (owner permission 2026-08-26)** — one `git_sources` table with a `kind` discriminator, one admin page, one Sync button; no `local_sources` table, no second page.
|
||||
- **A13 honoured** — the new columns land via a reversible Alembic migration.
|
||||
- **Phase-32 scope boundary kept** — adding a local dir does NOT import immediately; the Sync button performs the import (the page hint says so); removal prunes on the next sync (`prune=True` over the union).
|
||||
- **Local sources are DB-registered only** — no env var for local paths (the DB is the registry; `BOR_GIT_SOURCES` remains the git-only fallback while the table is empty).
|
||||
- **A10 honoured** — the API stays behind the phase-16 `require_admin` pattern; no new session state.
|
||||
- **A16/A17 honoured** — one new story E2E suite + one atomic `--no-gpg-sign` commit.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task 05 — Story E2E + docs + commit
|
||||
|
||||
**Phase:** `38_local_directory_sources` · **Source:** `TODO.md:11 — "Also need a way to import from existing directory if it's not a git repo"`
|
||||
**Story:** `.agents/user_stories/local-directory-sources.md`
|
||||
|
||||
## Objective
|
||||
The story's isolated Playwright suite (add → validate → sync-import → prune-remove), the README update, and the phase commit.
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_local_directory_sources.py` (the story gate; the DB prerequisite per the conftest):
|
||||
- Fixture: a host temp dir (e.g. `tmp_path` — the app server runs on the same host, so the path is visible to it) containing one fixture `.md` with distinctive tokens.
|
||||
- As admin (the `auth_helpers.py` form-login): open the sources page → **Local directory** form → add the temp dir → the row appears with the Local badge; add a missing path (`/nonexistent/bor-e2e`) → the inline error names it, no row added; add the same temp dir again → the 409 duplicate error.
|
||||
- Sync: click the Sync button (the phase-32 pattern) → poll `/api/sync/status` until success → the fixture doc appears in `GET /api/docs`; delete the fixture file from the temp dir, sync again → the doc is pruned (union prune); then Remove the row on the page → the row disappears.
|
||||
- Anonymous: the page soft-gates and the API 403s (the phase-35 assertions, regression).
|
||||
- Keep the file self-contained (one story, one file, isolated run — A16).
|
||||
2. README: the "Sources" section — the two kinds (git = clone/pull mirror; local = direct import of an existing directory), add-time validation, union pruning; note that the DB is the local-source registry (no env var for local paths).
|
||||
3. `.env.example` — no new variable; extend phase 35's `BOR_GIT_SOURCES` note if needed ("env fallback is git-only — local directories are registered on the admin page").
|
||||
4. Regression pass: `uv run pytest` + the coverage gate (>90%) + the isolated story E2E + `tests/e2e/test_git_sources_admin.py` (phase 35's suite — its page assertions must survive the new form; if a selector collided, scope the test to the git form and note it in the commit message) + `tests/e2e/test_sync_button.py`.
|
||||
5. Commit — one atomic `--no-gpg-sign` Conventional Commits commit for the whole phase (AGENTS.md rule 8), e.g. `feat(admin): local directory sources — kind/path on git_sources, combined sync + import, page form + badges`; move the phase directory to `.agents/phases/complete/`.
|
||||
|
||||
## Testing & Quality
|
||||
- The gates above are this task's quality bar (A16: one story, one isolated E2E file, coverage >90%).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] The story E2E is green in isolation, deterministic across two consecutive runs.
|
||||
- [ ] The step-4 regression list green; coverage >90%.
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase directory moved to `.agents/phases/complete/`.
|
||||
Reference in New Issue
Block a user