phase: 135_git_source_branch
All verification complete. No edits were needed by this pass — the phase work and docs were already consistent and green. **Phase 135 final verification pass — report** - Verified all 5 tasks' work present & correct: migration 0024 (reversible), `GitSource.branch` + 4 schema shapes, `validate_branch_name` (full check order per design), `clone_or_pull(…, branch)` (HEAD guard first, `--branch` clone, fetch+ff merge), both call sites pass `row.branch`, UI (add-form field, editor field, row tag), `file://` URL transport. - Completion criteria: (1) branch-mirrored through Sync + CLI — **pass** (E2E + integration); (2) `branch=None` byte-identical — **pass** (`test_branch_none_argv_is_byte_identical_pre_phase_sequence` green); (3) fixed 422s / local-row 422 / tri-state — **pass**; (4) mismatch fails loudly with remove-and-re-add — **pass** (E2E); (5) coverage 99% >90%, E2E isolated green — **pass**; (6) no regressions — **pass**. - Tests: `uv run pytest --cov=app --cov-report=term-missing` → 3223 passed, app/ 99%; `uv run pytest tests/e2e/test_git_source_branch.py -v --no-cov` → 4 passed (16.4 s); `uv run ruff check .` clean; `uv run pyright` → 0 errors, 0 warnings. - Documentation: README.md reviewed in full — phase-135 branch section accurate; the phase's README diff also correctly fixed a pre-existing stale line (shallow `--depth 1` clone vs full-history, phase 107). Routes, env defaults, feature lists verified against code. AGENTS.md — all 10 rules + quick-reference verified (referenced tests, PLAN.md, phases dirs, commands all accurate). **Verified consistent — no changes needed.** - No live-infrastructure changes (dev DB already running and healthy; no live steps in this phase). - Next pending phase: `.agents/phases/todo/136_force_push_reset`.
This commit is contained in:
@@ -1,40 +0,0 @@
|
||||
# Phase 135 — A git source can pin the branch it mirrors
|
||||
|
||||
**Source:** `TODO.md` L4 — "Need to be able to specify the branch for a git source"
|
||||
**Story:** n/a (TODO-derived phase, Protocol B append).
|
||||
**Context:** `app/models.py::GitSource` (id, url, kind, path, ignore_paths, include_hidden, token, added_at — no branch column; the phase-121 `token` column is the nullable-write-only precedent); `alembic/versions/` (0023 is the newest; every migration is reversible — the house rule); `app/schemas.py` — `GitSourceIn` / `GitSourceOut` / `GitSourceRow` / `GitSourcePatchIn` (the token field is the write-trimmed precedent; the output shapes use `extra="forbid"`); `app/api/git_sources.py` — `POST` (the `_create_git_row` / `_create_local_row` split, the fixed-detail 422s, the token normalization), `PATCH /{source_id}` (the per-field present-wins tri-state — the token's absent/None = no change, `""` = clear, non-empty = replace), `GET` (DB rows vs the env-fallback synthetic rows); `scripts/git_sync.py::clone_or_pull(url, dest)` (fresh clone with full history; the shallow probe + one-time `git fetch --unshallow` self-heal (phase 107 D11); `git pull --ff-only` — the fail-loud D12 contract); the TWO call sites that resolve effective sources and clone git rows: `app/api/sync.py::_run_sync` and `scripts/import_docs.py::_resolve_sources` (both already pass `clone_url_for(row)` + `repo_name(row.url)`); `file_commit_dates` (walks the checkout — untouched); the UI: `frontend/index.html` — the add form `#git-source-form` (the url + token inputs) and the per-row editor dialog `#ignore-editor-dialog` (the source line, the ignore textarea, the token field); `frontend/assets/git-sources.js` — `loadSources` / `makeRow` (the "hidden on" text-tag idiom in the source cell), `wireAddForm` (`opts.body(value, token)` builds the POST body), the dialog open/save handlers (the PATCH body builder), the `toggleHidden` never-stale lifecycle as the per-row control precedent.
|
||||
|
||||
## Objective
|
||||
A git source can specify the branch it mirrors: the owner gives a branch when adding a git source (or edits it per-row later), the clone/pull honors it (a fresh clone checks the branch out; an existing checkout tracks it), BOTH entry points — the Sync button and the `import_docs` CLI — honor it, and every row without a branch (all existing rows + the env-fallback rows) behaves byte-identically to today (default branch, `git pull --ff-only`).
|
||||
|
||||
## Dependencies
|
||||
- `134_sync_cancel` (todo) — pipeline order only; no code dependency (independent surfaces).
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
- **The column** (ASSUMPTION B1): `git_sources.branch` — `TEXT NULL`, server default NULL; alembic **0024** (reversible — `add_column` / `drop_column`, no data migration: NULL = "the repo's default branch", so every existing row is unchanged by definition). `GitSource.branch: Mapped[str | None] = mapped_column(Text, nullable=True)`.
|
||||
- **Validation** (ASSUMPTION B2): a pure helper `validate_branch_name(raw: str) -> str` in `app/rag/git_sources.py` (next to `normalize_credential` — the write-path helper home; the API layer maps its `ValueError` to the fixed-detail 422). Contract: trim → reject empty; reject length > 200; reject anything not matching `^[A-Za-z0-9._/-]+$` (alphanumerics + dot, underscore, hyphen, slash — whitespace/`@`/`~`/`^`/`:`/`?`/`*`/`[`/backslash are all out); reject a name containing `..` or `//`; reject a leading `-`, `.`, or `/`; reject a trailing `/` or `.`; reject `HEAD` (case-insensitive — git's refname; a branch literally named `head` is a documented cost of the simple rule). Returns the trimmed value. The API's 422 detail is the FIXED string `not a valid branch name` (never echoes the input — the house credential-safety discipline, applied for consistency).
|
||||
- **The write paths** (ASSUMPTION B3): `GitSourceIn.branch: str | None` (optional at create — absent/empty → NULL; trimmed in the schema before the length constraint, the `_trim_url` precedent). `GitSourcePatchIn.branch: str | None = None` — TRI-STATE exactly like the token: absent/None = no change; `""` = clear (stored NULL); non-empty = set (re-validated). A `branch` on a LOCAL row (POST or PATCH) → 422 fixed detail `branch is only valid for git sources` (local rows are walked, not cloned). `GitSourceOut.branch: str | None` + `GitSourceRow.branch: str | None` (env-fallback synthetic rows report `None` — no DB row to store one on, the `ignore_paths: []` precedent).
|
||||
- **`clone_or_pull(url, dest, branch: str | None = None)`** (ASSUMPTION B4): `branch=None` → the EXISTING code path byte-identical (same argv: clone without `--branch`; probe; unshallow-if-shallow; `git pull --ff-only`). A fresh clone with a branch → `git clone --branch <b> url dest`. An existing checkout with a branch → (1) the HEAD GUARD first: `git rev-parse --abbrev-ref HEAD` — if the result is not `<b>` → `GitSyncError("checkout <dest> is on branch '<current>' but the source specifies branch '<b>' — remove and re-add the source to switch branches")` (ASSUMPTION B5 — fail loud, the D12 convention; NO silent checkout switch in this phase: the checkout may serve other rows' documents, and a switch would silently re-map the source's file set; phase 136's explicit-ref reset makes the ref unambiguous later without changing this guard); (2) the shallow probe + one-time unshallow (unchanged); (3) `git fetch origin <b>`; (4) `git merge --ff-only FETCH_HEAD` (the single-fetch ff pull — the sequence phase 136 extends with its reset fallback).
|
||||
- **Wiring:** both call sites pass `row.branch` — `app/api/sync.py::_run_sync` and `scripts/import_docs.py::_resolve_sources`: `clone_or_pull(clone_url_for(row), sources_root / repo_name(row.url), row.branch)`. Env-fallback synthetic rows carry `branch=None` (the `GitSource(url=…, kind="git")` constructor default). `file_commit_dates`, the checkout naming (`repo_name` stays on the BARE url), and the ignore/hidden/date maps are untouched.
|
||||
- **The UI** (ASSUMPTION B6): the add form gains an optional `#git-source-branch` text input (maxlength 200, `autocomplete="off"`, hint "optional — default: the repo's default branch") between the URL and Token fields; the submit body becomes `(url, token, branch) => ({ url, ...(branch ? { branch } : {}), ...(token ? { token } : {}) })` — a blank branch OMITS the key (NULL). The per-row editor dialog gains a `#ignore-editor-branch` field (visible for GIT rows only — local rows' dialogs hide it; it opens prefilled from `s.branch ?? ""`; the PATCH body for a git row ALWAYS includes `branch: <trimmed value>` — empty = the tri-state's clear leg, so clearing a branch is a deliberate empty save, unlike the token's blank = keep). Every STORED git row with a branch shows a `branch: <name>` text tag in the source cell (the "hidden on" tag idiom — text + background, WCAG: the text IS the signal, never color alone).
|
||||
|
||||
## Tasks
|
||||
1. `01_branch_column.md` — the alembic 0024 migration, the `GitSource.branch` column, the schema fields, the migration round-trip test.
|
||||
2. `02_branch_api.md` — `validate_branch_name` + POST/PATCH/GET wiring (the tri-state, the local-row 422, the env rows).
|
||||
3. `03_clone_pull_branch.md` — `clone_or_pull(…, branch)` (the `--branch` clone, the HEAD guard, the fetch+ff sequence) + both call sites.
|
||||
4. `04_branch_ui.md` — the add-form field, the per-row editor field, the row tag.
|
||||
5. `05_e2e_git_source_branch.md` — the dedicated Playwright E2E (the two-branch `file://` fixture, the UI add-with-branch → sync → branch content, the fail-loud mismatch, the API contracts).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `validate_branch_name` (the full accept/reject matrix); `clone_or_pull`'s argv sequences with mocked `subprocess` (the `tests/unit/test_git_sync.py` pattern — fresh-clone `--branch`, the existing-checkout guard → probe → fetch → merge, the mismatch raise, `branch=None` byte-identical).
|
||||
- Integration: the migration round-trip (0024); the API round-trips (create/patch/list, the tri-state, the 422s); a real `file://` two-branch clone through `clone_or_pull` (the `tests/integration/test_import_docs_git.py` pattern) + the sync pipeline importing the branch's content.
|
||||
- E2E: one dedicated Playwright file, run in isolation.
|
||||
- Coverage: **>90%** on new/modified code (`app/`).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] a git source added with a branch mirrors THAT branch (fresh clone + subsequent pulls) through both the Sync button and the CLI — pinned by the E2E + the integration test
|
||||
- [ ] a row with no branch (all existing rows + env rows) behaves byte-identically to pre-phase (the `branch=None` argv pin is green)
|
||||
- [ ] invalid branch names are rejected with the fixed 422 (never echoing input); a branch on a local row is a 422; the PATCH tri-state (set/change/clear) round-trips
|
||||
- [ ] setting a branch on an existing checkout on a different branch fails the sync LOUDLY with the remove-and-re-add message (no silent switch)
|
||||
- [ ] `uv run pytest --cov=app --cov-report=term-missing` green, `app/` coverage >90%; `uv run pytest tests/e2e/test_git_source_branch.py -v --no-cov` green in isolation
|
||||
- [ ] no behavior change in completed phases (the token/ignore/hidden round-trips, the sync pipeline, the CLI — all existing tests green untouched)
|
||||
@@ -1,38 +0,0 @@
|
||||
# Task 01 — The `git_sources.branch` column: migration 0024, the model field, the schema fields
|
||||
|
||||
**Phase:** `135_git_source_branch` · **Source:** `TODO.md:4` — "Need to be able to specify the branch for a git source"
|
||||
|
||||
## Objective
|
||||
The registry can store a per-source branch: a reversible alembic 0024 adds the nullable `branch` column, the ORM model carries it, and the four API schema shapes report/accept it — with NULL meaning "the repo's default branch" so every existing row is unchanged.
|
||||
|
||||
## Work
|
||||
1. `alembic/versions/0024_git_source_branch.py` (new) — follow the shape of `0021_git_source_token.py` (read it first):
|
||||
- `revision = "0024"`, `down_revision = "0023"` (verify the actual latest revision id in `alembic/versions/` at execution time — the file numbering is the convention, the `down_revision` chain is the truth).
|
||||
- `upgrade`: `op.add_column("git_sources", sa.Column("branch", sa.Text(), nullable=True))`.
|
||||
- `downgrade`: `op.drop_column("git_sources", "branch")` (reversible — the house rule; NULL = default branch means no data migration in either direction).
|
||||
2. `app/models.py::GitSource` — add:
|
||||
```python
|
||||
#: The branch to mirror (phase 135): NULL = the repo's default branch
|
||||
#: (every pre-phase-135 row and every env-fallback row — byte-identical
|
||||
#: clone/pull behavior). A non-NULL value is validated at the API
|
||||
#: boundary (app.rag.git_sources.validate_branch_name) — the column
|
||||
#: itself is plain Text.
|
||||
branch: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
```
|
||||
3. `app/schemas.py` (the four shapes — each docstring gains its phase-135 paragraph):
|
||||
- `GitSourceIn.branch: str | None = Field(default=None, max_length=200)` — optional at create (absent → NULL); a `field_validator` trims before the length constraint runs (the `_trim_url` precedent) — note the API layer treats a trimmed-EMPTY value as absent at create and as the tri-state's clear leg at PATCH (document that split in the docstring).
|
||||
- `GitSourceOut.branch: str | None` — the stored value (NULL → null in the response); the shape keeps `extra="forbid"` (no new omission risk — the field is explicit).
|
||||
- `GitSourceRow.branch: str | None` — DB rows report the stored value; env-fallback synthetic rows report `None` (the `ignore_paths: []` / `include_hidden: False` precedent — no DB row to store one on).
|
||||
- `GitSourcePatchIn.branch: str | None = None` — the tri-state (absent/None = no change; `""` = clear; non-empty = set — the token's precedent, documented in the docstring).
|
||||
4. `uv run alembic upgrade head` against the dev DB (and `downgrade -1` + `upgrade head` once — the reversibility proof).
|
||||
5. `tests/integration/test_migration_0024.py` (new — the `test_migration_0021.py` pattern): the upgrade adds the nullable column to a table WITH pre-existing rows (all report NULL — the byte-identical guarantee), a row can take a branch value, the downgrade drops the column.
|
||||
|
||||
## Testing & Quality
|
||||
- Integration: the migration round-trip (the suite above).
|
||||
- Coverage: **>90%** on new/modified code (the model/schema surface is exercised by task 02's API tests — this task's own gate is the migration test + the suite run).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run alembic upgrade head && uv run alembic downgrade -1 && uv run alembic upgrade head` succeeds on a scratch DB (the reversibility proof)
|
||||
- [ ] pre-existing rows read `branch: None` (the integration test asserts on seeded rows — no backfill, no default)
|
||||
- [ ] `uv run pytest tests/integration/test_migration_0024.py -v` green
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean
|
||||
@@ -1,52 +0,0 @@
|
||||
# Task 02 — API: branch validation, POST/PATCH/GET wiring (the tri-state, the local-row 422, the env rows)
|
||||
|
||||
**Phase:** `135_git_source_branch` · **Source:** `TODO.md:4` — "Need to be able to specify the branch for a git source"
|
||||
|
||||
## Objective
|
||||
The admin API accepts, validates, stores, and reports a branch per git source: a pure `validate_branch_name` helper is the single choke point, POST stores it (git rows only), PATCH edits it with the token's tri-state, and GET/POST-201/PATCH-200 report it — invalid names get a fixed-detail 422 that never echoes the input.
|
||||
|
||||
## Work
|
||||
1. `app/rag/git_sources.py` — the pure validator (next to `normalize_credential`; no DB, no logging of the raw value beyond the fixed reason):
|
||||
```python
|
||||
_BRANCH_RE = re.compile(r"^[A-Za-z0-9._/-]+$")
|
||||
|
||||
def validate_branch_name(raw: str) -> str:
|
||||
"""Trim + validate a branch name (phase 135, ASSUMPTION B2).
|
||||
|
||||
Returns the trimmed name; raises ValueError (fixed reason strings,
|
||||
for logs only — the API maps to the fixed-detail 422) when the
|
||||
name is empty, > 200 chars, outside the safe charset
|
||||
([A-Za-z0-9._/-]+), contains ".." or "//", starts with "-"/"."/"",
|
||||
ends with "/" or ".", or is HEAD (case-insensitive).
|
||||
"""
|
||||
```
|
||||
(The leading-slash check is `startswith("/")`; write the helper with the checks in this order: trim → empty → length → charset → `..`/`//` → leading → trailing → HEAD.)
|
||||
2. `app/api/git_sources.py`:
|
||||
- `_create_git_row`: after the URL contract (and the token normalization — order: URL shape → credential normalization → duplicate check → branch): `branch = (payload.branch or "").strip()`; if `branch`: `validate_branch_name` → store; else store `None`. A `ValueError` → `HTTPException(422, detail="not a valid branch name")` (FIXED detail — the input is never echoed).
|
||||
- `_create_local_row`: `payload.branch` present and non-empty after strip → `HTTPException(422, detail="branch is only valid for git sources")` (local rows are walked, not cloned — the fixed detail, the kind-field rules' precedent).
|
||||
- `patch_git_source`: a new independent present-wins field — `if payload.branch is not None:` the row must be `kind == "git"` (local → the same fixed 422); `""` → `row.branch = None` (the tri-state's clear leg); non-empty → `validate_branch_name` → set. No Integrity interaction (branch never touches a unique index) — no 409 path. Update the docstring's field list (ignore_paths / include_hidden / token / branch).
|
||||
- `list_git_sources`: DB rows report `branch=row.branch`; env-fallback rows report `branch=None`. Update the docstring.
|
||||
- `create_git_source` / `patch_git_source` responses: include `branch=row.branch` in the `GitSourceOut` construction.
|
||||
- The module docstring's route summary gains the branch clauses (the house one-paragraph-per-route contract).
|
||||
3. Unit tests — `tests/unit/test_git_sources_branch.py` (new; the validator has no existing home):
|
||||
- Accept: `main`, `release/1.2`, `feature_x`, `a.b/c`, `v2-rc.1` (trim applied: `" main "` → `main`).
|
||||
- Reject (each → `ValueError`): `""`, `" "`, 201 chars, `a b`, `a@b`, `a~b`, `a:b`, `a?b`, `a*b`, `a[b`, `a\b`, `a..b`, `a//b`, `-x`, `.x`, `/x`, `x/`, `x.`, `HEAD`, `head`, `HeAd`.
|
||||
4. Integration tests — `tests/integration/test_git_sources_api.py` (extend — the file already owns this router's contract):
|
||||
- POST a git source with `branch: "release"` → 201 reports `branch: "release"`; GET reports it.
|
||||
- POST with an invalid branch (`"a..b"`) → 422 `not a valid branch name` (the detail is EXACT — no input echo); the row is not created.
|
||||
- POST a LOCAL source with a branch → 422 `branch is only valid for git sources`.
|
||||
- PATCH the tri-state: set (`"hotfix/x"`) → 200 reports it; change (`"hotfix/y"`); clear (`""`) → 200 reports `null`; absent (PATCH `{"include_hidden": true}` only) → the branch is untouched.
|
||||
- PATCH a local row's branch → 422.
|
||||
- Env-fallback rows (empty table + `BOR_GIT_SOURCES` env): GET reports `branch: null` for every synthetic row.
|
||||
- The existing token/ignore/hidden round-trips stay green (no regression).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the validator matrix (step 3).
|
||||
- Integration: the API round-trips (step 4).
|
||||
- Coverage: **>90%** on new/modified code.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_git_sources_branch.py tests/integration/test_git_sources_api.py -v` green
|
||||
- [ ] every 422 detail is a fixed string (a grep of the new code shows no f-string echoing the branch input)
|
||||
- [ ] the tri-state matches the token's semantics field-for-field (the PATCH docstring documents both)
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean
|
||||
@@ -1,48 +0,0 @@
|
||||
# Task 03 — `clone_or_pull(…, branch)` and both call sites (sync pipeline + CLI)
|
||||
|
||||
**Phase:** `135_git_source_branch` · **Source:** `TODO.md:4` — "Need to be able to specify the branch for a git source"
|
||||
|
||||
## Objective
|
||||
The git plumbing honors a row's branch: a fresh clone checks it out (`--branch`), an existing checkout is tracked (fetch + ff-merge of that ref) with a fail-loud guard when the checkout is on a different branch, and `branch=None` keeps the pre-phase argv byte-identical — wired through BOTH entry points (the Sync button and the `import_docs` CLI).
|
||||
|
||||
## Work
|
||||
1. `scripts/git_sync.py::clone_or_pull` — the new optional parameter (the module docstring's clone/pull paragraph + the function docstring gain the branch contract):
|
||||
```python
|
||||
def clone_or_pull(url: str, dest: Path | str, branch: str | None = None) -> Path:
|
||||
```
|
||||
- **`branch is None`** — the EXISTING code path, byte-identical: clone without `--branch`; `rev-parse --is-shallow-repository` probe; `fetch --unshallow` while shallow; `git pull --ff-only`. (The argv pin in the unit tests is the proof.)
|
||||
- **Fresh clone + branch** (`dest` absent or no `.git`): `git clone --branch <b> url dest`.
|
||||
- **Existing checkout + branch**, in order:
|
||||
1. HEAD GUARD: `git rev-parse --abbrev-ref HEAD` → let `current` be the result. If `current != branch` → `GitSyncError(f"checkout {dest} is on branch '{current}' but the source specifies branch '{branch}' — remove and re-add the source to switch branches")` (ASSUMPTION B5 — fail loud per D12; the message names the checkout and both branch names — no URL, so no credential exposure).
|
||||
2. the shallow probe + one-time `git fetch --unshallow` self-heal (UNCHANGED from today — the phase-107 D11 contract).
|
||||
3. `git fetch origin <b>` → `git merge --ff-only FETCH_HEAD` (the single-fetch ff pull of the specified ref — the sequence phase 136 extends with its reset fallback; a non-ff merge still fails loud HERE — the force-push fix is phase 136, not this one).
|
||||
- All commands go through `run_git` (the A11 single invocation site — unchanged).
|
||||
2. The call sites — pass `row.branch`:
|
||||
- `app/api/sync.py::_run_sync` (the per-source loop, the `kind == "git"` arm): `root = clone_or_pull(clone_url_for(row), sources_root / repo_name(row.url), row.branch)`.
|
||||
- `scripts/import_docs.py::_resolve_sources` (the same shape): `root = clone_or_pull(clone_url_for(row), sources_root / repo_name(row.url), row.branch)`.
|
||||
- Both files' docstrings/comments name the branch argument (the phase-106/121 comment style). Env-fallback synthetic rows carry `branch=None` (the `GitSource(url=…, kind="git")` constructor — no argument needed, the model default is None — verified by the env-fallback integration path).
|
||||
- `file_commit_dates(root)` is untouched (it walks whatever the checkout holds — for a branch checkout that is the branch's tree, exactly the head-of-branch semantics the owner asked for).
|
||||
3. Unit tests — `tests/unit/test_git_sync.py` (extend — the mocked-`subprocess` argv pattern the file already uses):
|
||||
- `branch=None` fresh + existing: the argv sequence is byte-identical to the pre-phase expectations (the existing tests stay green + add an explicit argv-equality pin if the file doesn't already pin it).
|
||||
- Fresh clone + `branch="release"`: `["git", "clone", "--branch", "release", url, dest]`.
|
||||
- Existing + matching HEAD (`rev-parse --abbrev-ref HEAD` → `"release"`): the sequence `rev-parse --abbrev-ref HEAD` → `rev-parse --is-shallow-repository` → (`fetch --unshallow` when the probe says shallow) → `fetch origin release` → `merge --ff-only FETCH_HEAD`.
|
||||
- Existing + mismatched HEAD (`"main"` vs `branch="release"`): raises `GitSyncError` whose message contains both branch names and the checkout path — and NO fetch/merge argv was invoked (the guard is first).
|
||||
- The mismatch message never contains the URL (credential-safety — even for a plain URL the message is URL-free).
|
||||
4. Integration tests — `tests/integration/test_import_docs_git.py` (extend — the real-`git`-`file://` pattern the file already uses):
|
||||
- Build a two-branch fixture repo (real git: `main` with file A, `release` with file B — different content per branch).
|
||||
- `clone_or_pull(url, dest, "release")` fresh → the checkout's HEAD branch is `release` and file B is present (file A absent).
|
||||
- A second `clone_or_pull` after the origin gains a commit on `release` → ff-merged (file B updated, no error).
|
||||
- The mismatch case: clone fresh with no branch (HEAD = `main`), then `clone_or_pull(url, dest, "release")` → `GitSyncError` (the remove-and-re-add message), the checkout untouched (still on `main`).
|
||||
- The full sync pipeline with a branch row: a `git_sources` row (`kind="git"`, `branch="release"`) → `_run_sync`'s source resolution imports file B (the `tests/integration/test_sync_api.py` fixture shape) — or at minimum `_resolve_sources`-level for the CLI path (follow whichever seam the file already exercises; the E2E, task 05, pins the button path end to end).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the argv matrix (step 3) — the `branch=None` byte-identical pin is the regression gate for every existing deployment.
|
||||
- Integration: the real-git two-branch scenarios (step 4).
|
||||
- Coverage: **>90%** on new/modified code.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_git_sync.py tests/integration/test_import_docs_git.py -v` green
|
||||
- [ ] `branch=None` produces the exact pre-phase argv (the pin is in the suite — a future refactor that rewrites the default path breaks it)
|
||||
- [ ] the mismatch fails loud BEFORE any fetch (the unit test asserts no fetch argv after the guard)
|
||||
- [ ] both entry points pass `row.branch` (a grep of the two call sites + the integration test covering each)
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean
|
||||
@@ -1,43 +0,0 @@
|
||||
# Task 04 — UI: the add-form field, the per-row editor field, and the row tag
|
||||
|
||||
**Phase:** `135_git_source_branch` · **Source:** `TODO.md:4` — "Need to be able to specify the branch for a git source"
|
||||
|
||||
## Objective
|
||||
The Sources manager (git-sources view) lets the owner see, set, change, and clear a source's branch: an optional Branch input in the add form, a Branch field in the per-row editor dialog (git rows only, prefilled, empty-save = clear), and a `branch: <name>` text tag on every stored git row that has one — all on the §7.4 never-stale lifecycle.
|
||||
|
||||
## Work
|
||||
1. `frontend/index.html` (the `#view-git-sources` section):
|
||||
- **The add form** (`#git-source-form`) — between the URL field and the Token field:
|
||||
```html
|
||||
<label for="git-source-branch">Branch <span class="field-hint">optional — default: the repo's default branch</span></label>
|
||||
<input id="git-source-branch" name="branch" type="text" maxlength="200" autocomplete="off" placeholder="main, release/1.2, …">
|
||||
```
|
||||
(The visible `<label for=…>` is the WCAG input-label rule, same as the url/token fields.)
|
||||
- **The per-row editor dialog** (`#ignore-editor-dialog`) — after the source line, before the ignore textarea:
|
||||
```html
|
||||
<label for="ignore-editor-branch">Branch</label>
|
||||
<input id="ignore-editor-branch" type="text" maxlength="200" autocomplete="off" placeholder="(default branch)">
|
||||
<p class="ignore-editor-branch-hint">Leave empty for the repo's default branch.</p>
|
||||
```
|
||||
The hint uses the existing `.field-hint` / muted-copy idiom (match the token field's hint styling — no new CSS class unless the existing one doesn't fit).
|
||||
2. `frontend/assets/git-sources.js`:
|
||||
- **Element lookups:** `const branchInput = root.querySelector("#git-source-branch");` and `const ignoreBranchInput = root.querySelector("#ignore-editor-branch");` (next to the existing token lookups).
|
||||
- **Add form:** thread the branch through `wireAddForm` — the opts object gains `branchInput`, and the git-kind body builder becomes `body: (url, token, branch) => ({ url, ...(branch ? { branch } : {}), ...(token ? { token } : {}) })` (a blank branch OMITS the key — the token's blank-omits precedent; the submit handler reads `branchInput.value.trim()` and clears it on 201 alongside the url/token — the credential-free inputs clear together).
|
||||
- **Per-row editor dialog:**
|
||||
- Open: for a git row (`s.kind === "git"`) — prefill `ignoreBranchInput.value = s.branch ?? ""` and show the field; for a local row — hide the field (label + input + hint) and the save body OMITS `branch` (local rows can't take one — the API's 422 is the backstop, the UI simply doesn't offer it).
|
||||
- Save: the PATCH body for a git row ALWAYS includes `branch: ignoreBranchInput.value.trim()` (empty = the tri-state's clear leg — a deliberate empty save clears the branch, unlike the token's blank = keep; the docstring/section comment documents the asymmetry), composed with the existing `ignore_paths` + conditional `token` (the token stays the blank = keep tri-state — untouched).
|
||||
- The never-stale lifecycle mirrors the existing save handler: the Save button disables in flight, 200 → close the dialog + `loadSources()` (the row re-renders from the server — the tag appears/disappears), failure → the dialog's error line (the existing pattern) + the inputs keep the user's text.
|
||||
- **The row tag** (`makeRow` / the source-cell builder): when `s.id && s.branch` — append a text tag `branch: <name>` in the source cell, next to the existing "hidden on" tag (the SAME tag idiom: a span with the tag class — text + background, never color alone; the tag text is the signal). Env-fallback rows (no `s.id`) and NULL-branch rows show no tag. Update the source cell's `aria-label`/`title` to include the branch (the hover precision idiom).
|
||||
- The module's section-header docstring gains the phase-135 paragraph (the house pattern — the field, the tri-state asymmetry, the tag rule).
|
||||
3. No new API surface, no new fetches — the branch rides the existing POST/PATCH/GET payloads (task 02's shapes).
|
||||
|
||||
## Testing & Quality
|
||||
- No JS unit layer in this repo — the UI contract is pinned by the phase's E2E (task 05).
|
||||
- Coverage: **>90%** on new/modified code (frontend-only task — the bar is carried by the suite run + the E2E).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] adding a git source with a branch sends `branch` (non-blank) / omits it (blank) — pinned by the E2E's request assertions
|
||||
- [ ] the dialog prefills the stored branch; saving an empty value clears it (the row's tag disappears after `loadSources`); local rows' dialogs never show the field
|
||||
- [ ] a stored git row with a branch shows the `branch: <name>` text tag (visible, text-based, in the source cell)
|
||||
- [ ] the 360 px case renders (the form + dialog don't overflow — verified in the E2E's viewport assertions where applicable)
|
||||
- [ ] no console errors on the view (the E2E pins it)
|
||||
@@ -1,44 +0,0 @@
|
||||
# Task 05 — E2E: `tests/e2e/test_git_source_branch.py` (the dedicated Playwright suite, run in isolation)
|
||||
|
||||
**Phase:** `135_git_source_branch` · **Source:** `TODO.md:4` — "Need to be able to specify the branch for a git source"
|
||||
|
||||
## Objective
|
||||
The branch feature is pinned end to end against the REAL pipeline: a two-branch `file://` fixture repo, the owner adding a source WITH a branch through the UI (→ sync → the branch's content is indexed, not the default branch's), the row tag, the per-row editor's branch field (change + the fail-loud mismatch), and the API contracts (the tri-state, the fixed 422s, the local-row rejection).
|
||||
|
||||
## Work
|
||||
1. `tests/e2e/test_git_source_branch.py` (new) — the per-module app pattern (`test_git_sources_admin.py` + `test_sync_button.py` combined):
|
||||
- **Ports/env:** `APP_PORT = int(os.environ.get("E2E_APP_PORT_GITBR", "8134"))`; the standard module env (`BOR_STATIC_DIR`, the mock LLM base URL, `BOR_RELEVANCE_THRESHOLD=0.30`, `BOR_ADMIN_PASSWORD` / `BOR_SESSION_SECRET`, a dedicated `BOR_SOURCES_DIR`). **No `BOR_GIT_SOURCES` env** — the sources under test are DB rows (the UI adds them), so the env fallback stays empty.
|
||||
- **The fixture repo** (module-scoped, `tmp_path_factory`, real `git`): two branches — `main` (file `notes/main-only.md` with `Marker: RESE-BRANCH-MAIN-7a31`) and `release` (file `notes/release-only.md` with `Marker: RESE-BRANCH-REL-9c2d`, plus a shared `notes/common.md` whose content DIFFERS between the branches — `main` says `VERSION: 1`, `release` says `VERSION: 2`). Both branches pushed to the repo (a local bare-ish fixture: the test builds `main`, commits, then `git checkout -b release`, edits, commits — a normal two-branch repo; the app clones it via `file://`).
|
||||
- **Helpers:** `add_git_source(page, url, branch=None, token=None)` (fills the UI form — the branch field when given), `sync_and_wait(page, app_url)` (clicks `#sync-btn` and polls the status endpoint to a terminal state — the `_wait_sync_done` pattern), `catalog_has_doc(page, doc_name)` (the drill-down tree assertion, the `_expect_fixture_row` pattern).
|
||||
2. Tests:
|
||||
- `test_add_with_branch_syncs_that_branch` — the owner's exact flow:
|
||||
1. login (lands on the git-sources view — the `login` helper's `next` param or a nav click, per the existing suites);
|
||||
2. add a source: url = `file://<fixture>`, branch = `release` → 201; the row appears with the `branch: release` tag (the text is in the source cell);
|
||||
3. switch to the Sources/RAG view, `sync_and_wait` → `state == "success"`;
|
||||
4. the catalog contains `release-only.md` (the branch's doc) AND `common.md` shows `VERSION: 2` (drill in and assert on the document's stored content via the document modal or the `GET /api/docs` row — follow the assertion idiom the dates suite uses for content checks); `main-only.md` is ABSENT (the default branch's doc was never indexed);
|
||||
5. re-sync (idempotent): "0 added · 2 unchanged" (release's two files).
|
||||
- `test_default_branch_source_unaffected` — a SECOND source row for the SAME url is impossible (the unique URL 409) — so this test uses a CLONE of the fixture (a second `file://` repo, identical history, `main` only): add it with NO branch → sync → `main-only.md` present, `release-only.md` absent (the `branch=None` path mirrors the pre-phase behavior — the byte-identical guarantee, observable through the UI).
|
||||
- `test_row_editor_branch_change_fails_loud` — the fail-loud mismatch (ASSUMPTION B5, end to end):
|
||||
1. take the `release` source from test 1's state (or a fresh module state — order the tests so the fixture state is explicit; the KB-truncate autouse fixture keeps them independent: re-add the source fresh);
|
||||
2. open the per-row editor, change the branch field to `main`, save → the dialog closes, the row re-renders with the `branch: main` tag (the PATCH landed — the registry is updated);
|
||||
3. sync → `state == "failed"`; the sanitized error banner/modal shows the fixed mismatch copy (assert on the user-visible fragment `remove and re-add the source` — the message is a fixed string, safe to pin);
|
||||
4. restore: editor → branch field `release` → save → sync → success again (the guard is symmetric — back to the matching branch, the ff path works).
|
||||
- `test_api_branch_contracts` — the API layer through `page.request` (admin cookie):
|
||||
- POST a git source with `branch: "a..b"` → 422, detail EXACTLY `not a valid branch name` (no echo); the list is unchanged;
|
||||
- POST a git source with `branch: "HEAD"` → 422 (same detail);
|
||||
- PATCH the tri-state on a stored row: set `hotfix/x` → 200 reports it; clear with `""` → 200 reports `null`; a PATCH of `{"include_hidden": true}` leaves the branch untouched;
|
||||
- POST a LOCAL source (an existing tmp dir) with `branch: "x"` → 422 `branch is only valid for git sources`;
|
||||
- the token field is unaffected by all of the above (the existing token tests stay green — no regression pin needed here).
|
||||
- Wall-time guard: the whole module < 180 s (three full syncs of a 3-file repo against the mock LLM — the `slow_llm` proxy is NOT needed: the branch assertions are about WHICH files land, not about a running window).
|
||||
3. Header docstring: the story (the TODO item, L4), the run command (`uv run pytest tests/e2e/test_git_source_branch.py -v --no-cov`), the isolation note (per-module app, dedicated port, DB rows not env), the fixture's two-branch shape.
|
||||
|
||||
## Testing & Quality
|
||||
- E2E: this file IS the task's suite; it runs in ISOLATION (`--no-cov`, the AGENTS.md rule 9 gate) with the DB up (`podman compose up -d db`) and git on PATH.
|
||||
- Coverage: **>90%** on new/modified code (test-only task — the bar is carried by the suite run).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/e2e/test_git_source_branch.py -v --no-cov` green in isolation
|
||||
- [ ] the branch's content is what gets indexed (the sentinel docs on the right branch present, the wrong branch's doc absent — the feature's core promise)
|
||||
- [ ] the `branch=None` path imports the default branch exactly as pre-phase (the unaffected-source test)
|
||||
- [ ] the mismatch is LOUD (the failed sync names the remove-and-re-add fix — the owner is never left with a silently-wrong checkout)
|
||||
- [ ] every 422 detail is the fixed string (the test asserts on the exact detail, not a substring of the input)
|
||||
Reference in New Issue
Block a user