phase: 135_git_source_branch
Build and Push Containers / build-and-push-app (push) Successful in 2m34s
Build and Push Containers / build-and-push-db (push) Successful in 14s

All verification complete. No edits were needed by this pass — the phase work and docs were already consistent and green.

**Phase 135 final verification pass — report**

- Verified all 5 tasks' work present & correct: migration 0024 (reversible), `GitSource.branch` + 4 schema shapes, `validate_branch_name` (full check order per design), `clone_or_pull(…, branch)` (HEAD guard first, `--branch` clone, fetch+ff merge), both call sites pass `row.branch`, UI (add-form field, editor field, row tag), `file://` URL transport.
- Completion criteria: (1) branch-mirrored through Sync + CLI — **pass** (E2E + integration); (2) `branch=None` byte-identical — **pass** (`test_branch_none_argv_is_byte_identical_pre_phase_sequence` green); (3) fixed 422s / local-row 422 / tri-state — **pass**; (4) mismatch fails loudly with remove-and-re-add — **pass** (E2E); (5) coverage 99% >90%, E2E isolated green — **pass**; (6) no regressions — **pass**.
- Tests: `uv run pytest --cov=app --cov-report=term-missing` → 3223 passed, app/ 99%; `uv run pytest tests/e2e/test_git_source_branch.py -v --no-cov` → 4 passed (16.4 s); `uv run ruff check .` clean; `uv run pyright` → 0 errors, 0 warnings.
- Documentation: README.md reviewed in full — phase-135 branch section accurate; the phase's README diff also correctly fixed a pre-existing stale line (shallow `--depth 1` clone vs full-history, phase 107). Routes, env defaults, feature lists verified against code. AGENTS.md — all 10 rules + quick-reference verified (referenced tests, PLAN.md, phases dirs, commands all accurate). **Verified consistent — no changes needed.**
- No live-infrastructure changes (dev DB already running and healthy; no live steps in this phase).
- Next pending phase: `.agents/phases/todo/136_force_push_reset`.
This commit is contained in:
2026-09-28 21:15:03 -04:00
parent fcac876d40
commit d3f8c7f2c8
50 changed files with 3955 additions and 140 deletions
+136 -45
View File
@@ -16,19 +16,33 @@ Routes: ``GET`` (DB rows oldest-first, or the env list with
``from_env: true`` while the table is empty; rows carry ``kind`` +
``path``, git rows — and env rows — report ``path: null``; every row
reports ``ignore_paths`` — DB rows their stored normalized list, env
rows ``[]``, phase 89 — and ``include_hidden`` — DB rows their stored
flag, env rows ``False``, phase 105), ``POST``
(201, validated create; ``kind`` selects the validation: git → exactly
the phase-35 URL contract, local → an existing absolute directory, else
422 naming the path; both kinds accept ``ignore_paths`` — optional,
absent → ``[]``, stored normalized, phase 89 — and ``include_hidden``
— optional, absent → ``False``, stored as sent, phase 105),
``PATCH /{source_id}`` (phase 89, A5 + phase 105 — the ignore list
(replace) and/or the hidden-folders flag (phase 105) — each optional,
present-wins: 404 unknown id; ``ignore_paths`` when present is
normalized + A4-validated with fixed-detail 422s and REPLACES the
row's list wholesale — an empty list clears all; ``include_hidden``
when present sets the flag; both absent → 200 no-op; 200 → the
rows ``[]``, phase 89 — ``include_hidden`` — DB rows their stored
flag, env rows ``False``, phase 105 — and ``branch`` — DB rows their
stored value (NULL = the repo's default branch), env rows ``null``,
phase 135), ``POST``
(201, validated create; ``kind`` selects the validation: git → the
phase-35 URL contract + the ``file://`` local transport (phase 135,
task 05 — the E2E fixture repos the UI adds end to end), local → an
existing absolute directory, else 422 naming the path; both kinds
accept ``ignore_paths`` — optional, absent → ``[]``, stored
normalized, phase 89 — and ``include_hidden``
— optional, absent → ``False``, stored as sent, phase 105; git rows
accept ``branch`` — optional, absent/empty → NULL (the default
branch), non-empty validated by
:func:`app.rag.git_sources.validate_branch_name` with the fixed-detail
422 ``not a valid branch name`` (input never echoed, phase 135) — a
``branch`` on a LOCAL row is 422 ``branch is only valid for git
sources``),
``PATCH /{source_id}`` (phase 89, A5 + phase 105 + phase 121 + phase
135 — the ignore list (replace), the hidden-folders flag (phase 105),
the token (phase 121 tri-state), and the branch (phase 135 tri-state)
— each optional, present-wins: 404 unknown id; ``ignore_paths`` when
present is normalized + A4-validated with fixed-detail 422s and
REPLACES the row's list wholesale — an empty list clears all;
``include_hidden`` when present sets the flag; ``token`` and ``branch``
when present are tri-state — absent/None = no change, non-empty =
set/replace (``branch`` re-validated, ``local`` rows 422), empty
string = clear (stored NULL); all absent → 200 no-op; 200 → the
``GitSourceOut`` shape),
``POST /upload`` (phase 49, backgrounded in phase
64 task 03, scan deferred in phase 90 — admin archive upload: the
@@ -115,7 +129,11 @@ from app.rag.archive_upload import (
swap_in,
unpack_archive,
)
from app.rag.git_sources import normalize_credential, sanitize_url
from app.rag.git_sources import (
normalize_credential,
sanitize_url,
validate_branch_name,
)
from app.rag.importer import normalize_ignore_path
from app.rag.llm import LLMClient
from app.rag.overview import regenerate_overview
@@ -196,9 +214,12 @@ _upload_status = UploadStatus()
#: Accepted git URL shapes — the trimmed URL must *start* with one of them.
#: Covers the phase-28 real URLs (HTTPS + ``git@`` SSH); scp-style
#: ``host:repo`` is deliberately rejected (422). ASSUMPTION (task 02): the
#: accepted shapes are exactly these four prefixes.
URL_RE = re.compile(r"^(https?://|ssh://|git@)")
#: ``host:repo`` is deliberately rejected (422). ``file://`` (phase 135,
#: task 05) is the deterministic local-transport source the E2E fixture
#: repos use — a real ``git clone`` of a local repo (the phase-28/32/107
#: transport-true E2E idiom, promoted from the ``BOR_GIT_SOURCES`` env
#: fallback to a stored row the UI can add).
URL_RE = re.compile(r"^(https?://|ssh://|file://|git@)")
#: Phase 89, A4 — the per-source ignore-list limits, enforced in
#: :func:`_validate_ignore_paths` (shared by POST and PATCH): at most
@@ -219,13 +240,14 @@ def list_git_sources(
same-timestamp inserts) with ``from_env: false`` — each row carries
its ``kind``, its ``ignore_paths`` (phase 89 — the stored,
normalized list; ``or []`` guards a row that predated the column),
its ``include_hidden`` (phase 105 — the stored flag), and, for
local rows, the stored ``path`` (git rows and env rows report
``path: null``); while the table is empty, the
its ``include_hidden`` (phase 105 — the stored flag), its
``branch`` (phase 135 — the stored value; NULL = the repo's default
branch), and, for local rows, the stored ``path`` (git rows and env
rows report ``path: null``); while the table is empty, the
``BOR_GIT_SOURCES`` env URLs as git rows (the env fallback is
git-only) with null ``id``/``added_at``, ``ignore_paths: []`` and
``include_hidden: False`` (no DB row to store a list or a flag on),
and ``from_env: true``.
git-only) with null ``id``/``added_at``, ``ignore_paths: []``,
``include_hidden: False`` and ``branch: None`` (no DB row to store a
list, a flag or a branch on), and ``from_env: true``.
Every URL is masked on the way out (phase 121, LOCKED A2 —
:func:`sanitize_url`): an env value or a legacy stored URL may
@@ -248,6 +270,7 @@ def list_git_sources(
added_at=row.added_at,
ignore_paths=row.ignore_paths or [],
include_hidden=row.include_hidden,
branch=row.branch, # phase 135: NULL = default branch
)
for row in rows
],
@@ -263,6 +286,7 @@ def list_git_sources(
added_at=None,
ignore_paths=[],
include_hidden=False,
branch=None, # phase 135: no DB row to store a branch on
)
for url in get_settings().git_source_list
],
@@ -310,6 +334,16 @@ def create_git_source(
credential in the token column, an explicit ``token`` wins over
the embedded one (LOCKED A6), and the duplicate check runs on the
bare URL.
``branch`` (phase 135) — git rows only: absent/None → stored NULL
(the repo's DEFAULT branch — the pre-phase-135 behavior), a
trimmed-EMPTY value likewise stores NULL (at CREATE, empty is
absent — the split from PATCH's clear leg, see :class:`GitSourceIn`),
non-empty → validated by :func:`validate_branch_name` and stored
(a ``ValueError`` → 422 ``not a valid branch name`` — FIXED detail,
the input is never echoed). A ``branch`` on a LOCAL row is 422
``branch is only valid for git sources`` (local rows are walked,
not cloned).
"""
row = _create_git_row(payload, db) if payload.kind == "git" else _create_local_row(payload, db)
return GitSourceOut(
@@ -318,6 +352,7 @@ def create_git_source(
added_at=row.added_at,
ignore_paths=row.ignore_paths,
include_hidden=row.include_hidden,
branch=row.branch, # phase 135: NULL = default branch
)
@@ -355,16 +390,21 @@ def _commit_new(row: GitSource, duplicate_detail: str, db: Session) -> GitSource
def _create_git_row(payload: GitSourceIn, db: Session) -> GitSource:
"""``kind=git`` — the phase-35 URL contract, unchanged (A10: no
credential echo, so every detail is a fixed string)."""
"""``kind=git`` — the phase-35 URL contract, extended with the
``file://`` local transport (phase 135, task 05 — the E2E fixture
repos the UI adds end to end); A10: no credential echo, so every
detail is a fixed string."""
if payload.path is not None:
raise HTTPException(status_code=422, detail="a git source takes a url, not a path")
if payload.url is None:
raise HTTPException(status_code=422, detail="a git source requires a url")
url = payload.url
if not URL_RE.match(url):
# The detail lists the accepted prefixes (fixed string — the
# input is never echoed; phase 135 task 05 added file://).
raise HTTPException(
status_code=422, detail="not a valid git URL (expected https://, ssh:// or git@…)"
status_code=422,
detail="not a valid git URL (expected https://, ssh://, file:// or git@…)",
)
# Phase 121 (task 02, LOCKED A6): normalize the credential — an
# old-style embedded ``user:pass@`` URL is stored BARE and the
@@ -375,6 +415,18 @@ def _create_git_row(payload: GitSourceIn, db: Session) -> GitSource:
url, effective_token = normalize_credential(url, payload.token)
if db.scalar(select(GitSource).where(GitSource.url == url)) is not None:
raise HTTPException(status_code=409, detail="a git source with this URL already exists")
# Phase 135: the branch pin (ASSUMPTION B2/B3) — absent or
# trimmed-EMPTY stores NULL (the repo's default branch, the
# pre-phase-135 behavior); non-empty is validated at the boundary
# with the FIXED-detail 422 (the raw name never echoes).
branch = (payload.branch or "").strip()
if branch:
try:
branch = validate_branch_name(branch)
except ValueError:
raise HTTPException(
status_code=422, detail="not a valid branch name"
) from None
return _commit_new(
GitSource(
url=url,
@@ -382,6 +434,7 @@ def _create_git_row(payload: GitSourceIn, db: Session) -> GitSource:
ignore_paths=_validate_ignore_paths(payload.ignore_paths),
include_hidden=bool(payload.include_hidden),
token=effective_token or None,
branch=branch or None,
),
"a git source with this URL already exists",
db,
@@ -391,9 +444,17 @@ def _create_git_row(payload: GitSourceIn, db: Session) -> GitSource:
def _create_local_row(payload: GitSourceIn, db: Session) -> GitSource:
"""``kind=local`` — fail-loud add-time validation (phase 38):
trimmed → ``expanduser()`` → absolute + existing directory, else 422
naming the path (not a secret, unlike a git URL)."""
naming the path (not a secret, unlike a git URL). A ``branch`` on a
local row (phase 135) is 422 — local rows are walked, not cloned."""
if payload.url is not None:
raise HTTPException(status_code=422, detail="a local source takes a path, not a url")
if (payload.branch or "").strip():
# Phase 135: local rows are walked, not cloned — a branch pin
# has no meaning for them (the kind-field rules' precedent:
# fixed detail, no input echo).
raise HTTPException(
status_code=422, detail="branch is only valid for git sources"
)
if payload.path is None:
raise HTTPException(status_code=422, detail="a local source requires a path")
expanded = Path(payload.path).expanduser()
@@ -432,26 +493,37 @@ def patch_git_source(
payload: GitSourcePatchIn,
db: Session = Depends(get_db), # noqa: B008
) -> GitSourceOut:
"""Edit one source's ignore list, hidden-folders flag, and/or
private-repo token.
"""Edit one source's ignore list, hidden-folders flag, private-
repo token, and/or branch.
Phase 89 A5 (ignore list) + phase 105 (the flag) + phase 121
(the token): 404 unknown id; each PRESENT body field applies
independently — ``ignore_paths`` REPLACES the list (normalized +
A4-validated, fixed 422 details); ``include_hidden`` sets the
flag; ``token`` is TRI-STATE (LOCKED A2): absent/None = no change
(the row's stored credential survives an edit that does not touch
the masked field), non-empty = replace, empty string = clear
(stored NULL). A PRESENT token also re-normalizes the (current
url, new token) pair with the POST write-path rules — a legacy
embedded-token URL gets its userinfo stripped (moved to the
column) the first time an explicit credential is written; a clean
URL comes back untouched. The 409 backstop: re-normalizing can
make the stored URL collide with another row's bare URL (a
legacy ``user:pass@`` row and a bare row for the same repo) — the
unique index yields the generic 409, never a 500. Returns the
updated row's public shape (id, url — masked, added_at,
ignore_paths, include_hidden); the token is never echoed.
(the token) + phase 135 (the branch): 404 unknown id; each PRESENT
body field applies independently — ``ignore_paths`` REPLACES the
list (normalized + A4-validated, fixed 422 details);
``include_hidden`` sets the flag; ``token`` is TRI-STATE (LOCKED
A2): absent/None = no change (the row's stored credential survives
an edit that does not touch the masked field), non-empty = replace,
empty string = clear (stored NULL). A PRESENT token also
re-normalizes the (current url, new token) pair with the POST
write-path rules — a legacy embedded-token URL gets its userinfo
stripped (moved to the column) the first time an explicit
credential is written; a clean URL comes back untouched. The 409
backstop: re-normalizing can make the stored URL collide with
another row's bare URL (a legacy ``user:pass@`` row and a bare row
for the same repo) — the unique index yields the generic 409,
never a 500. ``branch`` (phase 135) is an independent PRESENT-
wins field with the token's EXACT tri-state semantics:
absent/None = no change (the row's stored branch survives an edit
that does not touch it), non-empty = set (re-validated by
:func:`validate_branch_name` — a ``ValueError`` is the FIXED-detail
422 ``not a valid branch name``, input never echoed), empty string
= clear (stored NULL = the repo's default branch). A PRESENT
branch on a LOCAL row is 422 ``branch is only valid for git
sources`` (local rows are walked, not cloned). Branch never
touches a unique index — no Integrity interaction, no 409 path.
Returns the updated row's public shape (id, url — masked,
added_at, ignore_paths, include_hidden, branch); the token is
never echoed.
"""
row = db.get(GitSource, source_id)
if row is None:
@@ -467,6 +539,24 @@ def patch_git_source(
# bare + column credential.
row.url, effective = normalize_credential(row.url, payload.token)
row.token = effective or None
if payload.branch is not None:
# Phase 135: the branch tri-state (the token's exact semantics
# — see the docstring). Local rows are walked, not cloned — a
# branch pin is meaningless there (the same fixed 422 as POST).
if row.kind != "git":
raise HTTPException(
status_code=422, detail="branch is only valid for git sources"
)
branch = payload.branch.strip()
if not branch:
row.branch = None # the tri-state's clear leg (stored NULL)
else:
try:
row.branch = validate_branch_name(branch)
except ValueError:
raise HTTPException(
status_code=422, detail="not a valid branch name"
) from None
try:
db.commit()
except IntegrityError:
@@ -485,6 +575,7 @@ def patch_git_source(
added_at=row.added_at,
ignore_paths=row.ignore_paths,
include_hidden=row.include_hidden,
branch=row.branch, # phase 135: NULL = default branch
)
+19 -2
View File
@@ -37,7 +37,14 @@ decisions):
— the row's ``token`` column injected as
``https://x-access-token:<token>@…`` only for https? rows; NULL
token → the bare stored URL verbatim, so public repos and legacy
embedded-token rows clone exactly as before) into
embedded-token rows clone exactly as before) and the row's
BRANCH PIN as the third argument (phase 135 — a fresh clone checks
it out, ``git clone --branch``; an existing checkout is guarded — a
mismatched branch fails loudly, the remove-and-re-add error — and
tracked, ``git fetch origin <b>`` + ``git merge --ff-only
FETCH_HEAD``; ``branch=None``, i.e. every pre-phase-135 row and
every env-fallback row, keeps the pre-phase argv byte-identical)
into
``BOR_SOURCES_DIR/<repo-name>/`` (phase 28 — reused, not
re-implemented; the checkout name stays on the bare URL —
credential-free); ``kind=local`` → the stored directory, re-verified
@@ -498,7 +505,17 @@ async def _run_sync() -> None:
# URL ONLY here (clone_url_for — NULL token → the bare
# stored URL verbatim); repo_name stays on the bare URL
# so the checkout directory name is credential-free.
root = clone_or_pull(clone_url_for(row), sources_root / repo_name(row.url))
# Phase 135: the row's branch pin is the third
# argument — NULL (every pre-phase-135 row + every
# env-fallback row) keeps the pre-phase argv
# byte-identical; a pinned branch is checked out on a
# fresh clone (--branch) and guarded + tracked on an
# existing checkout (fetch + ff-merge of that ref).
root = clone_or_pull(
clone_url_for(row),
sources_root / repo_name(row.url),
row.branch,
)
# Phase 106 (D2): the checkout's per-file last-commit
# dates, keyed by the SAME root string the importer
# sees (full-history checkouts → true per-file