phase: 135_git_source_branch
All verification complete. No edits were needed by this pass — the phase work and docs were already consistent and green. **Phase 135 final verification pass — report** - Verified all 5 tasks' work present & correct: migration 0024 (reversible), `GitSource.branch` + 4 schema shapes, `validate_branch_name` (full check order per design), `clone_or_pull(…, branch)` (HEAD guard first, `--branch` clone, fetch+ff merge), both call sites pass `row.branch`, UI (add-form field, editor field, row tag), `file://` URL transport. - Completion criteria: (1) branch-mirrored through Sync + CLI — **pass** (E2E + integration); (2) `branch=None` byte-identical — **pass** (`test_branch_none_argv_is_byte_identical_pre_phase_sequence` green); (3) fixed 422s / local-row 422 / tri-state — **pass**; (4) mismatch fails loudly with remove-and-re-add — **pass** (E2E); (5) coverage 99% >90%, E2E isolated green — **pass**; (6) no regressions — **pass**. - Tests: `uv run pytest --cov=app --cov-report=term-missing` → 3223 passed, app/ 99%; `uv run pytest tests/e2e/test_git_source_branch.py -v --no-cov` → 4 passed (16.4 s); `uv run ruff check .` clean; `uv run pyright` → 0 errors, 0 warnings. - Documentation: README.md reviewed in full — phase-135 branch section accurate; the phase's README diff also correctly fixed a pre-existing stale line (shallow `--depth 1` clone vs full-history, phase 107). Routes, env defaults, feature lists verified against code. AGENTS.md — all 10 rules + quick-reference verified (referenced tests, PLAN.md, phases dirs, commands all accurate). **Verified consistent — no changes needed.** - No live-infrastructure changes (dev DB already running and healthy; no live steps in this phase). - Next pending phase: `.agents/phases/todo/136_force_push_reset`.
This commit is contained in:
+136
-45
@@ -16,19 +16,33 @@ Routes: ``GET`` (DB rows oldest-first, or the env list with
|
||||
``from_env: true`` while the table is empty; rows carry ``kind`` +
|
||||
``path``, git rows — and env rows — report ``path: null``; every row
|
||||
reports ``ignore_paths`` — DB rows their stored normalized list, env
|
||||
rows ``[]``, phase 89 — and ``include_hidden`` — DB rows their stored
|
||||
flag, env rows ``False``, phase 105), ``POST``
|
||||
(201, validated create; ``kind`` selects the validation: git → exactly
|
||||
the phase-35 URL contract, local → an existing absolute directory, else
|
||||
422 naming the path; both kinds accept ``ignore_paths`` — optional,
|
||||
absent → ``[]``, stored normalized, phase 89 — and ``include_hidden``
|
||||
— optional, absent → ``False``, stored as sent, phase 105),
|
||||
``PATCH /{source_id}`` (phase 89, A5 + phase 105 — the ignore list
|
||||
(replace) and/or the hidden-folders flag (phase 105) — each optional,
|
||||
present-wins: 404 unknown id; ``ignore_paths`` when present is
|
||||
normalized + A4-validated with fixed-detail 422s and REPLACES the
|
||||
row's list wholesale — an empty list clears all; ``include_hidden``
|
||||
when present sets the flag; both absent → 200 no-op; 200 → the
|
||||
rows ``[]``, phase 89 — ``include_hidden`` — DB rows their stored
|
||||
flag, env rows ``False``, phase 105 — and ``branch`` — DB rows their
|
||||
stored value (NULL = the repo's default branch), env rows ``null``,
|
||||
phase 135), ``POST``
|
||||
(201, validated create; ``kind`` selects the validation: git → the
|
||||
phase-35 URL contract + the ``file://`` local transport (phase 135,
|
||||
task 05 — the E2E fixture repos the UI adds end to end), local → an
|
||||
existing absolute directory, else 422 naming the path; both kinds
|
||||
accept ``ignore_paths`` — optional, absent → ``[]``, stored
|
||||
normalized, phase 89 — and ``include_hidden``
|
||||
— optional, absent → ``False``, stored as sent, phase 105; git rows
|
||||
accept ``branch`` — optional, absent/empty → NULL (the default
|
||||
branch), non-empty validated by
|
||||
:func:`app.rag.git_sources.validate_branch_name` with the fixed-detail
|
||||
422 ``not a valid branch name`` (input never echoed, phase 135) — a
|
||||
``branch`` on a LOCAL row is 422 ``branch is only valid for git
|
||||
sources``),
|
||||
``PATCH /{source_id}`` (phase 89, A5 + phase 105 + phase 121 + phase
|
||||
135 — the ignore list (replace), the hidden-folders flag (phase 105),
|
||||
the token (phase 121 tri-state), and the branch (phase 135 tri-state)
|
||||
— each optional, present-wins: 404 unknown id; ``ignore_paths`` when
|
||||
present is normalized + A4-validated with fixed-detail 422s and
|
||||
REPLACES the row's list wholesale — an empty list clears all;
|
||||
``include_hidden`` when present sets the flag; ``token`` and ``branch``
|
||||
when present are tri-state — absent/None = no change, non-empty =
|
||||
set/replace (``branch`` re-validated, ``local`` rows 422), empty
|
||||
string = clear (stored NULL); all absent → 200 no-op; 200 → the
|
||||
``GitSourceOut`` shape),
|
||||
``POST /upload`` (phase 49, backgrounded in phase
|
||||
64 task 03, scan deferred in phase 90 — admin archive upload: the
|
||||
@@ -115,7 +129,11 @@ from app.rag.archive_upload import (
|
||||
swap_in,
|
||||
unpack_archive,
|
||||
)
|
||||
from app.rag.git_sources import normalize_credential, sanitize_url
|
||||
from app.rag.git_sources import (
|
||||
normalize_credential,
|
||||
sanitize_url,
|
||||
validate_branch_name,
|
||||
)
|
||||
from app.rag.importer import normalize_ignore_path
|
||||
from app.rag.llm import LLMClient
|
||||
from app.rag.overview import regenerate_overview
|
||||
@@ -196,9 +214,12 @@ _upload_status = UploadStatus()
|
||||
|
||||
#: Accepted git URL shapes — the trimmed URL must *start* with one of them.
|
||||
#: Covers the phase-28 real URLs (HTTPS + ``git@`` SSH); scp-style
|
||||
#: ``host:repo`` is deliberately rejected (422). ASSUMPTION (task 02): the
|
||||
#: accepted shapes are exactly these four prefixes.
|
||||
URL_RE = re.compile(r"^(https?://|ssh://|git@)")
|
||||
#: ``host:repo`` is deliberately rejected (422). ``file://`` (phase 135,
|
||||
#: task 05) is the deterministic local-transport source the E2E fixture
|
||||
#: repos use — a real ``git clone`` of a local repo (the phase-28/32/107
|
||||
#: transport-true E2E idiom, promoted from the ``BOR_GIT_SOURCES`` env
|
||||
#: fallback to a stored row the UI can add).
|
||||
URL_RE = re.compile(r"^(https?://|ssh://|file://|git@)")
|
||||
|
||||
#: Phase 89, A4 — the per-source ignore-list limits, enforced in
|
||||
#: :func:`_validate_ignore_paths` (shared by POST and PATCH): at most
|
||||
@@ -219,13 +240,14 @@ def list_git_sources(
|
||||
same-timestamp inserts) with ``from_env: false`` — each row carries
|
||||
its ``kind``, its ``ignore_paths`` (phase 89 — the stored,
|
||||
normalized list; ``or []`` guards a row that predated the column),
|
||||
its ``include_hidden`` (phase 105 — the stored flag), and, for
|
||||
local rows, the stored ``path`` (git rows and env rows report
|
||||
``path: null``); while the table is empty, the
|
||||
its ``include_hidden`` (phase 105 — the stored flag), its
|
||||
``branch`` (phase 135 — the stored value; NULL = the repo's default
|
||||
branch), and, for local rows, the stored ``path`` (git rows and env
|
||||
rows report ``path: null``); while the table is empty, the
|
||||
``BOR_GIT_SOURCES`` env URLs as git rows (the env fallback is
|
||||
git-only) with null ``id``/``added_at``, ``ignore_paths: []`` and
|
||||
``include_hidden: False`` (no DB row to store a list or a flag on),
|
||||
and ``from_env: true``.
|
||||
git-only) with null ``id``/``added_at``, ``ignore_paths: []``,
|
||||
``include_hidden: False`` and ``branch: None`` (no DB row to store a
|
||||
list, a flag or a branch on), and ``from_env: true``.
|
||||
|
||||
Every URL is masked on the way out (phase 121, LOCKED A2 —
|
||||
:func:`sanitize_url`): an env value or a legacy stored URL may
|
||||
@@ -248,6 +270,7 @@ def list_git_sources(
|
||||
added_at=row.added_at,
|
||||
ignore_paths=row.ignore_paths or [],
|
||||
include_hidden=row.include_hidden,
|
||||
branch=row.branch, # phase 135: NULL = default branch
|
||||
)
|
||||
for row in rows
|
||||
],
|
||||
@@ -263,6 +286,7 @@ def list_git_sources(
|
||||
added_at=None,
|
||||
ignore_paths=[],
|
||||
include_hidden=False,
|
||||
branch=None, # phase 135: no DB row to store a branch on
|
||||
)
|
||||
for url in get_settings().git_source_list
|
||||
],
|
||||
@@ -310,6 +334,16 @@ def create_git_source(
|
||||
credential in the token column, an explicit ``token`` wins over
|
||||
the embedded one (LOCKED A6), and the duplicate check runs on the
|
||||
bare URL.
|
||||
|
||||
``branch`` (phase 135) — git rows only: absent/None → stored NULL
|
||||
(the repo's DEFAULT branch — the pre-phase-135 behavior), a
|
||||
trimmed-EMPTY value likewise stores NULL (at CREATE, empty is
|
||||
absent — the split from PATCH's clear leg, see :class:`GitSourceIn`),
|
||||
non-empty → validated by :func:`validate_branch_name` and stored
|
||||
(a ``ValueError`` → 422 ``not a valid branch name`` — FIXED detail,
|
||||
the input is never echoed). A ``branch`` on a LOCAL row is 422
|
||||
``branch is only valid for git sources`` (local rows are walked,
|
||||
not cloned).
|
||||
"""
|
||||
row = _create_git_row(payload, db) if payload.kind == "git" else _create_local_row(payload, db)
|
||||
return GitSourceOut(
|
||||
@@ -318,6 +352,7 @@ def create_git_source(
|
||||
added_at=row.added_at,
|
||||
ignore_paths=row.ignore_paths,
|
||||
include_hidden=row.include_hidden,
|
||||
branch=row.branch, # phase 135: NULL = default branch
|
||||
)
|
||||
|
||||
|
||||
@@ -355,16 +390,21 @@ def _commit_new(row: GitSource, duplicate_detail: str, db: Session) -> GitSource
|
||||
|
||||
|
||||
def _create_git_row(payload: GitSourceIn, db: Session) -> GitSource:
|
||||
"""``kind=git`` — the phase-35 URL contract, unchanged (A10: no
|
||||
credential echo, so every detail is a fixed string)."""
|
||||
"""``kind=git`` — the phase-35 URL contract, extended with the
|
||||
``file://`` local transport (phase 135, task 05 — the E2E fixture
|
||||
repos the UI adds end to end); A10: no credential echo, so every
|
||||
detail is a fixed string."""
|
||||
if payload.path is not None:
|
||||
raise HTTPException(status_code=422, detail="a git source takes a url, not a path")
|
||||
if payload.url is None:
|
||||
raise HTTPException(status_code=422, detail="a git source requires a url")
|
||||
url = payload.url
|
||||
if not URL_RE.match(url):
|
||||
# The detail lists the accepted prefixes (fixed string — the
|
||||
# input is never echoed; phase 135 task 05 added file://).
|
||||
raise HTTPException(
|
||||
status_code=422, detail="not a valid git URL (expected https://, ssh:// or git@…)"
|
||||
status_code=422,
|
||||
detail="not a valid git URL (expected https://, ssh://, file:// or git@…)",
|
||||
)
|
||||
# Phase 121 (task 02, LOCKED A6): normalize the credential — an
|
||||
# old-style embedded ``user:pass@`` URL is stored BARE and the
|
||||
@@ -375,6 +415,18 @@ def _create_git_row(payload: GitSourceIn, db: Session) -> GitSource:
|
||||
url, effective_token = normalize_credential(url, payload.token)
|
||||
if db.scalar(select(GitSource).where(GitSource.url == url)) is not None:
|
||||
raise HTTPException(status_code=409, detail="a git source with this URL already exists")
|
||||
# Phase 135: the branch pin (ASSUMPTION B2/B3) — absent or
|
||||
# trimmed-EMPTY stores NULL (the repo's default branch, the
|
||||
# pre-phase-135 behavior); non-empty is validated at the boundary
|
||||
# with the FIXED-detail 422 (the raw name never echoes).
|
||||
branch = (payload.branch or "").strip()
|
||||
if branch:
|
||||
try:
|
||||
branch = validate_branch_name(branch)
|
||||
except ValueError:
|
||||
raise HTTPException(
|
||||
status_code=422, detail="not a valid branch name"
|
||||
) from None
|
||||
return _commit_new(
|
||||
GitSource(
|
||||
url=url,
|
||||
@@ -382,6 +434,7 @@ def _create_git_row(payload: GitSourceIn, db: Session) -> GitSource:
|
||||
ignore_paths=_validate_ignore_paths(payload.ignore_paths),
|
||||
include_hidden=bool(payload.include_hidden),
|
||||
token=effective_token or None,
|
||||
branch=branch or None,
|
||||
),
|
||||
"a git source with this URL already exists",
|
||||
db,
|
||||
@@ -391,9 +444,17 @@ def _create_git_row(payload: GitSourceIn, db: Session) -> GitSource:
|
||||
def _create_local_row(payload: GitSourceIn, db: Session) -> GitSource:
|
||||
"""``kind=local`` — fail-loud add-time validation (phase 38):
|
||||
trimmed → ``expanduser()`` → absolute + existing directory, else 422
|
||||
naming the path (not a secret, unlike a git URL)."""
|
||||
naming the path (not a secret, unlike a git URL). A ``branch`` on a
|
||||
local row (phase 135) is 422 — local rows are walked, not cloned."""
|
||||
if payload.url is not None:
|
||||
raise HTTPException(status_code=422, detail="a local source takes a path, not a url")
|
||||
if (payload.branch or "").strip():
|
||||
# Phase 135: local rows are walked, not cloned — a branch pin
|
||||
# has no meaning for them (the kind-field rules' precedent:
|
||||
# fixed detail, no input echo).
|
||||
raise HTTPException(
|
||||
status_code=422, detail="branch is only valid for git sources"
|
||||
)
|
||||
if payload.path is None:
|
||||
raise HTTPException(status_code=422, detail="a local source requires a path")
|
||||
expanded = Path(payload.path).expanduser()
|
||||
@@ -432,26 +493,37 @@ def patch_git_source(
|
||||
payload: GitSourcePatchIn,
|
||||
db: Session = Depends(get_db), # noqa: B008
|
||||
) -> GitSourceOut:
|
||||
"""Edit one source's ignore list, hidden-folders flag, and/or
|
||||
private-repo token.
|
||||
"""Edit one source's ignore list, hidden-folders flag, private-
|
||||
repo token, and/or branch.
|
||||
|
||||
Phase 89 A5 (ignore list) + phase 105 (the flag) + phase 121
|
||||
(the token): 404 unknown id; each PRESENT body field applies
|
||||
independently — ``ignore_paths`` REPLACES the list (normalized +
|
||||
A4-validated, fixed 422 details); ``include_hidden`` sets the
|
||||
flag; ``token`` is TRI-STATE (LOCKED A2): absent/None = no change
|
||||
(the row's stored credential survives an edit that does not touch
|
||||
the masked field), non-empty = replace, empty string = clear
|
||||
(stored NULL). A PRESENT token also re-normalizes the (current
|
||||
url, new token) pair with the POST write-path rules — a legacy
|
||||
embedded-token URL gets its userinfo stripped (moved to the
|
||||
column) the first time an explicit credential is written; a clean
|
||||
URL comes back untouched. The 409 backstop: re-normalizing can
|
||||
make the stored URL collide with another row's bare URL (a
|
||||
legacy ``user:pass@`` row and a bare row for the same repo) — the
|
||||
unique index yields the generic 409, never a 500. Returns the
|
||||
updated row's public shape (id, url — masked, added_at,
|
||||
ignore_paths, include_hidden); the token is never echoed.
|
||||
(the token) + phase 135 (the branch): 404 unknown id; each PRESENT
|
||||
body field applies independently — ``ignore_paths`` REPLACES the
|
||||
list (normalized + A4-validated, fixed 422 details);
|
||||
``include_hidden`` sets the flag; ``token`` is TRI-STATE (LOCKED
|
||||
A2): absent/None = no change (the row's stored credential survives
|
||||
an edit that does not touch the masked field), non-empty = replace,
|
||||
empty string = clear (stored NULL). A PRESENT token also
|
||||
re-normalizes the (current url, new token) pair with the POST
|
||||
write-path rules — a legacy embedded-token URL gets its userinfo
|
||||
stripped (moved to the column) the first time an explicit
|
||||
credential is written; a clean URL comes back untouched. The 409
|
||||
backstop: re-normalizing can make the stored URL collide with
|
||||
another row's bare URL (a legacy ``user:pass@`` row and a bare row
|
||||
for the same repo) — the unique index yields the generic 409,
|
||||
never a 500. ``branch`` (phase 135) is an independent PRESENT-
|
||||
wins field with the token's EXACT tri-state semantics:
|
||||
absent/None = no change (the row's stored branch survives an edit
|
||||
that does not touch it), non-empty = set (re-validated by
|
||||
:func:`validate_branch_name` — a ``ValueError`` is the FIXED-detail
|
||||
422 ``not a valid branch name``, input never echoed), empty string
|
||||
= clear (stored NULL = the repo's default branch). A PRESENT
|
||||
branch on a LOCAL row is 422 ``branch is only valid for git
|
||||
sources`` (local rows are walked, not cloned). Branch never
|
||||
touches a unique index — no Integrity interaction, no 409 path.
|
||||
Returns the updated row's public shape (id, url — masked,
|
||||
added_at, ignore_paths, include_hidden, branch); the token is
|
||||
never echoed.
|
||||
"""
|
||||
row = db.get(GitSource, source_id)
|
||||
if row is None:
|
||||
@@ -467,6 +539,24 @@ def patch_git_source(
|
||||
# bare + column credential.
|
||||
row.url, effective = normalize_credential(row.url, payload.token)
|
||||
row.token = effective or None
|
||||
if payload.branch is not None:
|
||||
# Phase 135: the branch tri-state (the token's exact semantics
|
||||
# — see the docstring). Local rows are walked, not cloned — a
|
||||
# branch pin is meaningless there (the same fixed 422 as POST).
|
||||
if row.kind != "git":
|
||||
raise HTTPException(
|
||||
status_code=422, detail="branch is only valid for git sources"
|
||||
)
|
||||
branch = payload.branch.strip()
|
||||
if not branch:
|
||||
row.branch = None # the tri-state's clear leg (stored NULL)
|
||||
else:
|
||||
try:
|
||||
row.branch = validate_branch_name(branch)
|
||||
except ValueError:
|
||||
raise HTTPException(
|
||||
status_code=422, detail="not a valid branch name"
|
||||
) from None
|
||||
try:
|
||||
db.commit()
|
||||
except IntegrityError:
|
||||
@@ -485,6 +575,7 @@ def patch_git_source(
|
||||
added_at=row.added_at,
|
||||
ignore_paths=row.ignore_paths,
|
||||
include_hidden=row.include_hidden,
|
||||
branch=row.branch, # phase 135: NULL = default branch
|
||||
)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user