phase: 135_git_source_branch
Build and Push Containers / build-and-push-app (push) Successful in 2m34s
Build and Push Containers / build-and-push-db (push) Successful in 14s

All verification complete. No edits were needed by this pass — the phase work and docs were already consistent and green.

**Phase 135 final verification pass — report**

- Verified all 5 tasks' work present & correct: migration 0024 (reversible), `GitSource.branch` + 4 schema shapes, `validate_branch_name` (full check order per design), `clone_or_pull(…, branch)` (HEAD guard first, `--branch` clone, fetch+ff merge), both call sites pass `row.branch`, UI (add-form field, editor field, row tag), `file://` URL transport.
- Completion criteria: (1) branch-mirrored through Sync + CLI — **pass** (E2E + integration); (2) `branch=None` byte-identical — **pass** (`test_branch_none_argv_is_byte_identical_pre_phase_sequence` green); (3) fixed 422s / local-row 422 / tri-state — **pass**; (4) mismatch fails loudly with remove-and-re-add — **pass** (E2E); (5) coverage 99% >90%, E2E isolated green — **pass**; (6) no regressions — **pass**.
- Tests: `uv run pytest --cov=app --cov-report=term-missing` → 3223 passed, app/ 99%; `uv run pytest tests/e2e/test_git_source_branch.py -v --no-cov` → 4 passed (16.4 s); `uv run ruff check .` clean; `uv run pyright` → 0 errors, 0 warnings.
- Documentation: README.md reviewed in full — phase-135 branch section accurate; the phase's README diff also correctly fixed a pre-existing stale line (shallow `--depth 1` clone vs full-history, phase 107). Routes, env defaults, feature lists verified against code. AGENTS.md — all 10 rules + quick-reference verified (referenced tests, PLAN.md, phases dirs, commands all accurate). **Verified consistent — no changes needed.**
- No live-infrastructure changes (dev DB already running and healthy; no live steps in this phase).
- Next pending phase: `.agents/phases/todo/136_force_push_reset`.
This commit is contained in:
2026-09-28 21:15:03 -04:00
parent fcac876d40
commit d3f8c7f2c8
50 changed files with 3955 additions and 140 deletions
+94 -28
View File
@@ -1,9 +1,17 @@
"""Git source sync for import_docs (phase 28).
clone_or_pull(url, dest) clones ``url`` into ``dest`` with FULL history
the first time, or — for an existing checkout — unshallows a shallow
one first (``git fetch --unshallow``, the one-time self-heal) and then
fast-forwards it with ``git pull --ff-only``.
clone_or_pull(url, dest, branch=None) clones ``url`` into ``dest`` with
FULL history the first time — ``git clone --branch <b>`` when a branch
is pinned (phase 135: the source's branch is checked out from the
first commit), plain ``git clone`` when not — or, for an existing
checkout, runs the branch guard first when a branch is pinned (a
checkout on any other branch fails loudly — the source must be removed
and re-added to switch, phase 135), unshallows a shallow one first
(``git fetch --unshallow``, the one-time self-heal), and then
fast-forwards it — ``git pull --ff-only`` with no branch, or the
single-fetch ``git fetch origin <b>`` + ``git merge --ff-only
FETCH_HEAD`` of the pinned ref with one (``branch=None`` keeps the
pre-phase argv byte-identical — the unit argv pins are the proof).
Auth: nothing special — an ``https://…`` URL uses the OS credential
helper / prompts; a ``git@host:repo.git`` URL uses the machine's SSH key.
@@ -12,11 +20,16 @@ used.
This module is the only place the ``git`` CLI is invoked (A11: stdlib
``subprocess`` only, no new packages) — every git command goes through
:func:`run_git`: the clone, the shallow probe, the one-time
``git fetch --unshallow``, and the ``git pull --ff-only`` in
:func:`clone_or_pull`, the per-file last-commit-date walk in
:func:`file_commit_dates` (phase 106), and the docs-push sequence in
:mod:`app.core.docs_push` (phase 59).
:func:`run_git`: the clone (with ``--branch`` when a branch is
pinned, phase 135), the branch guard's ``git rev-parse --abbrev-ref
HEAD`` probe, the shallow probe, the one-time ``git fetch
--unshallow``, the pinned branch's ``git fetch origin <b>`` +
``git merge --ff-only FETCH_HEAD`` pair, and the ``git pull
--ff-only`` in :func:`clone_or_pull`; the per-file last-commit-date
walk in :func:`file_commit_dates` (phase 106 — it walks the
checkout's own log, so a branch checkout yields the BRANCH's history:
the head-of-branch semantics phase 135 pins, untouched); and the
docs-push sequence in :mod:`app.core.docs_push` (phase 59).
Per-file last-commit dates (phase 106 D2; phase 107 D11 supersedes
phase 106 D10) — behavior verified against scratch and live repos
@@ -25,11 +38,13 @@ phase 106 D10) — behavior verified against scratch and live repos
* every :func:`clone_or_pull` checkout is FULL history for EVERY
transport (https/ssh/``file://``/local-path): a fresh clone carries
no ``--depth`` (D11 — D10's ``--depth 1`` shallow URL clones are
gone), and an existing shallow checkout (made pre-phase 107 — i.e.
every deployed one, live + dev included) is probed with ``git
gone; ``--branch`` clones, phase 135, included), and an existing
shallow checkout (made pre-phase 107 — i.e. every deployed one,
live + dev included) is probed with ``git
rev-parse --is-shallow-repository`` and, while shallow, self-healed
with the ONE-TIME ``git fetch --unshallow`` before the usual ``git
pull --ff-only`` — no re-clone;
pull --ff-only`` (no branch) / ``git merge --ff-only FETCH_HEAD``
(pinned branch) — no re-clone;
* on a full-history checkout :func:`file_commit_dates` yields the
TRUE per-file last-commit date for ALL git sources. The old
shallow-clone behavior was the bug the owner reported 2026-09-16:
@@ -57,45 +72,96 @@ class GitSyncError(RuntimeError):
"""A git clone/pull failed (or git is missing); carries git's stderr."""
def clone_or_pull(url: str, dest: Path | str) -> Path:
def clone_or_pull(url: str, dest: Path | str, branch: str | None = None) -> Path:
"""Clone ``url`` into ``dest`` (full history, first run) or fast-forward it.
- dest without a ``.git`` (or absent) → ``git clone url dest`` (full
history for every transport — phase 107 D11: no ``--depth``, so a
URL-transport checkout carries the whole commit log, not just the
tip).
- dest with a ``.git`` → probe ``git rev-parse
tip); with a pinned ``branch`` → ``git clone --branch <b> url
dest`` (phase 135: the source's branch is checked out from the
first commit — no post-clone checkout; ``branch=None`` keeps the
two-argument argv byte-identical).
- dest with a ``.git`` and no ``branch`` → probe ``git rev-parse
--is-shallow-repository``; while shallow, ``git fetch
--unshallow`` (the ONE-TIME self-heal for checkouts made shallow
pre-phase 107 — the next sync of a deployed checkout becomes
full-history without a re-clone), then ``git pull --ff-only``
(refuses to merge unrelated histories — a broken checkout fails
loudly rather than producing a dirty index).
- dest with a ``.git`` and a pinned ``branch`` (phase 135), in
order: the HEAD GUARD first — ``git rev-parse --abbrev-ref
HEAD``; a checkout sitting on any other branch raises the
remove-and-re-add :class:`GitSyncError` (naming the checkout and
BOTH branch names — never the URL, so no credential exposure)
BEFORE any fetch or merge (no silent branch switch: the checkout
may serve other rows' documents, and a switch would silently
re-map the source's file set); a matching checkout then runs the
SAME shallow probe + one-time unshallow self-heal, and pulls the
pinned ref with the single-fetch sequence ``git fetch origin
<b>`` + ``git merge --ff-only FETCH_HEAD`` (a non-ff merge still
fails loudly HERE — the force-push reset fallback is phase 136's
extension of this sequence, not this phase's).
Returns the destination path. Raises :class:`GitSyncError` when git is
missing or a git invocation exits non-zero (with git's stderr in the
message, so the caller can name the failing repo + reason). D12
fail-loud: a failed probe/unshallow/pull propagates exactly like a
clone failure — never a silent fallback to tip dates or mtimes (a
continued shallow checkout would silently re-serve the uniform tip
date, i.e. the bug D11 fixes).
fail-loud: a failed probe/unshallow/pull/merge propagates exactly
like a clone failure — never a silent fallback to tip dates or
mtimes (a continued shallow checkout would silently re-serve the
uniform tip date, i.e. the bug D11 fixes) and never a silent branch
switch (phase 135 — the fail-loud guard, above).
"""
dest = Path(dest)
if not dest.exists() or not (dest / ".git").exists():
dest.parent.mkdir(parents=True, exist_ok=True)
run_git(["git", "clone", url, str(dest)], cwd=dest.parent)
else:
shallow = (
run_git(["git", "rev-parse", "--is-shallow-repository"], cwd=dest)
.strip()
== "true"
)
if shallow:
run_git(["git", "fetch", "--unshallow"], cwd=dest)
argv = ["git", "clone"]
if branch is not None:
argv += ["--branch", branch]
argv += [url, str(dest)]
run_git(argv, cwd=dest.parent)
elif branch is None:
_unshallow_if_shallow(dest)
run_git(["git", "pull", "--ff-only"], cwd=dest)
else:
# Phase 135 HEAD guard (first — before the probe, any fetch,
# or any merge): a mismatched checkout fails loudly. The
# message names the checkout and both branch names; it never
# carries the URL (the only place a token could ride).
current = run_git(["git", "rev-parse", "--abbrev-ref", "HEAD"], cwd=dest).strip()
if current != branch:
raise GitSyncError(
f"checkout {dest} is on branch '{current}' but the source "
f"specifies branch '{branch}' — remove and re-add the "
"source to switch branches"
)
_unshallow_if_shallow(dest)
# The single-fetch ff pull of the pinned ref (phase 136
# extends this sequence with its reset fallback).
run_git(["git", "fetch", "origin", branch], cwd=dest)
run_git(["git", "merge", "--ff-only", "FETCH_HEAD"], cwd=dest)
return dest
def _unshallow_if_shallow(dest: Path) -> None:
"""The one-time shallow self-heal (phase 107 D11).
Probe ``git rev-parse --is-shallow-repository``; while shallow,
``git fetch --unshallow`` (a pre-phase-107 deployed checkout
becomes full-history on its next sync without a re-clone). BOTH
pull paths of :func:`clone_or_pull` (no branch / pinned branch)
run this before their pull — the argv sequence is the pre-phase
one in both (the unit argv pins are the proof).
"""
shallow = (
run_git(["git", "rev-parse", "--is-shallow-repository"], cwd=dest)
.strip()
== "true"
)
if shallow:
run_git(["git", "fetch", "--unshallow"], cwd=dest)
def _parse_commit_dates(output: str) -> dict[str, datetime]:
"""Parse ``git log --name-only --format=@@%cI`` output (newest first).