phase: 135_git_source_branch
All verification complete. No edits were needed by this pass — the phase work and docs were already consistent and green. **Phase 135 final verification pass — report** - Verified all 5 tasks' work present & correct: migration 0024 (reversible), `GitSource.branch` + 4 schema shapes, `validate_branch_name` (full check order per design), `clone_or_pull(…, branch)` (HEAD guard first, `--branch` clone, fetch+ff merge), both call sites pass `row.branch`, UI (add-form field, editor field, row tag), `file://` URL transport. - Completion criteria: (1) branch-mirrored through Sync + CLI — **pass** (E2E + integration); (2) `branch=None` byte-identical — **pass** (`test_branch_none_argv_is_byte_identical_pre_phase_sequence` green); (3) fixed 422s / local-row 422 / tri-state — **pass**; (4) mismatch fails loudly with remove-and-re-add — **pass** (E2E); (5) coverage 99% >90%, E2E isolated green — **pass**; (6) no regressions — **pass**. - Tests: `uv run pytest --cov=app --cov-report=term-missing` → 3223 passed, app/ 99%; `uv run pytest tests/e2e/test_git_source_branch.py -v --no-cov` → 4 passed (16.4 s); `uv run ruff check .` clean; `uv run pyright` → 0 errors, 0 warnings. - Documentation: README.md reviewed in full — phase-135 branch section accurate; the phase's README diff also correctly fixed a pre-existing stale line (shallow `--depth 1` clone vs full-history, phase 107). Routes, env defaults, feature lists verified against code. AGENTS.md — all 10 rules + quick-reference verified (referenced tests, PLAN.md, phases dirs, commands all accurate). **Verified consistent — no changes needed.** - No live-infrastructure changes (dev DB already running and healthy; no live steps in this phase). - Next pending phase: `.agents/phases/todo/136_force_push_reset`.
This commit is contained in:
+94
-28
@@ -1,9 +1,17 @@
|
||||
"""Git source sync for import_docs (phase 28).
|
||||
|
||||
clone_or_pull(url, dest) clones ``url`` into ``dest`` with FULL history
|
||||
the first time, or — for an existing checkout — unshallows a shallow
|
||||
one first (``git fetch --unshallow``, the one-time self-heal) and then
|
||||
fast-forwards it with ``git pull --ff-only``.
|
||||
clone_or_pull(url, dest, branch=None) clones ``url`` into ``dest`` with
|
||||
FULL history the first time — ``git clone --branch <b>`` when a branch
|
||||
is pinned (phase 135: the source's branch is checked out from the
|
||||
first commit), plain ``git clone`` when not — or, for an existing
|
||||
checkout, runs the branch guard first when a branch is pinned (a
|
||||
checkout on any other branch fails loudly — the source must be removed
|
||||
and re-added to switch, phase 135), unshallows a shallow one first
|
||||
(``git fetch --unshallow``, the one-time self-heal), and then
|
||||
fast-forwards it — ``git pull --ff-only`` with no branch, or the
|
||||
single-fetch ``git fetch origin <b>`` + ``git merge --ff-only
|
||||
FETCH_HEAD`` of the pinned ref with one (``branch=None`` keeps the
|
||||
pre-phase argv byte-identical — the unit argv pins are the proof).
|
||||
|
||||
Auth: nothing special — an ``https://…`` URL uses the OS credential
|
||||
helper / prompts; a ``git@host:repo.git`` URL uses the machine's SSH key.
|
||||
@@ -12,11 +20,16 @@ used.
|
||||
|
||||
This module is the only place the ``git`` CLI is invoked (A11: stdlib
|
||||
``subprocess`` only, no new packages) — every git command goes through
|
||||
:func:`run_git`: the clone, the shallow probe, the one-time
|
||||
``git fetch --unshallow``, and the ``git pull --ff-only`` in
|
||||
:func:`clone_or_pull`, the per-file last-commit-date walk in
|
||||
:func:`file_commit_dates` (phase 106), and the docs-push sequence in
|
||||
:mod:`app.core.docs_push` (phase 59).
|
||||
:func:`run_git`: the clone (with ``--branch`` when a branch is
|
||||
pinned, phase 135), the branch guard's ``git rev-parse --abbrev-ref
|
||||
HEAD`` probe, the shallow probe, the one-time ``git fetch
|
||||
--unshallow``, the pinned branch's ``git fetch origin <b>`` +
|
||||
``git merge --ff-only FETCH_HEAD`` pair, and the ``git pull
|
||||
--ff-only`` in :func:`clone_or_pull`; the per-file last-commit-date
|
||||
walk in :func:`file_commit_dates` (phase 106 — it walks the
|
||||
checkout's own log, so a branch checkout yields the BRANCH's history:
|
||||
the head-of-branch semantics phase 135 pins, untouched); and the
|
||||
docs-push sequence in :mod:`app.core.docs_push` (phase 59).
|
||||
|
||||
Per-file last-commit dates (phase 106 D2; phase 107 D11 supersedes
|
||||
phase 106 D10) — behavior verified against scratch and live repos
|
||||
@@ -25,11 +38,13 @@ phase 106 D10) — behavior verified against scratch and live repos
|
||||
* every :func:`clone_or_pull` checkout is FULL history for EVERY
|
||||
transport (https/ssh/``file://``/local-path): a fresh clone carries
|
||||
no ``--depth`` (D11 — D10's ``--depth 1`` shallow URL clones are
|
||||
gone), and an existing shallow checkout (made pre-phase 107 — i.e.
|
||||
every deployed one, live + dev included) is probed with ``git
|
||||
gone; ``--branch`` clones, phase 135, included), and an existing
|
||||
shallow checkout (made pre-phase 107 — i.e. every deployed one,
|
||||
live + dev included) is probed with ``git
|
||||
rev-parse --is-shallow-repository`` and, while shallow, self-healed
|
||||
with the ONE-TIME ``git fetch --unshallow`` before the usual ``git
|
||||
pull --ff-only`` — no re-clone;
|
||||
pull --ff-only`` (no branch) / ``git merge --ff-only FETCH_HEAD``
|
||||
(pinned branch) — no re-clone;
|
||||
* on a full-history checkout :func:`file_commit_dates` yields the
|
||||
TRUE per-file last-commit date for ALL git sources. The old
|
||||
shallow-clone behavior was the bug the owner reported 2026-09-16:
|
||||
@@ -57,45 +72,96 @@ class GitSyncError(RuntimeError):
|
||||
"""A git clone/pull failed (or git is missing); carries git's stderr."""
|
||||
|
||||
|
||||
def clone_or_pull(url: str, dest: Path | str) -> Path:
|
||||
def clone_or_pull(url: str, dest: Path | str, branch: str | None = None) -> Path:
|
||||
"""Clone ``url`` into ``dest`` (full history, first run) or fast-forward it.
|
||||
|
||||
- dest without a ``.git`` (or absent) → ``git clone url dest`` (full
|
||||
history for every transport — phase 107 D11: no ``--depth``, so a
|
||||
URL-transport checkout carries the whole commit log, not just the
|
||||
tip).
|
||||
- dest with a ``.git`` → probe ``git rev-parse
|
||||
tip); with a pinned ``branch`` → ``git clone --branch <b> url
|
||||
dest`` (phase 135: the source's branch is checked out from the
|
||||
first commit — no post-clone checkout; ``branch=None`` keeps the
|
||||
two-argument argv byte-identical).
|
||||
- dest with a ``.git`` and no ``branch`` → probe ``git rev-parse
|
||||
--is-shallow-repository``; while shallow, ``git fetch
|
||||
--unshallow`` (the ONE-TIME self-heal for checkouts made shallow
|
||||
pre-phase 107 — the next sync of a deployed checkout becomes
|
||||
full-history without a re-clone), then ``git pull --ff-only``
|
||||
(refuses to merge unrelated histories — a broken checkout fails
|
||||
loudly rather than producing a dirty index).
|
||||
- dest with a ``.git`` and a pinned ``branch`` (phase 135), in
|
||||
order: the HEAD GUARD first — ``git rev-parse --abbrev-ref
|
||||
HEAD``; a checkout sitting on any other branch raises the
|
||||
remove-and-re-add :class:`GitSyncError` (naming the checkout and
|
||||
BOTH branch names — never the URL, so no credential exposure)
|
||||
BEFORE any fetch or merge (no silent branch switch: the checkout
|
||||
may serve other rows' documents, and a switch would silently
|
||||
re-map the source's file set); a matching checkout then runs the
|
||||
SAME shallow probe + one-time unshallow self-heal, and pulls the
|
||||
pinned ref with the single-fetch sequence ``git fetch origin
|
||||
<b>`` + ``git merge --ff-only FETCH_HEAD`` (a non-ff merge still
|
||||
fails loudly HERE — the force-push reset fallback is phase 136's
|
||||
extension of this sequence, not this phase's).
|
||||
|
||||
Returns the destination path. Raises :class:`GitSyncError` when git is
|
||||
missing or a git invocation exits non-zero (with git's stderr in the
|
||||
message, so the caller can name the failing repo + reason). D12
|
||||
fail-loud: a failed probe/unshallow/pull propagates exactly like a
|
||||
clone failure — never a silent fallback to tip dates or mtimes (a
|
||||
continued shallow checkout would silently re-serve the uniform tip
|
||||
date, i.e. the bug D11 fixes).
|
||||
fail-loud: a failed probe/unshallow/pull/merge propagates exactly
|
||||
like a clone failure — never a silent fallback to tip dates or
|
||||
mtimes (a continued shallow checkout would silently re-serve the
|
||||
uniform tip date, i.e. the bug D11 fixes) and never a silent branch
|
||||
switch (phase 135 — the fail-loud guard, above).
|
||||
"""
|
||||
dest = Path(dest)
|
||||
if not dest.exists() or not (dest / ".git").exists():
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
run_git(["git", "clone", url, str(dest)], cwd=dest.parent)
|
||||
else:
|
||||
shallow = (
|
||||
run_git(["git", "rev-parse", "--is-shallow-repository"], cwd=dest)
|
||||
.strip()
|
||||
== "true"
|
||||
)
|
||||
if shallow:
|
||||
run_git(["git", "fetch", "--unshallow"], cwd=dest)
|
||||
argv = ["git", "clone"]
|
||||
if branch is not None:
|
||||
argv += ["--branch", branch]
|
||||
argv += [url, str(dest)]
|
||||
run_git(argv, cwd=dest.parent)
|
||||
elif branch is None:
|
||||
_unshallow_if_shallow(dest)
|
||||
run_git(["git", "pull", "--ff-only"], cwd=dest)
|
||||
else:
|
||||
# Phase 135 HEAD guard (first — before the probe, any fetch,
|
||||
# or any merge): a mismatched checkout fails loudly. The
|
||||
# message names the checkout and both branch names; it never
|
||||
# carries the URL (the only place a token could ride).
|
||||
current = run_git(["git", "rev-parse", "--abbrev-ref", "HEAD"], cwd=dest).strip()
|
||||
if current != branch:
|
||||
raise GitSyncError(
|
||||
f"checkout {dest} is on branch '{current}' but the source "
|
||||
f"specifies branch '{branch}' — remove and re-add the "
|
||||
"source to switch branches"
|
||||
)
|
||||
_unshallow_if_shallow(dest)
|
||||
# The single-fetch ff pull of the pinned ref (phase 136
|
||||
# extends this sequence with its reset fallback).
|
||||
run_git(["git", "fetch", "origin", branch], cwd=dest)
|
||||
run_git(["git", "merge", "--ff-only", "FETCH_HEAD"], cwd=dest)
|
||||
return dest
|
||||
|
||||
|
||||
def _unshallow_if_shallow(dest: Path) -> None:
|
||||
"""The one-time shallow self-heal (phase 107 D11).
|
||||
|
||||
Probe ``git rev-parse --is-shallow-repository``; while shallow,
|
||||
``git fetch --unshallow`` (a pre-phase-107 deployed checkout
|
||||
becomes full-history on its next sync without a re-clone). BOTH
|
||||
pull paths of :func:`clone_or_pull` (no branch / pinned branch)
|
||||
run this before their pull — the argv sequence is the pre-phase
|
||||
one in both (the unit argv pins are the proof).
|
||||
"""
|
||||
shallow = (
|
||||
run_git(["git", "rev-parse", "--is-shallow-repository"], cwd=dest)
|
||||
.strip()
|
||||
== "true"
|
||||
)
|
||||
if shallow:
|
||||
run_git(["git", "fetch", "--unshallow"], cwd=dest)
|
||||
|
||||
|
||||
def _parse_commit_dates(output: str) -> dict[str, datetime]:
|
||||
"""Parse ``git log --name-only --format=@@%cI`` output (newest first).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user