diff --git a/.agents/phases/todo/107_git_full_history_dates/00_phase.md b/.agents/phases/todo/107_git_full_history_dates/00_phase.md new file mode 100644 index 0000000..b0b66c7 --- /dev/null +++ b/.agents/phases/todo/107_git_full_history_dates/00_phase.md @@ -0,0 +1,58 @@ +# Phase 107 — True per-file document dates for URL git sources: full-history checkouts (revisits phase 106 D10) + +**Source:** Owner bug report 2026-09-16 (chat): after the phase-106 re-sync of brain.reeseapps.com, the git source `https://gitea.reeseapps.com/services/homelab.git` shows `active/container_bifrost` created **8/16/2026** — "completely wrong, container_bifrost is > 6 months old at this point"; "This is a git source, so it should be easy to tell when a document was last edit[ed]". + +**Story:** n/a (owner bug report, phase-106 follow-up — the phase's E2E suite proves the fix end to end). + +**Context / verified root cause (2026-09-16, scratch + dev checkouts of the live homelab repo):** `clone_or_pull` (`scripts/git_sync.py` L63) clones URL-transport sources with `--depth 1` — phase 28's strategy, which phase 106's **D10 explicitly locked** ("No clone-strategy change… `--depth 1` stays"), predicting the consequence: "URL git sources are shallow → every file carries the repo's TIP-commit date (uniform within the repo…); revisit only if the owner later wants intra-repo recency on URL sources." In a shallow checkout git cannot see history past the shallow boundary (= the tip commit), so `file_commit_dates` (phase 106, `git log --name-only --format=@@%cI` first-sighting-wins) returns the **tip commit's date for EVERY file in the repo** — that uniform tip date is what the live site now displays for `container_bifrost` (the live clone's tip; the dev clone of the same repo shows 2026-09-07 for every file, `git rev-parse --is-shallow-repository` → `true`, exactly ONE `@@` line in the date walk). After `git fetch --unshallow` (438 commits visible) the TRUE last-commit date of `active/container_bifrost/bifrost.md` is **2026-05-05T06:26:40-04:00** — months older than the displayed date. Local-PATH git sources were never affected (git ignores `--depth` for local clones → full history → true dates — which is why the bug only surfaced on the URL source). Everything DOWNSTREAM of the checkout (the importer's `doc_dates_by_root` map + first-sighting-wins walk, D3 normalization, D4 sync semantics, the API/LLM/UI date surfaces, the recency boost) is CORRECT as built — the wrong value is produced at the checkout, so the fix is confined to `clone_or_pull` plus the docstrings/tests that enshrine the shallow assumption. + +**⚠ LOCKED-DECISION REVISIT (AGENTS.md rule 3 — flagged, not silent):** this phase REVISITS phase 106's locked decision **D10** (shallow clone strategy). The revisit is owner-authorized: D10 itself names this exact trigger ("revisit only if the owner later wants intra-repo recency on URL sources") and the owner's 2026-09-16 report is precisely that request. D10's tip-date expectation is hereby SUPERSEDED; every other phase-106 decision (D1–D9 — storage/flag, provenance walk, normalization, sync semantics, LLM surfaces, recency boost, UI columns, derived folder dates) stands unchanged and simply receives true dates. No `PLAN.md` change is involved (D10 is a phase-level decision, not a PLAN §2 anchor). + +## Objective +Git-source checkouts keep FULL history — a fresh `clone_or_pull` clones without `--depth 1`, and any EXISTING shallow checkout (including the live + dev homelab checkouts) self-heals via `git fetch --unshallow` on its next sync — so `file_commit_dates` yields the TRUE per-file last-commit date for every git source (local and URL). A `file://` E2E proves it end to end: an old file shows its old commit date and a tip-touched file shows the tip date, in the API, the Sources tables, and the document viewer — instead of the uniform tip date the bug produced. + +## Dependencies +- `106_document_dates` (complete) — the entire date pipeline this phase corrects AT THE SOURCE: `file_commit_dates` (task 03), the importer's `doc_dates_by_root` map + D4 refresh semantics (task 04), D3 normalization, and the API/LLM/UI surfaces. Only D10's shallow-clone assumption is revised; its suites are the regression gate. +- `28_git_based_sources` (complete) — `scripts/git_sync.py` (the ONLY git-invocation site, A11 — `run_git` contract, `GitSyncError` semantics) and `clone_or_pull` itself. `app/core/docs_push.py`'s own `--depth 1/100` fetches operate on the DOCS repo (phase 18/59) and are NOT touched. +- `32_admin_sync_button` (complete) — `tests/e2e/test_sync_button.py`'s app-server idiom: per-module env with `BOR_GIT_SOURCES=file://` + its own `BOR_SOURCES_DIR`, a real fixture repo built via `git` subprocesses with controlled commit dates, the real in-app sync. The new E2E copies it. +- `99_kb_tree_table_and_back_nav` (complete) — the Sources RAG-view tables + `GET /api/docs/tree` the E2E asserts on (the `Created`/`Updated` columns phase 106 D8/D9 added). + +## Design (shared by all tasks — the executor reads this, not the chat) + +- **The fix — `scripts/git_sync.py::clone_or_pull` (task 01; the single clone/pull entry point BOTH sync entry points call — `app/api/sync.py` L296 and `scripts/import_docs.py` L237 — so one change fixes the UI Sync button and the CLI at once):** + 1. **Fresh checkout** (dest absent or without `.git`): `run_git(["git", "clone", url, str(dest)], cwd=dest.parent)` — the `--depth 1` flags are REMOVED (full history on the first clone for every transport: https/ssh/`file://`/local-path). One-time cost only — subsequent syncs are incremental; the KB repos are small homelab-docs repos. Deliberately NO env knob to restore shallow: shallow == the bug (D12). + 2. **Existing checkout** (dest has `.git`): probe first — `run_git(["git", "rev-parse", "--is-shallow-repository"], cwd=dest)`; stdout.strip() == `"true"` → `run_git(["git", "fetch", "--unshallow"], cwd=dest)` (the ONE-TIME self-heal for checkouts that are already shallow — live + dev homelab included — so the fix reaches deployed sites WITHOUT a re-clone: the next sync after deploy unshallows, pulls, and re-dates), then `run_git(["git", "pull", "--ff-only"], cwd=dest)` as today. Probe `"false"` → straight to `git pull --ff-only` (the common steady-state path — one extra cheap probe per sync, no network fetch). + 3. **Fail loud (D12):** ANY of the probe/unshallow/pull steps raising `GitSyncError` propagates exactly like today's clone/pull failures — the sync aborts with the named repo + reason (the `app/api/sync.py` 502 surface, the CLI traceback). NEVER a silent fallback: continuing a failed unshallow would silently re-serve tip dates (the bug), and falling back to mtimes would be worse. A broken checkout failing loudly is the phase-28 contract. + - Every git invocation still goes through `run_git` (A11 — the module docstring's git-inventory sentence lists the three new/changed commands). + - `file_commit_dates` is UNCHANGED in code — on a full-history checkout its existing newest-first, first-sighting-wins walk already returns the true per-file last commit (verified 2026-09-16: 438 commits, `bifrost.md` → 2026-05-05). ONLY its docstring + the module docstring lose the "shallow URL → uniform tip date" narrative and state the new guarantee: every `clone_or_pull` checkout is full-history → TRUE per-file dates for ALL git sources. +- **Downstream — deliberately untouched:** the importer's `doc_dates_by_root` plumbing, D1/D3/D4 (storage, normalization, sync-refresh semantics — including "a date may go OLDER", which is exactly how the wrong tip dates self-correct on the first post-fix sync: every git document's stored date is refreshed to its true commit date, `dates_updated` counts them, `sources_meta` does NOT bump — a date-only refresh is `unchanged` per D4), the admin date API, the LLM surfaces, the UI columns, the recency boost. No migration, no data fix — the sync IS the fix. `created_at_manual = true` rows keep the owner's corrections (D1) — correct as designed. +- **Stale narratives to correct (same task as the code — a comment that lies is a bug):** `scripts/git_sync.py` module docstring (the L18-28 "Per-file last-commit dates" block: the URL-shallow bullet is false after this phase), `clone_or_pull`'s docstring (L48-51 "shallow, depth 1" + the behavior list), `file_commit_dates`'s docstring (L104-111), `scripts/import_docs.py` docstring L19 ("first run, shallow ``--depth 1``" → "first run, full history"), `app/api/sync.py` per-row comment (L298-303, "shallow URL checkouts → the tip date, D10" → true per-file dates for all git sources). + +## Tasks +1. `01_full_history_checkouts.md` — `clone_or_pull`: no-`--depth` clone + the shallow-probe/`fetch --unshallow` self-heal + fail-loud; the unit argv pins, the real-git integration pins (true per-file dates over `file://` + the existing-shallow self-heal), and the five stale docstring/comment sites. +2. `02_e2e_git_source_dates.md` — dedicated Playwright suite `tests/e2e/test_git_source_dates.py` (isolation): a two-commit `file://` fixture repo, a real in-app sync, true per-file dates asserted in the API + the Sources tables + the viewer badge. +3. `03_gates_and_commit.md` — full gate (unit + integration, coverage >90%, the new E2E + the three regression suites in isolation, ruff + pyright), one atomic `--no-gpg-sign` commit, phase dir → `complete/`. + +## Testing & Quality +- Unit — `tests/unit/test_git_sync.py` (task 01): fresh-clone argv re-pinned to `["git", "clone", url, str(dest)]` (NO `--depth`); the existing-checkout path asserts probe-then-pull; NEW: probe `"true"` → `fetch --unshallow` THEN `pull --ff-only` (argv + order); NEW: `fetch --unshallow` failure → `GitSyncError` propagates (D12 fail-loud); the `git clone --depth 1 … failed (exit 128)` match-string updated to the new argv. +- Integration — `tests/integration/test_git_file_dates.py` (task 01, real `git`, DB-free, the git-availability skip pattern): the module docstring's D10 tip-date expectation is REPLACED with the full-history guarantee; `test_shallow_file_clone_yields_tip_date_for_every_file` (which pinned THE BUG) is REPLACED by `test_url_clone_yields_true_per_file_dates` — the same two-commit recipe (`a.md`/`docs/deep.md` committed 2020-01-02, `b.md` touched again at the 2024-06-15 tip), but `clone_or_pull(f"file://{scratch_repo}", dest)` (a URL transport, through the real function) must yield `{"a.md": DATE_A, "b.md": DATE_B, "docs/deep.md": DATE_A}` — the regression pin: pre-fix this returned `DATE_B` for all three; NEW `test_existing_shallow_checkout_self_heals` — the harness builds a `--depth 1` `file://` clone directly (simulating the deployed checkouts: dates uniform tip, `is-shallow` true), then `clone_or_pull(url, dest)` → no longer shallow + true per-file dates; the local-clone true-date test and every fail-soft test stay green unchanged. +- E2E (mandatory, A16) — `tests/e2e/test_git_source_dates.py` (task 02): `uv run pytest tests/e2e/test_git_source_dates.py -v --no-cov` with the DB up. +- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing` — the validate.sh gate; `scripts/` is outside the `app/` denominator but fully pinned by the suites above). + +## Completion Criteria +- [ ] A fresh `clone_or_pull` over a URL transport yields a NON-shallow checkout (unit argv pin: no `--depth`; integration: `git rev-parse --is-shallow-repository` on the `file://` clone → `false`) with TRUE per-file last-commit dates (2020 file stays 2020, tip-touched file gets the tip date — NOT uniform). +- [ ] An EXISTING shallow checkout (made with `--depth 1`, like every deployed one) unshallows on its next `clone_or_pull` (integration pin: probe → `fetch --unshallow` → pull; dates true afterwards) and a non-shallow checkout takes the plain pull path (unit argv pins); a failed unshallow aborts the sync with `GitSyncError` (fail loud, D12). +- [ ] `tests/integration/test_git_file_dates.py` green with the D10 tip-date test replaced by the true-date regression pin; `tests/unit/test_git_sync.py` green with the updated argv pins. +- [ ] E2E green in isolation: after a real in-app sync of a two-commit `file://` fixture, `GET /api/docs` carries `created_at[:10]` = the OLD commit date for the old file and the TIP date for the new file (the two DIFFER — the bug made them identical); the Sources file table's `Created` column renders the two different years; the old document's viewer `Created` badge carries the old date (ISO `title`); the folder `Updated` columns are the subtree maxes (old folder 2020, new folder 2024). +- [ ] The phase-106 regression suites green in isolation: `uv run pytest tests/e2e/test_document_dates.py -v --no-cov`, `tests/e2e/test_sync_button.py -v --no-cov`, `tests/e2e/test_git_sources_admin.py -v --no-cov`; `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL >90%; `uv run ruff check . && uv run pyright` clean. +- [ ] The five stale shallow/tip-date narratives (git_sync module + two function docstrings, import_docs docstring, sync.py comment) now describe full-history checkouts — no remaining claim that URL sources carry tip dates. +- [ ] One `--no-gpg-sign` Conventional Commit; phase dir moved to `.agents/phases/complete/` by the pipeline gate. + +## Locked decisions +- **D11 — Full-history git checkouts (SUPersedes phase 106 D10; owner-authorized 2026-09-16 — D10's own revisit clause).** Fresh `clone_or_pull` checkouts clone WITHOUT `--depth 1`; existing checkouts are probed with `git rev-parse --is-shallow-repository` and, while shallow, `git fetch --unshallow` before the usual `git pull --ff-only` (one-time self-heal of deployed checkouts, no re-clone). Consequence: `file_commit_dates` returns the TRUE per-file last-commit date for EVERY git source — local and URL — and the next sync after deploy refreshes every git document's stored `created_at` to its true commit date (D4's "may go older" makes this a plain date-only refresh: no `sources_meta` bump, manual corrections survive, `dates_updated` counts them). +- **D12 — Fail loud, never silently re-shallow (the house fail-loud rule applied to the self-heal).** A probe/unshallow/pull failure raises `GitSyncError` and aborts the sync exactly like any clone/pull failure; there is no env knob to restore shallow clones or to skip the self-heal — a shallow checkout would silently re-serve the uniform tip date, i.e. the bug this phase fixes. + +## Commit +```bash +git add scripts/git_sync.py scripts/import_docs.py app/api/sync.py tests/ .agents/phases/ && git commit --no-gpg-sign -m "fix(git): full-history checkouts so URL sources get true per-file document dates" +``` diff --git a/.agents/phases/todo/107_git_full_history_dates/01_full_history_checkouts.md b/.agents/phases/todo/107_git_full_history_dates/01_full_history_checkouts.md new file mode 100644 index 0000000..9f8f0e5 --- /dev/null +++ b/.agents/phases/todo/107_git_full_history_dates/01_full_history_checkouts.md @@ -0,0 +1,43 @@ +# Task 01 — `clone_or_pull` full-history checkouts: no `--depth 1` + the shallow self-heal (D11/D12) + +**Phase:** `107_git_full_history_dates` · **Source:** owner bug report 2026-09-16 — URL git sources show the repo TIP date for every document (`container_bifrost` "created 8/16/2026", months off) because phase 106 D10 locked `--depth 1` shallow URL clones. + +## Objective +Make every `clone_or_pull` checkout carry FULL git history — fresh clones without `--depth 1`, existing shallow checkouts self-healing via `git fetch --unshallow` before the usual fast-forward — so `file_commit_dates` (unchanged) yields true per-file last-commit dates for all git sources; and correct every docstring/comment that enshrines the old shallow assumption. + +## Work +1. `scripts/git_sync.py::clone_or_pull` — the behavior change (keep the signature, the `dest`/`.git` dispatch, and the `GitSyncError` semantics; EVERY invocation through `run_git`, A11): + - Fresh checkout (dest absent or without `.git`): `dest.parent.mkdir(parents=True, exist_ok=True)` then `run_git(["git", "clone", url, str(dest)], cwd=dest.parent)` — the `"--depth", "1"` arguments are REMOVED (full history for every transport). + - Existing checkout (`.git` present): probe `shallow = run_git(["git", "rev-parse", "--is-shallow-repository"], cwd=dest).strip() == "true"`; if `shallow` → `run_git(["git", "fetch", "--unshallow"], cwd=dest)` (the one-time self-heal — deployed checkouts, live + dev included, become full-history on their NEXT sync with no re-clone); then `run_git(["git", "pull", "--ff-only"], cwd=dest)` as today. Probe `"false"` → straight to the pull (steady state: one cheap local probe, no network). + - **D12 fail-loud:** do NOT catch `GitSyncError` — probe, unshallow, and pull failures all propagate exactly like today's clone/pull failures (the sync aborts with the named repo + git's stderr; never a silent fallback to tip dates or mtimes). + - Docstrings (the lies are the rest of the bug): + - module docstring: L3 "(shallow, depth 1)" → full-history clone; the L18-28 "Per-file last-commit dates (phase 106, D2/D10)" block → rewrite for phase 107: a `clone_or_pull` checkout is FULL-history for every transport (fresh: no `--depth`; existing: `--is-shallow-repository` probe + one-time `git fetch --unshallow` self-heal before the `--ff-only` pull) → `file_commit_dates` yields TRUE per-file last-commit dates for ALL git sources (local AND URL); cite the 2026-09-16 verification (shallow walk = one tip commit; after `--unshallow` the true per-file dates, e.g. `bifrost.md` 2026-05-05 not the 2026-09-07 tip). + - `clone_or_pull` docstring (L48-51 + the behavior bullets): "shallow, first run" → "full history, first run"; the bullet list gains the probe/unshallow step; drop "(shallow: the KB is re-imported incrementally anyway)". + - `file_commit_dates` docstring (L104-111): the "local FULL history / URL shallow → uniform TIP date (D10)" paragraph → "every `clone_or_pull` checkout is full history → true per-file last-commit dates for all git sources (phase 107 D11 — supersedes phase 106 D10)"; the fail-soft paragraph stays. +2. `scripts/import_docs.py` — docstring L19: "(first run, shallow ``--depth 1``) or fast-forwarded" → "(first run, full history — no ``--depth``; an existing shallow checkout is unshallowed first, phase 107) or fast-forwarded". No code change here (it already calls `clone_or_pull` and then `file_commit_dates` — the fix flows through). +3. `app/api/sync.py` — per-row comment (L298-303): replace "(local-path checkouts → true per-file dates, shallow URL checkouts → the tip date, D10)" with "(full-history checkouts → true per-file last-commit dates for every git source — phase 107 D11 supersedes phase 106 D10's shallow tip-date behavior)". No code change. +4. Tests (run `uv run pytest tests/unit/test_git_sync.py tests/integration/test_git_file_dates.py -v` — DB-free): + - `tests/unit/test_git_sync.py` (subprocess fully faked, argv pins — update the module docstring's first lines too: "``git clone`` (full history, fresh dest) or a shallow-probe + optional ``git fetch --unshallow`` + ``git pull --ff-only`` (existing checkout)"): + - `test_clone_or_pull_clones_when_dest_has_no_git_dir` → `call["argv"] == ["git", "clone", url, str(dest)]` (NO `--depth`) — same cwd-parent assertion. + - `test_clone_or_pull_creates_missing_parent_before_clone` → same argv re-pin. + - `test_clone_or_pull_pulls_when_git_dir_exists` → the fake `run_git` must now answer the probe (stdout `"false\n"`) before the pull: `calls[0]["argv"] == ["git", "rev-parse", "--is-shallow-repository"]`, `calls[1]["argv"] == ["git", "pull", "--ff-only"]`, both cwd=dest. + - NEW `test_clone_or_pull_unshallows_existing_shallow_checkout` → probe stdout `"true\n"` → `calls[0]` probe, `calls[1]["argv"] == ["git", "fetch", "--unshallow"]`, `calls[2]["argv"] == ["git", "pull", "--ff-only"]` (order pinned: unshallow BEFORE pull). + - NEW `test_clone_or_pull_unshallow_failure_propagates` → probe `"true"`, unshallow raises `GitSyncError("git fetch --unshallow failed (exit 128): fatal: …")` → `pytest.raises(GitSyncError, match="--unshallow")` (D12 fail-loud) and the pull is NEVER called. + - the error-path test's match string `r"git clone --depth 1 .* failed \(exit 128\): fatal: repository not found"` → `r"git clone .* failed \(exit 128\): fatal: repository not found"`. + - The fake-run helper must dispatch by argv (clone/probe/unshallow/pull) — extend it, keep the existing `_FakeProc` shape. + - `tests/integration/test_git_file_dates.py` (real `git`, DB-free, keep the `GIT` skip mark + the `scratch_repo` recipe + every fail-soft/parser test): + - module docstring: replace the "shallow URL-transport clone → TIP date for EVERY file (D10)" bullet with the phase-107 guarantee (a `clone_or_pull` `file://` checkout is full history → true per-file dates; an existing shallow checkout self-heals) and note the D10 supersession. + - REPLACE `test_shallow_file_clone_yields_tip_date_for_every_file` with `test_url_clone_yields_true_per_file_dates(scratch_repo, tmp_path)`: `clone_or_pull(f"file://{scratch_repo}", tmp_path / "url-clone")` → `file_commit_dates(dest) == {"a.md": DATE_A, "b.md": DATE_B, "docs/deep.md": DATE_A}` (the REGRESSION PIN — pre-fix this returned `DATE_B` for all three files) AND `run_git(["git", "rev-parse", "--is-shallow-repository"], cwd=dest).strip() == "false"` (the checkout is not shallow). + - NEW `test_existing_shallow_checkout_self_heals(scratch_repo, tmp_path)`: the harness builds a shallow checkout directly (`_git(tmp_path, "clone", "-q", "--depth", "1", f"file://{scratch_repo}", str(dest))` — simulating the pre-phase deployed checkouts) → pre-heal: `is-shallow` true and `file_commit_dates` uniform `DATE_B` for all three files; then `clone_or_pull(f"file://{scratch_repo}", dest)` → `is-shallow` false and true per-file dates (same dict as the previous test). + - `test_local_clone_yields_true_per_file_dates` stays green unchanged (local path was always full history). + +## Testing & Quality +- Unit: argv + order pins for the fresh-clone (no `--depth`), probe-then-pull, probe→unshallow→pull, and fail-loud paths (the fake subprocess never sees the real git — deterministic). +- Integration: real `git` scratch repos — URL-transport true dates, the self-heal lifecycle, the pre-existing fail-soft battery. +- Coverage: **>90%** on `app/` (no `app/` code changes this task — the gate still passes; `scripts/` is pinned by the suites above). + +## Completion Criteria +- [ ] `clone_or_pull` fresh-clone argv is `["git", "clone", url, str(dest)]` (unit-pinned); existing checkouts take probe → (unshallow iff shallow) → `pull --ff-only` (unit-pinned, order asserted); a failed unshallow raises `GitSyncError` before the pull runs (D12) +- [ ] `uv run pytest tests/unit/test_git_sync.py tests/integration/test_git_file_dates.py -v` green, including `test_url_clone_yields_true_per_file_dates` (per-file true dates over `file://` — the bug's regression pin) and `test_existing_shallow_checkout_self_heals` +- [ ] No remaining "shallow"/"tip date" narrative in `scripts/git_sync.py`, `scripts/import_docs.py`, or `app/api/sync.py` describing CURRENT behavior (phase-106 phase records in `complete/` are history — untouched) +- [ ] `uv run ruff check . && uv run pyright` clean; full `uv run pytest` green diff --git a/.agents/phases/todo/107_git_full_history_dates/02_e2e_git_source_dates.md b/.agents/phases/todo/107_git_full_history_dates/02_e2e_git_source_dates.md new file mode 100644 index 0000000..a46e0de --- /dev/null +++ b/.agents/phases/todo/107_git_full_history_dates/02_e2e_git_source_dates.md @@ -0,0 +1,30 @@ +# Task 02 — E2E: true per-file git dates end to end (`tests/e2e/test_git_source_dates.py`) + +**Phase:** `107_git_full_history_dates` · **Source:** owner bug report 2026-09-16 — the UI must show the document's true last-commit date (a `file://` URL git source is the transport-true stand-in for the live https source; pre-fix it rendered the uniform tip date). + +## Objective +Add the phase's dedicated Playwright suite (A16, run in isolation) that proves the fix end to end: a real `file://` git fixture with two commits of controlled dates, a real in-app admin sync, and assertions that the OLD file shows its OLD commit date and the NEW file shows the TIP date — in `GET /api/docs` / `GET /api/docs/tree` (deterministic ISO) and in the Sources tables + document viewer (locale-tolerant). + +## Work +1. `tests/e2e/test_git_source_dates.py` (NEW — copy the app-server + fixture idiom from `tests/e2e/test_sync_button.py` verbatim in shape: module-scoped `app_server` with per-module env, `tmp_path_factory`-scoped real git fixture, per-test fresh-KB fixture, `e2e.auth_helpers.login`, the conftest `_wait_http`/`USE_REAL_LLM` imports; module docstring: story n/a — owner bug report 2026-09-16; the isolation command; what each test pins): + - **Fixture repo** (real `git` subprocesses, controlled `GIT_COMMITTER_DATE`/`GIT_AUTHOR_DATE` + fixed identity, the `sync_git_repo` helper shape — two commits, MID-YEAR dates so the browser's locale/TZ rendering of the year is stable in any timezone, the `test_document_dates.py` L455 lesson): + - commit one @ `2020-06-15T12:00:00Z` adds `old/old-note.md` (body: a sentence about an old, stable note). + - commit two (the tip) @ `2024-06-15T12:00:00Z` adds `recent/recent-note.md` (body: a sentence about a recent note). + - App env: `BOR_GIT_SOURCES=file://` + its own `BOR_SOURCES_DIR` (fresh dir, the `test_sync_button.py` L141-143 pattern); mock LLM; the session app (no git sources) is never started in this isolated run (no port clash). + - **Per-test sync helper:** admin login → the admin Sources page → click "Sync sources" → wait for the success detail (the `test_sync_button.py` lifecycle wait, generous timeout — real clone + mock-LLM embed). + - **Tests:** + 1. `test_api_created_dates_are_true_per_file` — after a sync: `GET /api/docs` (admin, the page context's request client) — the row with `path == "old/old-note.md"` has `created_at[:10] == "2020-06-15"`, the row with `path == "recent/recent-note.md"` has `created_at[:10] == "2024-06-15"`, and the two values DIFFER (the regression assertion — the phase-106 bug made every URL-source file carry the tip date, i.e. both `2024-06-15`). `GET /api/docs/tree`: the file nodes carry those `created_at` verbatim; the `old` folder's `updated_at[:10] == "2020-06-15"`, the `recent` folder's `updated_at[:10] == "2024-06-15"`, the source node's `updated_at[:10] == "2024-06-15"` (subtree max, phase 106 D9 — now over TRUE dates). + 2. `test_sources_tables_render_distinct_created_dates` — the RAG view's FILE table: the row whose path cell contains `old/old-note.md` has its `Created` cell (the column phase 106 D8 placed between `Chunks` and `Indexed`) matching a regex for the year `2020`; the `recent/recent-note.md` row's `Created` cell matches `2024`; the two cells' text differs. The FOLDER table: the `old` row's `Updated` cell (between `Documents` and `Description`) matches `2020`, the `recent` row's matches `2024`. (Year-regex assertions — `toLocaleString` rendering is locale/TZ-dependent; the year is stable for the mid-year fixture dates. `textContent` reads only — never set innerHTML.) + 3. `test_viewer_created_badge_is_the_true_git_date` — open the old document (click its row/title, the phase-26 same-page modal): the viewer's `.doc-created` badge (phase 106, before the `Indexed` badge) has `title == "2020-06-15T12:00:00+00:00"` (the raw ISO — `metaBadge`'s title, deterministic under any locale/TZ, the house solution to the L455 rendering trap) and its visible text starts with `Created `; close the modal (Escape) — the viewer reverts cleanly. + 4. `test_page_a11y_and_no_cdn_basics` — the standard light pass (AGENTS.md rules 5/6, the `test_git_sources_admin.py` a11y test shape): landmarks on the Sources view, the `Created`/`Updated` `` cells present in both tables, ≥4.5:1-free text (no new color), 3px `:focus-visible` on a table row link, same-origin assets only (no external `src`/`href`). + - Fresh-KB fixture: truncate `documents`, `chunks`, `git_sources`, `sources_meta`, `kb_overview` (the `test_sync_button.py` `clean_kb` shape) so each test's sync counts are its own. +2. Run `uv run pytest tests/e2e/test_git_source_dates.py -v --no-cov` (DB up: `podman compose up -d db`) — all four tests green in isolation. + +## Testing & Quality +- E2E (the task IS the test): real `file://` URL-transport clone through `clone_or_pull` (task 01's full-history path) + the real importer + the real API + the real UI — the owner's scenario, transport-true. +- Coverage: the suite is `--no-cov`; it exercises `app/` (sync, docs API, tree builder) and `scripts/git_sync.py` for real — the `app/` >90% gate is unaffected. + +## Completion Criteria +- [ ] `uv run pytest tests/e2e/test_git_source_dates.py -v --no-cov` green in isolation (DB up) — all four tests +- [ ] The regression assertion holds: the two fixture documents' `created_at` values DIFFER (2020 vs 2024) in the API, the tables, and the viewer — the uniform-tip-date bug is provably gone for URL transports +- [ ] The suite leaves no other suite's fixtures touched (isolation: the session app never starts, own `BOR_SOURCES_DIR`, per-test DB reset) diff --git a/.agents/phases/todo/107_git_full_history_dates/03_gates_and_commit.md b/.agents/phases/todo/107_git_full_history_dates/03_gates_and_commit.md new file mode 100644 index 0000000..92c029d --- /dev/null +++ b/.agents/phases/todo/107_git_full_history_dates/03_gates_and_commit.md @@ -0,0 +1,36 @@ +# Task 03 — Full gate: suites, coverage, lint/types, regression E2Es, atomic commit + +**Phase:** `107_git_full_history_dates` · **Source:** AGENTS.md rules 8/9 — the test gates are non-negotiable; one atomic, professional commit per completed phase. + +## Objective +Run the complete quality gate for the phase — unit + integration green, `app/` coverage >90%, the new E2E + the three phase-106/28/35 regression E2E suites green in isolation, ruff + pyright clean — then land the single `--no-gpg-sign` Conventional Commit and move the phase directory to `complete/`. + +## Work +1. DB up: `podman compose up -d db` (and `uv run alembic upgrade head` if the dev DB is behind — no NEW migration this phase, so head is unchanged). +2. Unit + integration: `uv run pytest` — green. +3. Coverage: `uv run pytest --cov=app --cov-report=term-missing` — TOTAL >90% (no `app/` code changed this phase; this is the regression check on the phase-106 suites + the new E2E's `app/` exercise). +4. E2E, each in isolation (`--no-cov`): + - the NEW suite: `uv run pytest tests/e2e/test_git_source_dates.py -v --no-cov` + - regressions (the three suites whose behavior this phase touches or that pin phase-106 dates end to end): + - `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` + - `uv run pytest tests/e2e/test_sync_button.py -v --no-cov` + - `uv run pytest tests/e2e/test_git_sources_admin.py -v --no-cov` +5. Lint + types: `uv run ruff check . && uv run pyright` — clean. +6. UI Structure Check (AGENTS.md rule 5): the phase adds NO UI of its own — the date columns/badge asserted by task 02 are phase-106 UI, re-verified only; confirm nothing in `frontend/` changed this phase (`git status` shows none) so the byte-identical contracts are untouched. +7. Commit (exactly one atomic commit, `--no-gpg-sign`, the 00_phase.md message): + ```bash + git add scripts/git_sync.py scripts/import_docs.py app/api/sync.py tests/ .agents/phases/ && git commit --no-gpg-sign -m "fix(git): full-history checkouts so URL sources get true per-file document dates" + ``` +8. Move the phase directory: `mv .agents/phases/todo/107_git_full_history_dates .agents/phases/complete/` (the pipeline gate does this on success — do it only after step 7 succeeds and include the move in the SAME commit's tree if the gate script does not, per the house protocol; check `.agents/pipeline.log` / the phased-execution gate for how prior phases recorded the move and match it). + +## Testing & Quality +- This task runs, not writes, the gate: every command above must pass before the commit exists. +- Coverage: **>90%** on `app/` (TOTAL line of the `term-missing` report). + +## Completion Criteria +- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL >90% +- [ ] `uv run pytest tests/e2e/test_git_source_dates.py -v --no-cov` green in isolation (DB up) +- [ ] `tests/e2e/test_document_dates.py`, `tests/e2e/test_sync_button.py`, `tests/e2e/test_git_sources_admin.py` each green in isolation +- [ ] `uv run ruff check . && uv run pyright` clean +- [ ] Exactly one new commit, message `fix(git): full-history checkouts so URL sources get true per-file document dates`, signed with `--no-gpg-sign`; `git status` clean afterwards (only gitignored runtime artifacts aside) +- [ ] Phase dir at `.agents/phases/complete/107_git_full_history_dates/`