Files
brain-of-reese/tests/integration/test_import_docs_git.py
T
ducoterra 0035bdf533
Build and Push Containers / build-and-push-app (push) Successful in 2m25s
Build and Push Containers / build-and-push-db (push) Successful in 12s
phase: 136_force_push_reset
Everything is verified green and consistent. Final report:

**Phase 136 — final verification pass (all tasks already complete in `complete/136_force_push_reset/`):**

- **Verified implemented:** unified ff-or-reset update sequence in `scripts/git_sync.py::clone_or_pull` (phase-135 HEAD guard → shallow self-heal → `fetch origin <ref>` → `merge --ff-only` → on non-ff: SHA capture + `reset --hard FETCH_HEAD` + one WARNING line, returns normally); detached-HEAD fail-loud; `branch=None` legacy `git pull` deliberately gone
- **Completion criteria — all met:**
  1. ✅ Force push no longer fails sync (integration `test_clone_or_pull_force_push_resets_to_the_new_head` + API-seam `test_sync_pipeline_survives_a_force_push` + E2E run 2 asserts `success` and KB serves the V2 sentinel)
  2. ✅ Unrelated-history force push handled by the same reset (`test_clone_or_pull_unrelated_history_force_push_resets`)
  3. ✅ Broken fetch still fails loud (fetch-failure unit pins raise with no merge/reset argv; `test_main_git_failure_aborts_before_import`)
  4. ✅ WARNING line pinned: one line, 7-char SHAs, checkout path, `origin/<ref>`, no URL (unit exact-string pin + integration caplog pin)
  5. ✅ `uv run pytest --cov=app --cov-report=term-missing` → **3231 passed, app/ 99%**; `uv run pytest tests/e2e/test_force_push_resync.py -v --no-cov` → **2 passed in isolation** (10.6 s)
  6. ✅ Phase-135 pins preserved (mismatch guard, `--branch` clone argv, `branch=None` fresh-clone argv all green in the 53-test run)
- **Test/lint results:** `uv run pytest tests/unit/test_git_sync.py tests/integration/test_import_docs_git.py -v` → 53 passed; full suite 3231 passed (72 s); `uv run ruff check . && uv run pyright` → clean (0 errors)
- **Documentation:** README.md git-sources section already updated this phase to the ff-or-reset contract; all `.env.example` vars documented; pages table matches router; AGENTS.md rules and quick-reference commands verified accurate (rule 5 correctly documents the phase-100 §7.1 supersession) — **no changes needed**
- **Notable:** E2E "teeth" verification (fails on pre-phase code) is recorded in the task report; nothing was committed per harness protocol
- **Next pending phase:** `todo/137_pdf_image_context`
2026-09-28 22:38:54 -04:00

1334 lines
54 KiB
Python

"""Integration test: ``import_docs`` source resolution (phase 28, task
03; phase 35 re-points at the shared resolver; phase 38 adds the local
kind).
Drives ``scripts.import_docs`` end to end with a fake ``clone_or_pull``
(no real git, no network) and a recording fake ``import_sources`` (no
real DB), covering: the phase-53 version bump is stubbed in the
``main()`` tests the same way (the ``sources_version=`` summary token
is asserted against the canned value).
- Effective sources set (phase 35: the shared resolver — stubbed here,
keeping this file's no-real-DB style) → each git URL is cloned/pulled
into ``BOR_SOURCES_DIR/<repo-name>/``; local rows are their existing
directories, walked directly; exactly those dirs are imported.
- DB rows (both kinds) win over ``BOR_GIT_SOURCES`` (the resolver's
``db`` origin — the env list is ignored; the env fallback stays
git-only).
- ``--source`` still wins over the DB rows (no git at all, no resolver
call).
- No sources configured + no ``--source`` → the legacy
``DEFAULT_SOURCES``.
- A failing git sync → exit code 1, an error naming the failing repo on
stderr, and **zero** import attempts.
- A missing local directory → the same pre-import fail-loud: exit code
1, ``local source missing: <path>`` on stderr, zero import attempts.
Phase 135 (the branch pin) adds the REAL-git section at the bottom:
a two-branch ``file://`` fixture repo (``main`` with file A, ``release``
with file B) drives the real :func:`scripts.git_sync.clone_or_pull` —
the fresh ``--branch`` checkout, the ff-merge pull of a new commit on
the pinned branch, the fail-loud branch MISMATCH (checkout untouched),
and the ``_resolve_sources`` seam with a ``branch='release'`` row
(importing file B, not file A) + an env-origin row without a branch
(the default branch, byte-identical pre-phase behavior). Skipped, not
failed, on a machine without the git CLI (the house pattern).
Phase 136 (the ff-or-reset) extends the REAL-git section: the owner's
force-push repro (the origin's history REWRITTEN to a new root commit
between two ``clone_or_pull`` runs — normal divergence AND fully
unrelated history) now resets the checkout to the origin's new head
instead of failing (the loud WARNING line pinned alongside), a
branched row's force push lands ``origin/<branch>``'s head (main's
content not involved), ``file_commit_dates`` over the post-reset
checkout reflects the NEW head's dates, and the SYNC PIPELINE END-
TO-END over the API seam (``POST /api/sync`` + the status poll — the
seam ``test_sync_api.py`` exercises) lands ``success`` after a force
push, with the checkout (and therefore the KB) holding the new head.
"""
from __future__ import annotations
import asyncio
import logging
import os
import re
import subprocess
import time
from collections.abc import Iterator
from datetime import datetime
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import text
from sqlalchemy.orm import Session
from app.api import sync as sync_api
from app.config import Settings
from app.db import SessionLocal
from app.main import app as fastapi_app
from app.models import GitSource
from app.rag import git_sources as git_sources_resolver
from app.rag.importer import ImportSummary
from scripts import import_docs
from scripts.git_sync import GitSyncError, clone_or_pull, file_commit_dates
from tests.conftest import ADMIN_PASSWORD
from tests.fakes import FakeEmbedder
def _settings(git_sources: str = "", sources_dir: str = "~/bor-sources") -> Settings:
"""Fresh settings (no .env file); explicit kwargs beat any env leaks."""
return Settings(_env_file=None, git_sources=git_sources, sources_dir=sources_dir) # pyright: ignore[reportCallIssue]
def _git_row(url: str, ignore_paths: list[str] | None = None) -> GitSource:
return GitSource(url=url, kind="git", ignore_paths=ignore_paths or [])
def _local_row(
path: str,
ignore_paths: list[str] | None = None,
include_hidden: bool = False,
) -> GitSource:
"""A local row as the phase-38 API stores it: the expanded path in
both ``path`` and the NOT-NULL ``url`` location column (plus the
phase-89 ignore list and the phase-105 hidden-folders flag, both
defaulting off — A4)."""
return GitSource(
url=path, kind="local", path=path,
ignore_paths=ignore_paths or [], include_hidden=include_hidden,
)
@pytest.fixture()
def clean_documents(db: Session) -> Iterator[None]:
"""Phase 105: the real-import CLI tests write ``documents``/
``chunks`` (the canonical KB state) — global, truncated around
every such test."""
db.execute(text("TRUNCATE chunks, documents"))
db.commit()
yield
db.execute(text("TRUNCATE chunks, documents"))
db.commit()
class FakeImportSources:
"""Records every ``import_sources`` call instead of touching a DB."""
def __init__(self) -> None:
self.calls: list[dict] = []
async def __call__(
self,
sources: list[Path],
llm: object,
*,
prune: bool = False,
limit: int | None = None,
ignore_by_root: dict[str, list[str]] | None = None, # phase 89
include_hidden_by_root: dict[str, bool] | None = None, # phase 105
doc_dates_by_root: dict[str, dict[str, datetime]] | None = None, # phase 106
) -> ImportSummary:
self.calls.append(
{"sources": list(sources), "prune": prune, "limit": limit,
"ignore_by_root": ignore_by_root,
"include_hidden_by_root": include_hidden_by_root,
"doc_dates_by_root": doc_dates_by_root}
)
return ImportSummary(files=1, added=1)
def _fake_clone_factory() -> tuple[list[tuple[str, Path]], object]:
"""A ``clone_or_pull`` that materialises a checkout with one .md
file. Phase 135: accepts the row's branch pin (``None`` for every
row in this file's fixtures — the real-git branch behavior is
pinned by the phase-135 section below, which uses the REAL
``clone_or_pull``)."""
calls: list[tuple[str, Path]] = []
def fake_clone_or_pull(url: str, dest: Path | str, branch: str | None = None) -> Path:
assert branch is None # every row in this file's fixtures pins no branch
dest = Path(dest)
dest.mkdir(parents=True, exist_ok=True)
(dest / "notes.md").write_text(f"# {dest.name}\ncontent for the KB\n", encoding="utf-8")
calls.append((url, dest))
return dest
return calls, fake_clone_or_pull
def _stub_bump(monkeypatch: pytest.MonkeyPatch) -> list[None]:
"""Stub the phase-53 version bump (this file keeps its no-real-DB
style for the counter — the fake import already avoids the KB
tables). Returns the call record; the canned new version is 1."""
bumps: list[None] = []
def fake_bump(session: object) -> int:
bumps.append(None)
return 1
monkeypatch.setattr(import_docs, "bump_sources_version", fake_bump)
return bumps
def _stub_folder_summaries(monkeypatch: pytest.MonkeyPatch) -> list[dict]:
"""Stub the phase-94 folder-summary regeneration (this file keeps
its no-real-DB / no-network style — the real generator would read
the global ``documents`` table and burn ``lite`` calls). Returns
the call record; the canned stats are the zero dict."""
calls: list[dict] = []
async def fake_generate(
db: object, llm: object, *, skip: bool = False, only_missing: bool = False
) -> dict[str, int]:
calls.append({"skip": skip, "only_missing": only_missing})
return {"generated": 0, "failed": 0, "pruned": 0}
monkeypatch.setattr(import_docs, "generate_folder_summaries", fake_generate)
return calls
# --- repo_name -------------------------------------------------------------
@pytest.mark.parametrize(
("url", "name"),
[
("https://github.com/user/homelab.git", "homelab"),
("https://github.com/user/homelab", "homelab"),
("git@github.com:user/homelab.git", "homelab"),
("git@github.com:homelab.git", "homelab"),
("ssh://git@host:2222/group/deployments.git", "deployments"),
("https://git.reeseapps.com:8443/proj/notes", "notes"),
],
)
def test_repo_name(url: str, name: str) -> None:
assert import_docs.repo_name(url) == name
def test_repo_name_slug_fallback() -> None:
# No usable basename (path ends in the .git suffix itself) → slug.
assert import_docs.repo_name("https://host/.git") == "https-host"
# --- _resolve_sources ------------------------------------------------------
def test_resolve_sources_git_urls_cloned_into_sources_dir(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
calls, fake = _fake_clone_factory()
monkeypatch.setattr(import_docs, "clone_or_pull", fake)
# Phase 35: resolution goes through the shared resolver (stubbed —
# this file keeps its no-real-DB style); the rows are the env list,
# surfaced as synthetic git rows.
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: (
[_git_row("https://host/a/homelab.git"), _git_row("git@host:user/deploy.git")],
"env",
),
)
settings = _settings(
git_sources="https://host/a/homelab.git, git@host:user/deploy.git ,",
sources_dir=str(tmp_path / "bor"),
)
sources, ignore_map, hidden_map, date_map = import_docs._resolve_sources(None, settings)
assert sources == [tmp_path / "bor" / "homelab", tmp_path / "bor" / "deploy"]
assert ignore_map == {} # phase 89: no row carries a list → empty map
# Phase 105: flag-off rows still contribute their root — with False
# (A4), keyed by the SAME root string the importer sees.
assert hidden_map == {
str(tmp_path / "bor" / "homelab"): False,
str(tmp_path / "bor" / "deploy"): False,
}
# Phase 106: git rows are listed with their checkout's date walk —
# the fake checkouts are not git repos, so the walk fails soft to
# ``{}`` (the importer would then take the mtime fallback).
assert date_map == {
str(tmp_path / "bor" / "homelab"): {},
str(tmp_path / "bor" / "deploy"): {},
}
assert calls == [
("https://host/a/homelab.git", tmp_path / "bor" / "homelab"),
("git@host:user/deploy.git", tmp_path / "bor" / "deploy"),
]
def test_resolve_sources_cli_source_wins(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
calls, fake = _fake_clone_factory()
monkeypatch.setattr(import_docs, "clone_or_pull", fake)
settings = _settings(git_sources="https://host/a/repo.git")
manual = tmp_path / "Manual"
sources, ignore_map, hidden_map, date_map = import_docs._resolve_sources([manual], settings)
assert sources == [manual]
assert ignore_map == {} # phase 89: manual dirs have no rows → no ignore
assert hidden_map == {} # phase 105: manual dirs have no rows → hidden skipped
# Phase 106: manual dirs have no rows (no clone) → no date map
# entries (the importer's mtime fallback applies).
assert date_map == {}
assert calls == [] # git is never touched when --source is given
def test_resolve_sources_db_rows_win_over_env(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Phase 35: the resolver's ``db`` origin (table has rows) — the
``BOR_GIT_SOURCES`` list must be ignored; only the DB repo is cloned."""
calls, fake = _fake_clone_factory()
monkeypatch.setattr(import_docs, "clone_or_pull", fake)
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: ([_git_row("https://db.example/only.git")], "db"),
)
settings = _settings(
git_sources="https://env.example/ignored.git",
sources_dir=str(tmp_path / "bor"),
)
sources, ignore_map, hidden_map, date_map = import_docs._resolve_sources(None, settings)
assert sources == [tmp_path / "bor" / "only"]
assert ignore_map == {} # phase 89: no row carries a list → empty map
# Phase 105: the default-flag row contributes its root with False (A4).
assert hidden_map == {str(tmp_path / "bor" / "only"): False}
# Phase 106: the git row's (fake, non-repo) checkout fails soft → {}.
assert date_map == {str(tmp_path / "bor" / "only"): {}}
assert calls == [("https://db.example/only.git", tmp_path / "bor" / "only")]
def test_resolve_sources_defaults_when_nothing_configured(
monkeypatch: pytest.MonkeyPatch,
) -> None:
# Both origins empty (the resolver's ``([], "env")``) → legacy dirs.
monkeypatch.setattr(import_docs, "effective_sources", lambda db: ([], "env"))
sources, ignore_map, hidden_map, date_map = import_docs._resolve_sources(None, _settings())
assert sources == [p.expanduser() for p in import_docs.DEFAULT_SOURCES]
assert ignore_map == {} # phase 89: the legacy fallback has no rows
assert hidden_map == {} # phase 105: the legacy fallback has no rows
# Phase 106: the legacy fallback has no rows (no clone) → mtime
# fallback for every file.
assert date_map == {}
def test_resolve_sources_rows_branch_builds_ignore_map(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Phase 89: the rows branch returns each row's ignore list keyed by
the resolved root string — the local row's directory, the git row's
checkout dir; a row without a list contributes nothing to the map."""
calls, fake = _fake_clone_factory()
monkeypatch.setattr(import_docs, "clone_or_pull", fake)
local_dir = tmp_path / "LocalDocs"
local_dir.mkdir()
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: (
[
_git_row("https://db.example/only.git"),
_local_row(str(local_dir), ignore_paths=["ignore/"]),
],
"db",
),
)
settings = _settings(sources_dir=str(tmp_path / "bor"))
sources, ignore_map, hidden_map, date_map = import_docs._resolve_sources(None, settings)
assert sources == [tmp_path / "bor" / "only", local_dir]
# Keyed by the SAME string the importer sees (the root, not the name).
assert ignore_map == {str(local_dir): ["ignore/"]}
# Phase 105: both rows are flag-off → per-root False entries (A4).
assert hidden_map == {str(tmp_path / "bor" / "only"): False, str(local_dir): False}
# Phase 106: ONLY the git row is listed (local rows take the mtime
# fallback); the fake checkout's date walk fails soft to ``{}``.
assert date_map == {str(tmp_path / "bor" / "only"): {}}
def test_resolve_sources_two_rows_sharing_root_string_extend(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Phase 89 collision rule: two rows resolving to the SAME root
string (the sibling/repo-name edge — ``…/shared`` and
``…/shared.git``) get the UNION of their lists (extend, not
replace), in row order."""
calls, fake = _fake_clone_factory()
monkeypatch.setattr(import_docs, "clone_or_pull", fake)
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: (
[
_git_row("https://a.example/shared", ignore_paths=["a/"]),
_git_row("https://a.example/shared.git", ignore_paths=["b"]),
],
"db",
),
)
settings = _settings(sources_dir=str(tmp_path / "bor"))
sources, ignore_map, hidden_map, date_map = import_docs._resolve_sources(None, settings)
shared = str(tmp_path / "bor" / "shared")
assert sources == [tmp_path / "bor" / "shared", tmp_path / "bor" / "shared"]
assert ignore_map == {shared: ["a/", "b"]} # union, row order
# Phase 105 collision: the shared root gets the OR of the flags —
# both rows off here, so one False entry for the one root string.
assert hidden_map == {shared: False}
# Phase 106 collision: both git rows resolve to the SAME root — one
# date walk for the one root string (last row's walk wins, both
# fail soft to ``{}`` for the fake checkout).
assert date_map == {shared: {}}
def test_main_rows_branch_passes_ignore_map_to_import(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
capsys: pytest.CaptureFixture[str],
) -> None:
"""Phase 89: a ``kind=local`` row carrying ``ignore_paths`` →
``main`` passes the per-root map to ``import_sources`` (keyed by
the directory string, prune flag unchanged)."""
settings = _settings(sources_dir=str(tmp_path / "bor"))
monkeypatch.setattr(import_docs, "get_settings", lambda: settings)
local_dir = tmp_path / "LocalDocs"
local_dir.mkdir()
(local_dir / "keep.md").write_text("# Keep\nin scope\n", encoding="utf-8")
(local_dir / "ignore").mkdir()
(local_dir / "ignore" / "secret.md").write_text("# Secret\nignored\n",
encoding="utf-8")
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: ([_local_row(str(local_dir), ignore_paths=["ignore/"])], "db"),
)
fake_import = FakeImportSources()
monkeypatch.setattr(import_docs, "import_sources", fake_import)
_stub_bump(monkeypatch)
_stub_folder_summaries(monkeypatch)
rc = import_docs.main([])
assert rc == 0
call = fake_import.calls[0]
assert call["sources"] == [local_dir]
assert call["ignore_by_root"] == {str(local_dir): ["ignore/"]}
# Phase 105: the default-flag row passes the per-root map too — a
# False entry, not an absent key (the importer reads it per root).
assert call["include_hidden_by_root"] == {str(local_dir): False}
# Phase 106: the local-only resolution contributes no date map
# (no clone — the importer's mtime fallback applies).
assert call["doc_dates_by_root"] == {}
assert call["prune"] is False # the CLI's no-prune default is unchanged
# --- phase 105: per-row hidden-folders flag ---------------------------------
def test_main_rows_branch_passes_include_hidden_map_to_import(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
capsys: pytest.CaptureFixture[str],
) -> None:
"""Phase 105: a ``kind=local`` row with ``include_hidden=True`` →
``main`` passes the per-root flag map to ``import_sources`` (keyed
by the directory string — the map is built, not lost)."""
settings = _settings(sources_dir=str(tmp_path / "bor"))
monkeypatch.setattr(import_docs, "get_settings", lambda: settings)
local_dir = tmp_path / "LocalDocs"
local_dir.mkdir()
(local_dir / "visible.md").write_text("# Visible\nin scope\n", encoding="utf-8")
(local_dir / ".hidden").mkdir()
(local_dir / ".hidden" / "note.md").write_text("# Note\nhidden\n", encoding="utf-8")
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: ([_local_row(str(local_dir), include_hidden=True)], "db"),
)
fake_import = FakeImportSources()
monkeypatch.setattr(import_docs, "import_sources", fake_import)
_stub_bump(monkeypatch)
_stub_folder_summaries(monkeypatch)
rc = import_docs.main([])
assert rc == 0
call = fake_import.calls[0]
assert call["sources"] == [local_dir]
assert call["include_hidden_by_root"] == {str(local_dir): True}
assert call["ignore_by_root"] == {} # the row carries no ignore list
assert call["prune"] is False # the CLI's no-prune default is unchanged
def _stub_overview(monkeypatch: pytest.MonkeyPatch) -> None:
"""Stub the phase-31 overview regeneration (the CLI's real-import
tests: the deterministic ``FakeEmbedder`` must not burn its canned
``chat`` on the KB outline — the import is what is under test)."""
async def fake_overview(llm: object, session: object = None) -> bool:
return False
monkeypatch.setattr(import_docs, "regenerate_overview", fake_overview)
def _kb_docs(db: Session) -> set[tuple[str, str]]:
"""The (source, path) pairs of the ``documents`` table."""
rows = db.execute(text("SELECT source, path FROM documents")).all()
return {(row.source, row.path) for row in rows}
def test_main_local_row_include_hidden_true_indexes_hidden_file(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
db: Session,
capsys: pytest.CaptureFixture[str],
clean_documents: None,
) -> None:
"""Phase 105 (A1): the CLI's DB-row path with the flag ON — the
file inside the hidden folder is indexed, embedded, and counted
like any visible file (real import over a host temp dir, the
deterministic ``FakeEmbedder``; the regression this phase most
plausibly breaks — the CLI's map built but lost — would leave it
out)."""
settings = _settings(sources_dir=str(tmp_path / "bor"))
monkeypatch.setattr(import_docs, "get_settings", lambda: settings)
local_dir = tmp_path / "LocalDocs"
local_dir.mkdir()
(local_dir / "visible.md").write_text("# Visible\nin scope\n", encoding="utf-8")
(local_dir / ".hidden").mkdir()
(local_dir / ".hidden" / "note.md").write_text("# Note\nhidden\n", encoding="utf-8")
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: ([_local_row(str(local_dir), include_hidden=True)], "db"),
)
monkeypatch.setattr(import_docs, "LLMClient", lambda: FakeEmbedder())
_stub_overview(monkeypatch)
_stub_bump(monkeypatch)
_stub_folder_summaries(monkeypatch)
rc = import_docs.main([])
assert rc == 0
# The hidden file has a documents row next to the visible one.
assert _kb_docs(db) == {
("LocalDocs", "visible.md"),
("LocalDocs", ".hidden/note.md"),
}
out = capsys.readouterr().out
assert "added=2" in out # both files were imported
def test_main_local_row_include_hidden_false_skips_hidden_file(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
db: Session,
capsys: pytest.CaptureFixture[str],
clean_documents: None,
) -> None:
"""Phase 105 (A4): the same fixture with the default flag (off) —
the hidden-folder file never enters the KB (the byte-identical
pre-phase-105 walk): no documents row, and the summary line counts
only the visible file."""
settings = _settings(sources_dir=str(tmp_path / "bor"))
monkeypatch.setattr(import_docs, "get_settings", lambda: settings)
local_dir = tmp_path / "LocalDocs"
local_dir.mkdir()
(local_dir / "visible.md").write_text("# Visible\nin scope\n", encoding="utf-8")
(local_dir / ".hidden").mkdir()
(local_dir / ".hidden" / "note.md").write_text("# Note\nhidden\n", encoding="utf-8")
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: ([_local_row(str(local_dir))], "db"), # flag defaults to False
)
monkeypatch.setattr(import_docs, "LLMClient", lambda: FakeEmbedder())
_stub_overview(monkeypatch)
_stub_bump(monkeypatch)
_stub_folder_summaries(monkeypatch)
rc = import_docs.main([])
assert rc == 0
assert _kb_docs(db) == {("LocalDocs", "visible.md")}
out = capsys.readouterr().out
assert "added=1" in out # the hidden file was never walked
# --- main() ----------------------------------------------------------------
def test_main_git_sources_clone_then_import(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
capsys: pytest.CaptureFixture[str],
) -> None:
settings = _settings(
git_sources="https://host/a/homelab.git,https://host/a/deploy.git",
sources_dir=str(tmp_path / "bor"),
)
monkeypatch.setattr(import_docs, "get_settings", lambda: settings)
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: (
[_git_row("https://host/a/homelab.git"), _git_row("https://host/a/deploy.git")],
"env",
),
)
calls, fake = _fake_clone_factory()
monkeypatch.setattr(import_docs, "clone_or_pull", fake)
fake_import = FakeImportSources()
monkeypatch.setattr(import_docs, "import_sources", fake_import)
bumps = _stub_bump(monkeypatch)
_stub_folder_summaries(monkeypatch)
rc = import_docs.main([])
assert rc == 0
assert [(url, dest) for url, dest in calls] == [
("https://host/a/homelab.git", tmp_path / "bor" / "homelab"),
("https://host/a/deploy.git", tmp_path / "bor" / "deploy"),
]
assert len(fake_import.calls) == 1
# The cloned checkouts are exactly what gets imported.
assert fake_import.calls[0]["sources"] == [
tmp_path / "bor" / "homelab",
tmp_path / "bor" / "deploy",
]
# Phase 106: the git rows' (fake, non-repo) checkouts fail soft to
# ``{}`` — but the roots ARE listed (the CLI feeds the map).
assert fake_import.calls[0]["doc_dates_by_root"] == {
str(tmp_path / "bor" / "homelab"): {},
str(tmp_path / "bor" / "deploy"): {},
}
for dest in (tmp_path / "bor" / "homelab", tmp_path / "bor" / "deploy"):
assert (dest / "notes.md").is_file()
# The final summary print reflects the import (added > 0).
out = capsys.readouterr().out
assert "added=1" in out
# Phase 53: a KB-changing run bumps the sources version exactly
# once and reports it (stubbed — this file keeps its no-real-DB
# style for the counter, like the fake import above).
assert len(bumps) == 1
assert "sources_version=1" in out
def test_main_cli_source_still_imports_manual_dir(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
capsys: pytest.CaptureFixture[str],
) -> None:
manual = tmp_path / "manual"
manual.mkdir()
(manual / "a.md").write_text("# A\nhi\n", encoding="utf-8")
# BOR_GIT_SOURCES is set but must be ignored — --source always wins.
settings = _settings(git_sources="https://host/a/repo.git")
monkeypatch.setattr(import_docs, "get_settings", lambda: settings)
calls, fake = _fake_clone_factory()
monkeypatch.setattr(import_docs, "clone_or_pull", fake)
fake_import = FakeImportSources()
monkeypatch.setattr(import_docs, "import_sources", fake_import)
bumps = _stub_bump(monkeypatch)
_stub_folder_summaries(monkeypatch)
rc = import_docs.main(["--source", str(manual)])
assert rc == 0
assert calls == []
assert fake_import.calls[0]["sources"] == [manual]
# Phase 106: manual --source has no rows (no clone) → no date map
# (the importer's mtime fallback applies).
assert fake_import.calls[0]["doc_dates_by_root"] == {}
assert fake_import.calls[0]["prune"] is False
# Phase 53: a manual --source run that changes the KB bumps exactly
# once (the CLI is the other canonical sync path).
assert len(bumps) == 1
assert "sources_version=1" in capsys.readouterr().out
def test_resolve_sources_mixed_git_and_local(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Phase 38: DB rows of both kinds — the git row is cloned into
``BOR_SOURCES_DIR``, the local row is its existing directory itself
(no clone), in row order; the env list is ignored."""
calls, fake = _fake_clone_factory()
monkeypatch.setattr(import_docs, "clone_or_pull", fake)
local_dir = tmp_path / "LocalDocs"
local_dir.mkdir()
(local_dir / "a.md").write_text("# A\nlocal fixture\n", encoding="utf-8")
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: (
[_git_row("https://db.example/only.git"), _local_row(str(local_dir))],
"db",
),
)
settings = _settings(
git_sources="https://env.example/ignored.git",
sources_dir=str(tmp_path / "bor"),
)
sources, ignore_map, hidden_map, date_map = import_docs._resolve_sources(None, settings)
assert sources == [tmp_path / "bor" / "only", local_dir]
assert ignore_map == {} # phase 89: neither row carries a list
# Phase 105: both rows default-flag → per-root False entries (A4).
assert hidden_map == {str(tmp_path / "bor" / "only"): False, str(local_dir): False}
# Phase 106: only the git row is listed (local takes the mtime
# fallback); the fake checkout's date walk fails soft to ``{}``.
assert date_map == {str(tmp_path / "bor" / "only"): {}}
assert calls == [("https://db.example/only.git", tmp_path / "bor" / "only")]
def test_resolve_sources_missing_local_dir_aborts(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Phase 38: a local row whose directory is gone at run time →
``GitSyncError`` naming the path, before any import (the same
pre-import fail-loud as a failing git clone)."""
monkeypatch.setattr(import_docs, "clone_or_pull", _fake_clone_factory()[1])
missing = tmp_path / "Gone"
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: ([_local_row(str(missing))], "db"),
)
settings = _settings(sources_dir=str(tmp_path / "bor"))
with pytest.raises(GitSyncError, match=f"local source missing: {re.escape(str(missing))}"):
import_docs._resolve_sources(None, settings)
def test_main_git_failure_aborts_before_import(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
settings = _settings(
git_sources="https://host/a/bad.git",
sources_dir=str(tmp_path / "bor"),
)
monkeypatch.setattr(import_docs, "get_settings", lambda: settings)
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: ([_git_row("https://host/a/bad.git")], "env"),
)
def failing_clone(url: str, dest: Path | str, branch: str | None = None) -> Path:
raise GitSyncError(
f"git clone {url} failed (exit 128): "
"fatal: repository not found"
)
monkeypatch.setattr(import_docs, "clone_or_pull", failing_clone)
fake_import = FakeImportSources()
monkeypatch.setattr(import_docs, "import_sources", fake_import)
rc = import_docs.main([])
assert rc == 1
err = capsys.readouterr().err
assert "import_docs: source sync failed" in err
assert "bad.git" in err # the failing repo is named
assert fake_import.calls == [] # no partial import
assert not (tmp_path / "bor").exists()
def test_main_missing_local_dir_aborts_before_import(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""Phase 38: a DB local row whose directory is missing → exit code
1, ``local source missing: <path>`` on stderr, zero import attempts
(no ``--source`` given, so the DB row is what should have been
imported)."""
settings = _settings(sources_dir=str(tmp_path / "bor"))
monkeypatch.setattr(import_docs, "get_settings", lambda: settings)
missing = tmp_path / "Gone"
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: ([_local_row(str(missing))], "db"),
)
fake_import = FakeImportSources()
monkeypatch.setattr(import_docs, "import_sources", fake_import)
rc = import_docs.main([])
assert rc == 1
err = capsys.readouterr().err
assert "import_docs: source sync failed" in err
assert f"local source missing: {missing}" in err # the path is named
assert fake_import.calls == [] # no partial import
# --- phase 135: the branch pin (real git, two-branch file:// origin) -------
def _git_available() -> bool:
try:
proc = subprocess.run(["git", "--version"], capture_output=True, check=False)
return proc.returncode == 0
except (FileNotFoundError, OSError):
return False
#: Real ``git`` in the test environment — skipped cleanly without it
#: (the ``test_git_file_dates.py`` house pattern).
GIT = _git_available()
@pytest.fixture()
def two_branch_repo(tmp_path: Path) -> Path:
"""A real-git scratch origin (the phase-135 recipe): ``main`` holds
``a.md``, ``release`` holds ``b.md`` — different content per branch
(a branch checkout is the branch's tree, not main's)."""
repo = tmp_path / "origin"
repo.mkdir()
_git(repo, "init", "-q", "-b", "main")
_git(repo, "config", "commit.gpgsign", "false")
(repo / "a.md").write_text("# A\nmain branch content\n", encoding="utf-8")
_git(repo, "add", "a.md")
_git(repo, "commit", "-q", "-m", "main: add a.md")
_git(repo, "checkout", "-q", "-b", "release")
(repo / "a.md").unlink()
(repo / "b.md").write_text("# B\nrelease branch content v1\n", encoding="utf-8")
_git(repo, "add", "-A")
_git(repo, "commit", "-q", "-m", "release: add b.md")
_git(repo, "checkout", "-q", "main")
return repo
def _git(cwd: Path, *argv: str, date: str | None = None) -> None:
"""Run one git command for the test harness (fixture setup); a
non-zero exit fails the fixture, not the test under test.
``date`` pins the commit date (``GIT_COMMITTER_DATE`` =
``GIT_AUTHOR_DATE``) — deterministic per-file dates for the
phase-136 ``file_commit_dates`` pin (scratch commits would
otherwise share the wall clock's second)."""
env = os.environ.copy()
env["GIT_AUTHOR_NAME"] = "T"
env["GIT_AUTHOR_EMAIL"] = "t@example.com"
env["GIT_COMMITTER_NAME"] = "T"
env["GIT_COMMITTER_EMAIL"] = "t@example.com"
if date is not None:
env["GIT_AUTHOR_DATE"] = date
env["GIT_COMMITTER_DATE"] = date
proc = subprocess.run(
["git", *argv], cwd=cwd, env=env, capture_output=True, text=True, check=False
)
assert proc.returncode == 0, f"git {' '.join(argv)} failed: {proc.stderr}"
def _git_out(cwd: Path, *argv: str) -> str:
"""One harness git read (a non-zero exit fails the test)."""
proc = subprocess.run(
["git", *argv], cwd=cwd, capture_output=True, text=True, check=False
)
assert proc.returncode == 0, f"git {' '.join(argv)} failed: {proc.stderr}"
return proc.stdout
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_clone_or_pull_fresh_branch_checkout_is_on_that_branch(
tmp_path: Path, two_branch_repo: Path
) -> None:
"""Phase 135: a FRESH ``clone_or_pull(url, dest, 'release')``
(real git, ``file://``) checks out the pinned branch — HEAD is
``release``, file B present, file A (main's tree) absent."""
url = f"file://{two_branch_repo}"
dest = tmp_path / "bor" / "origin"
result = clone_or_pull(url, dest, "release")
assert result == dest
assert _git_out(dest, "rev-parse", "--abbrev-ref", "HEAD").strip() == "release"
assert (dest / "b.md").is_file()
assert (dest / "b.md").read_text(encoding="utf-8") == "# B\nrelease branch content v1\n"
assert not (dest / "a.md").exists()
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_clone_or_pull_existing_branch_checkout_ff_merges_new_commits(
tmp_path: Path, two_branch_repo: Path
) -> None:
"""Phase 135: a SECOND ``clone_or_pull`` of the same pinned branch
after the origin gains a commit on it → the guarded checkout is
ff-merged onto the new tip (file B updated, no error, still on
``release``)."""
url = f"file://{two_branch_repo}"
dest = tmp_path / "bor" / "origin"
clone_or_pull(url, dest, "release")
# the origin gains a commit on release (v2 of file B)
_git(two_branch_repo, "checkout", "-q", "release")
(two_branch_repo / "b.md").write_text("# B\nrelease branch content v2\n", encoding="utf-8")
_git(two_branch_repo, "add", "b.md")
_git(two_branch_repo, "commit", "-q", "-m", "release: bump b.md")
_git(two_branch_repo, "checkout", "-q", "main")
clone_or_pull(url, dest, "release")
assert _git_out(dest, "rev-parse", "--abbrev-ref", "HEAD").strip() == "release"
assert (dest / "b.md").read_text(encoding="utf-8") == "# B\nrelease branch content v2\n"
assert not (dest / "a.md").exists()
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_clone_or_pull_branch_mismatch_fails_loud_checkout_untouched(
tmp_path: Path, two_branch_repo: Path
) -> None:
"""Phase 135 (ASSUMPTION B5): a checkout on the DEFAULT branch
(fresh clone with no branch → ``main``) + a source row pinning
``release`` → the remove-and-re-add ``GitSyncError`` — and the
checkout is UNTOUCHED (still on main, still main's tree: no silent
branch switch)."""
url = f"file://{two_branch_repo}"
dest = tmp_path / "bor" / "origin"
clone_or_pull(url, dest) # no branch → the origin's default (main)
assert _git_out(dest, "rev-parse", "--abbrev-ref", "HEAD").strip() == "main"
assert (dest / "a.md").is_file()
with pytest.raises(GitSyncError) as excinfo:
clone_or_pull(url, dest, "release")
msg = str(excinfo.value)
assert "remove and re-add the source to switch branches" in msg
assert "'main'" in msg and "'release'" in msg
assert url not in msg # the guard message never carries the URL
# the checkout is untouched: still on main, still main's tree
assert _git_out(dest, "rev-parse", "--abbrev-ref", "HEAD").strip() == "main"
assert (dest / "a.md").is_file()
assert not (dest / "b.md").exists()
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_resolve_sources_git_row_with_branch_checks_out_that_branch(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, two_branch_repo: Path
) -> None:
"""Phase 135, the CLI seam: a ``git_sources`` row carrying
``branch='release'`` → ``_resolve_sources`` passes ``row.branch`` to
the REAL ``clone_or_pull`` (no fake — the branch is honored
end to end): the resolved root is the RELEASE tree (file B imported,
file A absent), and the date walk over the real checkout lists ONLY
the branch's tree (head-of-branch semantics, ``file_commit_dates``
untouched)."""
url = f"file://{two_branch_repo}"
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: ([GitSource(url=url, kind="git", branch="release")], "db"),
)
settings = _settings(sources_dir=str(tmp_path / "bor"))
sources, ignore_map, hidden_map, date_map = import_docs._resolve_sources(None, settings)
root = tmp_path / "bor" / "origin"
assert sources == [root]
assert ignore_map == {}
assert hidden_map == {str(root): False}
assert (root / "b.md").is_file() # the branch's file is what gets imported
assert not (root / "a.md").exists()
# the (real, untouched) date walk over the branch checkout yields
# TRUE dates for the branch's files (not the fail-soft ``{}`` — the
# checkout is a real full-history repo). The walk reads the
# reachable LOG (``git log --name-only``), so it also lists files
# from shared ancestor commits (``a.md`` — release was cut after
# main's first commit); the importer only consults the map for the
# files it actually finds in the working tree, so that is harmless.
dates = date_map[str(root)]
assert "b.md" in dates
assert isinstance(dates["b.md"], datetime)
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_resolve_sources_row_without_branch_clones_default_branch(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, two_branch_repo: Path
) -> None:
"""Phase 135, the byte-identical leg: a row WITHOUT a branch (the
env-fallback synthetic rows' shape — ``GitSource(url=…, kind="git")``
carries the model default ``branch=None``) → the real
``clone_or_pull`` takes the pre-phase path: the origin's DEFAULT
branch (``main``) is checked out — file A present, file B absent."""
url = f"file://{two_branch_repo}"
monkeypatch.setattr(
import_docs,
"effective_sources",
lambda db: ([_git_row(url)], "env"), # branch defaults to None
)
settings = _settings(git_sources=url, sources_dir=str(tmp_path / "bor"))
sources, *_ = import_docs._resolve_sources(None, settings)
root = tmp_path / "bor" / "origin"
assert sources == [root]
assert (root / "a.md").is_file() # main's file — the default branch
assert not (root / "b.md").exists()
# --- phase 136: the ff-or-reset (real git, the force-push repro) -----------
@pytest.fixture()
def single_branch_repo(tmp_path: Path) -> Path:
"""A real-git scratch origin on ``main`` with one file (``a.md``)
(the phase-136 base shape — the owner's single-branch source;
the root commit's date is pinned for the deterministic
``file_commit_dates`` pin below)."""
repo = tmp_path / "origin"
repo.mkdir()
_git(repo, "init", "-q", "-b", "main")
_git(repo, "config", "commit.gpgsign", "false")
(repo / "a.md").write_text("# A\noriginal content\n", encoding="utf-8")
_git(repo, "add", "a.md")
_git(repo, "commit", "-q", "-m", "root: add a.md", date="2026-01-05T10:00:00+00:00")
return repo
def _force_push_new_root(
origin: Path,
files: dict[str, str | None],
message: str,
*,
branch: str = "main",
date: str | None = None,
) -> None:
"""Rewrite the origin's ``branch`` to a NEW ROOT commit (the
phase-136 force push): ``git checkout --orphan`` + commit, so the
branch now points at a root with NO shared ancestry with the old
history — the strongest divergence, exactly what ``--ff-only``
refuses. ``files`` maps name → new content (the orphan commit
holds exactly that set — the old files are dropped)."""
_git(origin, "checkout", "--orphan", "rewritten")
for p in sorted(origin.iterdir()):
if p.is_file() and p.name != ".git":
p.unlink()
for name, content in files.items():
if content is not None:
(origin / name).write_text(content, encoding="utf-8")
_git(origin, "add", "-A")
_git(origin, "commit", "-q", "-m", message, date=date)
_git(origin, "branch", "-D", branch)
_git(origin, "branch", "-m", "rewritten", branch)
_git(origin, "checkout", "-q", "main")
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_clone_or_pull_normal_ff_updates_without_reset(
tmp_path: Path, single_branch_repo: Path
) -> None:
"""Phase 136: a normal second commit (a true fast-forward) → the
unified sequence ff-merges: the checkout's HEAD is the origin's
head, the second commit's message is the HEAD subject, NO merge
commit (a pure ff), and the working tree holds the new content —
the reset tail never runs on a clean ff."""
origin = single_branch_repo
url = f"file://{origin}"
dest = tmp_path / "bor" / "origin"
clone_or_pull(url, dest)
(origin / "b.md").write_text("# B\nsecond commit\n", encoding="utf-8")
_git(origin, "add", "b.md")
_git(origin, "commit", "-q", "-m", "second: add b.md", date="2026-02-01T09:00:00+00:00")
clone_or_pull(url, dest)
dest_head = _git_out(dest, "rev-parse", "HEAD").strip()
origin_head = _git_out(origin, "rev-parse", "main").strip()
assert dest_head == origin_head
subjects = _git_out(dest, "log", "--format=%s").splitlines()
assert subjects[0] == "second: add b.md" # the second commit's message is the HEAD
assert subjects == ["second: add b.md", "root: add a.md"] # pure ff: no merge commit
assert (dest / "b.md").read_text(encoding="utf-8") == "# B\nsecond commit\n"
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_clone_or_pull_force_push_resets_to_the_new_head(
tmp_path: Path, single_branch_repo: Path, caplog: pytest.LogCaptureFixture
) -> None:
"""Phase 136 (the owner's repro, TODO L5): after the first clone,
the origin's history is REWRITTEN to a NEW ROOT commit (same
file, new content — the branch now diverges; pre-phase, the next
sync died on the non-ff ``git pull --ff-only`` → the whole sync
landed ``failed``) → ``clone_or_pull`` NO LONGER raises: the
checkout is reset to the origin's NEW head (the checkout's HEAD ==
the origin's head), the working tree holds the NEW content, and
ONE WARNING line (PLAN §9) is the only side effect — no
user-facing error (the log carries the checkout path, never the
URL)."""
origin = single_branch_repo
url = f"file://{origin}"
dest = tmp_path / "bor" / "origin"
new_content = "# A\nrewritten content\n"
clone_or_pull(url, dest)
_force_push_new_root(
origin, {"a.md": new_content}, "force: rewritten root",
date="2026-03-10T12:00:00+00:00",
)
with caplog.at_level(logging.WARNING, logger="scripts.git_sync"):
result = clone_or_pull(url, dest) # NO exception (the pre-phase break)
assert result == dest
dest_head = _git_out(dest, "rev-parse", "HEAD").strip()
origin_head = _git_out(origin, "rev-parse", "main").strip()
assert dest_head == origin_head
assert (dest / "a.md").read_text(encoding="utf-8") == new_content
# ONE WARNING line: the checkout path + the tracked ref, never the URL.
warnings = [r for r in caplog.records if r.levelno == logging.WARNING]
assert len(warnings) == 1
msg = warnings[0].getMessage()
assert "git sync: divergent remote head on " in msg
assert str(dest) in msg
assert "to origin/main" in msg
assert url not in msg # credential-safety: the log never carries the URL
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_clone_or_pull_unrelated_history_force_push_resets(
tmp_path: Path, single_branch_repo: Path
) -> None:
"""Phase 136: the rewritten origin has a DIFFERENT root commit
with a fully UNRELATED tree (a different file, NO shared ancestry
— the ff merge of an unrelated history is exactly what
``--ff-only`` refuses) → the SAME reset path handles it: the
checkout's HEAD is the origin's new head and the working tree IS
the new tree (the old file GONE — the reset is a hard one, not a
merge)."""
origin = single_branch_repo
url = f"file://{origin}"
dest = tmp_path / "bor" / "origin"
clone_or_pull(url, dest)
_force_push_new_root(
origin, {"z.md": "# Z\nunrelated root\n"}, "force: unrelated root",
date="2026-03-11T12:00:00+00:00",
)
result = clone_or_pull(url, dest) # NO exception — the same reset path
assert result == dest
dest_head = _git_out(dest, "rev-parse", "HEAD").strip()
origin_head = _git_out(origin, "rev-parse", "main").strip()
assert dest_head == origin_head
assert (dest / "z.md").read_text(encoding="utf-8") == "# Z\nunrelated root\n"
assert not (dest / "a.md").exists() # the hard reset dropped the old tree's file
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_clone_or_pull_branched_force_push_resets_to_the_branch_head(
tmp_path: Path, two_branch_repo: Path
) -> None:
"""Phase 136 + phase 135: a two-branch origin (the phase-135
shape), a row-shaped call ``clone_or_pull(url, dest,
"release")`` after a force push of ONLY ``release`` → the reset
lands ``origin/release``'s head (the owner's verbatim "reset
origin/<branch>") and ``main``'s content is not involved (file A
is still absent — the checkout holds the branch's tree only)."""
origin = two_branch_repo
url = f"file://{origin}"
dest = tmp_path / "bor" / "origin"
forced = "# B\nrelease branch content FORCED\n"
clone_or_pull(url, dest, "release")
_force_push_new_root(
origin, {"b.md": forced}, "release: force-pushed root",
branch="release", date="2026-03-15T08:00:00+00:00",
)
clone_or_pull(url, dest, "release") # NO exception — the reset landed
assert _git_out(dest, "rev-parse", "--abbrev-ref", "HEAD").strip() == "release"
dest_head = _git_out(dest, "rev-parse", "HEAD").strip()
release_head = _git_out(origin, "rev-parse", "release").strip()
assert dest_head == release_head
assert (dest / "b.md").read_text(encoding="utf-8") == forced
assert not (dest / "a.md").exists() # main's content is not involved
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_file_commit_dates_after_reset_reflects_the_new_head(
tmp_path: Path, single_branch_repo: Path
) -> None:
"""Phase 136: ``file_commit_dates`` over the post-reset checkout
yields the NEW head's per-file dates — the new root commit's date
is what the walk returns for its files (the head-of-branch dates
the owner asked for: the KB is rebuilt from the reset checkout on
the same run, so its document dates are the NEW history's, not
the old one's)."""
origin = single_branch_repo
url = f"file://{origin}"
dest = tmp_path / "bor" / "origin"
clone_or_pull(url, dest)
old_dates = file_commit_dates(dest)
assert old_dates["a.md"] == datetime.fromisoformat("2026-01-05T10:00:00+00:00")
_force_push_new_root(
origin, {"a.md": "# A\nrewritten content\n"}, "force: rewritten root",
date="2026-03-10T12:00:00+00:00",
)
clone_or_pull(url, dest)
new_dates = file_commit_dates(dest)
assert set(new_dates) == {"a.md"} # the new tree's files only
assert new_dates["a.md"] == datetime.fromisoformat("2026-03-10T12:00:00+00:00")
assert new_dates["a.md"] != old_dates["a.md"] # the NEW head dates the file
# --- phase 136: the sync pipeline end to end (the API seam) ---------------
def _reset_sources_meta() -> None:
"""Pin the single-row ``sources_meta`` counter to the
migration-0010 seed (0) — the runner bumps it through its own
short-lived ``SessionLocal`` (the ``test_sync_api`` pattern)."""
session = SessionLocal()
try:
session.execute(text("UPDATE sources_meta SET version = 0 WHERE id = 1"))
session.commit()
finally:
session.close()
@pytest.fixture()
def fresh_sync_state() -> Iterator[None]:
"""Phase 136 (the API-seam pin): the sync runner's module-level
status object + task + cancel event are process-global — reset
them around the test (the ``test_sync_api`` autouse pattern, the
full suite runs both files in one process)."""
sync_api._status = sync_api.SyncStatus()
sync_api._task = None
sync_api._cancel_event = asyncio.Event()
yield
sync_api._status = sync_api.SyncStatus()
sync_api._task = None
sync_api._cancel_event = asyncio.Event()
@pytest.fixture()
def clean_sync_seam(db: Session) -> Iterator[None]:
"""Phase 136 (the API-seam pin): the real sync runner writes
global tables — ``git_sources`` (seeded), ``documents``/
``chunks`` (the real import), and ``folder_summaries`` (the real
generator) — truncated around the test; the ``sources_meta``
counter is pinned to 0 before and after. Skips like the ``db``
fixture when Postgres is down."""
db.execute(text("TRUNCATE git_sources, chunks, documents, folder_summaries"))
db.commit()
_reset_sources_meta()
yield
db.execute(text("TRUNCATE git_sources, chunks, documents, folder_summaries"))
db.commit()
_reset_sources_meta()
@pytest.fixture()
def sync_client() -> Iterator[TestClient]:
"""Context-managed TestClient — one app event loop across
requests (the background sync task must survive between the POST
and the polls — the ``test_sync_api`` pattern; a TestClient
without the context manager would cancel the task on request
exit)."""
with TestClient(fastapi_app) as client:
yield client
def _poll_sync(client: TestClient, want: str, timeout: float = 10.0) -> dict:
"""Poll ``GET /api/sync/status`` until ``state == want``
(terminal). Any state other than ``running`` before the deadline
fails loudly — an unexpected ``failed`` must never be masked by
the wait (the pre-phase force push landed exactly that)."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
body = client.get("/api/sync/status").json()
if body["state"] == want:
return body
assert body["state"] == "running", (
f"unexpected state {body['state']!r} while waiting for {want!r}: {body}"
)
time.sleep(0.05)
raise AssertionError(f"sync did not reach {want!r} within {timeout}s")
@pytest.mark.skipif(not GIT, reason="git CLI not available")
def test_sync_pipeline_survives_a_force_push(
sync_client: TestClient,
monkeypatch: pytest.MonkeyPatch,
db: Session,
tmp_path: Path,
single_branch_repo: Path,
fresh_sync_state: None,
clean_sync_seam: None,
) -> None:
"""Phase 136, the SYNC PIPELINE END-TO-END (the seam
``tests/integration/test_sync_api.py`` already exercises — the
pre-phase force push landed the button sync in ``failed``; this
test is the API-seam regression pin that it now lands
``success``): a real-git ``file://`` origin + a seeded git row;
sync #1 clones and imports the original content; the origin's
history is REWRITTEN to a new root commit (the owner's exact
repro); sync #2 → ``state == "success"`` (NOT ``failed``), the
rewritten file is re-imported (``updated=1``), and the checkout
(and therefore the KB) holds the NEW head. The REAL
``clone_or_pull`` runs (no fake — the reset is what is under
test) over the real import + real overview/folder-summary steps
(deterministic in-process ``FakeEmbedder``, no network)."""
origin = single_branch_repo
url = f"file://{origin}"
db.add(GitSource(url=url, kind="git"))
db.commit()
# The resolver's env fallback pinned empty (the DB row wins — the
# dev ``.env`` never leaks in; the ``test_sync_api`` pattern).
monkeypatch.setattr(
git_sources_resolver,
"get_settings",
lambda: Settings(_env_file=None, git_sources=""), # pyright: ignore[reportCallIssue]
)
monkeypatch.setattr(
sync_api,
"get_settings",
lambda: Settings(_env_file=None, sources_dir=str(tmp_path / "bor")), # pyright: ignore[reportCallIssue]
)
monkeypatch.setattr(sync_api, "LLMClient", lambda: FakeEmbedder()) # probe passes against it
r = sync_client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login failed: {r.status_code} {r.text}"
# Sync #1: the fresh clone imports the original content.
assert sync_client.post("/api/sync").status_code == 202
body1 = _poll_sync(sync_client, "success")
assert body1["detail"]["added"] == 1
# The owner's repro: force push the origin (a NEW ROOT commit).
new_content = "# A\nrewritten content\n"
_force_push_new_root(origin, {"a.md": new_content}, "force: rewritten root")
# Sync #2: pre-phase this landed ``failed`` (the non-ff
# ``git pull --ff-only`` died) — the reset is now self-healing
# and the run SUCCEEDS.
assert sync_client.post("/api/sync").status_code == 202
body2 = _poll_sync(sync_client, "success") # NOT ``failed`` — the regression pin
assert body2["error"] is None
assert body2["detail"]["updated"] == 1 # the rewritten file is re-imported
# The checkout (and therefore the KB) holds the NEW head.
root = tmp_path / "bor" / "origin"
root_head = _git_out(root, "rev-parse", "HEAD").strip()
origin_head = _git_out(origin, "rev-parse", "main").strip()
assert root_head == origin_head
assert (root / "a.md").read_text(encoding="utf-8") == new_content