phase: 106_document_dates
Build and Push Containers / build-and-push-app (push) Successful in 4m35s
Build and Push Containers / build-and-push-db (push) Successful in 14s

Everything is verified green. Final report:

**Phase 106 — Document dates (final verification pass; all 10 tasks already complete)**

- Verified all phase artifacts: alembic `0020` (dev DB at `0020`), `doc_dates.py`, git `file_commit_dates`, importer `doc_dates_by_root`/`dates_updated`, both entry-point wirings, date APIs + tree `created_at`/`updated_at`, LLM surfaces (prompt block, `read` line 2, appended `ls` field), `apply_recency_boost` in `retrieve()`, UI columns/badge, admin editor, mock-LLM regex — all present and correct; no defects found, no fixes needed.
- `uv run pytest --cov=app --cov-report=term-missing` → **2299 passed, TOTAL 99%** (>90% ✓)
- `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` → **6/6 passed** in isolation (DB up)
- 12 regression E2E suites (retrieval_quality, whole_document_context, agent_document_tools, ls_tree_drilldown, read_truncation_cap, kb_tree, kb_tree_nav, document_viewer, edit_summaries, import_documents, sync_button, hidden_folders_toggle, smoke) → **all green in isolation**
- `uv run ruff check .` → clean; `uv run pyright` → **0 errors, 0 warnings**

**Completion criteria:** 1) non-null `created_at` + 0020 upgrade/downgrade on dev DB ✓ (real-Alembic integration tests) 2) sync refresh/older/manual-persists/content-reset/no sources_meta bump ✓ 3) zip/tar mtime + future→today ✓ 4) LLM date surfaces + cross-check ✓ 5) UI Created/Updated/badge positions ✓ 6) admin editor set+revert round-trip ✓ 7) old-correct-beats-new-similar (defaults & boost-off) + near-tie + `BOR_RECENCY_BOOST=0` byte-identical ✓ 8) full gate ✓ 9) commit/phase-move — left to harness per instructions.

- **Notable:** recency default tuned 0.001 → **0.0007** (task 07 step 5 explicitly permits; measured margins recorded in `test_recency_boost.py` docstring).
- **Next pending phase:** none — `todo/` holds only this phase.
This commit is contained in:
2026-09-13 19:28:05 -04:00
parent cec819743d
commit ee3efb28c9
113 changed files with 8228 additions and 344 deletions
+99 -2
View File
@@ -11,15 +11,33 @@ used.
This module is the only place the ``git`` CLI is invoked (A11: stdlib
``subprocess`` only, no new packages) — every git command goes through
:func:`run_git`: the clone/pull in :func:`clone_or_pull` and the
:func:`run_git`: the clone/pull in :func:`clone_or_pull`, the per-file
last-commit-date walk in :func:`file_commit_dates` (phase 106), and the
docs-push sequence in :mod:`app.core.docs_push` (phase 59).
Per-file last-commit dates (phase 106, D2/D10) — behavior verified
against scratch repos 2026-09-13:
* a LOCAL-PATH checkout made by :func:`clone_or_pull` keeps FULL
history (``git clone --depth 1 /local/path`` prints "--depth is
ignored in local clones" and does not shallow) →
:func:`file_commit_dates` yields TRUE per-file last-commit dates;
* a URL-TRANSPORT checkout (https/ssh/``file://``) is shallow, and in a
shallow clone git reports the TIP commit as every existing file's
last commit (the shallow boundary is each file's history root) → a
uniform per-repo tip date: no intra-repo distortion, a real
cross-source signal, refreshed on every pull.
"""
from __future__ import annotations
import logging
import subprocess
from datetime import datetime
from pathlib import Path
__all__ = ["GitSyncError", "clone_or_pull", "run_git"]
logger = logging.getLogger(__name__)
__all__ = ["GitSyncError", "clone_or_pull", "file_commit_dates", "run_git"]
class GitSyncError(RuntimeError):
@@ -48,6 +66,85 @@ def clone_or_pull(url: str, dest: Path | str) -> Path:
return dest
def _parse_commit_dates(output: str) -> dict[str, datetime]:
"""Parse ``git log --name-only --format=@@%cI`` output (newest first).
A ``@@<ISO-8601>`` line starts a commit (``%cI`` is ISO-strict, so
the date is always aware — parsed with ``datetime.fromisoformat``);
the following non-empty, non-``@@`` lines are repo-relative paths.
The FIRST sighting of a path wins (the walk is newest-first) — that
is the file's last-commit date. Paths are split on whitespace (like
name-only output), ``\\``-normalized to ``/``, and a leading ``/``
is stripped. Raises ``ValueError`` on a malformed commit date or a
path line before any commit header (the caller fails soft).
"""
dates: dict[str, datetime] = {}
commit: datetime | None = None
for line in output.splitlines():
line = line.strip()
if not line:
continue
if line.startswith("@@"):
commit = datetime.fromisoformat(line[2:])
continue
if commit is None:
raise ValueError(f"path line before any commit header: {line!r}")
for raw_path in line.split():
path = raw_path.replace("\\", "/").lstrip("/")
if path:
dates.setdefault(path, commit)
return dates
def file_commit_dates(dest: Path | str) -> dict[str, datetime]:
"""Per-file last-commit dates for one checkout (phase 106, D2).
ONE ``git log --name-only --format=@@%cI`` walk through
:func:`run_git` (the A11 single-invocation site, one git call per
source per sync) → ``{repo-relative POSIX path: last-commit
datetime}``, newest-first so the first sighting of a path wins.
The checkout behavior is pinned (verified 2026-09-13 — see the
module docstring): a local-path ``clone_or_pull`` checkout keeps
FULL history → TRUE per-file dates; a URL-transport checkout is
shallow → the repo's TIP-commit date for every working-tree file
(D10: uniform within the repo, real across sources).
Fail-soft (pinned): a missing/non-directory checkout, a git failure
(:class:`GitSyncError`), or ANY parse anomaly logs a warning and
returns ``{}`` — the importer falls back to file mtimes; a date
walk must never break a sync.
"""
dest = Path(dest)
if not dest.is_dir():
logger.warning(
"file_commit_dates: %s is not a directory — no git dates "
"(the importer will fall back to file mtimes)",
dest,
)
return {}
try:
output = run_git(["git", "log", "--name-only", "--format=@@%cI"], cwd=dest)
except GitSyncError as exc:
logger.warning(
"file_commit_dates: git log failed for %s: %s — the importer "
"will fall back to file mtimes",
dest,
exc,
)
return {}
try:
return _parse_commit_dates(output)
except ValueError as exc:
logger.warning(
"file_commit_dates: unparseable git log output for %s (%s) — the "
"importer will fall back to file mtimes",
dest,
exc,
)
return {}
def run_git(argv: list[str], cwd: Path) -> str:
"""Run one git command, capturing output; raise GitSyncError on failure.