phase: 106_document_dates
Everything is verified green. Final report: **Phase 106 — Document dates (final verification pass; all 10 tasks already complete)** - Verified all phase artifacts: alembic `0020` (dev DB at `0020`), `doc_dates.py`, git `file_commit_dates`, importer `doc_dates_by_root`/`dates_updated`, both entry-point wirings, date APIs + tree `created_at`/`updated_at`, LLM surfaces (prompt block, `read` line 2, appended `ls` field), `apply_recency_boost` in `retrieve()`, UI columns/badge, admin editor, mock-LLM regex — all present and correct; no defects found, no fixes needed. - `uv run pytest --cov=app --cov-report=term-missing` → **2299 passed, TOTAL 99%** (>90% ✓) - `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` → **6/6 passed** in isolation (DB up) - 12 regression E2E suites (retrieval_quality, whole_document_context, agent_document_tools, ls_tree_drilldown, read_truncation_cap, kb_tree, kb_tree_nav, document_viewer, edit_summaries, import_documents, sync_button, hidden_folders_toggle, smoke) → **all green in isolation** - `uv run ruff check .` → clean; `uv run pyright` → **0 errors, 0 warnings** **Completion criteria:** 1) non-null `created_at` + 0020 upgrade/downgrade on dev DB ✓ (real-Alembic integration tests) 2) sync refresh/older/manual-persists/content-reset/no sources_meta bump ✓ 3) zip/tar mtime + future→today ✓ 4) LLM date surfaces + cross-check ✓ 5) UI Created/Updated/badge positions ✓ 6) admin editor set+revert round-trip ✓ 7) old-correct-beats-new-similar (defaults & boost-off) + near-tie + `BOR_RECENCY_BOOST=0` byte-identical ✓ 8) full gate ✓ 9) commit/phase-move — left to harness per instructions. - **Notable:** recency default tuned 0.001 → **0.0007** (task 07 step 5 explicitly permits; measured margins recorded in `test_recency_boost.py` docstring). - **Next pending phase:** none — `todo/` holds only this phase.
This commit is contained in:
+18
-4
@@ -36,9 +36,13 @@ decisions):
|
||||
4. ``import_sources(..., prune=True)`` over the single combined list
|
||||
(git checkouts + local dirs), honoring each row's ``ignore_paths``
|
||||
(phase 89 — the per-root ignore map is built in the same per-row
|
||||
loop as the source list) and its ``include_hidden`` flag (phase 105
|
||||
— the per-root hidden-folders map, same per-row construction) —
|
||||
prune so files deleted upstream, out of
|
||||
loop as the source list), its ``include_hidden`` flag (phase 105 —
|
||||
the per-root hidden-folders map, same per-row construction), and
|
||||
feeding each git checkout's per-file last-commit dates (phase 106,
|
||||
D2 — the per-root date map, ``file_commit_dates`` after the clone,
|
||||
same per-row construction; local rows contribute nothing and take
|
||||
the importer's mtime fallback) — prune so files deleted upstream,
|
||||
out of
|
||||
a local dir, or newly matching an ignore pattern leave the index
|
||||
(pruning covers the union; the CLI's no-prune default is unchanged);
|
||||
5. when the import changed the KB (added + updated > 0),
|
||||
@@ -117,7 +121,7 @@ from app.rag.importer import ImportSummary, import_sources
|
||||
from app.rag.llm import LLMClient, check_models
|
||||
from app.rag.overview import regenerate_overview
|
||||
from app.rag.sources_meta import bump_sources_version, current_sources_version
|
||||
from scripts.git_sync import GitSyncError, clone_or_pull
|
||||
from scripts.git_sync import GitSyncError, clone_or_pull, file_commit_dates
|
||||
from scripts.import_docs import repo_name
|
||||
|
||||
logger = logging.getLogger("app.api.sync")
|
||||
@@ -286,9 +290,17 @@ async def _run_sync() -> None:
|
||||
sources: list[Path] = []
|
||||
ignore_by_root: dict[str, list[str]] = {}
|
||||
include_hidden_by_root: dict[str, bool] = {}
|
||||
doc_dates_by_root: dict[str, dict[str, datetime]] = {}
|
||||
for row in rows:
|
||||
if row.kind == "git":
|
||||
root = clone_or_pull(row.url, sources_root / repo_name(row.url))
|
||||
# Phase 106 (D2): the checkout's per-file last-commit
|
||||
# dates, keyed by the SAME root string the importer
|
||||
# sees (local-path checkouts → true per-file dates,
|
||||
# shallow URL checkouts → the tip date, D10). Local
|
||||
# rows contribute nothing — the importer's mtime
|
||||
# fallback applies to them.
|
||||
doc_dates_by_root[str(root)] = file_commit_dates(root)
|
||||
else:
|
||||
# kind=local — the stored expanded path (phase 38 also
|
||||
# mirrors it in the NOT-NULL ``url`` location column, the
|
||||
@@ -329,6 +341,7 @@ async def _run_sync() -> None:
|
||||
summary: ImportSummary = await import_sources(
|
||||
sources, llm, prune=True, progress=_hook, ignore_by_root=ignore_by_root,
|
||||
include_hidden_by_root=include_hidden_by_root,
|
||||
doc_dates_by_root=doc_dates_by_root,
|
||||
)
|
||||
overview = False
|
||||
if summary.added + summary.updated > 0:
|
||||
@@ -431,6 +444,7 @@ async def _run_sync() -> None:
|
||||
"chunks": summary.chunks,
|
||||
"summaries": summary.summaries,
|
||||
"summary_errors": summary.summary_errors,
|
||||
"dates_updated": summary.dates_updated,
|
||||
"overview": overview,
|
||||
"sources_version": sources_version,
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user