phase: 106_document_dates

Everything is verified green. Final report:

**Phase 106 — Document dates (final verification pass; all 10 tasks already complete)**

- Verified all phase artifacts: alembic `0020` (dev DB at `0020`), `doc_dates.py`, git `file_commit_dates`, importer `doc_dates_by_root`/`dates_updated`, both entry-point wirings, date APIs + tree `created_at`/`updated_at`, LLM surfaces (prompt block, `read` line 2, appended `ls` field), `apply_recency_boost` in `retrieve()`, UI columns/badge, admin editor, mock-LLM regex — all present and correct; no defects found, no fixes needed.
- `uv run pytest --cov=app --cov-report=term-missing` → **2299 passed, TOTAL 99%** (>90% ✓)
- `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` → **6/6 passed** in isolation (DB up)
- 12 regression E2E suites (retrieval_quality, whole_document_context, agent_document_tools, ls_tree_drilldown, read_truncation_cap, kb_tree, kb_tree_nav, document_viewer, edit_summaries, import_documents, sync_button, hidden_folders_toggle, smoke) → **all green in isolation**
- `uv run ruff check .` → clean; `uv run pyright` → **0 errors, 0 warnings**

**Completion criteria:** 1) non-null `created_at` + 0020 upgrade/downgrade on dev DB ✓ (real-Alembic integration tests) 2) sync refresh/older/manual-persists/content-reset/no sources_meta bump ✓ 3) zip/tar mtime + future→today ✓ 4) LLM date surfaces + cross-check ✓ 5) UI Created/Updated/badge positions ✓ 6) admin editor set+revert round-trip ✓ 7) old-correct-beats-new-similar (defaults & boost-off) + near-tie + `BOR_RECENCY_BOOST=0` byte-identical ✓ 8) full gate ✓ 9) commit/phase-move — left to harness per instructions.

- **Notable:** recency default tuned 0.001 → **0.0007** (task 07 step 5 explicitly permits; measured margins recorded in `test_recency_boost.py` docstring).
- **Next pending phase:** none — `todo/` holds only this phase.

app/rag/importer.py

@@ -31,6 +31,17 @@ previously-imported junk (e.g. dot-dir READMEs) leaves the index. Per-file
 logging uses the verbs ``added | updated | unchanged | pruned`` plus a
 summary line with per-format counts (PLAN §9).
 
+Document dates (phase 106, D2/D4): every import sources
+``documents.created_at`` from the file's source — the per-file git
+last-commit date when a ``doc_dates_by_root`` entry names the file,
+else the file's mtime — normalized by
+:func:`app.rag.doc_dates.normalize_doc_date` (undetermined or future →
+today, D3) on every add and update. On the unchanged path the stored
+date is REFRESHED from the same source (it may go OLDER — no monotonic
+guard) and counted in ``summary.dates_updated`` — unless the row
+carries the owner's manual correction (``created_at_manual``, D1), which
+the sync never touches.
+
 ``import_sources`` accepts an optional per-file ``progress`` callback
 (phase 64, task 01) reporting the file being processed right now.
 """
@@ -51,6 +62,7 @@ from app.config import Settings
 from app.db import SessionLocal
 from app.models import Chunk, Document
 from app.rag.chunker import chunk_document, extract_title
+from app.rag.doc_dates import file_mtime_datetime, normalize_doc_date
 from app.rag.llm import EmbeddingError, LLMError
 from app.rag.summarizer import generate_summary
 
@@ -93,6 +105,12 @@ class ImportSummary:
     #: Non-markdown files whose summary generation failed (best-effort —
     #: the document is still indexed, without a summary).
     summary_errors: int = 0
+    #: Files whose ``created_at`` was refreshed on the UNCHANGED path —
+    #: content untouched, date re-sourced (phase 106, D4: the date may
+    #: go OLDER; a date-only refresh NEVER counts added/updated/pruned,
+    #: so no ``sources_meta`` bump, no overview/folder-summary
+    #: regeneration).
+    dates_updated: int = 0
     #: Files walked, keyed by lowercased extension (``md``, ``yaml``, …).
     formats: dict[str, int] = field(default_factory=dict)
 
@@ -107,7 +125,7 @@ class ImportSummary:
         logger.info(
             "import: summary files=%d added=%d updated=%d unchanged=%d pruned=%d "
             "errors=%d chunks=%d embed_batches=%d summaries=%d summary_errors=%d "
-            "formats=%s",
+            "dates_updated=%d formats=%s",
             self.files,
             self.added,
             self.updated,
@@ -118,6 +136,7 @@ class ImportSummary:
             self.embed_batches,
             self.summaries,
             self.summary_errors,
+            self.dates_updated,
             self.format_counts(),
         )
 
@@ -248,6 +267,7 @@ async def import_sources(
     progress: Callable[[str, str, int, int], None] | None = None,
     ignore_by_root: dict[str, list[str]] | None = None,
     include_hidden_by_root: dict[str, bool] | None = None,
+    doc_dates_by_root: dict[str, dict[str, datetime]] | None = None,
 ) -> ImportSummary:
     """Import every A9-format file under *sources* (see module docstring).
 
@@ -288,6 +308,19 @@ async def import_sources(
     flag ON and is walked again with it OFF simply never enters
     ``seen``, so the next ``prune=True`` run deletes its row
     automatically (A2 — the A9/phase-89 precedent).
+
+    ``doc_dates_by_root`` (phase 106, D2/D4) maps ``str(root)`` — the
+    root path string exactly as passed in *sources* — to that source's
+    RAW per-file source dates: source-relative POSIX path → the git
+    last-commit datetime (task 03's ``file_commit_dates``). ONLY git
+    roots are listed — unlisted roots (local dirs, unpacked uploads)
+    take the mtime fallback, and a path missing from its root's map
+    does too. The map entry beats the file's mtime when present. The
+    progress pre-walk is untouched (dates change no file count).
+    ``None`` (the default) changes nothing for existing callers: the
+    mtime fallback applies to every file — which IS the behavior
+    change, D4: an unchanged file now refreshes its stored date from
+    its source on every run (the backfill-correction case).
     """
     if limit is not None and limit <= 0:
         raise ValueError("limit must be >= 1")
@@ -325,6 +358,10 @@ async def import_sources(
             source_names.add(source)
             ignore = _ignore_for_root(root, ignore_by_root)
             include_hidden = _include_hidden_for_root(root, include_hidden_by_root)
+            # Phase 106 (D2): the root's raw source dates (git last-commit
+            # for git roots, keyed by the same str(root) convention); {}
+            # for unlisted roots — every file then takes the mtime fallback.
+            dates_map = (doc_dates_by_root or {}).get(str(root), {})
             for path in iter_importable_files(
                 root,
                 llm.settings.import_extension_set,
@@ -353,7 +390,7 @@ async def import_sources(
                 try:
                     await _index_file(
                         session, source=source, rel=rel, full_path=path, llm=llm,
-                        summary=summary,
+                        summary=summary, raw_date=dates_map.get(rel),
                     )
                 except EmbeddingError as e:
                     # A pathological file (e.g. content the embedding endpoint
@@ -384,15 +421,45 @@ async def _index_file(
     full_path: Path,
     llm: Embedder,
     summary: ImportSummary,
+    raw_date: datetime | None = None,
 ) -> None:
-    """Upsert one file: doc row + chunk rows + embeddings, one transaction."""
+    """Upsert one file: doc row + chunk rows + embeddings, one transaction.
+
+    ``raw_date`` (phase 106, D2) is the file's RAW source date — the
+    git last-commit datetime from the caller's ``doc_dates_by_root``
+    map, or ``None`` (every non-git case): the file's mtime is read
+    here, once, and becomes the source date (the D2 fallback).
+    """
     settings = llm.settings
     content = full_path.read_text(encoding="utf-8", errors="replace").replace("\x00", "")
     digest = hashlib.sha256(content.encode("utf-8")).hexdigest()
     doc = session.scalar(select(Document).where(Document.source == source, Document.path == rel))
+    if raw_date is None:
+        # D2 fallback: no source date in the map → the file's mtime
+        # (one stat). Read before the unchanged early-return — the
+        # unchanged path refreshes the stored date from the same source.
+        raw_date = file_mtime_datetime(full_path)
     if doc is not None and doc.content_hash == digest:
         summary.unchanged += 1
         logger.info("import: unchanged source=%s path=%s", source, rel)
+        if doc.created_at_manual:
+            # D1/D4: the owner's correction survives the sync — no
+            # write at all (the phase-97 ``manually_edited`` precedent).
+            return
+        # D4: the date refreshes on every sync, including unchanged
+        # files, and may go OLDER (no monotonic guard). A date-only
+        # refresh is still counted ``unchanged`` — never added/updated/
+        # pruned, so no ``sources_meta`` bump and no regeneration.
+        target = normalize_doc_date(raw_date)
+        if target != doc.created_at:
+            doc.created_at = target
+            session.commit()
+            summary.dates_updated += 1
+            logger.info(
+                "import: date-refreshed source=%s path=%s date=%s",
+                source, rel,
+                doc.created_at.isoformat(),
+            )
         return
 
     verb = "updated" if doc is not None else "added"
@@ -411,6 +478,11 @@ async def _index_file(
             content=content,
             content_hash=digest,
             indexed_at=datetime.now(UTC),
+            # Phase 106 (D2/D3): the sourced creation date, normalized
+            # (undetermined or future → today). ``created_at_manual``
+            # stays the column default (False) — only the date-edit API
+            # (task 05) sets it.
+            created_at=normalize_doc_date(raw_date),
         )
         session.add(doc)
     else:
@@ -419,6 +491,11 @@ async def _index_file(
         doc.content = content
         doc.content_hash = digest
         doc.indexed_at = datetime.now(UTC)
+        # Phase 106 (D4): a content change is a new document version —
+        # the date is re-sourced and a previous manual correction is
+        # reset (it referred to the old content).
+        doc.created_at = normalize_doc_date(raw_date)
+        doc.created_at_manual = False
 
     session.flush()  # guarantees doc.id even for brand-new rows