ducoterra
ee3efb28c9
Everything is verified green. Final report: **Phase 106 — Document dates (final verification pass; all 10 tasks already complete)** - Verified all phase artifacts: alembic `0020` (dev DB at `0020`), `doc_dates.py`, git `file_commit_dates`, importer `doc_dates_by_root`/`dates_updated`, both entry-point wirings, date APIs + tree `created_at`/`updated_at`, LLM surfaces (prompt block, `read` line 2, appended `ls` field), `apply_recency_boost` in `retrieve()`, UI columns/badge, admin editor, mock-LLM regex — all present and correct; no defects found, no fixes needed. - `uv run pytest --cov=app --cov-report=term-missing` → **2299 passed, TOTAL 99%** (>90% ✓) - `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` → **6/6 passed** in isolation (DB up) - 12 regression E2E suites (retrieval_quality, whole_document_context, agent_document_tools, ls_tree_drilldown, read_truncation_cap, kb_tree, kb_tree_nav, document_viewer, edit_summaries, import_documents, sync_button, hidden_folders_toggle, smoke) → **all green in isolation** - `uv run ruff check .` → clean; `uv run pyright` → **0 errors, 0 warnings** **Completion criteria:** 1) non-null `created_at` + 0020 upgrade/downgrade on dev DB ✓ (real-Alembic integration tests) 2) sync refresh/older/manual-persists/content-reset/no sources_meta bump ✓ 3) zip/tar mtime + future→today ✓ 4) LLM date surfaces + cross-check ✓ 5) UI Created/Updated/badge positions ✓ 6) admin editor set+revert round-trip ✓ 7) old-correct-beats-new-similar (defaults & boost-off) + near-tie + `BOR_RECENCY_BOOST=0` byte-identical ✓ 8) full gate ✓ 9) commit/phase-move — left to harness per instructions. - **Notable:** recency default tuned 0.001 → **0.0007** (task 07 step 5 explicitly permits; measured margins recorded in `test_recency_boost.py` docstring). - **Next pending phase:** none — `todo/` holds only this phase.
66 lines
2.8 KiB
Python
66 lines
2.8 KiB
Python
"""Document-creation-date sourcing + normalization (phase 106, D2/D3).
|
|
|
|
Every document date the importer writes and every date the owner
|
|
edits passes through :func:`normalize_doc_date` — the single choke
|
|
point for the owner's rules: an UNDETERMINED date (no source signal)
|
|
and a FUTURE date (beyond a small clock-skew tolerance) both assume
|
|
the document was created TODAY (UTC). Naive source timestamps (zip
|
|
DOS mtimes, tar mtimes, git-free fallbacks) are tz-agnostic epoch-
|
|
based values rendered as UTC; aware ones are converted to UTC.
|
|
|
|
Pure and stdlib-only by contract (unit-pinned in
|
|
``tests/unit/test_doc_dates.py``): no database, no logging, no I/O
|
|
besides :func:`file_mtime_datetime`'s single ``stat`` — the callers
|
|
(importer task 04, the date-edit API task 05) own everything else.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
from datetime import UTC, datetime, timedelta
|
|
from pathlib import Path
|
|
|
|
__all__ = ["FUTURE_SKEW_TOLERANCE", "file_mtime_datetime", "normalize_doc_date"]
|
|
|
|
#: Clock-skew tolerance (D3): a source date up to this far in the
|
|
#: FUTURE is a drifting clock, not a future document — it keeps its
|
|
#: date. Beyond it, the owner's rule applies (→ today).
|
|
FUTURE_SKEW_TOLERANCE = timedelta(days=1)
|
|
|
|
|
|
def normalize_doc_date(raw: datetime | None, now: datetime | None = None) -> datetime:
|
|
"""*raw* → the stored UTC creation date (the D3 rule, pinned).
|
|
|
|
``now`` is injectable (tests); it defaults to
|
|
``datetime.now(UTC)``. ``raw=None`` (undetermined) → *now*;
|
|
naive *raw* → treated as UTC; aware *raw* → converted to UTC;
|
|
*raw* beyond *now* + :data:`FUTURE_SKEW_TOLERANCE` → *now*.
|
|
The result always carries full precision (no date-truncation —
|
|
the display formats, the storage doesn't).
|
|
"""
|
|
if now is None:
|
|
now = datetime.now(UTC)
|
|
elif now.tzinfo is None:
|
|
# The future check compares in AWARE space — a naive ``now``
|
|
# (callers/tests) is a UTC instant, like the naive ``raw``.
|
|
now = now.replace(tzinfo=UTC)
|
|
if raw is None:
|
|
return now
|
|
# Epoch-based source values (zip DOS times, tar mtimes) are
|
|
# tz-agnostic — attach UTC; never assume the host's local TZ.
|
|
# Aware values are converted to UTC (the comparison below is
|
|
# done in aware space).
|
|
raw = raw.replace(tzinfo=UTC) if raw.tzinfo is None else raw.astimezone(UTC)
|
|
if raw > now + FUTURE_SKEW_TOLERANCE:
|
|
# Genuinely future (beyond the clock-skew tolerance) → today.
|
|
return now
|
|
return raw
|
|
|
|
|
|
def file_mtime_datetime(path: Path) -> datetime:
|
|
"""The file's mtime as an aware UTC datetime (the D2 fallback).
|
|
|
|
Epoch mtimes are tz-agnostic — UTC is the correct rendering
|
|
(zip DOS timestamps and tar mtimes pass through the same
|
|
:func:`normalize_doc_date` after unpacking, task 03).
|
|
"""
|
|
return datetime.fromtimestamp(path.stat().st_mtime, tz=UTC)
|