phase: 106_document_dates
Build and Push Containers / build-and-push-app (push) Successful in 4m35s
Build and Push Containers / build-and-push-db (push) Successful in 14s

Everything is verified green. Final report:

**Phase 106 — Document dates (final verification pass; all 10 tasks already complete)**

- Verified all phase artifacts: alembic `0020` (dev DB at `0020`), `doc_dates.py`, git `file_commit_dates`, importer `doc_dates_by_root`/`dates_updated`, both entry-point wirings, date APIs + tree `created_at`/`updated_at`, LLM surfaces (prompt block, `read` line 2, appended `ls` field), `apply_recency_boost` in `retrieve()`, UI columns/badge, admin editor, mock-LLM regex — all present and correct; no defects found, no fixes needed.
- `uv run pytest --cov=app --cov-report=term-missing` → **2299 passed, TOTAL 99%** (>90% ✓)
- `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` → **6/6 passed** in isolation (DB up)
- 12 regression E2E suites (retrieval_quality, whole_document_context, agent_document_tools, ls_tree_drilldown, read_truncation_cap, kb_tree, kb_tree_nav, document_viewer, edit_summaries, import_documents, sync_button, hidden_folders_toggle, smoke) → **all green in isolation**
- `uv run ruff check .` → clean; `uv run pyright` → **0 errors, 0 warnings**

**Completion criteria:** 1) non-null `created_at` + 0020 upgrade/downgrade on dev DB ✓ (real-Alembic integration tests) 2) sync refresh/older/manual-persists/content-reset/no sources_meta bump ✓ 3) zip/tar mtime + future→today ✓ 4) LLM date surfaces + cross-check ✓ 5) UI Created/Updated/badge positions ✓ 6) admin editor set+revert round-trip ✓ 7) old-correct-beats-new-similar (defaults & boost-off) + near-tie + `BOR_RECENCY_BOOST=0` byte-identical ✓ 8) full gate ✓ 9) commit/phase-move — left to harness per instructions.

- **Notable:** recency default tuned 0.001 → **0.0007** (task 07 step 5 explicitly permits; measured margins recorded in `test_recency_boost.py` docstring).
- **Next pending phase:** none — `todo/` holds only this phase.
This commit is contained in:
2026-09-13 19:28:05 -04:00
parent cec819743d
commit ee3efb28c9
113 changed files with 8228 additions and 344 deletions
+60 -36
View File
@@ -70,9 +70,11 @@ task 04):
(the count is the subfolder's recursive subtree — every document
whose path equals the folder or starts with ``folder + "/"``, the
same set the sync-time folder summary describes — and the file
lines ``source: X | path: Y | title: Z`` (the canonical
``read``/``grep`` identity — the phase-63 labeled format,
unchanged) in path order (``GET /api/docs`` order), capped at
lines ``source: X | path: Y | title: Z | date: YYYY-MM-DD`` (the
canonical ``read``/``grep`` identity — the phase-63 labeled format —
plus the phase-106 D5 ``date`` field APPENDED after ``title``; only
FILE lines carry a date — source/folder lines are not documents)
in path order (``GET /api/docs`` order), capped at
:data:`LS_MAX_FILE_LINES` lines + one deterministic grep-pointer
note for the rest (a 500-file folder costs 50 lines, never 500);
a ``source/folder`` ``path``: that folder's subfolders + own file
@@ -463,7 +465,8 @@ NO_DOCUMENT_DID_YOU_MEAN_MANY = (
SUGGESTION_LIMIT = 3
#: The drill-down ``ls`` file-line cap (phase 94, task 03): a folder's
#: own files list at most this many ``source: X | path: Y | title: Z``
#: own files list at most this many
#: ``source: X | path: Y | title: Z | date: YYYY-MM-DD``
#: lines (path order), then one deterministic grep-pointer note — a
#: 500-file folder costs the model 50 lines + the note, never 500.
#: Pinned module constant (no env var — the phase-94 TODO asks for a
@@ -655,14 +658,17 @@ def _source_root_summaries(db: Session) -> list[tuple[str, str]]:
]
def _source_document_rows(db: Session, source: str) -> list[tuple[str, str]]:
"""``(path, title)`` of every document under *source*, ordered by
``path`` — the one bounded fetch a folder drill level lists (phase
94 task 03; one source's paths, not the whole KB)."""
def _source_document_rows(db: Session, source: str) -> list[tuple[str, str, str]]:
"""``(path, title, created_iso_date)`` of every document under
*source*, ordered by ``path`` — the one bounded fetch a folder
drill level lists (phase 94 task 03; one source's paths, not the
whole KB). The date is the row's ``created_at`` UTC date part
(``YYYY-MM-DD``, phase 106 D5 — the ``ls`` FILE line's appended
`` | date: …`` field; only file lines carry a date)."""
return [
(path, title)
for path, title in db.execute(
select(Document.path, Document.title)
(path, title, created_at.strftime("%Y-%m-%d"))
for path, title, created_at in db.execute(
select(Document.path, Document.title, Document.created_at)
.where(Document.source == source)
.order_by(Document.path)
)
@@ -707,15 +713,17 @@ def ls_top(db: Session) -> list[tuple[str, int, str | None]]:
def group_folder_listing(
source: str,
folder: str,
rows: Sequence[tuple[str, str]],
rows: Sequence[tuple[str, str, str]],
summaries: Mapping[str, str],
) -> tuple[list[tuple[str, int, str | None]], list[tuple[str, str, str]], int]:
) -> tuple[
list[tuple[str, int, str | None]], list[tuple[str, str, str, str]], int
]:
"""One level of the drill-down tree (phase 94, task 03) — pure.
Given *rows* — the source's ``(path, title)`` pairs in catalog
(path) order — and *summaries* (the source's stored
``folder_summaries`` rows: ``folder_path → summary``), the folder
level *folder* (source-relative; ``""`` = the source root):
Given *rows* — the source's ``(path, title, created_iso_date)``
triples in catalog (path) order — and *summaries* (the source's
stored ``folder_summaries`` rows: ``folder_path → summary``), the
folder level *folder* (source-relative; ``""`` = the source root):
* **(a) direct subfolders** — the folders whose parent is exactly
*folder*, in path order, each
@@ -732,8 +740,10 @@ def group_folder_listing(
prefix before the last ``/`` —
:func:`app.rag.folder_summaries.folder_of`, the shared notion) IS
*folder*, in path order (catalog order — the same order
``GET /api/docs`` serves), as ``(source, path, title)`` triples
— the canonical ``read``/``grep`` identity, capped at
``GET /api/docs`` serves), as ``(source, path, title, date)``
4-tuples — the canonical ``read``/``grep`` identity plus the
phase-106 D5 ``date`` field (the row's ``created_at`` UTC date
part, APPENDED — never inserted before ``title``), capped at
:data:`LS_MAX_FILE_LINES` (the rest fold into the renderer's
note; a 500-file folder never costs 500 lines).
* **(c) the TOTAL direct-file count** — pre-cap, for the note.
@@ -745,7 +755,7 @@ def group_folder_listing(
# indexed path (the existence rule's candidate set — a folder is
# present iff at least one path starts with ``folder + "/"``).
folders: set[str] = set()
for path, _title in rows:
for path, _title, _date in rows:
f = folder_of(path)
while f:
folders.add(f)
@@ -755,7 +765,7 @@ def group_folder_listing(
# folder + "/"`` arm (the folder's true descendants), one pass per
# document.
counts: dict[str, int] = {f: 0 for f in folders}
for path, _title in rows:
for path, _title, _date in rows:
if path in folders:
counts[path] += 1
f = folder_of(path)
@@ -767,8 +777,8 @@ def group_folder_listing(
for g in sorted(g for g in folders if folder_of(g) == folder)
]
files = [
(source, path, title)
for path, title in rows
(source, path, title, date)
for path, title, date in rows
if folder_of(path) == folder
]
return subfolders, files[:LS_MAX_FILE_LINES], len(files)
@@ -776,7 +786,9 @@ def group_folder_listing(
def ls_folder(
db: Session, source: str, folder: str
) -> tuple[list[tuple[str, int, str | None]], list[tuple[str, str, str]], int]:
) -> tuple[
list[tuple[str, int, str | None]], list[tuple[str, str, str, str]], int
]:
"""One folder level of the drill-down ``ls`` (phase 94, task 03).
The source's document rows (:func:`_source_document_rows`) and
@@ -793,7 +805,7 @@ def ls_folder(
)
def _folder_exists_in(rows: Sequence[tuple[str, str]], folder: str) -> bool:
def _folder_exists_in(rows: Sequence[tuple[str, str, str]], folder: str) -> bool:
"""The phase-94 folder-existence rule (``00_phase.md``), pure.
Folder *folder* (source-relative) under a registered source
@@ -805,11 +817,11 @@ def _folder_exists_in(rows: Sequence[tuple[str, str]], folder: str) -> bool:
if not folder:
return True
prefix = folder + "/"
return any(path.startswith(prefix) for path, _title in rows)
return any(path.startswith(prefix) for path, _title, _date in rows)
def _deepest_existing_ancestor(
rows: Sequence[tuple[str, str]], folder: str
rows: Sequence[tuple[str, str, str]], folder: str
) -> str:
"""The deepest EXISTING folder prefix of a missing *folder* (pure).
@@ -854,7 +866,7 @@ def render_ls_top(entries: Sequence[tuple[str, int, str | None]]) -> str:
def render_folder_listing(
identity: str,
subfolders: Sequence[tuple[str, int, str | None]],
files: Sequence[tuple[str, str, str]],
files: Sequence[tuple[str, str, str, str]],
total_files: int,
) -> str:
"""One folder level of the drill-down ``ls`` (phase 94, task 03) —
@@ -867,9 +879,10 @@ def render_folder_listing(
below the header — a blank line, the 2-space-indented subfolder
lines `` {sub}/ — {m} documents`` in path order (``: {summary}``
appended ONLY when the subfolder's summary is stored), a blank
line, the file lines in EXACTLY the existing
``source: X | path: Y | title: Z`` format (the canonical
``read``/``grep`` identity — unchanged), and the cap note
line, the file lines in EXACTLY the
``source: X | path: Y | title: Z | date: YYYY-MM-DD`` format (the
canonical ``read``/``grep`` identity plus the phase-106 D5
appended ``date`` field — the only changed part), and the cap note
``…and {hidden} more documents in this folder — use grep
(pattern) to find a specific one.`` ONLY when the folder's own
files outnumber :data:`LS_MAX_FILE_LINES` (*files* arrives capped;
@@ -891,8 +904,8 @@ def render_folder_listing(
if files or total_files > len(files):
body.append("")
body.extend(
f"source: {source} | path: {path} | title: {title}"
for source, path, title in files
f"source: {source} | path: {path} | title: {title} | date: {date}"
for source, path, title, date in files
)
hidden = total_files - len(files)
if hidden > 0:
@@ -1172,15 +1185,26 @@ def _execute_tool(
holder.read_truncations.append(
(cast("str", raw_path), cap, len(doc.content))
)
# Phase 106 (D5): the date rides every document the model
# sees — the ``read`` result's SECOND line; the FIRST line
# stays ``Document {source}/{path}:`` BYTE-IDENTICAL (the
# E2E mock's ``_READ_RESULT_PREFIX`` header contract).
return (
f"Document {doc.source}/{doc.path}:\n"
f"date: {doc.created_at:%Y-%m-%d}\n"
f"{doc.content[:cap]}\n"
f"{TRUNCATION_MARKER}\n"
f"{READ_TRUNCATION_NOTICE.format(shown=cap, total=len(doc.content))}"
)
# At or under the cap: byte-identical to the pre-phase-95 result
# (no marker, no notice, no holder entry, no ToolResultPiece).
return f"Document {doc.source}/{doc.path}:\n{doc.content}"
# At or under the cap: the pre-phase-95 result plus the
# phase-106 D5 date line (first line byte-identical — the
# mock's header contract; no marker, no notice, no holder
# entry, no ToolResultPiece).
return (
f"Document {doc.source}/{doc.path}:\n"
f"date: {doc.created_at:%Y-%m-%d}\n"
f"{doc.content}"
)
if call.name == "grep":
raw_pattern = call.arguments.get("pattern")
pattern = raw_pattern.strip() if isinstance(raw_pattern, str) else ""