phase: 106_document_dates
Everything is verified green. Final report: **Phase 106 — Document dates (final verification pass; all 10 tasks already complete)** - Verified all phase artifacts: alembic `0020` (dev DB at `0020`), `doc_dates.py`, git `file_commit_dates`, importer `doc_dates_by_root`/`dates_updated`, both entry-point wirings, date APIs + tree `created_at`/`updated_at`, LLM surfaces (prompt block, `read` line 2, appended `ls` field), `apply_recency_boost` in `retrieve()`, UI columns/badge, admin editor, mock-LLM regex — all present and correct; no defects found, no fixes needed. - `uv run pytest --cov=app --cov-report=term-missing` → **2299 passed, TOTAL 99%** (>90% ✓) - `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` → **6/6 passed** in isolation (DB up) - 12 regression E2E suites (retrieval_quality, whole_document_context, agent_document_tools, ls_tree_drilldown, read_truncation_cap, kb_tree, kb_tree_nav, document_viewer, edit_summaries, import_documents, sync_button, hidden_folders_toggle, smoke) → **all green in isolation** - `uv run ruff check .` → clean; `uv run pyright` → **0 errors, 0 warnings** **Completion criteria:** 1) non-null `created_at` + 0020 upgrade/downgrade on dev DB ✓ (real-Alembic integration tests) 2) sync refresh/older/manual-persists/content-reset/no sources_meta bump ✓ 3) zip/tar mtime + future→today ✓ 4) LLM date surfaces + cross-check ✓ 5) UI Created/Updated/badge positions ✓ 6) admin editor set+revert round-trip ✓ 7) old-correct-beats-new-similar (defaults & boost-off) + near-tie + `BOR_RECENCY_BOOST=0` byte-identical ✓ 8) full gate ✓ 9) commit/phase-move — left to harness per instructions. - **Notable:** recency default tuned 0.001 → **0.0007** (task 07 step 5 explicitly permits; measured margins recorded in `test_recency_boost.py` docstring). - **Next pending phase:** none — `todo/` holds only this phase.
This commit is contained in:
@@ -1,8 +1,11 @@
|
||||
"""Evaluate hybrid retrieval against the live knowledge base (phase 09).
|
||||
|
||||
Embeds each question via aipi, runs the same hybrid search the chat API
|
||||
uses (cosine top-N + FTS top-N, RRF-fused), and prints the top-5 documents
|
||||
with their cosine / fts / fused scores plus the honesty-gate verdict:
|
||||
uses (cosine top-N + FTS top-N, RRF-fused — plus the phase-106 recency
|
||||
boost, which ``retrieve()`` applies after the fusion), and prints the
|
||||
top-5 documents with their cosine / fts / fused scores (labelled
|
||||
``effective`` when the recency boost is on — the post-boost score)
|
||||
and the document's creation date, plus the honesty-gate verdict:
|
||||
|
||||
uv run python -m scripts.eval_retrieval "How did I install gitlab?"
|
||||
uv run python -m scripts.eval_retrieval --from-file questions.txt
|
||||
@@ -96,7 +99,9 @@ def main(argv: list[str] | None = None) -> int:
|
||||
print(
|
||||
f"eval: threshold={settings.relevance_threshold} "
|
||||
f"vector_candidates={settings.hybrid_vector_candidates} "
|
||||
f"lexical_candidates={settings.hybrid_lexical_candidates} rrf_k={settings.rrf_k}"
|
||||
f"lexical_candidates={settings.hybrid_lexical_candidates} rrf_k={settings.rrf_k} "
|
||||
f"recency_boost={settings.recency_boost} "
|
||||
f"recency_half_life_days={settings.recency_half_life_days}"
|
||||
)
|
||||
with SessionLocal() as db:
|
||||
for question, vec in zip(questions, vectors, strict=True):
|
||||
@@ -116,10 +121,17 @@ def main(argv: list[str] | None = None) -> int:
|
||||
key = f"{c.document.source}/{c.document.path}"
|
||||
if key not in best_by_doc:
|
||||
best_by_doc[key] = c
|
||||
# Phase 106, D6: the post-recency-boost score is the
|
||||
# effective score ``retrieve()`` ranked by — labelled
|
||||
# ``effective`` while the boost is on (``fused`` when the
|
||||
# kill switch is off), with the document's creation date
|
||||
# alongside (the boost's input).
|
||||
score_label = "effective" if settings.recency_boost > 0 else "fused"
|
||||
for i, c in enumerate(list(best_by_doc.values())[: args.top], start=1):
|
||||
print(
|
||||
f" {i}. {c.document.source}/{c.document.path} "
|
||||
f"cosine={c.cosine:.4f} fts={int(c.fts_hit)} fused={c.score:.5f} "
|
||||
f"cosine={c.cosine:.4f} fts={int(c.fts_hit)} "
|
||||
f"{score_label}={c.score:.5f} created={c.document.created_at:%Y-%m-%d} "
|
||||
f"({c.document.title})"
|
||||
)
|
||||
return 0
|
||||
|
||||
Reference in New Issue
Block a user