Files
ducoterra ee3efb28c9
Build and Push Containers / build-and-push-app (push) Successful in 4m35s
Build and Push Containers / build-and-push-db (push) Successful in 14s
phase: 106_document_dates
Everything is verified green. Final report:

**Phase 106 — Document dates (final verification pass; all 10 tasks already complete)**

- Verified all phase artifacts: alembic `0020` (dev DB at `0020`), `doc_dates.py`, git `file_commit_dates`, importer `doc_dates_by_root`/`dates_updated`, both entry-point wirings, date APIs + tree `created_at`/`updated_at`, LLM surfaces (prompt block, `read` line 2, appended `ls` field), `apply_recency_boost` in `retrieve()`, UI columns/badge, admin editor, mock-LLM regex — all present and correct; no defects found, no fixes needed.
- `uv run pytest --cov=app --cov-report=term-missing` → **2299 passed, TOTAL 99%** (>90% ✓)
- `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` → **6/6 passed** in isolation (DB up)
- 12 regression E2E suites (retrieval_quality, whole_document_context, agent_document_tools, ls_tree_drilldown, read_truncation_cap, kb_tree, kb_tree_nav, document_viewer, edit_summaries, import_documents, sync_button, hidden_folders_toggle, smoke) → **all green in isolation**
- `uv run ruff check .` → clean; `uv run pyright` → **0 errors, 0 warnings**

**Completion criteria:** 1) non-null `created_at` + 0020 upgrade/downgrade on dev DB ✓ (real-Alembic integration tests) 2) sync refresh/older/manual-persists/content-reset/no sources_meta bump ✓ 3) zip/tar mtime + future→today ✓ 4) LLM date surfaces + cross-check ✓ 5) UI Created/Updated/badge positions ✓ 6) admin editor set+revert round-trip ✓ 7) old-correct-beats-new-similar (defaults & boost-off) + near-tie + `BOR_RECENCY_BOOST=0` byte-identical ✓ 8) full gate ✓ 9) commit/phase-move — left to harness per instructions.

- **Notable:** recency default tuned 0.001 → **0.0007** (task 07 step 5 explicitly permits; measured margins recorded in `test_recency_boost.py` docstring).
- **Next pending phase:** none — `todo/` holds only this phase.
2026-09-13 19:28:05 -04:00

7.8 KiB

Task 05 — Date API surface: reads, the admin date edit, and the tree's dates (D7/D8/D9)

Phase: 106_document_dates · Source: owner request 2026-09-13 — "This timestamp should be editable so users can correct for errors"; the catalog needs the file dates + folder last-updated (the UI in task 08 renders exactly what this task serves).

Objective

Serve the date everywhere the UI (task 08) and the viewer need it — GET /api/docs, GET /api/documents/content, and GET /api/docs/tree (files: created_at; folders/sources: the derived subtree-max updated_at, D9) — and add the admin-only PATCH /api/documents/date (set + revert, the phase-57 gate/idiom, D7).

Work

  1. app/schemas.py:
    • DocSummary (L228-235): add created_at: str (ISO-8601 — the indexed_at docstring style: "verbatim from the row").
    • DocContent (L343-358): add created_at: str (after summary, before content — group the metadata).
    • NEW DateUpdate (the SummaryUpdate shape, L361-373): source: str, path: str, date: str | None (docstring: an ISO date YYYY-MM-DD or full ISO datetime; null/absent = the CLEAR — drop the manual flag, the stored date stands until the next sync refresh; a malformed non-null value 422s through Pydantic… correction: str passes any string — the handler parses (step 3); the 422 comes from the handler, not the model, so the error detail can name the field).
    • NEW DateResult: source: str, path: str, created_at: str, created_at_manual: bool (echoes the stored state — the viewer re-renders from it).
    • KbTreeFile (L241-256): add created_at: str (verbatim from the catalogue row — the indexed_at field's docstring pattern).
    • KbTreeFolder (L259-290) and KbTreeSource (L293-320): add updated_at: str | None (docstring: the subtree's MAX document created_at — D9, derived, never stored; null for a 0-document source, the summary: str | None shape).
  2. app/api/docs.py:
    • list_indexed_documents (L80-119): add Document.created_at to the select AND the group_by (the indexed_at twin, L104/L107); DocSummary(..., created_at=row.created_at.isoformat()).
    • get_document_content (L121-162): created_at=doc.created_at.isoformat() in the DocContent (L161 site).
    • NEW PATCH /api/documents/date (route order: next to update_document_summary, L164-232 — require_admin dependency, the phase-57 gate):
      @router.patch("/documents/date", response_model=DateResult)
      def update_document_date(payload: DateUpdate, db: Session = Depends(get_db),
                               _admin: None = Depends(require_admin)) -> DateResult:
      
      Logic (DB-only — the /documents/content row-lookup rule, no filesystem, no LLM/embedding call — a date is never embedded, the phase-57 no-LLM contrast): look up the row by (source, path) → none → 404 {"detail": "document not found"} (row-lookup semantics, the traversal-string-is-not-a-row note). payload.date truthy → parsed = datetime.fromisoformat(payload.date) (a bare YYYY-MM-DD and full ISO datetimes both parse; ValueError → 422 {"detail": "date must be an ISO date or datetime (e.g. 2024-06-15)"}) → doc.created_at = normalize_doc_date(parsed) (import from app.rag.doc_dates — D3: a manually set FUTURE date also folds to today, consistency with the sourced path) → doc.created_at_manual = True. payload.date falsy (null/absent — the CLEAR) → doc.created_at_manual = False only (the stored date stands; the next sync refreshes it — the API cannot re-read the source, D7). db.commit(); return DateResult with the stored created_at.isoformat() + flag.
    • The tree (task-05 half of D8/D9): TreeDocRow (L296-302) becomes the 6-tuple (source, path, title, chunks, indexed_at, created_at) (both ISO strings — the builder stays pure over plain types); _folder_counts / _level_children / _source_node thread a 6th element through their tuple unpacks (the _-named slots gain the date) and _level_children/_source_node compute each folder/source's updated_at: the MAX of the direct files' created_at and the children's updated_at values (ISO-8601 strings compare correctly lexicographically — they're all the same isoformat() shape; document that in the builder docstring) — None when the node has no documents at all (the 0-document registered source). KbTreeFolder(…, updated_at=…) / KbTreeFile(…, created_at=…) / KbTreeSource(…, updated_at=…) at their construction sites (L342-375, L467-490). list_kb_tree (L492-551): add Document.created_at to the query's select + group_by (the indexed_at twin, L543-546) and the doc_rows comprehension. The build_kb_tree docstring gains the D9 clause (updated_at = subtree max, derived, None for empty).
  3. Tests:
    • tests/unit/test_kb_tree_builder.py (extended — the pure builder): file nodes carry created_at verbatim; a nested fixture asserts each folder's + the source's updated_at = the subtree max (a deeper file's date wins over a shallow sibling's); a 0-document registered source → updated_at is None and no children; the ls↔group_folder_listing cross-check tests (L210-280) still pass with the extended tuples (task 06 changes the agent side — until then the rows stay 6-tuples on BOTH sides only after task 06; for THIS task the cross-check compares file (path, title[, chunks, indexed_at]) projections — read the current assertions and keep them green: the tree builder's file tuples are internal to the builder, the cross-check uses the builder's OUTPUT nodes, so it should pass unchanged — verify and pin).
    • tests/integration/test_docs_api_dates.py (NEW — the tests/integration/test_docs_api.py scaffolding: real app + db fixture, an admin cookie where that suite gets one): seed two documents in a nested folder (distinct created_ats via direct row writes):
      • GET /api/docs (admin) reports created_at per row (and indexed_at unchanged);
      • GET /api/documents/content carries created_at;
      • GET /api/docs/tree — the file node's created_at verbatim, the parent folder's and the source's updated_at = the max, a registered-but-empty source → updated_at: null;
      • the PATCH matrix — set 2020-01-02 → 200 + response echoes the stored ISO + created_at_manual: true + a re-GET confirms; set a full ISO datetime → accepted; malformed "not-a-date" → 422 (the detail names the field); date: null → 200 + created_at_manual: false + the stored date UNCHANGED; a future date "2999-01-01" → stored created_at folds to today (D3); unknown (source, path) → 404 document not found; anonymous → 403 (the gate).
  4. Run uv run pytest tests/unit/test_kb_tree_builder.py tests/integration/test_docs_api_dates.py -q (DB up) — green.

Testing & Quality

  • Unit: the pure builder's date threading (max computation, None-for-empty, verbatim file dates).
  • Integration: the full API matrix (reads + PATCH set/malformed/clear/future/404/403) against real Postgres.
  • Coverage: >90% on app/ (the new route + the builder branches covered — the validate.sh gate).

Completion Criteria

  • GET /api/docs, GET /api/documents/content, and GET /api/docs/tree serve created_at (files) and updated_at (folders/sources — subtree max, null when empty, derived in the pure builder, D9)
  • PATCH /api/documents/date is admin-only, DB-only, no-LLM: set (ISO date or datetime, future folds to today, created_at_manual=true), clear (null → flag drops, date stands), 422 malformed, 404 unknown pair, 403 anonymous — the phase-57 split intact (viewer stays user-gated)
  • tests/unit/test_kb_tree_builder.py + tests/integration/test_docs_api_dates.py pass; existing docs-API suites stay green
  • uv run ruff check . && uv run pyright clean