add phase 106 - document dates
This commit is contained in:
@@ -0,0 +1,45 @@
|
||||
# Task 05 — Date API surface: reads, the admin date edit, and the tree's dates (D7/D8/D9)
|
||||
|
||||
**Phase:** `106_document_dates` · **Source:** owner request 2026-09-13 — "This timestamp should be editable so users can correct for errors"; the catalog needs the file dates + folder last-updated (the UI in task 08 renders exactly what this task serves).
|
||||
|
||||
## Objective
|
||||
Serve the date everywhere the UI (task 08) and the viewer need it — `GET /api/docs`, `GET /api/documents/content`, and `GET /api/docs/tree` (files: `created_at`; folders/sources: the derived subtree-max `updated_at`, D9) — and add the admin-only `PATCH /api/documents/date` (set + revert, the phase-57 gate/idiom, D7).
|
||||
|
||||
## Work
|
||||
1. `app/schemas.py`:
|
||||
- `DocSummary` (L228-235): add `created_at: str` (ISO-8601 — the `indexed_at` docstring style: "verbatim from the row").
|
||||
- `DocContent` (L343-358): add `created_at: str` (after `summary`, before `content` — group the metadata).
|
||||
- NEW `DateUpdate` (the `SummaryUpdate` shape, L361-373): `source: str`, `path: str`, `date: str | None` (docstring: an ISO date `YYYY-MM-DD` or full ISO datetime; **null/absent = the CLEAR** — drop the manual flag, the stored date stands until the next sync refresh; a malformed non-null value 422s through Pydantic… correction: `str` passes any string — the handler parses (step 3); the 422 comes from the handler, not the model, so the error detail can name the field).
|
||||
- NEW `DateResult`: `source: str`, `path: str`, `created_at: str`, `created_at_manual: bool` (echoes the stored state — the viewer re-renders from it).
|
||||
- `KbTreeFile` (L241-256): add `created_at: str` (verbatim from the catalogue row — the `indexed_at` field's docstring pattern).
|
||||
- `KbTreeFolder` (L259-290) and `KbTreeSource` (L293-320): add `updated_at: str | None` (docstring: the subtree's MAX document `created_at` — D9, derived, never stored; `null` for a 0-document source, the `summary: str | None` shape).
|
||||
2. `app/api/docs.py`:
|
||||
- `list_indexed_documents` (L80-119): add `Document.created_at` to the select AND the `group_by` (the `indexed_at` twin, L104/L107); `DocSummary(..., created_at=row.created_at.isoformat())`.
|
||||
- `get_document_content` (L121-162): `created_at=doc.created_at.isoformat()` in the `DocContent` (L161 site).
|
||||
- NEW `PATCH /api/documents/date` (route order: next to `update_document_summary`, L164-232 — `require_admin` dependency, the phase-57 gate):
|
||||
```python
|
||||
@router.patch("/documents/date", response_model=DateResult)
|
||||
def update_document_date(payload: DateUpdate, db: Session = Depends(get_db),
|
||||
_admin: None = Depends(require_admin)) -> DateResult:
|
||||
```
|
||||
Logic (DB-only — the `/documents/content` row-lookup rule, no filesystem, no LLM/embedding call — a date is never embedded, the phase-57 no-LLM contrast): look up the row by `(source, path)` → none → 404 `{"detail": "document not found"}` (row-lookup semantics, the traversal-string-is-not-a-row note). `payload.date` truthy → `parsed = datetime.fromisoformat(payload.date)` (a bare `YYYY-MM-DD` and full ISO datetimes both parse; `ValueError` → 422 `{"detail": "date must be an ISO date or datetime (e.g. 2024-06-15)"}`) → `doc.created_at = normalize_doc_date(parsed)` (import from `app.rag.doc_dates` — D3: a manually set FUTURE date also folds to today, consistency with the sourced path) → `doc.created_at_manual = True`. `payload.date` falsy (null/absent — the CLEAR) → `doc.created_at_manual = False` only (the stored date stands; the next sync refreshes it — the API cannot re-read the source, D7). `db.commit()`; return `DateResult` with the stored `created_at.isoformat()` + flag.
|
||||
- The tree (task-05 half of D8/D9): `TreeDocRow` (L296-302) becomes the 6-tuple `(source, path, title, chunks, indexed_at, created_at)` (both ISO strings — the builder stays pure over plain types); `_folder_counts` / `_level_children` / `_source_node` thread a 6th element through their tuple unpacks (the `_`-named slots gain the date) and `_level_children`/`_source_node` compute each folder/source's `updated_at`: the MAX of the direct files' `created_at` and the children's `updated_at` values (ISO-8601 strings compare correctly lexicographically — they're all the same `isoformat()` shape; document that in the builder docstring) — `None` when the node has no documents at all (the 0-document registered source). `KbTreeFolder(…, updated_at=…)` / `KbTreeFile(…, created_at=…)` / `KbTreeSource(…, updated_at=…)` at their construction sites (L342-375, L467-490). `list_kb_tree` (L492-551): add `Document.created_at` to the query's select + group_by (the `indexed_at` twin, L543-546) and the `doc_rows` comprehension. The `build_kb_tree` docstring gains the D9 clause (updated_at = subtree max, derived, None for empty).
|
||||
3. Tests:
|
||||
- `tests/unit/test_kb_tree_builder.py` (extended — the pure builder): file nodes carry `created_at` verbatim; a nested fixture asserts each folder's + the source's `updated_at` = the subtree max (a deeper file's date wins over a shallow sibling's); a 0-document registered source → `updated_at is None` and no children; the ls↔`group_folder_listing` cross-check tests (L210-280) still pass with the extended tuples (task 06 changes the agent side — until then the rows stay 6-tuples on BOTH sides only after task 06; for THIS task the cross-check compares file `(path, title[, chunks, indexed_at])` projections — read the current assertions and keep them green: the tree builder's file tuples are internal to the builder, the cross-check uses the builder's OUTPUT nodes, so it should pass unchanged — verify and pin).
|
||||
- `tests/integration/test_docs_api_dates.py` (NEW — the `tests/integration/test_docs_api.py` scaffolding: real app + `db` fixture, an admin cookie where that suite gets one): seed two documents in a nested folder (distinct `created_at`s via direct row writes):
|
||||
- `GET /api/docs` (admin) reports `created_at` per row (and `indexed_at` unchanged);
|
||||
- `GET /api/documents/content` carries `created_at`;
|
||||
- `GET /api/docs/tree` — the file node's `created_at` verbatim, the parent folder's and the source's `updated_at` = the max, a registered-but-empty source → `updated_at: null`;
|
||||
- the PATCH matrix — set `2020-01-02` → 200 + response echoes the stored ISO + `created_at_manual: true` + a re-GET confirms; set a full ISO datetime → accepted; malformed `"not-a-date"` → 422 (the detail names the field); `date: null` → 200 + `created_at_manual: false` + the stored date UNCHANGED; a future date `"2999-01-01"` → stored `created_at` folds to today (D3); unknown `(source, path)` → 404 `document not found`; anonymous → 403 (the gate).
|
||||
4. Run `uv run pytest tests/unit/test_kb_tree_builder.py tests/integration/test_docs_api_dates.py -q` (DB up) — green.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: the pure builder's date threading (max computation, None-for-empty, verbatim file dates).
|
||||
- Integration: the full API matrix (reads + PATCH set/malformed/clear/future/404/403) against real Postgres.
|
||||
- Coverage: **>90%** on `app/` (the new route + the builder branches covered — the validate.sh gate).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `GET /api/docs`, `GET /api/documents/content`, and `GET /api/docs/tree` serve `created_at` (files) and `updated_at` (folders/sources — subtree max, `null` when empty, derived in the pure builder, D9)
|
||||
- [ ] `PATCH /api/documents/date` is admin-only, DB-only, no-LLM: set (ISO date or datetime, future folds to today, `created_at_manual=true`), clear (null → flag drops, date stands), 422 malformed, 404 unknown pair, 403 anonymous — the phase-57 split intact (viewer stays user-gated)
|
||||
- [ ] `tests/unit/test_kb_tree_builder.py` + `tests/integration/test_docs_api_dates.py` pass; existing docs-API suites stay green
|
||||
- [ ] `uv run ruff check . && uv run pyright` clean
|
||||
Reference in New Issue
Block a user