Files
brain-of-reese/tests/unit/test_sources_dates.py
T
ducoterra ee3efb28c9
Build and Push Containers / build-and-push-app (push) Successful in 4m35s
Build and Push Containers / build-and-push-db (push) Successful in 14s
phase: 106_document_dates
Everything is verified green. Final report:

**Phase 106 — Document dates (final verification pass; all 10 tasks already complete)**

- Verified all phase artifacts: alembic `0020` (dev DB at `0020`), `doc_dates.py`, git `file_commit_dates`, importer `doc_dates_by_root`/`dates_updated`, both entry-point wirings, date APIs + tree `created_at`/`updated_at`, LLM surfaces (prompt block, `read` line 2, appended `ls` field), `apply_recency_boost` in `retrieve()`, UI columns/badge, admin editor, mock-LLM regex — all present and correct; no defects found, no fixes needed.
- `uv run pytest --cov=app --cov-report=term-missing` → **2299 passed, TOTAL 99%** (>90% ✓)
- `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` → **6/6 passed** in isolation (DB up)
- 12 regression E2E suites (retrieval_quality, whole_document_context, agent_document_tools, ls_tree_drilldown, read_truncation_cap, kb_tree, kb_tree_nav, document_viewer, edit_summaries, import_documents, sync_button, hidden_folders_toggle, smoke) → **all green in isolation**
- `uv run ruff check .` → clean; `uv run pyright` → **0 errors, 0 warnings**

**Completion criteria:** 1) non-null `created_at` + 0020 upgrade/downgrade on dev DB ✓ (real-Alembic integration tests) 2) sync refresh/older/manual-persists/content-reset/no sources_meta bump ✓ 3) zip/tar mtime + future→today ✓ 4) LLM date surfaces + cross-check ✓ 5) UI Created/Updated/badge positions ✓ 6) admin editor set+revert round-trip ✓ 7) old-correct-beats-new-similar (defaults & boost-off) + near-tie + `BOR_RECENCY_BOOST=0` byte-identical ✓ 8) full gate ✓ 9) commit/phase-move — left to harness per instructions.

- **Notable:** recency default tuned 0.001 → **0.0007** (task 07 step 5 explicitly permits; measured margins recorded in `test_recency_boost.py` docstring).
- **Next pending phase:** none — `todo/` holds only this phase.
2026-09-13 19:28:05 -04:00

451 lines
20 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Unit: the phase-106 date COLUMNS in the UI (task 08, D8) + the
viewer's Created badge (display only — the admin date EDITOR is
task 09).
The owner asked for the date everywhere it is read: "For files,
include a date/timestamp before the 'indexed' column in the UI";
"I would also like to see a last updated dates/timestamps on folders
before the description column but after the documents column in the
UI"; "The UI must also show a date for every document at the top of
that document when the user clicks it." The values ride the task-05
APIs (``created_at`` on the tree's file nodes + ``/api/docs`` +
``/api/documents/content``; the derived subtree-max ``updated_at`` on
tree folders/sources, D9 — ``null`` for a 0-document source).
The browser behavior itself is E2E-gated by the phase's dedicated
suite (``tests/e2e/test_document_dates.py``, task 10); like the other
frontend-adjacent unit files (the ``test_source_ignore_paths.py`` /
``test_kb_tree_ui.py`` house pattern), this module pins the
source-level contract a silent regression would break:
* ``frontend/index.html`` — the header cell ORDER, pinned as the
exact ``<th>`` sequence in the RAG view:
``Source | Path | Title | Chunks | Created | Indexed`` (Created
BETWEEN Chunks and Indexed — D8 verbatim) and
``Folder | Documents | Updated | Description`` (Updated BETWEEN
Documents and Description — D8 verbatim);
* ``frontend/assets/sources.js`` — ``makeRow``'s cell order
(``created_at`` BEFORE ``indexed_at``; the Created cell built
explicitly — ``textContent`` = ``fmtDate(d.created_at)`` AND
``title`` = the full ISO value, the path-cell hover idiom the E2E
asserts on — never innerHTML), the file-row object fed from the
tree's file nodes carries ``created_at``, and the ``updatedTd``
null → ``"–"`` branch (the statLast idiom) is present in BOTH row
builders (makeSourceRow + makeFolderRow);
* ``frontend/assets/document.js`` — the ONE shared core's
``.doc-meta`` badge row: the ``doc-created`` badge BEFORE
``doc-indexed`` (modal + ``/document.html`` through the same
core — no per-surface copy), the ``Created `` label +
``fmtDate(doc.created_at)`` template, and the full ISO timestamp on
the badge's ``title`` (the ``titleEl`` ellipsis-precision idiom);
* ``frontend/assets/styles.css`` — the ``.doc-created`` rule with
the phase-106 D8 provenance comment + the recorded WCAG pair
(5.1:1 — the same family as the Indexed badge), and the
Description one-line clamp moved to ``td:nth-child(4)`` (the
Updated column took 3rd — no hard-coded column count left behind).
"""
from __future__ import annotations
import re
from pathlib import Path
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
SHELL_HTML = FRONTEND / "index.html"
SOURCES_JS = FRONTEND / "assets" / "sources.js"
DOCUMENT_JS = FRONTEND / "assets" / "document.js"
STYLES_CSS = FRONTEND / "assets" / "styles.css"
def _text(path: Path) -> str:
assert path.is_file(), f"missing frontend file: {path}"
return path.read_text(encoding="utf-8")
def _js() -> str:
return _text(SOURCES_JS)
def _doc_js() -> str:
return _text(DOCUMENT_JS)
def _css() -> str:
return _text(STYLES_CSS)
def _rag_view(html: str) -> str:
"""The RAG view section of the shell (view-scoped — the shell
carries many views, so whole-file matches hit the wrong one)."""
i = html.find('<section class="view" id="view-rag"')
assert i != -1, "the RAG view section must be in the shell"
j = html.find('<section class="view" id="view-git-sources"', i)
assert j != -1, "the Sources view section must follow the RAG view"
return html[i:j]
def _fn(js: str, name: str) -> str:
"""The source of a (possibly async, possibly nested) function via
balanced-brace counting (the test_kb_tree_ui.py helper). The brace
count starts AFTER the parameter list — a destructured parameter
(renderDocument's target object) may carry braces of its own."""
for prefix in ("async function ", "function "):
start = js.find(f"{prefix}{name}(")
if start != -1:
# Skip the parameter list (balanced parens).
depth = 0
i = js.find("(", start)
close = i
while i < len(js):
if js[i] == "(":
depth += 1
elif js[i] == ")":
depth -= 1
if depth == 0:
close = i
break
i += 1
# Then brace-count the body.
brace = js.find("{", close)
depth = 0
for j in range(brace, len(js)):
if js[j] == "{":
depth += 1
elif js[j] == "}":
depth -= 1
if depth == 0:
return js[start : j + 1]
raise AssertionError(f"unbalanced braces in {name}()")
raise AssertionError(f"{name}() must exist")
def _header_columns(html: str, table_id: str) -> list[str]:
"""The table's ``<th>`` cells, IN DOCUMENT ORDER (the ORDER is the
pin — D8's verbatim positions)."""
i = html.find(f'id="{table_id}"')
assert i != -1, f"#{table_id} must be in the RAG view"
head = re.search(r"<thead>(.*?)</thead>", html[i :], re.S)
assert head, f"#{table_id} must keep a static thead"
return [m.group(1) for m in re.finditer(r"<th scope=\"col\">([^<]*)</th>", head.group(1))]
def _css_rule(css: str, selector: str) -> str:
"""The declarations of a simple rule (comments stripped first — a
house comment may legally carry braces)."""
clean = re.sub(r"/\*.*?\*/", "", css, flags=re.S)
start = clean.find(f"{selector} {{")
assert start != -1, f"missing rule {selector} in styles.css"
brace = clean.find("{", start)
depth = 0
for i in range(brace, len(clean)):
if clean[i] == "{":
depth += 1
elif clean[i] == "}":
depth -= 1
if depth == 0:
return clean[start : i + 1]
raise AssertionError(f"unbalanced braces in {selector}")
# ---------- index.html: the two header rows (ORDER pinned) ----------
def test_file_table_header_order_is_source_path_title_chunks_created_indexed() -> None:
"""D8 verbatim: the file table's ``<th>`` sequence is EXACTLY
Source | Path | Title | Chunks | Created | Indexed — Created
BETWEEN Chunks and Indexed (a set-membership pin would let a
regression move the column; the sequence pins the position)."""
view = _rag_view(_text(SHELL_HTML))
assert _header_columns(view, "docs-table") == [
"Source",
"Path",
"Title",
"Chunks",
"Created",
"Indexed",
], "the file table's column order (Created BEFORE Indexed, D8)"
def test_folder_table_header_order_is_folder_documents_updated_description() -> None:
"""D8 verbatim: the folder/source table's ``<th>`` sequence is
EXACTLY Folder | Documents | Updated | Description — Updated
BETWEEN Documents and Description."""
view = _rag_view(_text(SHELL_HTML))
assert _header_columns(view, "folders-table") == [
"Folder",
"Documents",
"Updated",
"Description",
], "the folder table's column order (Updated BETWEEN the two, D8)"
def test_inserted_headers_carry_the_phase_106_comment() -> None:
"""House comment style: each inserted <th> is annotated with a
phase-106 provenance comment (D8 for the position; D9 for the
derived Updated value) — a bare <th> with no comment is the
regression this guards against."""
view = _rag_view(_text(SHELL_HTML))
for col in ("Created", "Updated"):
i = view.find(f"<th scope=\"col\">{col}</th>")
assert i != -1, f"the {col} header must be in the RAG view"
comment = view.rfind("<!--", 0, i)
end = view.find("-->", comment)
assert comment > -1 and "phase 106" in view[comment:end].lower(), (
f"a phase-106 comment must sit above the {col} header"
)
assert "D8" in view[comment:end], f"the {col} comment cites D8"
# ---------- sources.js: makeRow's Created cell (before Indexed) ----------
def test_make_row_cell_order_is_created_before_indexed() -> None:
"""makeRow (the file table's row builder): the cell order is
[title, chunks, created, indexed] — the Created cell lands BEFORE
the Indexed one (D8 verbatim). Both date cells are built
EXPLICITLY (the plain-td loop can't carry per-cell titles): the
Created cell has ``textContent = fmtDate(d.created_at)`` AND
``title = d.created_at`` — the full ISO value on hover (the
path-cell idiom; the E2E asserts on the locale-stable title, not
on the toLocaleString output)."""
make = _fn(_js(), "makeRow")
# The loop keeps the two plain cells (title, chunks).
assert "for (const value of [d.title, String(d.chunks)]) {" in make
# The Created cell: explicit, locale date + the ISO title.
created_i = make.find("const createdTd = document.createElement(\"td\");")
assert created_i != -1, "the Created cell is built explicitly"
assert "createdTd.textContent = fmtDate(d.created_at);" in make
assert "createdTd.title = d.created_at;" in make, (
"the full ISO value on hover (the E2E's locale-stable pin)"
)
# The value-list ORDER: created_at before indexed_at (text AND
# append — both orderings pinned, a regression can't swap them).
created_fmt = make.find("fmtDate(d.created_at)")
indexed_fmt = make.find("fmtDate(d.indexed_at)")
assert -1 < created_fmt < indexed_fmt, "created_at BEFORE indexed_at (D8)"
assert (
make.find("tr.appendChild(createdTd)")
< make.find("tr.appendChild(indexedTd)")
), "the Created cell appends BEFORE the Indexed one"
# textContent only — the XSS contract (never innerHTML with
# document-derived data).
assert "createdTd.innerHTML" not in make and "indexedTd.innerHTML" not in make
def test_tree_file_row_object_carries_created_at() -> None:
"""renderLevel's file loop feeds makeRow the FLAT row shape —
the tree's file node carries created_at (task 05's tree shape),
and the row object restores it BEFORE indexed_at (the order the
reader gets matches makeRow's cell order)."""
js = _js()
render = js[js.find("function renderLevel(") : js.find("function renderEmpty(")]
assert "makeRow({" in render, "renderLevel still feeds makeRow"
created_i = render.find("created_at: f.created_at,")
indexed_i = render.find("indexed_at: f.indexed_at,")
assert -1 < created_i < indexed_i, (
"the row object carries created_at (before indexed_at — task 05's tree shape)"
)
# ---------- sources.js: the Updated cell in BOTH row builders ----------
UPDATED_TAIL = "makeDescCell" # the Description cell follows Updated
def test_source_row_has_updated_cell_between_count_and_description() -> None:
"""makeSourceRow (top level — the rows ARE the sources): ONE new
td BETWEEN the count td and the Description cell — the
``updatedTd`` null → ``"–"`` branch (the statLast idiom: D9's
``None`` for a 0-document source), the locale date otherwise
(fmtDate), and the ISO value on the cell's title (hover
precision — the makeRow path-cell idiom)."""
make = _fn(_js(), "makeSourceRow")
count_i = make.find("countTd.textContent = String(s.documents);")
tail_i = make.find(UPDATED_TAIL, count_i)
updated_i = make.find("const updatedTd = document.createElement(\"td\");", count_i)
assert -1 < count_i < updated_i < tail_i, (
"the Updated cell sits BETWEEN the count and the Description (D8)"
)
assert 'updatedTd.textContent = s.updated_at ? fmtDate(s.updated_at) : "–";' in make, (
"the null → '–' branch (the statLast idiom — D9's None for a 0-document source)"
)
assert "if (s.updated_at) updatedTd.title = s.updated_at;" in make, (
"the ISO value on hover (only when there is one)"
)
append_i = make.find("tr.appendChild(updatedTd)", count_i)
desc_i = make.find(UPDATED_TAIL, append_i)
assert -1 < append_i < desc_i, "appended before the Description cell"
assert "updatedTd.innerHTML" not in make, "textContent only (XSS contract)"
def test_folder_row_has_updated_cell_between_count_and_description() -> None:
"""makeFolderRow (a level's subfolders): the SAME one td BETWEEN
the count td and the Description cell — the ``updatedTd`` null →
``"–"`` branch present in BOTH row builders (the phase pins it in
each — a source row and a folder row are separate code paths)."""
make = _fn(_js(), "makeFolderRow")
count_i = make.find("countTd.textContent = String(f.documents);")
tail_i = make.find(UPDATED_TAIL, count_i)
updated_i = make.find("const updatedTd = document.createElement(\"td\");", count_i)
assert -1 < count_i < updated_i < tail_i, (
"the Updated cell sits BETWEEN the count and the Description (D8)"
)
assert 'updatedTd.textContent = f.updated_at ? fmtDate(f.updated_at) : "–";' in make, (
"the null → '–' branch in the FOLDER builder too (both builders pinned)"
)
assert "if (f.updated_at) updatedTd.title = f.updated_at;" in make, (
"the ISO value on hover (only when there is one)"
)
append_i = make.find("tr.appendChild(updatedTd)", count_i)
desc_i = make.find(UPDATED_TAIL, append_i)
assert -1 < append_i < desc_i, "appended before the Description cell"
assert "updatedTd.innerHTML" not in make, "textContent only (XSS contract)"
def test_module_docstring_carries_the_phase_106_contract() -> None:
"""The house module-docstring convention: the phase-106 section
records the Created column (before Indexed), the Updated column
(between Documents and Description), and the untouched stat
cards (the indexed_at 'last indexed' semantics stay)."""
doc = _js()[: _js().find("import { fetchIsAdmin }")]
for frag in (
"Phase 106 (task 08, D8)",
"Created column BEFORE Indexed",
"BETWEEN\n * Documents and Description",
"D9",
"UNTOUCHED",
):
assert frag in doc, f"the module docstring lost: {frag!r}"
# ---------- document.js: the shared core's Created badge ----------
def test_meta_row_badge_order_is_created_before_indexed() -> None:
"""renderDocument (the ONE shared core — the modal AND
/document.html render through it, no per-surface copy): the
.doc-meta badge row carries the ``doc-created`` badge BEFORE the
``doc-indexed`` one (D8 verbatim — the date at the top of a
clicked document)."""
render = _fn(_doc_js(), "renderDocument")
block = render[render.find("metaEl.replaceChildren(") :]
created_i = block.find('metaBadge("doc-created"')
indexed_i = block.find('metaBadge("doc-indexed"')
chunks_i = block.find('metaBadge("doc-chunks"')
assert -1 < created_i < indexed_i < chunks_i, (
"the Created badge BEFORE Indexed, both before Chunks (D8)"
)
# The label + the locale-date template (the Indexed idiom).
assert "Created ${fmtDate(doc.created_at)}" in block, (
"the 'Created <date>' label + fmtDate(doc.created_at) template"
)
assert "Indexed ${fmtDate(doc.indexed_at)}" in block, (
"the Indexed badge is unchanged (the idiom the Created one copies)"
)
def test_created_badge_carries_the_full_iso_title() -> None:
"""The badge's ``title`` attribute carries the FULL ISO timestamp
(the titleEl ellipsis-precision idiom — the meta row may clip,
the exact value stays reachable): the created badge passes
``doc.created_at`` as metaBadge's title argument, and metaBadge
sets it via setAttribute (only when provided — the other badges
keep the two-argument shape, byte-identical)."""
js = _doc_js()
render = _fn(js, "renderDocument")
call_start = render.find("metaEl.replaceChildren(")
# The call ends at the first `);` AFTER the last badge (the
# doc-chunks one) — the comment above the created badge may carry
# parentheses, so slicing from the call top would be brittle.
chunks_i = render.find('metaBadge("doc-chunks"', call_start)
block = render[call_start : render.find(");", chunks_i) + 2]
created_i = block.find('metaBadge("doc-created"')
created_call = block[created_i : block.find("),", created_i) + 1]
assert created_call.endswith(", doc.created_at)"), (
"the created badge passes doc.created_at as its title"
)
badge = _fn(js, "metaBadge")
assert "function metaBadge(cls, text, title)" in badge, (
"metaBadge's optional title parameter"
)
assert 'el.setAttribute("title", title)' in badge
assert "title !== undefined" in badge, (
"the guard keeps the other badges' two-argument shape"
)
# No other badge passes a title (the pre-phase badges keep the
# two-argument shape — exactly one comma in the call).
for cls_ in ("doc-source-badge", "format-badge", "doc-indexed", "doc-chunks"):
i = block.find(f'metaBadge("{cls_}"')
assert i != -1, f"the {cls_} badge must stay in the meta row"
call = block[i : block.find("),", i) + 1]
assert call.count(",") == 1, f"{cls_} keeps the two-argument shape"
def test_core_docstring_and_comment_cite_phase_106_d8() -> None:
"""House comment style: the module docstring + the badge-row
comment record the phase-106 (task 08, D8) created-before-indexed
position and the ONE-shared-core guarantee (modal + page)."""
js = _doc_js()
doc = js.split("*/", 1)[0]
assert "Phase 106 (task 08, D8)" in doc
assert "BEFORE the Indexed one" in doc
assert "task 05" in doc, "the created_at payload provenance (task 05)"
render = _fn(js, "renderDocument")
assert "Phase 106 (task 08, D8)" in render, "the inline comment at the insertion site"
# ---------- styles.css: the .doc-created rule + the clamp move ----------
def test_doc_created_rule_present_with_provenance_and_contrast() -> None:
"""styles.css carries the ``.doc-created`` rule (the doc-indexed
badge family — the meta row's ink-soft text) with the phase-106
D8 provenance comment + the recorded WCAG pair (5.1:1 on
--surface — the same pair the Indexed badge inherits via
.doc-meta / .doc-modal-meta)."""
css = _css()
rule = _css_rule(css, ".doc-created")
assert "color: var(--ink-soft)" in rule, "the meta-row family (the Indexed look)"
# The provenance comment sits DIRECTLY above the rule (the
# house style: phase + decision + the verified contrast pair).
rule_i = css.find(".doc-created {")
comment_start = css.rfind("/*", 0, rule_i)
comment_end = css.find("*/", comment_start)
assert -1 < comment_start < rule_i and comment_end < rule_i, (
"a comment block must sit directly above the rule"
)
header = css[comment_start:comment_end]
assert "phase 106" in header.lower() and "D8" in header, (
"the provenance comment cites phase 106 + D8"
)
assert "5.1:1" in header, "the verified contrast pair is recorded (house style)"
# The rule sits next to the meta-row badge family (after
# .doc-chunks, before the .doc-shell block).
assert (
css.find(".doc-chunks {") < rule_i < css.find(".doc-shell {")
), "next to the existing meta-row badge rules"
def test_folders_description_clamp_follows_the_moved_cell() -> None:
"""The column-count change must not break the table's rules: the
Description one-line clamp (phase 99) follows the cell, which
moved to the 4th (the Updated column took 3rd) — no stale
``td:nth-child(3)`` folders rule may remain, and the new Updated
cell's ink pair (the table ink on --surface, 13.8:1) is recorded
in the house comment."""
css = _css()
assert ".kb-folders-table td:nth-child(4) {" in css, (
"the Description clamp moved with the cell (4th column)"
)
assert ".kb-folders-table td:nth-child(3)" not in css, (
"no stale 3rd-column folders rule (the Updated cell is there now)"
)
i = css.find(".kb-folders-table td:nth-child(4)")
comment = css.rfind("/*", 0, i)
end = css.find("*/", comment)
assert comment > -1 and "phase 106" in css[comment:end].lower(), (
"a phase-106 comment explains the Updated column + the clamp move"
)
assert "13.8:1" in css[comment:end], (
"the Updated cell's ink pair is recorded (table ink on --surface)"
)