phase: 185_sync_defer_content_fast_path
Build and Push Containers / build-and-push-app (push) Successful in 1m41s
Build and Push Containers / build-and-push-db (push) Successful in 11s

All gates green. Report:

**Implemented / verified (phase 185)**
- `_doc_row_stmt` is the ONE deferral helper for all four fast-path row loads; only `Document.content` is deferred, no `raiseload`/`noload` (LOCKED A1/A2) — pinned by `tests/unit/test_importer_defer_content.py`.
- `tests/integration/test_import_fast_path_columns.py` (5), `tests/e2e/test_fast_path_deferred_content.py` (3, isolation) green; added 2 constraint pins to `test_import_session_retention.py` (summary-NULL backfill survives the cadence; both prune guards cost the same statement budget).
- **Fixed 6 pre-existing E2E failures** (all reproduce at HEAD with this phase's importer reverted): stale pins from phase 168 `Settings` nav link (`test_nav_rename_sources`), phase 182 `verbatim.js` script count (`test_steering`), phase 175 `/api/chats?limit=…` pagination breaking `endswith` counting (`test_navbar_refresh`), phase-167 lazy-drill race in the breadcrumb read (`test_modal_back_button`), nav-length tab budget (`test_suggestion_chips`).
- **Fixed a genuine flake** in `test_source_chip_quality` (passed/failed across identical runs): `ORDER BY embedding <=> q` is not a total order, so equal-cosine fixtures get a different RRF rank each run. Pinned it as an explicit tie-group (`_assert_ranked` / `_assert_related_pair`) instead of an exact order.

**Tests / lint (all 100% green)**
- `uv run pytest --cov=app --cov-report=term-missing` → **4519 passed**, `app/rag/importer.py` **99%**, `TOTAL` **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- Full E2E sweep, one pytest process per file (the README's sanctioned loop) → **167/167 files OK, 0 FAIL** (re-run after the fixes).

**Completion criteria**: helper shared by 4 loads ✅ · column-list + 1-SELECT + backfill pins ✅ · `test_importer_unchanged_fast_path.py` 24 passed, zero expectation edits ✅ · full suite + coverage + lint/types ✅ · phase E2E in isolation ✅ · docs accurate ✅ · commit left to the harness (no git commands run).

**Documentation**: README + AGENTS.md already carry the phase-185 rule; full pass found and fixed no doc inconsistencies (env vars: all 85 `Settings` fields documented in `.env.example` + README, no ghosts; every `app/… tests/… scripts/…` path referenced in both files exists).

**Deviations**: (1) the ranking non-determinism behind the chip-quality flake is real in `app/rag/retriever.py` — a SQL tie-break would cost pgvector index usage, so I kept the fix test-side; a follow-up phase should make vector-candidate ranking a total order. (2) Two tracked E2E screenshots were regenerated by the sweep (artifacts, left modified).

**Next pending phase**: `186_git_log_streaming_dates`.
This commit is contained in:
2026-10-16 04:54:01 -04:00
parent 065304b52f
commit 632fb88e43
35 changed files with 2862 additions and 138 deletions
+68 -20
View File
@@ -114,6 +114,46 @@ def _ask(page: Page, message: str) -> None:
page.click("#send-btn")
def _assert_ranked(recorded: str, groups: list[list[str]]) -> None:
"""Compare a ``query_log.sources`` string against an expected rank list
whose trailing entries are TIE-GROUPS (order-free sets).
WHY the groups exist: the E2E mock embedder builds real token-overlap
vectors, so several tiny fixture documents end up on EXACTLY the same
fused score. ``fuse`` then sorts by ``(-score, -cosine, path,
position)``… but that only orders chunks whose SCORES differ; between
two documents tied in both lists the vector rank itself comes out of
``ORDER BY embedding <=> q``, which is not a total order, so Postgres
(a parallel seq scan handing its partial sorts back) may return the
tied pair the other way round between two runs of the SAME fixture.
That was this suite's flake. The rank of every block, and the exact
SET inside it, are still pinned — what the story pins is that the
record carries the FULL retrieval (LOCKED A3), not the luck of a tie.
"""
docs = [d.strip() for d in recorded.split(",")]
expected = [d for group in groups for d in group]
assert len(docs) == len(expected), f"recorded {docs}, expected {expected}"
at = 0
for group in groups:
window = docs[at : at + len(group)]
assert sorted(window) == sorted(group), (
f"rank block {at + 1}..{at + len(group)}: got {window}, "
f"expected the set {sorted(group)}"
)
at += len(group)
def _assert_related_pair(links: object, titles: set[str]) -> None:
"""The two related-doc links carry EXACTLY these two documents, in
either order (the same tie the record pins as a group — see
:func:`_assert_ranked`)."""
found = {
links.nth(i).get_attribute("title") # pyright: ignore[reportAttributeAccessIssue]
for i in range(2)
}
assert found == titles, f"related pair is {found}, expected {titles}"
def test_single_source_question_shows_exactly_one_citation_chip(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
@@ -179,15 +219,18 @@ def test_single_source_question_shows_exactly_one_citation_chip(
# (best eff 0.016277) now out-ranks ``kubernetes.md`` (0.016036);
# re-pinned against the post-phase walk (the record's CONTENT —
# suggested + related + read, deduped — is LOCKED A3, untouched).
assert row_log.sources == (
"docs/homelab/ssh/ssh_aliases.txt, "
"docs/homelab/container_gitlab/gitlab-compose.yaml, "
"docs/homelab/scripts/uptime_probe.py, "
"docs/homelab/backups.md, "
"docs/homelab/kubernetes.md, "
"docs/homelab/quadlet/compose.container, "
"docs/homelab/networking/static-dns.json"
), row_log.sources
_assert_ranked(
row_log.sources,
[
["docs/homelab/ssh/ssh_aliases.txt"],
["docs/homelab/container_gitlab/gitlab-compose.yaml"],
["docs/homelab/scripts/uptime_probe.py"],
# The tied pair this record used to pin in an arbitrary order.
["docs/homelab/backups.md", "docs/homelab/kubernetes.md"],
["docs/homelab/quadlet/compose.container"],
["docs/homelab/networking/static-dns.json"],
],
)
def test_deflected_question_shows_zero_citation_chips(
@@ -225,8 +268,9 @@ def test_deflected_question_shows_zero_citation_chips(
# record, but a deflected turn cites nothing, so they never render
# as chips.
expect(links).to_have_count(2)
expect(links.nth(0)).to_contain_text("backups.md")
expect(links.nth(1)).to_contain_text("kubernetes.md")
# The pair, as a SET: the two are tied in the fused score (see
# :func:`_assert_ranked`), so only their presence is a contract.
_assert_related_pair(links, {"docs/homelab/backups.md", "docs/homelab/kubernetes.md"})
# ZERO citation chips under the bubble — the weak hits are scored
# docs, not citations (the phase-112/113 contract on the wire).
@@ -243,12 +287,16 @@ def test_deflected_question_shows_zero_citation_chips(
assert row_log.deflected is True
assert 0.0 < row_log.top_score < get_settings().relevance_threshold
assert row_log.fts_hits == 0
assert row_log.sources == (
"docs/deployments/new-service.md, "
"docs/homelab/ssh/ssh_aliases.txt, "
"docs/homelab/quadlet/lan.network, "
"docs/homelab/scripts/uptime_probe.py, "
"docs/homelab/quadlet/compose.container, "
"docs/homelab/backups.md, "
"docs/homelab/kubernetes.md"
), row_log.sources
_assert_ranked(
row_log.sources,
[
["docs/deployments/new-service.md"],
["docs/homelab/ssh/ssh_aliases.txt"],
["docs/homelab/quadlet/lan.network"],
["docs/homelab/scripts/uptime_probe.py"],
["docs/homelab/quadlet/compose.container"],
# The related row IS this pair, and the row's order is the same
# tie — pinned as a set below (``_assert_related_pair``).
["docs/homelab/backups.md", "docs/homelab/kubernetes.md"],
],
)