phase: 119_name_signal_read_chips
Build and Push Containers / build-and-push-app (push) Successful in 2m1s
Build and Push Containers / build-and-push-db (push) Successful in 18s

All verification complete. Final report:

**Phase 119 final verification pass — all criteria verified, one stale pin fixed.**
- Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry.
- Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged.
- New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2.
- Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed).
- Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met).
- Next pending phase: **none** — `todo/` holds only phase 119.
This commit is contained in:
2026-09-16 15:50:48 -04:00
parent 795fb56425
commit a5b63f83ad
89 changed files with 4377 additions and 655 deletions
+72
View File
@@ -436,6 +436,29 @@ Implements just enough of the aipi surface:
phrases — the phase-71/72/94 ordering convention); verified
2026-09-16: no existing E2E question or fixture file contains the
phrase, so every other suite is unaffected.
- user message containing ``repeat your folder map``
(``FOLDER_MAP_TRIGGER``, phase 119, D3 — the suggested-folder
context's story suite ``tests/e2e/test_name_signal_read_chips.py``)
**and** the system prompt contains the ``<documents>`` section ->
the composed answer ends with `` (folders: <line 1>; <line 2>; …)``
echoing the suggested-folder lines VERBATIM — the plain lines
between ``SUGGEST_INTRO`` and the first ``<document `` in the
``<documents>`` section (``app.rag.prompts.build_high_prompt`` /
``app.rag.agent.suggested_folder_lines``, phase 119 LOCKED A4:
``<source>/<prefix>/: e1, e2, …``), joined with ``; ``. If no
folder lines are present (empty *folder_lines* — the
byte-identical phase-118 shape) the echo is omitted and the answer
is otherwise unchanged. The same prompt-injection-echo convention
as the ``<tuning>`` / ``<knowledge_base>`` triggers above (a SUFFIX
appended after both, so the folders suffix is the last thing
rendered): phase 119 D3's observability — the folder lines land in
the model's prompt only on grounded turns, and the echo is the
E2E's deterministic lens on their EXACT text (line identity shape
+ entry order + the owning-doc exclusion + the caps). It composes
with any base answer (it does not compete with a marker branch);
the trigger phrase is disjoint from every other trigger's; verified
2026-09-16: no existing E2E question or fixture file contains the
phrase, so every other suite is unaffected.
Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic
dead-endpoint behavior for the retry E2E suite (``tests/e2e/
@@ -675,6 +698,17 @@ TABLE_TRIGGER = "show me a table"
#: phrase, so every other suite is unaffected.
HISTORY_TRIGGER = "echo my history"
#: Phase 119 (D3, the suggested-folder context's story suite): a user
#: message containing this substring (case-insensitive) — combined
#: with the ``<documents>`` section in the system prompt (a grounded
#: turn) — gets the composed answer suffixed with the folder lines
#: VERBATIM (`` (folders: <line 1>; <line 2>; …)``, joined with
#: ``; ``; omitted when the section carries no folder lines) — see the
#: module docstring. Disjoint from every other trigger phrase; verified
#: 2026-09-16: no existing E2E question or fixture file contains the
#: phrase, so every other suite is unaffected.
FOLDER_MAP_TRIGGER = "repeat your folder map"
TABLE_ANSWER = (
"Here's the shape, in a table:\n"
"\n"
@@ -1657,6 +1691,32 @@ def first_kb_bullet(system: str) -> str | None:
return None
def folder_map_lines(system: str) -> list[str]:
"""The suggested-folder lines of the ``<documents>`` section
(phase 119, D3, LOCKED A4) — or ``[]`` when there are none.
``build_high_prompt`` lays the section body out as ``SUGGEST_INTRO``
(line 1), the folder lines (each on its own line, immediately after
the intro line), a blank line, then the first ``<document>`` block
— so the lines are exactly the non-blank lines between the intro
line and the first ``<document `` line of the block. With no folder
lines (the byte-identical phase-118 shape) there is only the blank
line there, and this returns ``[]`` (the echo is then omitted — the
answer is otherwise unchanged).
"""
block = _DOCUMENTS_BLOCK_RE.search(system)
if not block:
return []
lines = block.group(0).splitlines()
out: list[str] = []
for line in lines[2:]: # skip the tag line + the SUGGEST_INTRO line
if line.lstrip().startswith("<document"):
break
if line.strip():
out.append(line)
return out
def _history_echo(body: dict[str, Any]) -> str:
"""The phase-74 history echo (byte-stable, stateless over messages).
@@ -2093,6 +2153,18 @@ def compose_answer(body: dict[str, Any]) -> str:
bullet = first_kb_bullet(system)
if bullet:
answer = f"{answer} (kb: {bullet})"
# Suggested-folder context (phase 119, D3, LOCKED A4): when the user
# message carries FOLDER_MAP_TRIGGER and the system prompt carries
# the <documents> section, the answer ends with the folder lines
# VERBATIM (joined with "; ") — the E2E's deterministic lens on the
# exact lines build_high_prompt seeded (D3 observability; the same
# prompt-injection-echo convention as the two echoes above). Omitted
# when the section carries no folder lines (the byte-identical
# phase-118 shape) — the answer is otherwise unchanged.
if FOLDER_MAP_TRIGGER in user.lower() and "<documents>" in system:
folder_lines = folder_map_lines(system)
if folder_lines:
answer = f"{answer} (folders: {'; '.join(folder_lines)})"
return answer
+23 -14
View File
@@ -60,12 +60,15 @@ Test → story mapping (Playwright Mapping Rule):
``tool`` frames (``ls``, then ``read`` with the combined path, ahead
of any delta), the UI shows the transient calling-tool status while a
tool runs, the bubble shows both tool lines, the final answer quotes
the read document, and the source chips include the read document
(viewer link).
the read document, and the source chips are EXACTLY the read
document (phase 119, LOCKED A1 — chips cite read docs only; viewer
link).
2. ``test_tool_lines_re_render_after_reload`` — the persisted record
(phase 14) re-renders the tool lines.
3. ``test_plain_grounded_question_has_no_tool_frames`` — no marker → no
``tool`` frames, the answer renders exactly as today (regression
``tool`` frames, the answer renders exactly as today, and (phase 119,
LOCKED A1) the zero-read grounded turn chips NOTHING (the retrieval
doc is cited only in the durable record, never as a chip) (regression
inside the story file).
4. ``test_deflected_question_has_no_tool_frames`` — the tools are
grounded-only: a deflected turn runs none.
@@ -444,8 +447,12 @@ def test_marker_question_lists_reads_and_quotes(
)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False
# Phase 119 (LOCKED A1): done.sources = the READ docs only —
# exactly the scripted read; the retrieval (seed) doc is suggested
# context, never a citation chip (the retired phase-118 A4
# suggested+read union is gone). The durable record below still
# carries both (LOCKED A3, untouched).
assert [(s["source"], s["path"]) for s in done["sources"]] == [
(SEED_SOURCE, SEED_PATH),
(READ_SOURCE, READ_PATH),
]
@@ -465,16 +472,17 @@ def test_marker_question_lists_reads_and_quotes(
expect(bubble).to_contain_text(ANSWER_PREFIX)
expect(bubble).to_contain_text(ANSWER_QUOTE)
# Source chips: the retrieval doc AND the read doc (deduped, in
# order) — the read chip links to the viewer.
# Source chips: the read doc ONLY (phase 119, LOCKED A1 — the
# retrieval doc was never read, so it never chips; the retired
# phase-118 A4 union is gone) — the read chip links to the viewer.
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(2)
expect(chips.nth(0)).to_contain_text(SEED_SP)
expect(chips).to_have_count(1)
chip_read = page.locator(".msg.brain .source-chip", has_text=READ_PATH)
expect(chip_read).to_have_count(1)
expect(chip_read.first).to_have_attribute("href", READ_CHIP_HREF)
# Durable record: grounded, both sources logged (retrieval + read).
# Durable record: grounded, both sources logged (suggested + read —
# LOCKED A3, untouched by phase 119 A1).
row = _last_query_log()
assert row.question == MARKER_QUESTION
assert row.deflected is False
@@ -540,14 +548,15 @@ def test_plain_grounded_question_has_no_tool_frames(
assert _tool_frames(_frames(page)) == []
expect(page.locator(".tool-call")).to_have_count(0)
# The standard grounded answer, citing the retrieval doc only — the
# referenced JSON stays OUT of the sources (it was never read).
# The standard grounded answer, with ZERO citation chips — the
# turn read nothing, so (phase 119, LOCKED A1) the chip row is
# empty: the retrieval doc is cited in the durable record only
# (the retired phase-118 A4 union is gone); the referenced JSON
# stays out of the record too (it was never read).
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(PLAIN_QUESTION)
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER)
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(1)
expect(chips.first).to_contain_text(SEED_SP)
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
row = _last_query_log()
assert row.question == PLAIN_QUESTION
+36 -20
View File
@@ -65,9 +65,11 @@ Test → story mapping (Playwright Mapping Rule):
``I read <sp1> and <sp2>.`` line; the round cap (default 10) bounds
the turn, no budget refusal anywhere.
2. ``test_done_sources_include_reads`` — the source chips under the
answer list the retrieval doc PLUS both read documents, deduped
(the phase-37 ``done.sources`` extension contract, now with 2
reads); the same combined list lands in ``query_log.sources``.
answer list the BOTH read documents, deduped (phase 119, LOCKED A1:
chips cite read docs only — the retrieval doc was never read, so it
never chips; the retired phase-118 A4 suggested+read union is
gone); the retrieval doc PLUS both reads still land in
``query_log.sources`` (LOCKED A3, untouched).
3. ``test_relist_allowed`` — the listing tool ran (its line rendered)
and no pre-phase-45 budget refusal ("… budget left") appears
anywhere in the message bubble or tool lines: the old
@@ -195,14 +197,22 @@ BUDGET_REFUSAL_FRAGMENTS = (
"budget left",
)
# The combined source list the app reports (app/api/chat.py): retrieval
# docs first, then the agent's read docs, deduped by (source, path).
# The combined source list the DURABLE record reports
# (app/api/chat.py, LOCKED A3 — suggested + related + read, deduped):
# retrieval doc first, then the agent's read docs, deduped by
# (source, path). The citation surface (done.sources, the chips) is
# the READ set only (phase 119, LOCKED A1 — the retired phase-118 A4
# suggested+read union is gone): the retrieval doc was never read.
EXPECTED_SOURCES = [
(SEED_SOURCE, SEED_PATH),
(READ1_SOURCE, READ1_PATH),
(READ2_SOURCE, READ2_PATH),
]
EXPECTED_SOURCES_LINE = ", ".join(f"{s}/{p}" for s, p in EXPECTED_SOURCES)
READ_ONLY_SOURCES = [
(READ1_SOURCE, READ1_PATH),
(READ2_SOURCE, READ2_PATH),
]
# --------------------------------------------------------------------------
@@ -508,7 +518,7 @@ def test_multi_read_turn(
# --------------------------------------------------------------------------
# 2. done.sources / source chips: retrieval doc + BOTH reads, deduped
# 2. done.sources / source chips: BOTH reads, deduped (phase 119 A1)
# --------------------------------------------------------------------------
@@ -523,23 +533,28 @@ def test_done_sources_include_reads(
_submit(page, MULTI_QUESTION)
_wait_settled(page)
# Wire level: done.sources is the retrieval doc FIRST, then both
# read documents — deduped (the retrieval doc was never read, the
# reads are each read once; nothing appears twice).
# Wire level: done.sources is the READ documents only, in read
# order — deduped (the reads are each read once; nothing appears
# twice). Phase 119, LOCKED A1: the retrieval doc was never read,
# so it is NOT in the citation surface (the retired phase-118 A4
# suggested+read union is gone); it still lands in the durable
# record (LOCKED A3, pinned in test 1).
frames = _frames(page)
done = next(f for f in frames if f.get("type") == "done")
assert [(s["source"], s["path"]) for s in done["sources"]] == EXPECTED_SOURCES
assert [
(s["source"], s["path"]) for s in done["sources"]
] == READ_ONLY_SOURCES, done["sources"]
pairs = [(s["source"], s["path"]) for s in done["sources"]]
assert len(pairs) == len(set(pairs)), "done.sources must be deduped"
# UI: exactly three source chips under the answer, in the same
# order, each a viewer link — no duplicated chip.
# UI: exactly two source chips under the answer (the read docs,
# read order), each a viewer link — no duplicated chip, and no
# chip for the never-read retrieval doc (phase 119 A1).
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(3)
expect(chips.nth(0)).to_contain_text(SEED_SP)
expect(chips.nth(1)).to_contain_text(READ1_SP)
expect(chips.nth(2)).to_contain_text(READ2_SP)
for i, (source, path) in enumerate(EXPECTED_SOURCES):
expect(chips).to_have_count(2)
expect(chips.nth(0)).to_contain_text(READ1_SP)
expect(chips.nth(1)).to_contain_text(READ2_SP)
for i, (source, path) in enumerate(READ_ONLY_SOURCES):
expect(chips.nth(i)).to_have_attribute(
"href", f"/document.html?source={source}&path={path}&back=%2F"
)
@@ -631,12 +646,13 @@ def test_single_tool_flow_regression(
expect(bubble).not_to_contain_text(BOTH_READS_LINE)
expect(bubble).not_to_contain_text(READ2_SP)
# done: non-deflected; sources = retrieval doc + the single read
# (READ2 absent — it was never read).
# done: non-deflected; sources = the single READ doc only (phase
# 119, LOCKED A1 — READ2 absent: never read; the retrieval doc
# absent: never read — the retired phase-118 A4 union is gone; the
# durable record below keeps retrieval + read, LOCKED A3).
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False
assert [(s["source"], s["path"]) for s in done["sources"]] == [
(SEED_SOURCE, SEED_PATH),
(READ1_SOURCE, READ1_PATH),
]
+24 -9
View File
@@ -391,8 +391,11 @@ def test_admin_generates_token_in_ui(page: Page, app_url: str, db_ready: None) -
# ---------------------------------------------------------------------------
# 4. The token flow: a fresh context signs in through the real gate
# and USES the app — a grounded chat turn (mock LLM), a cited
# document opened in the same-page modal, the role-user header
# and USES the app — a grounded chat turn (mock LLM; phase 119,
# LOCKED A1: the turn is the scripted summary-read flow — chips
# cite READ docs only, and this turn's read earns the cited chip),
# a cited document opened in the same-page modal, the role-user
# header
# ---------------------------------------------------------------------------
@@ -417,15 +420,27 @@ def test_token_user_uses_the_app(
# task-04 helper: fill #auth-gate-input → submit → gate hides).
login_with_token(user, app_url, token)
# Use the app: a grounded turn against the seeded KB (mock
# LLM) — the brain bubble renders the deterministic answer.
_ask(user, "How is my Kubernetes cluster set up? (api-tokens-flow)")
# Use the app: a GROUNDED turn against the seeded KB (mock
# LLM). Phase 119 (LOCKED A1): chips cite READ docs only — a
# plain question would chip nothing (the retired phase-118 A4
# union is gone), so the turn is the mock's scripted
# summary-read flow (SUMMARY_SEED_READ_TRIGGER): it reads the
# kubernetes fixture, and that read is the chip's source.
q = (
"Read the suggested document: read docs/homelab/kubernetes.md — "
"how is my Kubernetes cluster set up? (api-tokens-flow)"
)
user.fill("#message-input", q)
user.click("#send-btn")
expect(user.locator(".msg.user .bubble").last).to_contain_text(q)
# A cited source chip opens the document in the SAME-PAGE
# modal (the require_user content endpoint passes for a
# live token session).
# The turn's cited source chip (its one READ doc — the scripted
# read, phase 119 A1) opens the document in the SAME-PAGE modal
# (the require_user content endpoint passes for a live token
# session).
chip = user.locator(".msg.brain a.source-chip").first
expect(chip).to_be_visible(timeout=15_000)
expect(chip).to_be_visible(timeout=30_000)
expect(user.locator("#send-label")).to_have_text("Send")
chip.click()
expect(user.locator("#doc-modal")).to_be_visible()
expect(
+3 -7
View File
@@ -63,9 +63,6 @@ from e2e.auth_helpers import login
REPO = Path(__file__).resolve().parents[2]
FIXTURES = REPO / "tests" / "fixtures" / "docs"
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
#: Phase 10 viewer URL + phase 13 back=/ (the restored chip must be
#: byte-identical to the live-rendered one).
CHIP_HREF = "/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
async def _import_fixtures(mock_port: int) -> ImportSummary:
@@ -279,10 +276,9 @@ def test_open_chat_returns_to_history(
# …the SAME answer text the History session saw (pixel-identical
# restore through renderStoredMessage)…
assert bubble.inner_text() == answer_before
# …with its source chip restored byte-identically.
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
expect(chip).to_have_count(1)
expect(chip.first).to_have_attribute("href", CHIP_HREF)
# …with ZERO citation chips restored — the saved turn read
# nothing, so its sources list is empty (phase 119, LOCKED A1;
# the retired phase-118 A4 suggested-chip pin is gone).
# The conversation continues: a new turn streams fine…
_ask(page, "How is my Kubernetes cluster set up? (hist-open-2)")
+11 -10
View File
@@ -42,9 +42,6 @@ OFF_TOPIC = "How do I bake sourdough bread?"
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
DEFLECT_PHRASE = r"haven't done anything like that"
STORAGE_KEY = "bor.chat.v1"
#: Phase 10 viewer URL + phase 13 back=/ (the restored chip must be
#: byte-identical to the live-rendered one).
CHIP_HREF = "/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
async def _import_fixtures(mock_port: int) -> ImportSummary:
@@ -143,22 +140,24 @@ def test_conversation_survives_reload(
assert MOCK_ANSWER_MARKER in brain["text"]
assert "<" not in brain["text"], "persisted brain text must be raw, not rendered HTML"
assert brain["deflected"] is False
assert any(s["path"] == "homelab/kubernetes.md" for s in brain["sources"])
# Phase 119 (LOCKED A1): the turn read nothing, so its done sources
# — and the persisted record — are EMPTY (the suggested kubernetes
# doc is context, not a citation; the retired phase-118 A4 union is
# gone).
# Refresh — the same context keeps its localStorage.
page.reload()
expect(page.locator("#empty-state")).to_be_hidden()
# Both bubbles restored: text + the source chip with the exact viewer URL.
# Both bubbles restored: the answer text, and ZERO citation chips
# (the zero-read turn persisted an empty sources list — phase 119,
# LOCKED A1; the retired phase-118 A4 chip pin is gone).
expect(page.locator(".msg.user .bubble")).to_have_count(1)
expect(page.locator(".msg.user .bubble")).to_contain_text(QUESTION)
bubble = page.locator(".msg.brain .bubble")
expect(bubble).to_have_count(1)
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER)
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
expect(chip).to_have_count(1)
expect(chip.first).to_have_attribute("href", CHIP_HREF)
expect(chip.first).not_to_have_attribute("target") # phase 26: modal, not a new tab
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# The restore is read-only: storage still holds the same two messages.
assert [m["who"] for m in _stored_parsed(page)["messages"]] == ["user", "brain"]
@@ -294,7 +293,9 @@ def test_persists_across_page_navigation(
expect(page.locator(".msg.user .bubble").first).to_contain_text(QUESTION)
expect(page.locator(".msg.user .bubble").nth(1)).to_contain_text(OFF_TOPIC)
expect(page.locator(".msg.brain .bubble").first).to_contain_text(MOCK_ANSWER_MARKER)
expect(page.locator(".msg.brain .source-chip", has_text="kubernetes.md")).to_have_count(1)
# Zero citation chips (phase 119, LOCKED A1 — the grounded turn read
# nothing; the retired phase-118 A4 union is gone).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
deflected = page.locator(".msg.brain.is-deflected .bubble")
expect(deflected).to_have_count(1)
expect(deflected.first).to_contain_text(re.compile(DEFLECT_PHRASE, re.IGNORECASE))
+21 -17
View File
@@ -97,18 +97,12 @@ def test_on_topic_question_streams_grounded_answer(
expect(bubble.first).to_contain_text(QUESTION, timeout=30_000)
expect(bubble.first).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
# Grounded: a kubernetes.md source chip renders under the bubble
# (top-N docs can add more chips; the question's doc must be among them).
# Phase 26: the chip opens the document in the SAME-PAGE modal — no new
# tab; the encoded href stays as the no-JS / context-menu escape hatch
# (phase 13's back=/ lets the viewer's back button return to chat).
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
expect(chip).to_have_count(1)
expect(chip.first).to_contain_text("kubernetes.md")
expect(chip.first).to_have_attribute(
"href", "/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
)
expect(chip.first).not_to_have_attribute("target") # phase 26: modal, not a new tab
# Phase 119 (LOCKED A1): a zero-read grounded turn chips NOTHING —
# the suggested kubernetes.md is seed context, not a citation chip
# (the retired phase-118 A4 suggested+read union is gone; the
# pre-phase-26 chip/contract pins retired with it). The grounding is
# pinned by the query_log row below (LOCKED A3, untouched).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# Button recovers: enabled + "Send" (never stale).
expect(page.locator("#send-btn")).to_be_enabled()
@@ -121,9 +115,16 @@ def test_chat_logs_query(page: Page, app_url: str, mock_llm: int, db_ready: None
login(page, app_url, next="/")
page.fill("#message-input", QUESTION)
page.click("#send-btn")
expect(
page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
).to_have_count(1, timeout=30_000)
# The turn settles with the mock answer (phase 119 A1: no chip to
# wait on — a zero-read turn chips nothing; the retired phase-118
# A4 union is gone). The button recovery is the settle sync: the
# client re-enables Send on the done frame, and the server writes
# the query_log row just before yielding it.
expect(page.locator(".msg.brain .bubble").last).to_contain_text(
"Deterministic mock answer for E2E", timeout=30_000
)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
# App still healthy after the turn.
r = httpx.get(f"{app_url}/api/health", timeout=5)
@@ -180,5 +181,8 @@ def test_sse_stream_shape(app_url: str, mock_llm: int, db_ready: None) -> None:
assert frames[-1]["type"] == "done" # done is the final event
assert done[0]["deflected"] is False
assert done[0]["suggestions"] == []
assert done[0]["sources"], "done must carry the cited sources"
assert any(s["path"] == "homelab/kubernetes.md" for s in done[0]["sources"])
# Phase 119 (LOCKED A1): done.sources = the READ docs only — this
# plain turn read nothing, so the frame carries an EMPTY sources
# list (the retired phase-118 A4 suggested+read union is gone; the
# suggested doc's durable record lives in query_log, not the frame).
assert done[0]["sources"] == [], done[0]["sources"]
+5 -3
View File
@@ -358,8 +358,9 @@ def test_behavior_unchanged_smoke(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
"""AC6: layout/behavior unchanged under the new skin — an on-topic
question streams a grounded answer, renders a source chip, and the send
button recovers (never stale)."""
question streams a grounded answer (phase 119, LOCKED A1: a zero-read
turn renders ZERO source chips — the retired phase-118 A4
suggested-chip is gone) and the send button recovers (never stale)."""
_seed_kb(mock_llm)
page.set_default_timeout(30_000)
login(page, app_url, next="/") # phase 79: chat is require_user-gated
@@ -373,7 +374,8 @@ def test_behavior_unchanged_smoke(
bubble.first.wait_for(state="visible", timeout=30_000)
expect(bubble.first).to_contain_text(QUESTION, timeout=30_000)
expect(bubble.first).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator(".msg.brain .source-chip", has_text="kubernetes.md")).to_have_count(1)
# Phase 119 (LOCKED A1): zero-read turn → zero citation chips.
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# The state machine settled: button re-enabled, label back to "Send".
expect(page.locator("#send-btn")).to_be_enabled()
+20 -4
View File
@@ -19,10 +19,16 @@ viewer URL survives as each link's ``href`` (the no-JS / context-menu
"open in new tab" escape hatch), so the back contract is asserted on
that exact href and verified by navigating to it directly.
Phase 119 re-target (LOCKED A1): chips cite READ docs only — a plain
question chips nothing, so test 1 drives the mock's scripted
summary-read flow (``SUMMARY_SEED_READ_TRIGGER``): the turn ``read``s
the kubernetes fixture, and its READ-doc chip is what carries the
``&back=%2F`` href the story asserts.
Test → story mapping (Playwright Mapping Rule):
1. ``test_back_from_chat_returns_to_chat`` — question → source chip href
(carries ``&back=%2F``) → viewer back link href ``/`` labeled "Chat"
→ click → the chat page.
1. ``test_back_from_chat_returns_to_chat`` — scripted-read question →
the read doc's source chip href (carries ``&back=%2F``) → viewer
back link href ``/`` labeled "Chat" → click → the chat page.
2. ``test_back_from_sources_returns_to_sources`` — Sources table link
href (no ``back`` param) → back link href ``/sources.html`` labeled
"Sources" → click → the Sources page.
@@ -49,7 +55,15 @@ from e2e.auth_helpers import login
REPO = Path(__file__).resolve().parents[2]
FIXTURES = REPO / "tests" / "fixtures" / "docs"
QUESTION = "How is my Kubernetes cluster set up?"
# Phase 119 (LOCKED A1): the chip is the turn's READ doc — the scripted
# summary-read flow (mock_llm.SUMMARY_SEED_READ_TRIGGER) reads the
# kubernetes fixture so the story's chip (with its back=%2F href)
# exists (a zero-read turn would chip nothing — the retired phase-118
# A4 suggested-chip is gone).
QUESTION = (
"Read the suggested document: read docs/homelab/kubernetes.md — "
"how is my Kubernetes cluster set up?"
)
# Seeded fixture doc (source=docs) shared by every test in this file.
DOC_SOURCE = "docs"
DOC_PATH = "homelab%2Fkubernetes.md"
@@ -111,6 +125,8 @@ def test_back_from_chat_returns_to_chat(
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
expect(chip).to_have_count(1, timeout=30_000)
# Phase 119 A1: the read doc is the turn's ONLY chip.
expect(page.locator(".msg.brain .source-chip")).to_have_count(1)
# Chat chips carry back=/ (encoded %2F) so the viewer knows where
# home is. Phase 26: the left click opens the same-page modal (no
# target=_blank); this href is what the no-JS / context-menu "open
+19 -18
View File
@@ -540,12 +540,14 @@ def test_old_correct_beats_new_similar(
page: Page, app_url: str, mock_llm: int, db_ready: None, dates_tree: Path
) -> None:
"""``How did I configure the backup retention policy?`` → grounded,
and the FIRST cited source is the OLDER correct doc (2020) — the
NEWER similar one (now, "under review") is cited second. Phase 118
(A4): the chip row IS the suggested tier (top-5, NO floor) — this
four-doc KB chips ALL four docs in rank order — and the related row
is absent (no rank-6+ doc). The real hybrid retriever + the DEFAULT
recency boost (0.0007 / 365 d) over the mock's token-overlap
and the FIRST logged source is the OLDER correct doc (2020) — the
NEWER similar one (now, "under review") is second. Phase 119
(LOCKED A1): the zero-read turn chips NOTHING (the chip row is the
READ docs only — the retired phase-118 A4 suggested-chip row is
gone), so the ordering assertion rides the durable record (LOCKED
A3 — all four docs, suggested rank order, untouched); the related
row is absent (no rank-6+ doc). The real hybrid retriever + the
DEFAULT recency boost (0.0007 / 365 d) over the mock's token-overlap
embeddings (the module docstring records the measured fused scores:
0.032523 vs 0.031498 — margin ≈ 0.001025 WITH the full zero-age
boost on the newer doc)."""
@@ -563,21 +565,20 @@ def test_old_correct_beats_new_similar(
# top_score 0.6222 ≥ the e2e threshold 0.30).
expect(page.locator(".msg.brain.is-deflected")).to_have_count(0)
# Phase 118 (A4): the chip row IS the suggested tier — top-5, NO
# floor, and this KB has exactly four docs, so ALL four are
# suggested (chipped) in fused rank order: the OLDER correct doc
# first, the NEWER similar one (the boost's intended beneficiary)
# second, then the two unrelated docs (no floor filters them — the
# LLM decides what the summaries earn).
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(4, timeout=30_000)
assert chips.nth(0).inner_text() == f"{dates_tree.name}/{RETENTION_MD}"
assert chips.nth(1).inner_text() == f"{dates_tree.name}/{DRAFT_MD}"
assert chips.nth(2).inner_text() == f"{dates_tree.name}/{FORWARD_MD}"
assert chips.nth(3).inner_text() == f"{dates_tree.name}/{OLDDOC_MD}"
# Phase 119 (LOCKED A1): the turn read nothing, so ZERO citation
# chips — the four suggested docs (all of them, top-5 NO floor on a
# four-doc KB: the OLDER correct doc first, the NEWER similar one
# second, then the two unrelated docs) seeded the prompt but never
# chip (the retired phase-118 A4 union is gone). Their rank order
# is pinned by the durable record below (LOCKED A3, untouched).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# No rank-6+ doc exists in this four-doc KB → the related row is
# absent (the de-emphasized row renders only when it has entries).
expect(page.locator(".msg.brain .related-docs")).to_have_count(0)
# The button label is the settle sync: the client re-labels Send on
# the done frame, and the server writes the query_log row just
# before yielding it.
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
# Durable record: one row, grounded, the FULL retrieval (suggested
# tier + related + read, deduped — here: all four docs) in rank
+10 -9
View File
@@ -171,8 +171,9 @@ def test_summary_hit_seeds_the_summary_not_the_full_text(
quotes the last suggested doc's summary tail (digest + pointer), and
the sentinel (the yaml doc's last line, outside the digest) is
ABSENT: the full source doc never reached the prompt (A6). Both
fixture docs are suggested (two-doc KB, no floor) and both chips
render (deflected: false)."""
fixture docs are suggested (two-doc KB, no floor) but chip
NOTHING — the turn read nothing, so (phase 119, LOCKED A1) the
citation surface is empty (deflected: false)."""
_reset_db()
summary = _run_in_thread(_import_fixtures(mock_llm))
assert summary.added == 2 # yaml + md control
@@ -225,13 +226,13 @@ def test_summary_hit_seeds_the_summary_not_the_full_text(
expect(bubble).to_contain_text(yaml_digest_line[-80:])
expect(bubble).not_to_contain_text(SENTINEL)
# Grounded: both fixture docs are suggested (two-doc KB — no floor)
# and both chips render, in rank order (md first, yaml last).
chip = page.locator(".msg.brain .source-chip", has_text=YAML_PATH)
expect(chip).to_have_count(1)
expect(chip.first).to_contain_text(f"{SOURCE}/{YAML_PATH}")
expect(page.locator(".msg.brain .source-chip", has_text=MD_PATH)).to_have_count(1)
expect(page.locator(".msg.brain .source-chip")).to_have_count(2)
# Phase 119 (LOCKED A1): the turn read nothing, so ZERO citation
# chips — both fixture docs are suggested (two-doc KB — no floor,
# md first, yaml last) but suggested docs are seed context, not
# citations (the retired phase-118 A4 suggested-chip union is gone);
# their rank order is pinned by the durable record below (LOCKED
# A3, untouched).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# No rank-6+ doc in a two-doc KB → the related row is absent.
expect(page.locator(".msg.brain .related-docs")).to_have_count(0)
+28 -4
View File
@@ -11,9 +11,18 @@ deterministic mock embeddings (same harness as the phase-10 suite — only
the assertions changed: chips/row links now open the SAME-PAGE modal,
no ``expect_popup``).
Phase 119 re-target (LOCKED A1): chips cite READ docs only — a plain
question chips nothing, so the chat-driven tests here run the
mock's scripted summary-read flow (``SUMMARY_SEED_READ_TRIGGER`` —
the house scripted-turn convention): the turn ``read``s
``docs/homelab/kubernetes.md``, whose read result is what earns the
kubernetes.md chip the story interacts with (the chip's viewer-link
contract is unchanged).
Test → story mapping (Playwright Mapping Rule):
1. ``test_source_chip_opens_modal`` — chat chip → modal opens in-page
(NO new tab, URL unchanged), title + ``.doc-md`` content + meta row.
1. ``test_source_chip_opens_modal`` — chat chip (the turn's READ doc —
phase 119 A1) → modal opens in-page (NO new tab, URL unchanged),
title + ``.doc-md`` content + meta row.
2. ``test_sources_row_opens_modal`` — Sources path link → modal, yaml in
``<pre.doc-raw>``, mono font, URL unchanged.
3. ``test_modal_closes_on_button_escape_and_backdrop`` — close via
@@ -56,7 +65,14 @@ from e2e.auth_helpers import login
REPO = Path(__file__).resolve().parents[2]
FIXTURES = REPO / "tests" / "fixtures" / "docs"
SCREENSHOTS = REPO / ".agents" / "screenshots" # house convention for visual records
QUESTION = "How is my Kubernetes cluster set up?"
# Phase 119 (LOCKED A1): the chip is the turn's READ doc — the scripted
# summary-read flow (mock_llm.SUMMARY_SEED_READ_TRIGGER) reads the
# kubernetes fixture so the story's chip exists (the retired phase-118
# A4 suggested-chip is gone: a zero-read turn would chip nothing).
QUESTION = (
"Read the suggested document: read docs/homelab/kubernetes.md — "
"how is my Kubernetes cluster set up?"
)
async def _import_fixtures(mock_port: int) -> ImportSummary:
@@ -98,12 +114,20 @@ def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
def _ask_for_chip(page: Page, app_url: str) -> Any:
"""Drive one chat turn and return the kubernetes.md source chip."""
"""Drive one chat turn and return the kubernetes.md source chip.
The scripted summary-read flow (phase 119, LOCKED A1): the turn
``read``s the kubernetes doc, so it is the turn's ONLY citation
chip (chips cite read docs only — the answer is the mock's
verbatim echo of the read result)."""
login(page, app_url, next="/") # phase 79: chat is require_user-gated
page.fill("#message-input", QUESTION)
page.click("#send-btn")
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
expect(chip).to_have_count(1, timeout=30_000)
# Phase 119 A1: the read doc is the turn's ONLY chip (exactly one
# .source-chip under the bubble — nothing suggested-but-unread).
expect(page.locator(".msg.brain .source-chip")).to_have_count(1)
return chip
+3 -3
View File
@@ -455,10 +455,10 @@ def test_read_flow_lines_answer_sources_no_raw_markup(
)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False
# Done-state sources include the read document (retrieval doc first,
# the agent's read doc after — the phase-37 extension contract).
# Phase 119 (LOCKED A1): done.sources = the READ docs only —
# exactly the scripted read; the retrieval (seed) doc was never
# read, so it never cites (the retired phase-118 A4 union is gone).
assert [(s["source"], s["path"]) for s in done["sources"]] == [
(SEED_SOURCE, SEED_PATH),
(READ_SOURCE, READ_PATH),
]
+22 -13
View File
@@ -23,10 +23,12 @@ assistant message — A4).
The marker is checked BEFORE the mock's ``DEFLECT_MODE`` branch, so
the echo fires on BOTH turn branches — the branch under test is
discriminated separately (the grounded source chip / the
``is-deflected`` bubble class). The echo answers carry no tool
markup, so no marker tool flow is re-triggered by the now-always-
present (user/assistant-only) history.
discriminated separately (the persisted record's ``deflected`` flag /
the ``is-deflected`` bubble class — phase 119, LOCKED A1: a zero-read
grounded turn chips nothing, so the retired chip discriminator is
gone). The echo answers carry no tool markup, so no marker tool flow
is re-triggered by the now-always-present (user/assistant-only)
history.
The file name deliberately differs from phase 50's
``test_chat_history.py`` (save & view chat history — a different
@@ -170,12 +172,15 @@ def test_followup_receives_history_and_thinking(
expect(bubble).to_contain_text(f"last answer tail: {answer_tail}")
expect(bubble).to_contain_text("thinking: yes")
# Grounded proof — the echo fires in BOTH branches, so the branch
# is discriminated by the kubernetes.md source chip (the deflected
# turn carries no cited sources). Scoped to the LAST brain message:
# turn 1 cited kubernetes.md too.
# is discriminated by the persisted record's ``deflected`` flag
# (phase 119, LOCKED A1: the turn read nothing, so it also chips
# nothing — the retired chip discriminator is gone).
brain2 = _wait_record(page, 4)["messages"][3]
assert brain2["who"] == "brain"
assert brain2["deflected"] is False, "the echo turn must be the grounded branch"
expect(
page.locator(".msg.brain").last.locator(".source-chip", has_text="kubernetes.md")
).to_have_count(1)
page.locator(".msg.brain").last.locator(".source-chip")
).to_have_count(0)
def test_first_question_has_no_history(
@@ -194,11 +199,15 @@ def test_first_question_has_no_history(
expect(bubble).to_contain_text("history: 0 prior messages", timeout=30_000)
expect(bubble).to_contain_text("last answer tail: none")
expect(bubble).to_contain_text("thinking: no")
# Grounded: the echo question is on-topic (the chip proves the
# HIGH gate, not a deflection).
# Grounded: the echo question is on-topic (the record's deflected
# flag proves the HIGH gate, not a deflection — phase 119, LOCKED
# A1: the zero-read turn chips nothing, so the retired chip
# discriminator is gone).
record = _wait_record(page, 2)
assert record["messages"][1]["deflected"] is False
expect(
page.locator(".msg.brain").last.locator(".source-chip", has_text="kubernetes.md")
).to_have_count(1)
page.locator(".msg.brain").last.locator(".source-chip")
).to_have_count(0)
def test_deflected_followup_receives_history(
+12 -12
View File
@@ -393,18 +393,18 @@ def test_dead_then_recovered_grounded(
_assert_no_error_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False
assert [(s["source"], s["path"]) for s in done["sources"]] == [
(SEED_SOURCE, SEED_PATH)
]
# Phase 119 (LOCKED A1): the grounded turn read NOTHING, so
# done.sources is empty (the suggested seed doc never chips — the
# retired phase-118 A4 union is gone); the durable record below
# still carries it (LOCKED A3, untouched).
assert done["sources"] == [], done["sources"]
# The grounded answer completed with the source chip — the agent
# round retried and the turn is intact.
# The grounded answer completed (no chips on a zero-read turn —
# phase 119 A1) — the agent round retried and the turn is intact.
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(GROUNDED_Q)
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER)
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(1)
expect(chips.first).to_contain_text(SEED_SP)
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
_assert_no_error_banner(page)
row = _last_query_log()
@@ -446,12 +446,12 @@ def test_embedding_retry_completes(
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False
# The turn completed normally with the grounded answer + chip.
# The turn completed normally with the grounded answer (zero chips
# — the turn read nothing, so the chip row is empty under phase
# 119, LOCKED A1; the retired phase-118 A4 union is gone).
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER)
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(1)
expect(chips.first).to_contain_text(SEED_SP)
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
_assert_no_error_banner(page)
+4 -4
View File
@@ -127,8 +127,8 @@ def test_normal_answer_unaffected(
bubble = page.locator(".msg.brain .bubble").last
expect(bubble).to_contain_text("Deterministic mock answer for E2E", timeout=30_000)
expect(bubble).not_to_contain_text("LONG-ANSWER-END")
# Grounded: the question's own document is cited as a chip.
expect(
page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
).to_have_count(1, timeout=30_000)
# Grounded (the bubble + marker above) — and ZERO citation chips:
# the turn read nothing, so (phase 119, LOCKED A1) the chip row is
# empty (the retired phase-118 A4 suggested-chip is gone).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
expect(page.locator("#send-btn")).to_be_enabled()
+2 -2
View File
@@ -678,8 +678,8 @@ def test_drill_down_sources_folders_files_and_read(
expect(bubble).to_contain_text(READ_ANSWER_QUOTE)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The read document is in the turn's sources (retrieval + agent-read,
# deduped — the grounded-turn record).
# The read document is in the turn's sources (phase 119, LOCKED A1:
# done.sources is the READ docs only — this turn's one read).
assert any(
s["path"] == TWO_A and s["source"] == ALPHA for s in done["sources"]
), done["sources"]
+5 -3
View File
@@ -185,9 +185,11 @@ def test_chat_table_renders(
assert "|---|" not in bubble_text, "the |---| separator leaked into the bubble"
assert "| Service | Port | Host |" not in bubble_text, "the raw header row leaked"
# Grounded retrieval: the table fixture is the top source chip.
chip = page.locator(".msg.brain .source-chip", has_text="homelab/tables.md")
expect(chip).to_have_count(1)
# Grounded (the table answer is the mock's non-deflected table
# branch) — and ZERO citation chips: the turn read nothing, so
# (phase 119, LOCKED A1) the chip row is empty (the retired
# phase-118 A4 suggested-chip is gone).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# ---------------------------------------------------------------------------
+794
View File
@@ -0,0 +1,794 @@
"""Phase 119 E2E (Playwright, mock-only): the name-signal seeds the
right document — and chips cite READ docs only.
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_name_signal_read_chips.py -v --no-cov
MOCK-ONLY suite: the story's gates are the deterministic contracts —
the mock's scripted-turn lenses (the tail echo, the single-read tool
flow, the new ``FOLDER_MAP_TRIGGER`` echo) make "what reached the
prompt / what got cited" assertable byte-exactly.
KB fixture — ``tests/fixtures/namekb`` (tracked; registered as a
local-directory source through the authenticated API + the REAL
in-process ``POST /api/sync``, the ``test_summary_seed_context.py``
pattern), seven markdown documents in one source (``namekb``) whose
bodies are token-controlled so the hybrid gate + the phase-119
name-hit bonus pick the intended tiers deterministically
(``synced_kb`` pins it with the app's REAL retrieval — a
fixture-text regression that breaks the design fails at setup with a
clear message, not at the wire assertions):
* every path carries a DIGITLESS question name token (the four
guides' stems hold ``install``, the two forgejo docs' stems +
folder hold ``forgejo``, ``zz-folder-map.md`` holds ``folder``) —
so the name-hit bonus applies to ALL seven docs (a uniform
``+0.005``) and the tier order reduces to the pure retrieval
signal (the bonus head-start is exercised, the ordering is not
distorted by it);
* the four guides' bodies repeat the question phrase a strength
gradient (×4/×3/×2/×1) so they OUT-RANK the name-hit docs on
vector/FTS; the forgejo docs and the filler doc carry NEUTRAL
bodies (their name is their main signal — the phase-119 owner
scenario: "the file's name is the distinctive part");
* engineered invariant (asserted in ``synced_kb`` for ALL FOUR
scripted questions, measured + probe-verified, stable across
re-imports): the name-hit doc ``forgejo/forgejo-home.md`` is the
LAST of the top-5 suggested (the name signal lifts it into the
tier, the weak body keeps it at the back — its ``Source:`` tail
line is the tail echo), and the related tier is the rank-6+
remainder (``forgejo-nginx`` + ``zz-folder-map``; for the folder
question the two swap order — both stay rank 6+).
Test → contract mapping (task 06 cases a–d; one Playwright file per
story, A16):
1. ``test_name_hit_doc_is_last_suggested`` — (a): the distinctive
question + ``show the end of your notes`` ⇒ the answer quotes the
last 160 chars of the seeded ``<documents>`` block — the
name-hit doc's SUMMARY tail (its ``Source: namekb/forgejo/
forgejo-home.md`` pointer line; no other doc's pointer line, no
doc's tail sentinel — the summary seed, not the full text). The
zero-read grounded turn chips NOTHING (LOCKED A1); the related
row renders rank 6+; the durable record carries suggested +
related (118-A3 untouched).
2. ``test_single_read_chips_only_the_read_doc`` — (b): the
``use your tools`` flow (the mock's single read: ``ls`` → drill
``ls(namekb)`` → read the FIRST file line —
``namekb/zz-folder-map.md``, the root file) ⇒ the bubble carries
EXACTLY ONE ``.source-chip`` — the read doc (LOCKED A1); none of
the five suggested docs chips; the de-emphasized related row
renders (rank 6+ deduped against the cited read doc — the read
doc is a chip, never a "nearby doc"); the durable record carries
suggested + related + read (118-A3 untouched).
3. ``test_zero_read_grounded_turn_chips_nothing`` — (c): a PLAIN
distinctive question (no tool trigger) ⇒ the ``done`` frame's
``sources == []`` and ZERO ``.source-chip`` elements in the
bubble — LOCKED A1's visible consequence; the answer still
renders grounded (deflected: false); the related row renders.
4. ``test_folder_lines_echoed_verbatim`` — (d): the distinctive
question + ``repeat your folder map`` (the new mock trigger) ⇒
the answer ends with the suggested-folder lines VERBATIM — the
name-hit doc's folder line ``namekb/forgejo/: forgejo-nginx.md``
(the line prefix + its known sibling entry) alongside the
guides' folder line.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
import time
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import httpx
import pytest
from playwright.sync_api import Locator, Page, expect
from sqlalchemy import select, text
from app.config import Settings as _Settings
from app.db import SessionLocal
from app.models import Document, QueryLog
from app.rag.retriever import TRUNCATION_MARKER, retrieve, select_related, select_suggested
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
from e2e.mock_llm import TOKEN_RE, embed_text
REPO = Path(__file__).resolve().parents[2]
# Phase 79 (task 04, full inventory): the conftest session app owns its
# port in a combined run — this module app binds its own port instead
# (a same-port second uvicorn dies on bind and would drive the wrong
# server). Env-overridable.
APP_PORT = int(os.environ.get("E2E_APP_PORT_NAMESIG", "8145"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"
SOURCE = "namekb" # the local directory's basename = the source name
FIXTURES = REPO / "tests" / "fixtures" / "namekb"
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
# --------------------------------------------------------------------------
# Fixture documents (tracked — tests/fixtures/namekb; deterministic,
# token-controlled — see the module docstring for the design)
# --------------------------------------------------------------------------
#: 22 neutral tokens (no question tokens) — the digest + filler
#: material of the neutral docs; the guides' strength phrase is
#: question-token-only.
PRELUDE = (
"fixture preamble block one two three four five six seven eight nine ten "
"eleven twelve thirteen fourteen fifteen sixteen seventeen eighteen nineteen"
)
assert len(TOKEN_RE.findall(PRELUDE.lower())) == 22
PHRASE = "install forgejo with the deployment steps"
#: (path, title, phrase repeats, filler line, tail sentinel) — the
#: guides carry the strength gradient (×4/×3/×2/×1); the forgejo docs
#: (name hits — distinctive digitless ``forgejo`` component) and the
#: root filler carry NEUTRAL bodies (0 repeats).
DOCS: list[tuple[str, str, int, str, str]] = [
("deploy/install-guide-a.md", "Install Forgejo Guide A", 4,
"alpha1 alpha2 alpha3 alpha4 alpha5 alpha6", "NAMEKB-TAIL-c5d6"),
("deploy/install-guide-b.md", "Install Forgejo Guide B", 3,
"beta1 beta2 beta3 beta4 beta5 beta6", "NAMEKB-TAIL-d7e8"),
("deploy/install-guide-c.md", "Install Forgejo Guide C", 2,
"gamma1 gamma2 gamma3 gamma4 gamma5 gamma6", "NAMEKB-TAIL-f9a0"),
("deploy/install-guide-d.md", "Install Forgejo Guide D", 1,
"delta1 delta2 delta3 delta4 delta5 delta6", "NAMEKB-TAIL-b1c2"),
("forgejo/forgejo-home.md", "Zeta Forge One", 0,
"kilo1 kilo2 kilo3 kilo4 kilo5 kilo6", "NAMEKB-TAIL-a1b2"),
("forgejo/forgejo-nginx.md", "Zeta Forge Two", 0,
"lambda1 lambda2 lambda3 lambda4 lambda5 lambda6", "NAMEKB-TAIL-b3c4"),
("zz-folder-map.md", "Zeta Misc Three", 0,
"micro1 micro2 micro3 micro4 micro5 micro6", "NAMEKB-TAIL-e3f4"),
]
#: The engineered invariant (measured, probe-verified, stable across
#: re-imports — pinned for ALL FOUR scripted questions): the name-hit
#: doc is the LAST of the top-5 suggested; the guides lead in
#: retrieval-strength order (the ×2/×1 docs swap under the md5
#: collision noise — pinned as measured, not as the gradient order).
SUGGESTED = [
"deploy/install-guide-a.md",
"deploy/install-guide-b.md",
"deploy/install-guide-d.md",
"deploy/install-guide-c.md",
"forgejo/forgejo-home.md",
]
NAME_HIT = "forgejo/forgejo-home.md" # the (a) tail-echo target
RELATED = ["forgejo/forgejo-nginx.md", "zz-folder-map.md"]
#: The folder question's rank-6+ order (the filler's name hit —
#: ``folder`` — lifts it above the sibling; both stay rank 6+).
RELATED_FOLDER = ["zz-folder-map.md", "forgejo/forgejo-nginx.md"]
READ_TARGET = "zz-folder-map.md" # the single-read's first file line
SENTINELS = [sentinel for _p, _t, _i, _f, sentinel in DOCS]
ALL_SOURCE_LINES = [f"Source: {SOURCE}/{p}" for p, _t, _i, _f, _s in DOCS]
#: The turn's questions (the mock's trigger phrases — see the module
#: docstring). The base question's name tokens (``install``,
#: ``forgejo``) ground every turn (best cosine ≈ 0.35–0.41 ≥ the E2E
#: 0.30 threshold) and fire the D1 name hits.
BASE_QUESTION = "How do I install forgejo?"
TAIL_QUESTION = BASE_QUESTION + " show the end of your notes"
TOOLS_QUESTION = BASE_QUESTION + " use your tools"
FOLDER_QUESTION = BASE_QUESTION + " repeat your folder map"
assert "show the end of your notes" in TAIL_QUESTION.lower()
assert "use your tools" in TOOLS_QUESTION.lower()
assert "repeat your folder map" in FOLDER_QUESTION.lower()
def _expected_summary(content: str, path: str) -> str:
"""The mock lite model's byte-stable digest + the code pointer line
(mirrors ``mock_llm.compose_answer``'s ``SUMMARY_MODE`` branch —
first 24 tokens of the document content — plus the summarizer's
deterministic ``Source:`` line; no model output is ever trusted)."""
digest = " ".join(TOKEN_RE.findall(content.lower())[:24])
return f"This document covers {digest}.\nSource: {SOURCE}/{path}"
# --------------------------------------------------------------------------
# Fixtures
# --------------------------------------------------------------------------
@pytest.fixture(scope="module")
def seed_dirs() -> Path:
"""The story's local-directory source: the tracked fixture dir
(the app server runs on the same host, so the path is visible to
it). The directory's basename is the source name (``kind=local``,
phase 38)."""
assert FIXTURES.is_dir(), "tests/fixtures/namekb is missing"
for path, _t, _i, _f, _s in DOCS:
assert (FIXTURES / path).is_file(), f"fixture doc {path} is missing"
return FIXTURES
@pytest.fixture(scope="module")
def app_server(mock_llm: int, seed_dirs: Path) -> Iterator[str]:
"""The real app under test — per-module app (the conftest pattern,
cf. ``test_summary_seed_context.py``): NO ``BOR_GIT_SOURCES`` (the
env fallback is git-only — the source here is a DB-registered local
directory), the mock LLM, the mock-calibrated threshold, and the
leak-guarded code defaults (the suggested/related tier settings
ride their code defaults — 5 / 2 — exactly like the production
``.env``-free defaults). The session app is never started in this
isolated run, so no port clash."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{mock_llm}/v1"
)
# Mock-calibrated gate (conftest pattern): the strength gradient
# keeps every scripted turn grounded (best cosine ≈ 0.35–0.41).
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
# Phase 67: instant retry waits + the code-default budget (the
# conftest leak-guard pattern).
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
# The repo's .env file carries the owner's BOR_GIT_SOURCES (the app
# reads it from cwd) — override it with an EMPTY value (the env var
# beats the .env file): the registry must hold EXACTLY the local
# directory this suite registers (a leftover env git list would
# pollute the KB the scripted turns run against).
env["BOR_GIT_SOURCES"] = ""
# Leak guards (conftest pattern): an operator's local (gitignored)
# .env cannot leak corpus-specific settings into the app under test.
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
_Settings.model_fields["suggestions"].default
)
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
def _truncate_all() -> None:
"""Fresh registry + KB (the E2E isolation pattern): the E2E suites
share one Postgres, so a leftover git_sources row or document would
pollute the retrieval the scripted turns run against (the
name-signal margins are pinned against EXACTLY these seven
documents)."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, steering_notes, "
"kb_overview, git_sources, folder_summaries"
)
)
db.commit()
def _wait_sync_done_http(client: httpx.Client, timeout_s: float = 180.0) -> dict[str, Any]:
"""Poll the (cookie-authenticated) status endpoint until the run
reaches a terminal state (the test_summary_seed_context pattern,
over plain httpx — this fixture has no browser page yet)."""
deadline = time.monotonic() + timeout_s
body: dict[str, Any] = {}
while time.monotonic() < deadline:
r = client.get("/api/sync/status")
assert r.status_code == 200, r.text
body = r.json()
if body["state"] in ("success", "failed"):
return body
time.sleep(0.5)
raise AssertionError(f"sync did not reach a terminal state: {body}")
def _assert_tiers(
question: str, related_paths: list[str]
) -> None:
"""Pin the name-signal design with the app's REAL hybrid retrieval
over the mock's embeddings (deterministic): the suggested tier is
exactly the four guides (retrieval-strength order) + the name-hit
doc LAST (LOCKED A3 — top-5, NO floor — the name-hit bonus, D2,
lifts it into the tier) and the related tier is the rank-6+
remainder (``related_max_docs`` = 2). A fixture-text regression
that breaks the design fails here, at setup, with a clear
message."""
with SessionLocal() as db:
chunks = retrieve(db, question, embed_text(question))
suggested = [f"{d.source}/{d.path}" for d in select_suggested(chunks)]
related = [
f"{d.source}/{d.path}"
for d in select_related(
chunks,
{d.id for d in select_suggested(chunks)},
_Settings.model_fields["related_max_docs"].default,
)
]
expected = [f"{SOURCE}/{p}" for p in SUGGESTED]
assert suggested == expected, (
f"suggested tier drifted for {question!r}: {suggested} "
f"(expected {expected})"
)
assert related == [f"{SOURCE}/{p}" for p in related_paths], (
f"related tier drifted for {question!r}: {related}"
)
@pytest.fixture(scope="module")
def synced_kb(app_server: str, seed_dirs: Path) -> None:
"""The story's precondition: the one-source KB synced under the
deterministic mock. Registers the fixture directory through the
authenticated API (the ``test_local_directory_sources.py``
pattern), runs the REAL in-process sync (``POST /api/sync`` — walk
→ chunk → embed → summaries → overview → folder summaries →
version bump), pins the stored content byte-identical to the
tracked fixture files, pins LOCKED A2 end-to-end (every doc stores
the mock's byte-stable digest + exactly one embedded
``is_summary`` chunk), and pins the engineered invariant (the
name-hit doc LAST of the top-5) for ALL FOUR scripted questions."""
_truncate_all()
with httpx.Client(base_url=app_server, timeout=30.0) as client:
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, r.text
r = client.post(
"/api/git-sources", json={"kind": "local", "path": str(seed_dirs)}
)
assert r.status_code == 201, r.text
r = client.post("/api/sync")
assert r.status_code == 202, r.text
body = _wait_sync_done_http(client)
assert body["state"] == "success", body
detail = body["detail"]
assert detail["added"] == len(DOCS), detail
assert detail["pruned"] == 0, detail
# The import stored the tracked fixture strings BYTE-IDENTICALLY
# and, for EVERY doc, the mock's byte-stable digest: the
# deterministic assertion surface of the whole suite.
with SessionLocal() as db:
for path, _title, _i, _f, _s in DOCS:
on_disk = (FIXTURES / path).read_text(encoding="utf-8")
stored = db.scalar(
select(Document).where(
Document.source == SOURCE, Document.path == path
)
)
assert stored is not None, f"fixture doc {path} was not imported"
assert stored.content == on_disk, f"stored content drifted for {path}"
assert stored.summary == _expected_summary(on_disk, path), (
f"summary is not the mock digest for {path}: {stored.summary!r}"
)
schunks = [c for c in stored.chunks if c.is_summary]
assert len(schunks) == 1 and schunks[0].position == -1, (
f"expected exactly one is_summary chunk for {path}"
)
assert schunks[0].embedding is not None, (
f"the is_summary chunk of {path} is not embedded"
)
_assert_tiers(TAIL_QUESTION, RELATED)
_assert_tiers(TOOLS_QUESTION, RELATED)
_assert_tiers(FOLDER_QUESTION, RELATED_FOLDER)
_assert_tiers(BASE_QUESTION, RELATED)
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
"""Per-test query_log isolation (the KB itself is module-scoped —
the scripted turns never change it, so the registry and the KB
persist across the tests of this module)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
yield
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
# --------------------------------------------------------------------------
# Page helpers (the test_summary_seed_context house patterns)
# --------------------------------------------------------------------------
#: Captures the raw SSE ``data:`` payloads of the /api/chat stream
#: (a response clone read in the background) — wire-level assertions
#: for the ``tool`` / ``done`` frames, independent of the UI rendering.
SSE_HOOK = """
() => {
if (window.__sseInstalled) return;
window.__sseInstalled = true;
window.__sseFrames = [];
const origFetch = window.fetch;
window.fetch = async function (...args) {
const res = await origFetch.apply(this, args);
try {
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
if (url.includes('/api/chat')) {
res.clone().text().then((bodyText) => {
for (const block of bodyText.split('\\n\\n')) {
const line = block.trim();
if (line.startsWith('data: ')) {
window.__sseFrames.push(line.slice(6));
}
}
});
}
} catch (e) { /* non-clonable responses: ignored */ }
return res;
};
}
"""
def _install_page_hooks(page: Page) -> None:
page.evaluate(SSE_HOOK)
def _frames(page: Page) -> list[dict]:
"""The SSE frames captured since the last submit (``_submit``
clears the buffer), once the hook's background read settles."""
deadline = time.monotonic() + 30.0
while True:
raw = page.evaluate("() => window.__sseFrames || []")
parsed = [json.loads(line) for line in raw if line]
if any(f.get("type") == "done" for f in parsed):
return parsed
if time.monotonic() > deadline:
raise AssertionError(
f"SSE hook captured no `done` frame (frames so far: "
f"{len(parsed)}) — hook install failed?"
)
time.sleep(0.05)
def _tool_frames(frames: list[dict]) -> list[dict]:
return [f for f in frames if f.get("type") == "tool"]
def _submit(page: Page, question: str) -> None:
page.evaluate("window.__sseFrames = []")
page.fill("#message-input", question)
page.click("#send-btn")
# The user bubble lands synchronously with the submit handler.
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
def _wait_settled(page: Page) -> None:
"""The turn is complete: answer text in the bubble, button recovered
(the phase-48 settle wait)."""
expect(page.locator(".msg.brain .bubble").last).not_to_have_text("", timeout=30_000)
expect(page.locator("#send-btn")).to_be_enabled(timeout=30_000)
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
def _last_brain(page: Page) -> Locator:
return page.locator(".msg.brain").last
def _last_query_log() -> QueryLog:
with SessionLocal() as db:
rows = db.scalars(select(QueryLog)).all()
assert len(rows) == 1, f"expected exactly one query_log row, got {len(rows)}"
return rows[0]
def _assert_chip_row(page: Page, read_paths: list[str]) -> None:
"""The citation surface of a grounded turn (phase 119, LOCKED A1 —
the retired phase-118 A4 union is gone): the chip row is the
AGENT-READ docs only (a zero-read turn chips NOTHING: the
suggested docs are seed context, not citations)."""
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(len(read_paths))
for path in read_paths:
expect(
chips.filter(has_text=path), message=f"chip for {path}"
).to_have_count(1)
def _assert_related_row(page: Page, related_paths: list[str]) -> None:
"""The de-emphasized ``related-docs`` row (phase-113 UI, untouched
by phase 119 — never a citation chip): the rank-6+ remainder
deduped against the cited read docs (a read related doc is a chip,
never a "nearby doc")."""
row = page.locator(".msg.brain .related-docs")
expect(row).to_have_count(1)
expect(row.first).to_have_attribute("aria-label", "Nearby docs, in case")
links = page.locator(".msg.brain .related-docs .related-doc")
expect(links).to_have_count(len(related_paths))
for i, path in enumerate(related_paths):
expect(links.nth(i)).to_contain_text(f"{SOURCE}/{path}")
# The related links are never citation chips.
expect(page.locator(".msg.brain .related-docs .source-chip")).to_have_count(0)
# --------------------------------------------------------------------------
# 1. (a) the name-signal doc is the LAST of the top-5 suggested — the
# tail echo proves it reached the prompt as the last seed block
# --------------------------------------------------------------------------
def test_name_hit_doc_is_last_suggested(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
"""The tail-echo question quotes the last 160 chars of the seeded
``<documents>`` block: they end in the LAST suggested doc's SUMMARY
— the name-hit doc's byte-stable digest tail + pointer line (the
digitless-component rule + the D2 bonus put ``forgejo-home`` in
the tier, its neutral body keeps it last). No other doc's pointer
line and no doc's tail sentinel are in the echoed context — the
summary seed, not the full text. The zero-read grounded turn
chips nothing (LOCKED A1); the related row renders rank 6+; the
durable record carries suggested + related (118-A3 untouched)."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, TAIL_QUESTION)
_wait_settled(page)
# No tools on this turn — the answer is the mock's direct tail echo
# of the seeded context (the summary-seed lens).
frames = _frames(page)
assert _tool_frames(frames) == [], _tool_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# Phase 119 (LOCKED A1): the grounded done frame cites the READ
# docs only — this turn read NOTHING (the summary-only fast path),
# so the citation surface is empty; the suggested set (name-hit
# doc included) is seed context, not citations.
assert done["sources"] == [], done["sources"]
bubble = _last_brain(page).locator(".bubble")
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
# The echoed tail ends in the name-hit doc's summary: its
# deterministic ``Source:`` pointer line (the digest is pinned in
# ``synced_kb`` — a content preview would carry the neutral-body
# text instead, and the pointer line only ever exists on a stored
# summary). The bubble renders the answer as markdown, which
# collapses the summary's newline — pin the pointer line alone
# (it sits inside the echoed 160 chars, byte-exact).
expect(bubble).to_contain_text(f"Source: {SOURCE}/{NAME_HIT}")
# No OTHER doc's pointer line is in the echoed tail — the
# name-hit doc was the LAST suggested block (the (a) invariant's
# visible proof).
for path, _t, _i, _f, _s in DOCS:
if path != NAME_HIT:
expect(bubble).not_to_contain_text(f"Source: {SOURCE}/{path}")
# And no document's tail sentinel: the full content of no
# suggested doc reached the model (summary seed only).
for sentinel in SENTINELS:
expect(bubble).not_to_contain_text(sentinel)
expect(bubble).not_to_contain_text(TRUNCATION_MARKER)
# The zero-read grounded turn chips NOTHING (LOCKED A1); the
# related row renders rank 6+ (the de-emphasized row, untouched).
_assert_chip_row(page, [])
_assert_related_row(page, RELATED)
# Durable record: grounded; suggested + related (LOCKED A3 — the
# log records retrieval, not citations; phase 119 A1 retires the
# chip surface only, not the record).
row = _last_query_log()
assert row.question == TAIL_QUESTION
assert row.deflected is False
assert row.sources == ", ".join(f"{SOURCE}/{p}" for p in SUGGESTED + RELATED)
# --------------------------------------------------------------------------
# 2. (b) chips = READ docs only — the single-read flow chips exactly
# the one read doc; the related row renders (deduped)
# --------------------------------------------------------------------------
def test_single_read_chips_only_the_read_doc(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
"""The ``use your tools`` flow drives the mock's single read:
``ls`` (the top level) → drill ``ls(namekb)`` → ``read`` the FIRST
file line (the root file ``zz-folder-map.md`` — subfolders list
first, files after). The bubble carries EXACTLY ONE
``.source-chip`` — the read doc (LOCKED A1); none of the five
suggested docs chips (the name-hit doc is seed context, not a
citation); the de-emphasized related row renders the rank-6+
remainder deduped against the cited read doc (the read doc is a
chip, never a "nearby doc")."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, TOOLS_QUESTION)
_wait_settled(page)
# Wire level: exactly three `tool` frames — ``ls`` (the top
# level), the drill ``ls`` scoped to the first (only) source
# (phase 94), then ``read`` the first file line's combined
# source/path — and all three ahead of the first `delta` frame.
frames = _frames(page)
assert _tool_frames(frames) == [
{"type": "tool", "name": "ls", "argument": None},
{"type": "tool", "name": "ls", "argument": SOURCE},
{"type": "tool", "name": "read", "argument": f"{SOURCE}/{READ_TARGET}"},
], _tool_frames(frames)
first_delta = next(i for i, f in enumerate(frames) if f.get("type") == "delta")
assert all(
i < first_delta for i, f in enumerate(frames) if f.get("type") == "tool"
)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# Phase 119 (LOCKED A1): done.sources = the READ docs only —
# exactly the one read; the five suggested docs (name-hit doc
# included) never chip.
assert [(s["source"], s["path"]) for s in done["sources"]] == [
(SOURCE, READ_TARGET)
], done["sources"]
# done.related = the rank-6+ remainder deduped against the cited
# read doc — the read doc is rank 7 (IN the remainder) and drops
# out: only the sibling name-hit doc remains.
assert [(s["source"], s["path"]) for s in done["related"]] == [
(SOURCE, "forgejo/forgejo-nginx.md")
], done["related"]
# The final answer quotes the read document (the mock's
# deterministic quote: "Read <source/path>. <first 80 chars of its
# content>").
bubble = _last_brain(page).locator(".bubble")
expect(
bubble,
).to_contain_text(f"Read {SOURCE}/{READ_TARGET}.", timeout=30_000)
expect(bubble).not_to_contain_text(TRUNCATION_MARKER)
# The UI chip row: EXACTLY the one read doc (LOCKED A1) — and
# explicitly NONE of the suggested docs (the name-hit doc first).
_assert_chip_row(page, [READ_TARGET])
expect(
page.locator(".msg.brain .source-chip", has_text="install-guide")
).to_have_count(0)
expect(
page.locator(".msg.brain .source-chip", has_text=NAME_HIT)
).to_have_count(0)
# The related row renders rank 6+ deduped against the cited read
# doc (phase-113 behavior untouched).
_assert_related_row(page, ["forgejo/forgejo-nginx.md"])
# Durable record: grounded; suggested + related + read (deduped —
# the read doc sits in the related slot, LOCKED A3 — phase 119 A1
# retires the chip surface only, not the record).
row = _last_query_log()
assert row.question == TOOLS_QUESTION
assert row.deflected is False
assert row.sources == ", ".join(
f"{SOURCE}/{p}" for p in SUGGESTED + RELATED
)
# --------------------------------------------------------------------------
# 3. (c) a zero-read grounded turn chips nothing (LOCKED A1's visible
# consequence)
# --------------------------------------------------------------------------
def test_zero_read_grounded_turn_chips_nothing(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
"""A PLAIN distinctive question (no tool trigger) grounds on the
name tokens: the ``done`` frame's ``sources == []`` and ZERO
``.source-chip`` elements in the bubble — LOCKED A1's visible
consequence (the suggested docs, name-hit doc included, are seed
context, never citations); the answer still renders grounded
(deflected: false); the related row renders rank 6+."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, BASE_QUESTION)
_wait_settled(page)
frames = _frames(page)
assert _tool_frames(frames) == [], _tool_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# LOCKED A1's wire-level consequence: a grounded turn that read
# nothing cites nothing.
assert done["sources"] == [], done["sources"]
bubble = _last_brain(page).locator(".bubble")
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
# Zero chips — not just "no suggested chips": the citation
# surface is empty.
_assert_chip_row(page, [])
_assert_related_row(page, RELATED)
# Durable record: grounded; suggested + related (LOCKED A3).
row = _last_query_log()
assert row.question == BASE_QUESTION
assert row.deflected is False
assert row.sources == ", ".join(f"{SOURCE}/{p}" for p in SUGGESTED + RELATED)
# --------------------------------------------------------------------------
# 4. (d) the suggested-folder lines ride the prompt — echoed verbatim
# by the new mock trigger
# --------------------------------------------------------------------------
def test_folder_lines_echoed_verbatim(
page: Page, app_url: str, synced_kb: None, db_ready: None
) -> None:
"""The distinctive question + ``repeat your folder map`` (the new
mock trigger — the prompt-injection-echo convention of
``<tuning>`` / ``<knowledge_base>``) ⇒ the answer ends with the
suggested-folder lines VERBATIM (joined with ``; ``): the guides'
folder line (its owner excluded from the entries) and the
name-hit doc's folder line — ``namekb/forgejo/: forgejo-nginx.md``
(the line prefix + its known sibling entry; the owner
``forgejo-home`` is excluded — its identity is already in its
``<document>`` block)."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, FOLDER_QUESTION)
_wait_settled(page)
frames = _frames(page)
assert _tool_frames(frames) == [], _tool_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
assert done["sources"] == [], done["sources"]
bubble = _last_brain(page).locator(".bubble")
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
# The folder lines VERBATIM — both lines, suggested-folder order
# (the guides' folder first — its first suggested doc leads; the
# name-hit doc's folder second), entries owner-excluded.
expect(bubble).to_contain_text(
"(folders: "
"namekb/deploy/: install-guide-b.md, install-guide-c.md, install-guide-d.md; "
"namekb/forgejo/: forgejo-nginx.md)",
)
# The task's pin: the name-hit doc's folder line prefix + one
# known sibling entry.
expect(bubble).to_contain_text(f"{SOURCE}/forgejo/: forgejo-nginx.md")
# Zero-read grounded turn — chips nothing (LOCKED A1).
_assert_chip_row(page, [])
# Durable record: grounded; suggested + related (LOCKED A3 — the
# folder question's rank-6+ order).
row = _last_query_log()
assert row.question == FOLDER_QUESTION
assert row.deflected is False
assert row.sources == ", ".join(
f"{SOURCE}/{p}" for p in SUGGESTED + RELATED_FOLDER
)
+13 -13
View File
@@ -472,13 +472,13 @@ def test_answer_content_intact(page: Page, app_url: str, seeded_kb: None) -> Non
page.set_default_timeout(30_000)
login(page, app_url, next="/")
# The long answer streams to completion with its sources ...
# The long answer streams to completion (phase 119, LOCKED A1: a
# zero-read turn chips nothing — the suggested kubernetes doc never
# chips; the retired phase-118 A4 union is gone) ...
submit(page, LONG_QUESTION)
wait_settled(page)
expect(page.locator(".msg.brain .bubble").last).to_contain_text(LONG_ANSWER_END)
expect(
page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
).to_have_count(1)
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# ... and a thinking turn completes with its block auto-collapsed
# (phase 17: open while streaming, closed from the first delta on).
@@ -489,25 +489,25 @@ def test_answer_content_intact(page: Page, app_url: str, seeded_kb: None) -> Non
expect(details.locator(".thinking-text")).to_contain_text(THINKING_FRAGMENT)
expect(page.locator(".msg.brain .bubble").last).to_contain_text(MOCK_ANSWER_MARKER)
# Persistence: four messages, the thinking text + sources stored raw.
# Persistence: four messages, the thinking text stored raw — and
# the thinking turn's sources EMPTY (phase 119, LOCKED A1: it read
# nothing, so the done frame — and the record — carry no sources;
# the retired phase-118 A4 union is gone).
raw = page.evaluate(f"() => localStorage.getItem('{STORAGE_KEY}')")
stored = json.loads(raw)
assert [m["who"] for m in stored["messages"]] == ["user", "brain", "user", "brain"]
assert LONG_ANSWER_END in stored["messages"][1]["text"]
assert THINKING_FRAGMENT in stored["messages"][3]["thinking"]
assert any(
s["path"] == "homelab/kubernetes.md" for s in stored["messages"][3]["sources"]
)
assert stored["messages"][3]["sources"] == []
# Restore: the long answer (with its chip) and the COLLAPSED thinking
# block come back intact.
# Restore: the long answer and the COLLAPSED thinking block come
# back intact — with ZERO citation chips (both turns read nothing;
# phase 119, LOCKED A1).
page.reload()
expect(page.locator(".msg.user .bubble")).to_have_count(2)
expect(page.locator(".msg.brain .bubble")).to_have_count(2)
expect(page.locator(".msg.brain .bubble").first).to_contain_text(LONG_ANSWER_END)
expect(
page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
).to_have_count(2)
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
restored = page.locator(".msg.brain").last.locator("details.thinking")
expect(restored).not_to_have_attribute("open")
expect(restored.locator(".thinking-text")).to_contain_text(THINKING_FRAGMENT)
+14 -6
View File
@@ -24,8 +24,10 @@ Test → story mapping (Playwright Mapping Rule):
4. ``test_jinja_retrievable_not_deflected`` — a question carrying the
``.j2`` sentinel FTS-matches the chunk (A8: LOW requires best cosine
below threshold **and** zero FTS hits) → honest-positive: the answer
bubble is not ``.is-deflected`` and a source chip names
``templates/deploy.j2``.
bubble is not ``.is-deflected`` — and (phase 119, LOCKED A1) the
zero-read turn chips NOTHING (the ``templates/deploy.j2`` retrieval
doc is suggested context, not a chip; the retired phase-118 A4
union is gone).
Phase 97 adaptation: the Sources table is the DRILL-DOWN TREE — the
rows live at their folder levels (``docs`` → ``homelab`` → ``quadlet``
@@ -218,10 +220,16 @@ def test_jinja_retrievable_not_deflected(
page.fill("#message-input", JINJA_QUESTION)
page.click("#send-btn")
# The done event appends source chips — waiting on the .j2 chip means
# the turn is finished and the retrieval doc reached the UI.
chip = page.locator(".msg.brain .source-chip", has_text="templates/deploy.j2")
expect(chip).to_have_count(1, timeout=30_000)
# Phase 119 (LOCKED A1): the turn read nothing, so ZERO citation
# chips — the .j2 retrieval doc is suggested context, not a chip
# (the retired phase-118 A4 union is gone). The turn is finished
# (and the grounded, non-deflected state is reached) when the
# answer settles with the send button recovered.
expect(page.locator(".msg.brain .bubble").last).to_contain_text(
"Deterministic mock answer for E2E", timeout=30_000
)
expect(page.locator("#send-label")).to_have_text("Send")
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# A8: LOW requires best cosine < threshold AND zero FTS hits — the
# question's sentinel tokens FTS-match the .j2 chunk, so the gate is
+7 -4
View File
@@ -658,10 +658,11 @@ def test_truncated_read_frame_order_live_marker_and_llm_notice(
assert i_tool < i_result < i_delta, (i_tool, i_result, i_delta)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The read document is the turn's cited source (retrieval +
# agent-read, deduped — the grounded-turn record; phase 118: the
# target may be a suggested (summary-seeded) document — its seed
# status never changes the cap's wire contract).
# The read document is in the turn's cited sources (phase 119,
# LOCKED A1: done.sources is the READ docs only — this turn's one
# read; phase 118: the target may be a suggested (summary-seeded)
# document — its seed status never changes the cap's wire
# contract).
assert any(
s["path"] == CAPPED_REL and s["source"] == SOURCE for s in done["sources"]
), done["sources"]
@@ -797,6 +798,8 @@ def test_short_read_control_no_frame_no_marker(
assert _result_frames(frames) == [], _result_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The read document is the turn's cited source (phase 119 A1 — the
# READ docs only; the target may be a suggested/summary-seeded doc).
assert any(
s["path"] == SHORT_REL and s["source"] == SOURCE for s in done["sources"]
), done["sources"]
+25 -10
View File
@@ -10,11 +10,15 @@ deterministic mock embeddings (same pattern as the earlier story suites).
The four tests map the story's acceptance criteria:
1. multi-format fixture import — hidden doc excluded, ``/api/docs`` counts
2. "How did I install gitlab?" — grounded (not deflected), gitlab chip,
``query_log`` row with the gitlab doc in ``sources``
2. "How did I install gitlab?" — grounded (not deflected), ZERO citation
chips (phase 119, LOCKED A1 — the turn reads nothing: the suggested
gitlab doc is context, not a chip), ``query_log`` row with the gitlab
doc in ``sources`` (LOCKED A3, untouched)
3. keyword-only question ("kafkabridge") beats the vector ranking — the
corroborated-lexical gate (A8 revised 2026-09-14) grounds it end to
end: weak cosine, but an FTS hit AND cosine >= lexical_support_floor
end: weak cosine, but an FTS hit AND cosine >= lexical_support_floor;
the FTS-matched doc tops the durable record (the chip pin is retired
with the phase-118 A4 union — a zero-read turn chips nothing)
4. "sourdough" — deflected bubble + ≥2 "Maybe try" chips
"""
from __future__ import annotations
@@ -156,9 +160,15 @@ def test_gitlab_question_is_grounded_with_gitlab_chip(
# Grounded: no deflected bubble at all.
expect(page.locator(".msg.brain.is-deflected")).to_have_count(0)
# The gitlab document is cited (a chip carrying its path).
chip = page.locator(".msg.brain .source-chip", has_text="container_gitlab/gitlab.md")
expect(chip).to_have_count(1, timeout=30_000)
# Phase 119 (LOCKED A1): the turn read nothing, so ZERO citation
# chips — the suggested gitlab doc is seed context, not a citation
# (the retired phase-118 A4 union is gone); the ranking assertion
# lives in the durable record below (LOCKED A3, untouched). The
# button label is the settle sync: the client re-labels Send on the
# done frame, and the server writes the query_log row just before
# yielding it.
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
# Durable record: not deflected, and the gitlab doc is in sources.
with SessionLocal() as db:
@@ -186,10 +196,15 @@ def test_keyword_only_question_beats_vector_ranking(
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator(".msg.brain.is-deflected")).to_have_count(0)
# The FTS-matched doc is the TOP source chip (it beats the vector rank).
first_chip = page.locator(".msg.brain .source-chip").first
first_chip.wait_for(state="visible", timeout=30_000)
expect(first_chip).to_contain_text("static-dns.json")
# Phase 119 (LOCKED A1): zero-read turn → ZERO chips (the
# FTS-matched doc beats the vector rank in the DURABLE record below
# — LOCKED A3, untouched; the retired phase-118 A4 chip pin is
# gone). Wait for the settle via the Send label instead of a chip
# (the in-flight button is the enabled Stop control — only the
# label proves the done frame landed and the query_log row was
# written).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
expect(page.locator("#send-label")).to_have_text("Send", timeout=30_000)
with SessionLocal() as db:
row = db.scalars(select(QueryLog)).one()
+15 -14
View File
@@ -50,9 +50,11 @@ Test → phase mapping:
(``Found …`` — the match reached the model), and the turn settles to
idle with no error banner.
2. ``test_search_adds_no_source_by_itself`` — context accounting
(locked A5): the search-only flow (no read) leaves
``done.sources`` / the source chips / ``query_log.sources`` at the
retrieval baseline — the search adds no source by itself.
(locked A5, phase-119 A1 shape): the search-only flow (no read)
leaves ``done.sources`` / the source chips EMPTY (chips cite read
docs only — the search adds no source by itself, and the retrieval
doc was never read) while ``query_log.sources`` keeps the retrieval
baseline (LOCKED A3, untouched).
3. ``test_search_tool_line_re_renders_after_reload`` — the persisted
record (phase 14 convention: the generic ``{name, argument}``
toolAcc) re-renders the search line through the same helper.
@@ -423,25 +425,24 @@ def test_search_adds_no_source_by_itself(
_wait_settled(page)
# The search really ran (its wire frame is present) — yet the
# search-only flow (no read) leaves done.sources at the RETRIEVAL
# baseline: the one fixture doc, nothing added by the search.
# search-only flow (no read) adds NO cited source: phase 119,
# LOCKED A1 — done.sources is the READ docs only, and nothing was
# read (the retrieval doc was never read, so it never chips — the
# retired phase-118 A4 suggested+read union is gone).
frames = _frames(page)
assert _tool_frames(frames) == [
{"type": "tool", "name": "grep", "argument": SEARCH_PATTERN}
]
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False
assert [(s["source"], s["path"]) for s in done["sources"]] == [
(SEED_SOURCE, SEED_PATH)
]
assert done["sources"] == [], done["sources"]
# UI: exactly one source chip — the retrieval doc (the search
# renders no chip of its own).
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(1)
expect(chips.nth(0)).to_contain_text(SEED_SP)
# UI: ZERO source chips — the search renders no chip of its own, and
# the never-read retrieval doc chips nothing (phase 119 A1).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# Durable record: the sources row is unchanged by the search alone.
# Durable record: the sources row keeps the retrieval baseline (the
# search adds nothing to it either — LOCKED A3, untouched).
row = _last_query_log()
assert row.deflected is False
assert row.sources == SEED_SP
+15 -17
View File
@@ -25,12 +25,14 @@ click is the idempotent share of the linked row):
cookies) opening ``/shared/<token>`` sees the full conversation
read-only through the same record shape: title = the auto-title,
user + brain bubbles (the same deterministic answer text the admin
session saw), the thinking block RESTORED COLLAPSED, the source
chips as PLAIN TEXT (zero ``a.source-chip`` — guests cannot open
documents, the documents API is admin-only), and ZERO interactive
controls anywhere (no composer, no Save/Share pills, no Tune/Retry,
no button chips); the nav's admin-only links stay hidden for a
guest;
session saw), the thinking block RESTORED COLLAPSED, ZERO source
chips of any kind — the shared turn read nothing, so (phase 119,
LOCKED A1) its ``done`` sources are empty and nothing re-renders as a
chip; where chips exist they are always PLAIN TEXT (zero
``a.source-chip`` — guests cannot open documents, the documents API
is admin-only) — and ZERO interactive controls anywhere (no
composer, no Save/Share pills, no Tune/Retry, no button chips); the
nav's admin-only links stay hidden for a guest;
* **Share from History + unshare** — the History row's Share column:
"Create link" → Copy + Unshare; Unshare is the inline two-step
(no native dialog); after Yes the cell returns to "Create link",
@@ -345,17 +347,13 @@ def test_anonymous_shared_view(
expect(think).to_have_count(1)
expect(think.first).not_to_have_attribute("open")
# Source chips are PLAIN TEXT: the on-topic turn carries its
# source chips (the suggested docs — the hybrid retrieval's
# summary-seed tier, phase 118), but every one as a <span>:
# zero <a.source-chip> anywhere (a guest cannot open documents;
# the documents API is admin-only).
assert (
anon.locator(".msg.brain .source-chip").count() >= 1
), "the grounded turn must carry its source chips"
expect(
anon.locator(".msg.brain .source-chip", has_text="kubernetes.md")
).to_have_count(1)
# Source chips: ZERO — the shared turn read nothing, so
# (phase 119, LOCKED A1) its done sources are empty and the
# shared page re-renders nothing (the retired phase-118 A4
# suggested+read union is gone); and where chips DO exist they
# are always plain text: zero <a.source-chip> anywhere (a
# guest cannot open documents; the documents API is admin-only).
expect(anon.locator(".msg.brain .source-chip")).to_have_count(0)
expect(anon.locator("a.source-chip")).to_have_count(0)
# ZERO interactive controls anywhere in the conversation: no
+58 -42
View File
@@ -1,22 +1,27 @@
"""Phase 113 E2E (Playwright) — phase 118 re-targeted: the citation-surface
contract (LOCKED A4) as VISIBLE chip counts — the summary-seed contract
replaced the phase-112/113 usefulness bar: the chip row is the
SUGGESTED tier (top-5 distinct docs, NO floor) + agent reads (none on
these turns), the related row is rank 6+ (capped at
``related_max_docs`` = 2), and a deflected turn still cites nothing
(done.sources = [] — its weak hits are suggested for the durable record
but are never citation chips).
"""Phase 113 E2E (Playwright) — phase 118 re-targeted, phase 119
re-targeted again: the citation-surface contract (phase 119, LOCKED
A1) as VISIBLE chip counts — chips are the AGENT-READ docs only (the
retired phase-118 A4 suggested+read union is gone): neither turn on
this suite reads anything, so a grounded turn shows ZERO citation
chips (the suggested tier is seed context, not citations — the
phase-112/113 usefulness bar was retired with the full-text seeds, and
the suggested chips were retired with the phase-118 A4 union), the
related row is rank 6+ (capped at ``related_max_docs`` = 2, untouched),
and a deflected turn still cites nothing (done.sources = [] — its weak
hits are suggested for the durable record but are never citation
chips).
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_source_chip_quality.py -v --no-cov
Acceptance (TODO L144–146, phase-118 shape): a grounded turn's chip row
is the suggested tier — the 5 docs the model was seeded with (its
"start here" set — the old "one chip per bar-clearing doc" is retired
with the full-text seeds; the LLM decides what the summaries earn);
a deflected turn shows zero citation chips (its weak hits, if any, live
in the de-emphasized related row — ``.related-doc`` links, NEVER
Acceptance (TODO L144–146, phase-119 shape): a grounded turn's chip row
is the READ set — empty on these no-read turns (the visible consequence
of the owner decision 2026-09-16: the suggested docs the model was
seeded with are context, not citations — the LLM decides what the
summaries earn, and a zero-read turn chips nothing); a deflected turn
shows zero citation chips (its weak hits, if any, live in the
de-emphasized related row — ``.related-doc`` links, NEVER
``.source-chip``; L2c: never render weak hits as answer citations).
The fixture KB's tier shapes are deterministic under the E2E mock's
@@ -25,14 +30,20 @@ here against the wire):
* **grounded question** — "What SSH aliases do I have?": best cosine
0.352 ≥ 0.30 → grounded; the suggested tier (top-5, NO floor) is
ssh_aliases.txt > gitlab-compose.yaml > uptime_probe.py > kubernetes.md
> backups.md (fused rank — the lexical-only cosine-0.0 docs rank when
they rank: no floor filters them, LOCKED A3) and the done frame
carries EXACTLY those five cited refs; the rank-6+ remainder
(compose.container, static-dns.json) rides the related row, capped at
two. (The four OBSERVED live shapes are unit-pinned at plan level in
``tests/unit/test_source_chip_quality.py`` — the fixture KB
reproduces the tiered shape live, so no docstring caveat is needed.)
ssh_aliases.txt > gitlab-compose.yaml > uptime_probe.py > backups.md
> kubernetes.md (fused rank — the lexical-only cosine-0.0 docs rank
when they rank: no floor filters them, LOCKED A3; phase 119, D1 —
``aliases`` is a stem sub-component of ``ssh_aliases.txt``, so the
doc name-hits, its name-hit row LEADS the lexical list, and the FTS
rank shift re-orders the fused scores — the pre-phase
kubernetes-before-backups order is retired) and the done frame
carries NO cited refs (phase 119 A1 — nothing was read: the
suggested tier seeds the prompt but never chips); the rank-6+
remainder (compose.container, static-dns.json) rides the related
row, capped at two. (The four OBSERVED live shapes are unit-pinned at
plan level in ``tests/unit/test_source_chip_quality.py`` — the
fixture KB reproduces the tiered shape live, so no docstring caveat
is needed.)
* **deflected question** — "How do I bake sourdough bread?": best cosine
0.109 < 0.30 and zero FTS hits → honest deflection → zero cited refs
(done.sources = []); the weak hits ARE suggested (no floor) for the
@@ -106,14 +117,15 @@ def _ask(page: Page, message: str) -> None:
def test_single_source_question_shows_exactly_one_citation_chip(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
"""The acceptance pin (phase 118, A4): a grounded question → the done
bubble carries EXACTLY the suggested tier as ``.source-chip``s (the
five docs the model was seeded with — top-5, NO floor: the old
usefulness-bar "one chip" is retired with the full-text seeds) and
the rank-6+ remainder renders only in the de-emphasized
``.related-docs`` row (``.related-doc`` links — never
``.source-chip``), labeled "Nearby docs, in case:". The durable
record keeps the FULL retrieval (LOCKED A3)."""
"""The acceptance pin (phase 119, A1): a grounded, NO-READ question →
the done bubble carries ZERO ``.source-chip``s (the chip row is the
agent-read docs only — nothing was read here: the five suggested
docs the model was seeded with are context, not citations — the
retired phase-118 A4 union is gone) and the rank-6+ remainder
renders only in the de-emphasized ``.related-docs`` row
(``.related-doc`` links — never ``.source-chip``), labeled "Nearby
docs, in case:" (phase-113 behavior, untouched). The durable record
keeps the FULL retrieval (LOCKED A3, untouched)."""
_reset_db(mock_llm)
page.set_default_timeout(30_000)
login(page, app_url, next="/") # phase 79: chat is require_user-gated
@@ -127,16 +139,14 @@ def test_single_source_question_shows_exactly_one_citation_chip(
# Grounded: no deflected bubble at all.
expect(page.locator(".msg.brain.is-deflected")).to_have_count(0)
# The citation surface IS the suggested tier (LOCKED A4 — top-5,
# NO floor): five chips, in fused rank order — the model was
# seeded with exactly these five docs' summaries.
chip = page.locator(".msg.brain .source-chip")
expect(chip).to_have_count(5, timeout=30_000)
expect(chip.nth(0)).to_contain_text("ssh_aliases.txt")
expect(chip.nth(1)).to_contain_text("gitlab-compose.yaml")
expect(chip.nth(2)).to_contain_text("uptime_probe.py")
expect(chip.nth(3)).to_contain_text("kubernetes.md")
expect(chip.nth(4)).to_contain_text("backups.md")
# Phase 119 (LOCKED A1): the citation surface is the READ docs only
# — this turn read nothing, so ZERO chips; the five suggested docs
# (ssh_aliases.txt > gitlab-compose.yaml > uptime_probe.py >
# backups.md > kubernetes.md — the phase-119 D1 re-rank, pinned in
# the durable record below) seeded the prompt but never chip —
# the retired phase-118 A4 union is gone. The suggested tier itself
# is pinned below by the durable record (LOCKED A3, untouched).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# The rank-6+ remainder rides the related row: a labeled,
# de-emphasized list — one .related-doc link per doc (rank order,
@@ -162,13 +172,19 @@ def test_single_source_question_shows_exactly_one_citation_chip(
row_log = db.scalars(select(QueryLog)).one()
assert row_log.question == SINGLE_SOURCE_QUESTION
assert row_log.deflected is False
# Suggested tier + related remainder, in the logged order.
# Suggested tier + related remainder, in the logged order. Phase
# 119 (D1): the stem sub-component ``aliases`` name-hits
# ``ssh_aliases.txt`` — its name-hit row leads the lexical list,
# the FTS rank shift re-orders the fused scores, and ``backups.md``
# (best eff 0.016277) now out-ranks ``kubernetes.md`` (0.016036);
# re-pinned against the post-phase walk (the record's CONTENT —
# suggested + related + read, deduped — is LOCKED A3, untouched).
assert row_log.sources == (
"docs/homelab/ssh/ssh_aliases.txt, "
"docs/homelab/container_gitlab/gitlab-compose.yaml, "
"docs/homelab/scripts/uptime_probe.py, "
"docs/homelab/kubernetes.md, "
"docs/homelab/backups.md, "
"docs/homelab/kubernetes.md, "
"docs/homelab/quadlet/compose.container, "
"docs/homelab/networking/static-dns.json"
), row_log.sources
+17 -11
View File
@@ -89,10 +89,6 @@ HESITATE_QUESTION = (
#: pre-content pause (SLOW_PRETOKEN_TRIGGER) is running.
THINKING_TAIL = "nothing is invented"
#: Phase-10 viewer URL + phase-13 back=/ (byte-identical to the chip the
#: persistence suite pins — grounded-turn sources are unchanged by 20).
CHIP_HREF = "/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
async def _import_fixtures(mock_port: int) -> ImportSummary:
kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
@@ -334,7 +330,10 @@ def test_full_answer_completes_after_rag_nav_midstream(
brain = msgs[1]
assert brain["text"] == FULL_LONG
assert brain["deflected"] is False
assert any(s["path"] == "homelab/kubernetes.md" for s in brain["sources"])
# Phase 119 (LOCKED A1): the turn read nothing, so its done sources
# — and the persisted record — are EMPTY (the retired phase-118 A4
# union is gone).
assert brain["sources"] == []
# The turn SETTLED — the phase-48 query_log row exists (a
# cancelled turn would leave no row at all).
@@ -408,7 +407,11 @@ def test_nav_switch_before_first_token_completes(
assert [m["who"] for m in msgs] == ["user", "brain"]
assert MOCK_ANSWER_MARKER in msgs[1]["text"]
assert msgs[1]["deflected"] is False
assert any(s["path"] == "homelab/kubernetes.md" for s in msgs[1]["sources"])
# Phase 119 (LOCKED A1): the turn read nothing, so its done sources
# — the stored record's sources list — are EMPTY (the retired
# phase-118 A4 suggested-citation is gone; the full retrieval
# stays in query_log, pinned below).
assert msgs[1]["sources"] == []
# The turn settled — one finalized row (a cancelled turn would
# leave no row at all).
@@ -495,22 +498,25 @@ def test_completed_turn_unaffected(
brain = before["messages"][1]
assert MOCK_ANSWER_MARKER in brain["text"]
assert brain["deflected"] is False
assert any(s["path"] == "homelab/kubernetes.md" for s in brain["sources"])
# Phase 119 (LOCKED A1): the turn read nothing, so its done sources
# — and the persisted record — are EMPTY (the retired phase-118 A4
# union is gone).
assert brain["sources"] == []
# A trip to Sources and back (the turn finished long ago — uiState is
# idle, so the pagehide save point must be a no-op).
page.goto(app_url + "/sources.html")
page.goto(app_url + "/")
# Full answer + source chip rendered; no error banner.
# Full answer rendered with ZERO citation chips (the zero-read turn
# persisted an empty sources list — phase 119, LOCKED A1; the
# retired phase-118 A4 chip pin is gone); no error banner.
expect(page.locator("#empty-state")).to_be_hidden()
expect(page.locator(".msg.user .bubble")).to_contain_text(QUESTION)
bubble = page.locator(".msg.brain .bubble")
expect(bubble).to_have_count(1)
expect(bubble.first).to_contain_text(MOCK_ANSWER_MARKER)
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
expect(chip).to_have_count(1)
expect(chip.first).to_have_attribute("href", CHIP_HREF)
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
_no_error_banner(page)
# Storage is byte-identical to the pre-navigation payload — the
+44 -36
View File
@@ -59,19 +59,21 @@ TODO item — one Playwright file per story, A16):
The inverse of the retired phase-24 pin: NO document's tail
sentinel is in the echoed context — the full content of no
suggested doc ever reached the model (case b). The grounded turn
cites the 5 suggested docs (case e's chip surface, no read yet) and
cites NOTHING (phase 119, LOCKED A1 — a zero-read turn chips
nothing; the retired phase-118 A4 suggested+read union is gone) and
renders the de-emphasized related row (rank 6+); the durable
record carries suggested + related.
record carries suggested + related (118-A3 untouched).
2. ``test_read_suggested_doc_adds_full_text_and_cites`` — cases (c) +
(e): the scripted flow ``read``s the rank-1 suggested doc — the
read SUCCEEDS (phase 118: the seeds are summaries, not full text —
the retired top-2 seed-read refusal is gone) and the mock's
verbatim echo of the read result lands the tail sentinel in the
answer (case c: the full text now arrives through the ``read``
tool, not the seed); the citation chips = suggested + read,
deduped — the read doc is among the suggested, so the chip row is
exactly the 5 suggested docs, and the related row renders rank 6+
(case e); the durable record carries suggested + related + read.
tool, not the seed); the citation chips = the READ docs only
(phase 119, LOCKED A1 — exactly the one read doc; none of the
four other suggested docs chips), and the related row renders
rank 6+ (case e); the durable record carries suggested + related +
read (118-A3 untouched).
"""
from __future__ import annotations
@@ -503,14 +505,16 @@ def _last_query_log() -> QueryLog:
return rows[0]
def _assert_suggested_chips_and_related_row(page: Page, suggested_paths: list[str]) -> None:
"""The citation surface of a grounded turn (LOCKED A4): the chip row
is the suggested set (+ agent reads, deduped — asserted per test)
and the de-emphasized ``related-docs`` row carries the rank-6+
remainder (phase-113 UI, never a citation chip)."""
def _assert_read_chips_and_related_row(page: Page, read_paths: list[str]) -> None:
"""The citation surface of a grounded turn (phase 119, LOCKED A1 —
the retired phase-118 A4 union is gone): the chip row is the
AGENT-READ docs only (deduped, read order — a zero-read turn chips
NOTHING: the suggested docs are seed context, not citations), and
the de-emphasized ``related-docs`` row carries the rank-6+ remainder
(phase-113 UI, untouched — never a citation chip)."""
chips = page.locator(".msg.brain .source-chip")
expect(chips).to_have_count(len(suggested_paths))
for path in suggested_paths:
expect(chips).to_have_count(len(read_paths))
for path in read_paths:
expect(
chips.filter(has_text=path), message=f"chip for {path}"
).to_have_count(1)
@@ -553,11 +557,12 @@ def test_summaries_seed_the_prompt_not_the_full_text(
assert _tool_frames(frames) == [], _tool_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The grounded turn cites the 5 suggested docs (LOCKED A4 — no read
# yet: the chip row IS the suggested set, in rank order).
assert [
(s["source"], s["path"]) for s in done["sources"]
] == [(SOURCE, p) for p in SUGGESTED_TAIL], done["sources"]
# Phase 119 (LOCKED A1): the grounded done frame cites the READ docs
# only — this turn read NOTHING (the summary-only fast path), so the
# citation surface is empty; the suggested set is seed context, not
# citations (the retired phase-118 A4 union is gone). The durable
# record below still carries suggested + related (118-A3 untouched).
assert done["sources"] == [], done["sources"]
bubble = _last_brain(page).locator(".bubble")
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
@@ -583,12 +588,13 @@ def test_summaries_seed_the_prompt_not_the_full_text(
expect(bubble).not_to_contain_text(sentinel)
expect(bubble).not_to_contain_text(TRUNCATION_MARKER)
# Case (e)'s chip surface (no read yet): chips = the 5 suggested
# docs; the related row renders rank 6+ (the de-emphasized row).
_assert_suggested_chips_and_related_row(page, SUGGESTED_TAIL)
# Case (e)'s chip surface (phase 119 A1 — no read yet): ZERO chips;
# the related row renders rank 6+ (the de-emphasized row, untouched).
_assert_read_chips_and_related_row(page, [])
# Durable record: grounded; suggested + related (LOCKED A3 — the log
# records retrieval, not citations).
# records retrieval, not citations; phase 119 A1 retires the chip
# union only, not the record).
row = _last_query_log()
assert row.question == TAIL_QUESTION
assert row.deflected is False
@@ -609,10 +615,10 @@ def test_read_suggested_doc_adds_full_text_and_cites(
retired top-2 seed-read refusal is gone) and the mock's verbatim
echo of the read result lands the document's tail sentinel in the
answer (case c: the full text reached the model through the
``read`` tool, not the seed). The citation chips = suggested +
read, deduped — the read doc is among the suggested, so the chip
row is exactly the 5 suggested docs; the related row renders rank
6+ (case e)."""
``read`` tool, not the seed). The citation chips = the READ docs
only (phase 119, LOCKED A1 — exactly the one read doc; none of the
other four suggested docs chips; the retired phase-118 A4 union is
gone); the related row renders rank 6+ (case e, untouched)."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
_install_page_hooks(page)
@@ -628,13 +634,15 @@ def test_read_suggested_doc_adds_full_text_and_cites(
], _tool_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# LOCKED A4: done.sources = suggested + read, deduped — the read doc
# is already among the suggested, so the citation surface is
# exactly the 5 suggested docs (in rank order).
# Phase 119 (LOCKED A1): done.sources = the READ docs only —
# exactly the one scripted read; the other four suggested docs
# never chip (the retired phase-118 A4 suggested+read union is
# gone).
assert [
(s["source"], s["path"]) for s in done["sources"]
] == [(SOURCE, p) for p in SUGGESTED_READ], done["sources"]
# done.related = the rank-6+ remainder (deduped against the cited).
] == [(SOURCE, READ_TARGET)], done["sources"]
# done.related = the rank-6+ remainder (deduped against the cited —
# the read doc is rank 1, not in the remainder).
assert [
(s["source"], s["path"]) for s in done["related"]
] == [(SOURCE, p) for p in RELATED_PATHS], done["related"]
@@ -658,13 +666,13 @@ def test_read_suggested_doc_adds_full_text_and_cites(
if path != READ_TARGET:
expect(bubble).not_to_contain_text(sentinel)
# Case (e): the UI chip row = suggested + read (deduped — the read
# doc is among the suggested, so chips = the 5 suggested docs), and
# the related row renders rank 6+.
_assert_suggested_chips_and_related_row(page, SUGGESTED_READ)
# Case (e): the UI chip row = the READ docs only (phase 119 A1 —
# exactly the one read doc), and the related row renders rank 6+.
_assert_read_chips_and_related_row(page, [READ_TARGET])
# Durable record: grounded; suggested + related + read (deduped,
# LOCKED A3).
# LOCKED A3 — phase 119 A1 retires the chip union only, not the
# record).
row = _last_query_log()
assert row.question == READ_QUESTION
assert row.deflected is False
+14 -15
View File
@@ -55,9 +55,6 @@ DEFLECT_PHRASE = r"haven't done anything like that"
#: suite keys off it (mock_llm.compose_thinking).
THINKING_FRAGMENT = "Step 2: Check my notes"
STORAGE_KEY = "bor.chat.v1"
#: Phase-10 viewer URL + phase-13 back=/ (byte-identical to the chip the
#: persistence suite pins — grounded-turn sources are unchanged by 17).
CHIP_HREF = "/document.html?source=docs&path=homelab%2Fkubernetes.md&back=%2F"
async def _import_fixtures(mock_port: int) -> ImportSummary:
@@ -154,13 +151,13 @@ def test_thinking_block_streams_open_then_collapses(
expect(bubble).not_to_have_text("", timeout=30_000)
expect(details).not_to_have_attribute("open")
# Settled: full scratchpad, grounded mock answer, source chip(s),
# and the re-enabled send button.
# Settled: full scratchpad, grounded mock answer, ZERO citation
# chips (phase 119, LOCKED A1 — the zero-read turn chips nothing;
# the retired phase-118 A4 union is gone), and the re-enabled send
# button.
expect(details.locator(".thinking-text")).to_contain_text(THINKING_FRAGMENT)
expect(bubble).to_contain_text(MOCK_ANSWER_MARKER)
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
expect(chip.first).to_be_visible()
expect(chip.first).to_have_attribute("href", CHIP_HREF)
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
expect(page.locator("#send-btn")).to_be_enabled()
expect(page.locator("#send-label")).to_have_text("Send")
@@ -232,10 +229,12 @@ def test_thinking_restored_after_reload(page: Page, app_url: str, seeded_kb: Non
expect(restored).not_to_have_attribute("open") # restored COLLAPSED
expect(restored.locator(".thinking-text")).to_have_text(captured)
# Answer bubble + source chip are intact (phase-14 restore path).
# Answer bubble is intact after the restore (phase-14 restore path)
# — with ZERO citation chips (phase 119, LOCKED A1: the zero-read
# turn persisted an empty sources list; the retired phase-118 A4
# union is gone).
expect(page.locator(".msg.brain .bubble").last).to_contain_text(MOCK_ANSWER_MARKER)
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
expect(chip.first).to_have_attribute("href", CHIP_HREF)
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# ---------------------------------------------------------------------------
@@ -252,10 +251,10 @@ def test_no_thinking_block_without_trigger(page: Page, app_url: str, seeded_kb:
# No trigger → no thinking events → no block anywhere on the page.
expect(page.locator("details.thinking")).to_have_count(0)
# The turn itself is complete and grounded, exactly as before phase 17.
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
expect(chip.first).to_be_visible()
expect(chip.first).to_have_attribute("href", CHIP_HREF)
# The turn itself is complete and grounded, exactly as before phase
# 17 — with ZERO citation chips (phase 119, LOCKED A1: the zero-read
# turn chips nothing; the retired phase-118 A4 union is gone).
expect(page.locator(".msg.brain .source-chip")).to_have_count(0)
# ---------------------------------------------------------------------------