Files
brain-of-reese/tests/e2e/test_chat_chip_cap.py
T
ducoterra 3f752fb77f
Build and Push Containers / build-and-push-app (push) Successful in 2m19s
Build and Push Containers / build-and-push-db (push) Successful in 11s
phase: 157_chat_chip_cap
All verification is green. Final report:

**Phase 157 — final verification pass (all 3 tasks already complete in `complete/`)**

- Verified the implemented work matches the design: `CHIP_CAP = 90` constant + display-only cap in `renderChips` (`slice(0,90).trimEnd() + "…"`), `title`/`aria-label`/click payload byte-unchanged; unit pins (89/90/91 letter & space, full-text invariant, short-string guard, Maybe-try single-path guard); phase-104 E2E updated to the cap contract
- Full suite `uv run pytest --cov=app --cov-report=term-missing`: **3578 passed**, app/ coverage **99%** (>90% ✓)
- `uv run ruff check .` clean; `uv run pyright`: 0 errors
- Dedicated E2E `uv run pytest tests/e2e/test_chat_chip_cap.py -v --no-cov`: **4/4 passed** in isolation (DB up)
- Cross-phase regression: phase-104 E2E 6/6, phase 05/80/103 chip suite 8/8 (seed chips <90 chars — cap is a no-op, assertions hold)
- Criteria: ≤91-char capped display ✓ (byte-for-byte pin) · click submits FULL 120-char question (user bubble) ✓ · `title` + clipped-case `aria-label` carry full text ✓ (375px viewport) · <90-char byte-identical ✓ (unit + live DOM) · commit: deferred to the harness (per harness protocol, overrides AGENTS.md rule 8 for this pass)
- Documentation: README.md reviewed in full — every `.env.example` var documented, all referenced scripts/pages/commands verified against the tree, the phase's chip-cap note is in the Chat section; AGENTS.md verified (commands, test files, layout-rule citations — phase 157 adds no new layout principle) → **verified consistent, no changes needed**
- No live-infrastructure changes were required or made
- Next pending phase: **158_thinking_timeline**
2026-10-04 22:18:31 -04:00

419 lines
18 KiB
Python

"""Phase 157 E2E (Playwright): the hero chip's DISPLAY cap (90 chars +
``…``) with the phase-104 full-text contract extended (owner-locked
A1, 2026-10-03 audit: "long opening questions get hard-truncated
mid-word… the chips stretch near full container width before
clipping").
The contract this suite pins end to end (the user-facing half of the
phase — the unit pins in ``tests/unit/test_chip_sizing_question_cap.py``
are the source-level half):
* **capped display** — a >90-char saved-chat opener renders a
COMPACT chip: the displayed text is
``slice(0, 90).trimEnd() + "…"`` — at most 91 chars, ending in
``…`` (the phase-104 single-line CSS clip demoted to the secondary
guard); the display is the EXACT pinned string, asserted
byte-for-byte on the live DOM, not just by length.
* **full text reachable** — the chip's ``title`` (the hover reveal)
and, in the CLIPPED case (a 91-char display overflows the
single-line box — the ≤640px viewport, where the chip's
``max-width: 100%`` binds and ``text-overflow: ellipsis``
engages), the ``aria-label`` accessible name both carry the FULL
question (phase 104, extended by the cap).
* **the click submits the FULL question** (the load-bearing
assertion) — clicking a capped chip sends the UN-capped 120-char
question: the user bubble is the full question, not the 91-char
display (the cap is display-only — the phase-80 contract).
* **short-chip regression guard** — a <90-char opener renders
byte-identical to pre-phase on the live DOM: the
``textContent`` is the full question with NO ``…`` appended, the
``title`` is the full question, and (unclipped at the 1280px
viewport) no ``aria-label`` — the cap is a no-op below 90.
Seeding idiom (mirrored from the phase-80/103 suite
``tests/e2e/test_suggestion_chips.py``, not invented): the chips are
the session openers read from ``saved_chats`` — each test saves its
chat(s) via the signed-in admin's ``POST /api/chats`` and loads a
FRESH page, whose boot fetch renders the chips. The mock LLM is the
deterministic answer source (no ``E2E_REAL_LLM`` dependency); the
seeded questions are FIXED strings (the pins, not generated
lengths). The click test grounds on the fixture KB (the 120-char
opener carries the "kubernetes" token — the mock-calibrated cosine
clears the 0.30 gate, non-deflected). ``saved_chats`` is global state
on the shared e2e Postgres AND the state this contract reads — the
autouse fixture truncates it before and after EVERY test (including
the click test's auto-saved row), so each test starts from — and
leaves — an empty deployment.
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_chat_chip_cap.py -v --no-cov
"""
from __future__ import annotations
import asyncio
import json
import time
from collections.abc import Iterator
from pathlib import Path
from threading import Thread
from typing import Any
import pytest
from playwright.sync_api import Browser, Page, expect
from sqlalchemy import text
from app.config import Settings
from app.db import SessionLocal
from app.rag.importer import ImportSummary, import_sources
from app.rag.llm import LLMClient
from e2e.auth_helpers import login
REPO = Path(__file__).resolve().parents[2]
FIXTURES = REPO / "tests" / "fixtures" / "docs"
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
#: The FIXED 120-char opener (90+ — the cap engages deterministically;
#: the suite's pin, not a generated length). It carries the
#: "kubernetes" token (``tests/fixtures/docs/homelab/kubernetes.md``
#: + the deployments fixture), so the click test's turn grounds on
#: the mock-calibrated gate (0.30) — non-deflected, marker answer.
LONG_Q = (
"What are the correct arguments for running a kubernetes node pool "
"with qwen 3.8 27b on llama.cpp in the homelab cluster?"
)
assert len(LONG_Q) == 120, "the pin is a FIXED 120-char string"
#: The phase-157 display of LONG_Q (the exact-string pin — char 90 is
#: the "a" of "llama", so the trimEnd is a no-op here; the
#: space-at-char-90 trimEnd case is the unit pins' job). 91 chars.
LONG_Q_CAPPED = LONG_Q[:90].rstrip() + "…"
assert len(LONG_Q_CAPPED) == 91 and LONG_Q_CAPPED.endswith("…")
#: The follow-up (phase-103 semantics: it must NEVER surface as a
#: chip — a follow-up is meaningless as a conversation starter).
LONG_FOLLOW_UP = "What about the node pool's GPU drivers?"
#: Two <90-char openers for the three-chip state (the row fills to
#: the cap of 3 with the long opener NEWEST — the first chip).
SHORT_A = "Which provider is the primary DNS for reeseapps.com?"
SHORT_A_FOLLOW_UP = "What about the secondary DNS for the LAN?"
SHORT_B = "What cron schedule runs the restic prune?"
SHORT_B_FOLLOW_UP = "And where do the restic lock files live?"
#: The STANDALONE <90-char regression-guard question.
SHORT_Q = "How do I rotate the WireGuard keys on the VPN node?"
SHORT_Q_FOLLOW_UP = "What about the peers' allowed-ips?"
@pytest.fixture(autouse=True)
def clean_chats(db_ready: None) -> Iterator[None]:
"""``saved_chats`` is the state the onboarding chips read (the
phase-103 openers) AND the click test's turn auto-saves a row —
truncate it before and after EVERY test so each starts from (and
leaves) an empty deployment (the seed-chip state), exactly the
phase-80/103/104 autouse pattern."""
with SessionLocal() as db:
db.execute(text("TRUNCATE saved_chats"))
db.commit()
yield
with SessionLocal() as db:
db.execute(text("TRUNCATE saved_chats"))
db.commit()
async def _import_fixtures(mock_port: int) -> ImportSummary:
kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
return await import_sources([FIXTURES], LLMClient(settings))
def _run_in_thread(coro: Any) -> Any:
"""Run a coroutine on a worker thread (Playwright owns the test loop)."""
box: dict[str, Any] = {}
def runner() -> None:
try:
box["value"] = asyncio.run(coro)
except BaseException as e: # noqa: BLE001 — re-raised on the test thread
box["error"] = e
t = Thread(target=runner)
t.start()
t.join()
if "error" in box:
raise box["error"]
return box["value"]
def _seed_kb(mock_port: int) -> ImportSummary:
"""Deterministic KB: truncate the KB tables, import the fixture
docs (needed by the click test's grounded answer).
``saved_chats`` is the autouse fixture's job."""
with SessionLocal() as db:
db.execute(text("TRUNCATE chunks, documents, query_log"))
db.commit()
summary = _run_in_thread(_import_fixtures(mock_port))
assert summary is not None and summary.added == 13 # A9 formats (phase 47 added quadlet+j2)
return summary
def _user(q: str) -> dict[str, Any]:
return {"who": "user", "text": q}
def _brain(text: str = "Grounded mock brain reply.") -> dict[str, Any]:
return {"who": "brain", "text": text}
def _save_chat(page: Page, app_url: str, messages: list[dict[str, Any]]) -> dict[str, Any]:
"""Save one conversation as the signed-in admin (``POST /api/chats``)
and return the 201 body. ``updated_at`` is the API's stamp (server
``now()`` at INSERT) — the save ORDER is what makes the chip-walk
order deterministic in the multi-chat test."""
r = page.request.post(
f"{app_url}/api/chats",
data=json.dumps({"messages": messages}),
headers={"Content-Type": "application/json"},
timeout=10_000,
)
assert r.status == 201, r.text
return r.json()
def _chips(page: Page) -> Any:
return page.locator("#suggestions .suggestion-chip")
def test_capped_display(page: Page, app_url: str, db_ready: None) -> None:
"""The >90-char opener renders a COMPACT chip: the displayed text
is the EXACT ``slice(0, 90).trimEnd() + "…"`` pin (91 chars,
ending in ``…``), and the row is the phase-80/103 three-chip
contract — the three seeded openers, newest ``updated_at``
first; the <90-char openers render byte-identical (no ``…``), and
none of the chats' follow-ups surfaces."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
# Save oldest→newest: the API stamps ``updated_at`` (server
# now()), so save order IS walk order (newest first). The short
# pauses keep the three stamps strictly apart.
stamps: list[str] = []
for opener, follow_up in (
(SHORT_A, SHORT_A_FOLLOW_UP),
(SHORT_B, SHORT_B_FOLLOW_UP),
(LONG_Q, LONG_FOLLOW_UP),
):
body = _save_chat(
page,
app_url,
[
_user(opener), _brain(),
_user(follow_up), _brain(),
],
)
stamps.append(body["updated_at"])
time.sleep(0.05)
assert stamps == sorted(stamps) and len(set(stamps)) == 3, (
"the three API-stamped updated_at values must be strictly increasing"
)
# A FRESH page load (a new boot fetch): EXACTLY the three
# openers, newest first — the long one capped, the shorts not.
page.goto(app_url + "/")
chips = _chips(page)
expect(chips.first).to_be_visible(timeout=30_000)
expect(chips).to_have_count(3, timeout=15_000)
# The CAP (the phase-157 A1 contract, live DOM, byte-for-byte):
# the 120-char opener shows as EXACTLY slice(0, 90).trimEnd() +
# "…" — 91 chars, ending in the ellipsis, not the 120-char
# pre-phase display (which stretched the chip near full row
# width before the CSS clip engaged — the audit's observation).
display = chips.nth(0).text_content()
assert display is not None
assert len(display) <= 91, (
f"a >90-char opener must display ≤91 chars — the cap + the "
f"… — got {len(display)}"
)
assert display.endswith("…"), "a capped chip ends with the ellipsis"
assert display == LONG_Q_CAPPED, (
"the display is the EXACT slice(0, 90).trimEnd() + '…' pin, "
"not just a length"
)
# The row order (phase 80/103 — newest updated_at first, cap 3)
# and the <90-char openers' byte-identity (no cap, no "…").
assert chips.nth(1).text_content() == SHORT_B, "a <90-char opener is byte-identical (no …)"
assert chips.nth(2).text_content() == SHORT_A, "a <90-char opener is byte-identical (no …)"
for follow_up in (SHORT_A_FOLLOW_UP, SHORT_B_FOLLOW_UP, LONG_FOLLOW_UP):
assert follow_up not in [
chips.nth(i).text_content() for i in range(3)
], "a chat's follow-up must never chip"
def test_full_text_reachable(browser: Browser, app_url: str, db_ready: None) -> None:
"""The full text stays one hover away AND accessible in the
CLIPPED case: at the ≤640px viewport the 91-char display
overflows the single-line box (the chip's ``max-width: 100%``
binds, ``text-overflow: ellipsis`` engages — the phase-104 CSS,
the secondary guard), so the phase-104 idiom applies: the
``title`` carries the FULL question (universal — every chip) and
the ``aria-label`` (set ONLY when visually clipped, the
``scrollWidth > clientWidth`` check) carries the FULL question
— never the 91-char display (phase 104, extended by the cap)."""
page = browser.new_page(viewport={"width": 375, "height": 720})
try:
page.set_default_timeout(30_000)
login(page, app_url, next="/")
# ONE multi-turn chat: the 120-char opener + the follow-up
# (phase-103: the follow-up never surfaces).
_save_chat(
page,
app_url,
[
_user(LONG_Q), _brain(),
_user(LONG_FOLLOW_UP), _brain(),
],
)
# A FRESH page load: EXACTLY ONE chip — the opener.
page.goto(app_url + "/")
chips = _chips(page)
expect(chips.first).to_be_visible(timeout=30_000)
expect(chips).to_have_count(1, timeout=15_000)
chip = chips.first
# Precondition: the CLIPPED case at this viewport (the 91-char
# display overflows the single-line box — pin the state the
# aria-label branch keys on, so a layout regression cannot
# silently flip the attribute's contract).
dims = chip.evaluate(
"el => ({ sw: el.scrollWidth, cw: el.clientWidth })"
)
assert dims["sw"] > dims["cw"], (
"at the ≤640px viewport the 91-char display must overflow "
"the single-line box (the clipped case the aria-label "
"idiom governs)"
)
# The hover reveal (phase 104, universal): the FULL question.
assert chip.get_attribute("title") == LONG_Q, (
"the title tooltip must carry the FULL question (phase 104)"
)
# The clipped-case accessible name (phase 104 idiom — the
# scrollWidth > clientWidth check): the FULL question.
assert chip.get_attribute("aria-label") == LONG_Q, (
"a clipped chip's aria-label must carry the FULL "
"question — never the 91-char display"
)
finally:
page.close()
def test_click_submits_full(
page: Page, app_url: str, mock_llm: int, db_ready: None
) -> None:
"""The load-bearing assertion: clicking a CAPPED chip submits the
FULL (120-char) question — the cap is display-only. The user
bubble is the EXACT full question (byte-for-byte, not the 91-char
display), and the turn is the grounded mock's (the opener carries
the "kubernetes" token — non-deflected, marker answer); the turn
clears the input and the send button recovers (never stale)."""
_seed_kb(mock_llm)
page.set_default_timeout(30_000)
login(page, app_url, next="/")
# ONE multi-turn chat: the 120-char opener + the follow-up.
_save_chat(
page,
app_url,
[
_user(LONG_Q), _brain(),
_user(LONG_FOLLOW_UP), _brain(),
],
)
# A FRESH page load: EXACTLY ONE chip — the capped opener.
page.goto(app_url + "/")
chips = _chips(page)
expect(chips.first).to_be_visible(timeout=30_000)
expect(chips).to_have_count(1, timeout=15_000)
chip = chips.first
# Precondition: the chip that gets clicked IS the capped display
# (the 91-char pin — the click test is about the CAPPED chip).
assert chip.text_content() == LONG_Q_CAPPED
# One tap = one question: the click fills AND submits — with the
# FULL text (the phase-80 contract, the load-bearing assertion).
chip.click()
expect(page.locator("#empty-state")).to_be_hidden()
expect(page.locator("#message-input")).to_have_value("") # submitted, not queued
bubble = page.locator(".msg.user .bubble")
expect(bubble).to_have_count(1, timeout=30_000)
# The user bubble is the EXACT FULL question — byte-for-byte, not
# the 91-char capped display (the cap is display-only).
assert bubble.inner_text() == LONG_Q, (
"the capped chip must submit the FULL question — the user "
"bubble is the un-capped 120-char text, not the display"
)
assert len(bubble.inner_text()) == 120
# The grounded mock reply follows (the seeded KB answers this
# topic — the "kubernetes" token grounds, non-deflected).
brain = page.locator(".msg.brain .bubble").first
expect(brain).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
expect(page.locator(".msg.brain.is-deflected")).to_have_count(0)
# Never stale: the turn cleared the input and the send button
# recovered.
expect(page.locator("#send-btn")).to_be_enabled()
expect(page.locator("#send-label")).to_have_text("Send")
def test_short_chip_unchanged(page: Page, app_url: str, db_ready: None) -> None:
"""The regression guard on the live DOM: a <90-char opener renders
byte-identical to pre-phase — the ``textContent`` is the full
question with NO ``…`` appended (the cap is a no-op below 90),
the ``title`` is the full question (phase 104, unchanged), and
(unclipped at the 1280px viewport) no ``aria-label`` (the
``scrollWidth > clientWidth`` check — the attribute is set only
when visually clipped, phase 104)."""
page.set_default_timeout(30_000)
login(page, app_url, next="/")
# ONE multi-turn chat: the 49-char opener + the follow-up.
_save_chat(
page,
app_url,
[
_user(SHORT_Q), _brain(),
_user(SHORT_Q_FOLLOW_UP), _brain(),
],
)
# A FRESH page load: EXACTLY ONE chip — the short opener.
page.goto(app_url + "/")
chips = _chips(page)
expect(chips.first).to_be_visible(timeout=30_000)
expect(chips).to_have_count(1, timeout=15_000)
chip = chips.first
# Byte-identical to pre-phase: the full question, no "…".
assert chip.text_content() == SHORT_Q, (
"a <90-char chip's display is byte-identical to pre-phase — "
"the full question, no ellipsis appended"
)
assert "…" not in (chip.text_content() or "")
# The hover reveal: the full question (phase 104, unchanged).
assert chip.get_attribute("title") == SHORT_Q
# Unclipped at the 1280px viewport (pin the state — the
# pre-phase display fit, so the cap must not change the
# aria-label contract either): no aria-label.
dims = chip.evaluate(
"el => ({ sw: el.scrollWidth, cw: el.clientWidth })"
)
assert dims["sw"] <= dims["cw"], (
"a 49-char chip must NOT be clipped at the 1280px viewport "
"(the pre-phase state — byte-identical)"
)
assert chip.get_attribute("aria-label") is None, (
"an unclipped chip stays attribute-free (phase 104)"
)
# The follow-up never surfaces (phase 103 — the row is the
# opener only).
assert SHORT_Q_FOLLOW_UP not in [chip.text_content()]