All verification is green. Final report: **Phase 157 — final verification pass (all 3 tasks already complete in `complete/`)** - Verified the implemented work matches the design: `CHIP_CAP = 90` constant + display-only cap in `renderChips` (`slice(0,90).trimEnd() + "…"`), `title`/`aria-label`/click payload byte-unchanged; unit pins (89/90/91 letter & space, full-text invariant, short-string guard, Maybe-try single-path guard); phase-104 E2E updated to the cap contract - Full suite `uv run pytest --cov=app --cov-report=term-missing`: **3578 passed**, app/ coverage **99%** (>90% ✓) - `uv run ruff check .` clean; `uv run pyright`: 0 errors - Dedicated E2E `uv run pytest tests/e2e/test_chat_chip_cap.py -v --no-cov`: **4/4 passed** in isolation (DB up) - Cross-phase regression: phase-104 E2E 6/6, phase 05/80/103 chip suite 8/8 (seed chips <90 chars — cap is a no-op, assertions hold) - Criteria: ≤91-char capped display ✓ (byte-for-byte pin) · click submits FULL 120-char question (user bubble) ✓ · `title` + clipped-case `aria-label` carry full text ✓ (375px viewport) · <90-char byte-identical ✓ (unit + live DOM) · commit: deferred to the harness (per harness protocol, overrides AGENTS.md rule 8 for this pass) - Documentation: README.md reviewed in full — every `.env.example` var documented, all referenced scripts/pages/commands verified against the tree, the phase's chip-cap note is in the Chat section; AGENTS.md verified (commands, test files, layout-rule citations — phase 157 adds no new layout principle) → **verified consistent, no changes needed** - No live-infrastructure changes were required or made - Next pending phase: **158_thinking_timeline**
419 lines
18 KiB
Python
419 lines
18 KiB
Python
"""Phase 157 E2E (Playwright): the hero chip's DISPLAY cap (90 chars +
|
|
``…``) with the phase-104 full-text contract extended (owner-locked
|
|
A1, 2026-10-03 audit: "long opening questions get hard-truncated
|
|
mid-word… the chips stretch near full container width before
|
|
clipping").
|
|
|
|
The contract this suite pins end to end (the user-facing half of the
|
|
phase — the unit pins in ``tests/unit/test_chip_sizing_question_cap.py``
|
|
are the source-level half):
|
|
|
|
* **capped display** — a >90-char saved-chat opener renders a
|
|
COMPACT chip: the displayed text is
|
|
``slice(0, 90).trimEnd() + "…"`` — at most 91 chars, ending in
|
|
``…`` (the phase-104 single-line CSS clip demoted to the secondary
|
|
guard); the display is the EXACT pinned string, asserted
|
|
byte-for-byte on the live DOM, not just by length.
|
|
* **full text reachable** — the chip's ``title`` (the hover reveal)
|
|
and, in the CLIPPED case (a 91-char display overflows the
|
|
single-line box — the ≤640px viewport, where the chip's
|
|
``max-width: 100%`` binds and ``text-overflow: ellipsis``
|
|
engages), the ``aria-label`` accessible name both carry the FULL
|
|
question (phase 104, extended by the cap).
|
|
* **the click submits the FULL question** (the load-bearing
|
|
assertion) — clicking a capped chip sends the UN-capped 120-char
|
|
question: the user bubble is the full question, not the 91-char
|
|
display (the cap is display-only — the phase-80 contract).
|
|
* **short-chip regression guard** — a <90-char opener renders
|
|
byte-identical to pre-phase on the live DOM: the
|
|
``textContent`` is the full question with NO ``…`` appended, the
|
|
``title`` is the full question, and (unclipped at the 1280px
|
|
viewport) no ``aria-label`` — the cap is a no-op below 90.
|
|
|
|
Seeding idiom (mirrored from the phase-80/103 suite
|
|
``tests/e2e/test_suggestion_chips.py``, not invented): the chips are
|
|
the session openers read from ``saved_chats`` — each test saves its
|
|
chat(s) via the signed-in admin's ``POST /api/chats`` and loads a
|
|
FRESH page, whose boot fetch renders the chips. The mock LLM is the
|
|
deterministic answer source (no ``E2E_REAL_LLM`` dependency); the
|
|
seeded questions are FIXED strings (the pins, not generated
|
|
lengths). The click test grounds on the fixture KB (the 120-char
|
|
opener carries the "kubernetes" token — the mock-calibrated cosine
|
|
clears the 0.30 gate, non-deflected). ``saved_chats`` is global state
|
|
on the shared e2e Postgres AND the state this contract reads — the
|
|
autouse fixture truncates it before and after EVERY test (including
|
|
the click test's auto-saved row), so each test starts from — and
|
|
leaves — an empty deployment.
|
|
|
|
Run in isolation (DB must be up: ``podman compose up -d db``):
|
|
|
|
uv run pytest tests/e2e/test_chat_chip_cap.py -v --no-cov
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import json
|
|
import time
|
|
from collections.abc import Iterator
|
|
from pathlib import Path
|
|
from threading import Thread
|
|
from typing import Any
|
|
|
|
import pytest
|
|
from playwright.sync_api import Browser, Page, expect
|
|
from sqlalchemy import text
|
|
|
|
from app.config import Settings
|
|
from app.db import SessionLocal
|
|
from app.rag.importer import ImportSummary, import_sources
|
|
from app.rag.llm import LLMClient
|
|
from e2e.auth_helpers import login
|
|
|
|
REPO = Path(__file__).resolve().parents[2]
|
|
FIXTURES = REPO / "tests" / "fixtures" / "docs"
|
|
MOCK_ANSWER_MARKER = "Deterministic mock answer for E2E"
|
|
|
|
#: The FIXED 120-char opener (90+ — the cap engages deterministically;
|
|
#: the suite's pin, not a generated length). It carries the
|
|
#: "kubernetes" token (``tests/fixtures/docs/homelab/kubernetes.md``
|
|
#: + the deployments fixture), so the click test's turn grounds on
|
|
#: the mock-calibrated gate (0.30) — non-deflected, marker answer.
|
|
LONG_Q = (
|
|
"What are the correct arguments for running a kubernetes node pool "
|
|
"with qwen 3.8 27b on llama.cpp in the homelab cluster?"
|
|
)
|
|
assert len(LONG_Q) == 120, "the pin is a FIXED 120-char string"
|
|
#: The phase-157 display of LONG_Q (the exact-string pin — char 90 is
|
|
#: the "a" of "llama", so the trimEnd is a no-op here; the
|
|
#: space-at-char-90 trimEnd case is the unit pins' job). 91 chars.
|
|
LONG_Q_CAPPED = LONG_Q[:90].rstrip() + "…"
|
|
assert len(LONG_Q_CAPPED) == 91 and LONG_Q_CAPPED.endswith("…")
|
|
#: The follow-up (phase-103 semantics: it must NEVER surface as a
|
|
#: chip — a follow-up is meaningless as a conversation starter).
|
|
LONG_FOLLOW_UP = "What about the node pool's GPU drivers?"
|
|
#: Two <90-char openers for the three-chip state (the row fills to
|
|
#: the cap of 3 with the long opener NEWEST — the first chip).
|
|
SHORT_A = "Which provider is the primary DNS for reeseapps.com?"
|
|
SHORT_A_FOLLOW_UP = "What about the secondary DNS for the LAN?"
|
|
SHORT_B = "What cron schedule runs the restic prune?"
|
|
SHORT_B_FOLLOW_UP = "And where do the restic lock files live?"
|
|
#: The STANDALONE <90-char regression-guard question.
|
|
SHORT_Q = "How do I rotate the WireGuard keys on the VPN node?"
|
|
SHORT_Q_FOLLOW_UP = "What about the peers' allowed-ips?"
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def clean_chats(db_ready: None) -> Iterator[None]:
|
|
"""``saved_chats`` is the state the onboarding chips read (the
|
|
phase-103 openers) AND the click test's turn auto-saves a row —
|
|
truncate it before and after EVERY test so each starts from (and
|
|
leaves) an empty deployment (the seed-chip state), exactly the
|
|
phase-80/103/104 autouse pattern."""
|
|
with SessionLocal() as db:
|
|
db.execute(text("TRUNCATE saved_chats"))
|
|
db.commit()
|
|
yield
|
|
with SessionLocal() as db:
|
|
db.execute(text("TRUNCATE saved_chats"))
|
|
db.commit()
|
|
|
|
|
|
async def _import_fixtures(mock_port: int) -> ImportSummary:
|
|
kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
|
|
settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
|
|
return await import_sources([FIXTURES], LLMClient(settings))
|
|
|
|
|
|
def _run_in_thread(coro: Any) -> Any:
|
|
"""Run a coroutine on a worker thread (Playwright owns the test loop)."""
|
|
box: dict[str, Any] = {}
|
|
|
|
def runner() -> None:
|
|
try:
|
|
box["value"] = asyncio.run(coro)
|
|
except BaseException as e: # noqa: BLE001 — re-raised on the test thread
|
|
box["error"] = e
|
|
|
|
t = Thread(target=runner)
|
|
t.start()
|
|
t.join()
|
|
if "error" in box:
|
|
raise box["error"]
|
|
return box["value"]
|
|
|
|
|
|
def _seed_kb(mock_port: int) -> ImportSummary:
|
|
"""Deterministic KB: truncate the KB tables, import the fixture
|
|
docs (needed by the click test's grounded answer).
|
|
``saved_chats`` is the autouse fixture's job."""
|
|
with SessionLocal() as db:
|
|
db.execute(text("TRUNCATE chunks, documents, query_log"))
|
|
db.commit()
|
|
summary = _run_in_thread(_import_fixtures(mock_port))
|
|
assert summary is not None and summary.added == 13 # A9 formats (phase 47 added quadlet+j2)
|
|
return summary
|
|
|
|
|
|
def _user(q: str) -> dict[str, Any]:
|
|
return {"who": "user", "text": q}
|
|
|
|
|
|
def _brain(text: str = "Grounded mock brain reply.") -> dict[str, Any]:
|
|
return {"who": "brain", "text": text}
|
|
|
|
|
|
def _save_chat(page: Page, app_url: str, messages: list[dict[str, Any]]) -> dict[str, Any]:
|
|
"""Save one conversation as the signed-in admin (``POST /api/chats``)
|
|
and return the 201 body. ``updated_at`` is the API's stamp (server
|
|
``now()`` at INSERT) — the save ORDER is what makes the chip-walk
|
|
order deterministic in the multi-chat test."""
|
|
r = page.request.post(
|
|
f"{app_url}/api/chats",
|
|
data=json.dumps({"messages": messages}),
|
|
headers={"Content-Type": "application/json"},
|
|
timeout=10_000,
|
|
)
|
|
assert r.status == 201, r.text
|
|
return r.json()
|
|
|
|
|
|
def _chips(page: Page) -> Any:
|
|
return page.locator("#suggestions .suggestion-chip")
|
|
|
|
|
|
def test_capped_display(page: Page, app_url: str, db_ready: None) -> None:
|
|
"""The >90-char opener renders a COMPACT chip: the displayed text
|
|
is the EXACT ``slice(0, 90).trimEnd() + "…"`` pin (91 chars,
|
|
ending in ``…``), and the row is the phase-80/103 three-chip
|
|
contract — the three seeded openers, newest ``updated_at``
|
|
first; the <90-char openers render byte-identical (no ``…``), and
|
|
none of the chats' follow-ups surfaces."""
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/")
|
|
|
|
# Save oldest→newest: the API stamps ``updated_at`` (server
|
|
# now()), so save order IS walk order (newest first). The short
|
|
# pauses keep the three stamps strictly apart.
|
|
stamps: list[str] = []
|
|
for opener, follow_up in (
|
|
(SHORT_A, SHORT_A_FOLLOW_UP),
|
|
(SHORT_B, SHORT_B_FOLLOW_UP),
|
|
(LONG_Q, LONG_FOLLOW_UP),
|
|
):
|
|
body = _save_chat(
|
|
page,
|
|
app_url,
|
|
[
|
|
_user(opener), _brain(),
|
|
_user(follow_up), _brain(),
|
|
],
|
|
)
|
|
stamps.append(body["updated_at"])
|
|
time.sleep(0.05)
|
|
assert stamps == sorted(stamps) and len(set(stamps)) == 3, (
|
|
"the three API-stamped updated_at values must be strictly increasing"
|
|
)
|
|
|
|
# A FRESH page load (a new boot fetch): EXACTLY the three
|
|
# openers, newest first — the long one capped, the shorts not.
|
|
page.goto(app_url + "/")
|
|
chips = _chips(page)
|
|
expect(chips.first).to_be_visible(timeout=30_000)
|
|
expect(chips).to_have_count(3, timeout=15_000)
|
|
|
|
# The CAP (the phase-157 A1 contract, live DOM, byte-for-byte):
|
|
# the 120-char opener shows as EXACTLY slice(0, 90).trimEnd() +
|
|
# "…" — 91 chars, ending in the ellipsis, not the 120-char
|
|
# pre-phase display (which stretched the chip near full row
|
|
# width before the CSS clip engaged — the audit's observation).
|
|
display = chips.nth(0).text_content()
|
|
assert display is not None
|
|
assert len(display) <= 91, (
|
|
f"a >90-char opener must display ≤91 chars — the cap + the "
|
|
f"… — got {len(display)}"
|
|
)
|
|
assert display.endswith("…"), "a capped chip ends with the ellipsis"
|
|
assert display == LONG_Q_CAPPED, (
|
|
"the display is the EXACT slice(0, 90).trimEnd() + '…' pin, "
|
|
"not just a length"
|
|
)
|
|
# The row order (phase 80/103 — newest updated_at first, cap 3)
|
|
# and the <90-char openers' byte-identity (no cap, no "…").
|
|
assert chips.nth(1).text_content() == SHORT_B, "a <90-char opener is byte-identical (no …)"
|
|
assert chips.nth(2).text_content() == SHORT_A, "a <90-char opener is byte-identical (no …)"
|
|
for follow_up in (SHORT_A_FOLLOW_UP, SHORT_B_FOLLOW_UP, LONG_FOLLOW_UP):
|
|
assert follow_up not in [
|
|
chips.nth(i).text_content() for i in range(3)
|
|
], "a chat's follow-up must never chip"
|
|
|
|
|
|
def test_full_text_reachable(browser: Browser, app_url: str, db_ready: None) -> None:
|
|
"""The full text stays one hover away AND accessible in the
|
|
CLIPPED case: at the ≤640px viewport the 91-char display
|
|
overflows the single-line box (the chip's ``max-width: 100%``
|
|
binds, ``text-overflow: ellipsis`` engages — the phase-104 CSS,
|
|
the secondary guard), so the phase-104 idiom applies: the
|
|
``title`` carries the FULL question (universal — every chip) and
|
|
the ``aria-label`` (set ONLY when visually clipped, the
|
|
``scrollWidth > clientWidth`` check) carries the FULL question
|
|
— never the 91-char display (phase 104, extended by the cap)."""
|
|
page = browser.new_page(viewport={"width": 375, "height": 720})
|
|
try:
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/")
|
|
|
|
# ONE multi-turn chat: the 120-char opener + the follow-up
|
|
# (phase-103: the follow-up never surfaces).
|
|
_save_chat(
|
|
page,
|
|
app_url,
|
|
[
|
|
_user(LONG_Q), _brain(),
|
|
_user(LONG_FOLLOW_UP), _brain(),
|
|
],
|
|
)
|
|
|
|
# A FRESH page load: EXACTLY ONE chip — the opener.
|
|
page.goto(app_url + "/")
|
|
chips = _chips(page)
|
|
expect(chips.first).to_be_visible(timeout=30_000)
|
|
expect(chips).to_have_count(1, timeout=15_000)
|
|
chip = chips.first
|
|
# Precondition: the CLIPPED case at this viewport (the 91-char
|
|
# display overflows the single-line box — pin the state the
|
|
# aria-label branch keys on, so a layout regression cannot
|
|
# silently flip the attribute's contract).
|
|
dims = chip.evaluate(
|
|
"el => ({ sw: el.scrollWidth, cw: el.clientWidth })"
|
|
)
|
|
assert dims["sw"] > dims["cw"], (
|
|
"at the ≤640px viewport the 91-char display must overflow "
|
|
"the single-line box (the clipped case the aria-label "
|
|
"idiom governs)"
|
|
)
|
|
# The hover reveal (phase 104, universal): the FULL question.
|
|
assert chip.get_attribute("title") == LONG_Q, (
|
|
"the title tooltip must carry the FULL question (phase 104)"
|
|
)
|
|
# The clipped-case accessible name (phase 104 idiom — the
|
|
# scrollWidth > clientWidth check): the FULL question.
|
|
assert chip.get_attribute("aria-label") == LONG_Q, (
|
|
"a clipped chip's aria-label must carry the FULL "
|
|
"question — never the 91-char display"
|
|
)
|
|
finally:
|
|
page.close()
|
|
|
|
|
|
def test_click_submits_full(
|
|
page: Page, app_url: str, mock_llm: int, db_ready: None
|
|
) -> None:
|
|
"""The load-bearing assertion: clicking a CAPPED chip submits the
|
|
FULL (120-char) question — the cap is display-only. The user
|
|
bubble is the EXACT full question (byte-for-byte, not the 91-char
|
|
display), and the turn is the grounded mock's (the opener carries
|
|
the "kubernetes" token — non-deflected, marker answer); the turn
|
|
clears the input and the send button recovers (never stale)."""
|
|
_seed_kb(mock_llm)
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/")
|
|
|
|
# ONE multi-turn chat: the 120-char opener + the follow-up.
|
|
_save_chat(
|
|
page,
|
|
app_url,
|
|
[
|
|
_user(LONG_Q), _brain(),
|
|
_user(LONG_FOLLOW_UP), _brain(),
|
|
],
|
|
)
|
|
|
|
# A FRESH page load: EXACTLY ONE chip — the capped opener.
|
|
page.goto(app_url + "/")
|
|
chips = _chips(page)
|
|
expect(chips.first).to_be_visible(timeout=30_000)
|
|
expect(chips).to_have_count(1, timeout=15_000)
|
|
chip = chips.first
|
|
# Precondition: the chip that gets clicked IS the capped display
|
|
# (the 91-char pin — the click test is about the CAPPED chip).
|
|
assert chip.text_content() == LONG_Q_CAPPED
|
|
|
|
# One tap = one question: the click fills AND submits — with the
|
|
# FULL text (the phase-80 contract, the load-bearing assertion).
|
|
chip.click()
|
|
expect(page.locator("#empty-state")).to_be_hidden()
|
|
expect(page.locator("#message-input")).to_have_value("") # submitted, not queued
|
|
bubble = page.locator(".msg.user .bubble")
|
|
expect(bubble).to_have_count(1, timeout=30_000)
|
|
# The user bubble is the EXACT FULL question — byte-for-byte, not
|
|
# the 91-char capped display (the cap is display-only).
|
|
assert bubble.inner_text() == LONG_Q, (
|
|
"the capped chip must submit the FULL question — the user "
|
|
"bubble is the un-capped 120-char text, not the display"
|
|
)
|
|
assert len(bubble.inner_text()) == 120
|
|
|
|
# The grounded mock reply follows (the seeded KB answers this
|
|
# topic — the "kubernetes" token grounds, non-deflected).
|
|
brain = page.locator(".msg.brain .bubble").first
|
|
expect(brain).to_contain_text(MOCK_ANSWER_MARKER, timeout=30_000)
|
|
expect(page.locator(".msg.brain.is-deflected")).to_have_count(0)
|
|
|
|
# Never stale: the turn cleared the input and the send button
|
|
# recovered.
|
|
expect(page.locator("#send-btn")).to_be_enabled()
|
|
expect(page.locator("#send-label")).to_have_text("Send")
|
|
|
|
|
|
def test_short_chip_unchanged(page: Page, app_url: str, db_ready: None) -> None:
|
|
"""The regression guard on the live DOM: a <90-char opener renders
|
|
byte-identical to pre-phase — the ``textContent`` is the full
|
|
question with NO ``…`` appended (the cap is a no-op below 90),
|
|
the ``title`` is the full question (phase 104, unchanged), and
|
|
(unclipped at the 1280px viewport) no ``aria-label`` (the
|
|
``scrollWidth > clientWidth`` check — the attribute is set only
|
|
when visually clipped, phase 104)."""
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/")
|
|
|
|
# ONE multi-turn chat: the 49-char opener + the follow-up.
|
|
_save_chat(
|
|
page,
|
|
app_url,
|
|
[
|
|
_user(SHORT_Q), _brain(),
|
|
_user(SHORT_Q_FOLLOW_UP), _brain(),
|
|
],
|
|
)
|
|
|
|
# A FRESH page load: EXACTLY ONE chip — the short opener.
|
|
page.goto(app_url + "/")
|
|
chips = _chips(page)
|
|
expect(chips.first).to_be_visible(timeout=30_000)
|
|
expect(chips).to_have_count(1, timeout=15_000)
|
|
chip = chips.first
|
|
# Byte-identical to pre-phase: the full question, no "…".
|
|
assert chip.text_content() == SHORT_Q, (
|
|
"a <90-char chip's display is byte-identical to pre-phase — "
|
|
"the full question, no ellipsis appended"
|
|
)
|
|
assert "…" not in (chip.text_content() or "")
|
|
# The hover reveal: the full question (phase 104, unchanged).
|
|
assert chip.get_attribute("title") == SHORT_Q
|
|
# Unclipped at the 1280px viewport (pin the state — the
|
|
# pre-phase display fit, so the cap must not change the
|
|
# aria-label contract either): no aria-label.
|
|
dims = chip.evaluate(
|
|
"el => ({ sw: el.scrollWidth, cw: el.clientWidth })"
|
|
)
|
|
assert dims["sw"] <= dims["cw"], (
|
|
"a 49-char chip must NOT be clipped at the 1280px viewport "
|
|
"(the pre-phase state — byte-identical)"
|
|
)
|
|
assert chip.get_attribute("aria-label") is None, (
|
|
"an unclipped chip stays attribute-free (phase 104)"
|
|
)
|
|
# The follow-up never surfaces (phase 103 — the row is the
|
|
# opener only).
|
|
assert SHORT_Q_FOLLOW_UP not in [chip.text_content()]
|