All verification complete. Final report: **Phase 119 final verification pass — all criteria verified, one stale pin fixed.** - Verified implementation of all 6 tasks: D1 component name-hit rule (`name_hit` flag, titles never matched, retired length tie-break), D2 `BOR_NAME_HIT_BONUS` (0.005 default, 0 = byte-identical kill switch, negative fails startup, selection-layer only, `eval_retrieval` `suggested:` line), D3 suggested-folder lines (after `SUGGEST_INTRO`, before first block), D4 cite-discipline `SUGGEST_INTRO` sentence (PERSONA/LOW/`TOOLS_SECTION` byte-pins intact), D5 `done.sources` = read docs only (frontend no-op on empty confirmed), D6 mock `repeat your folder map` echo + new suite + telemetry. - Battery (replica restored per skill, fingerprint docs=1000/chunks=8866 verified, `eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` re-run): **GATE PASS** — gitea README #4 in suggested top-5, forgejo 5/5 (README #1), gateway README in top-5 (#4), qwen3.8-27b quadlets top-5, Mongolia HIGH/fts=5 unchanged. - New E2E in isolation: `4 passed` ×2 (deterministic). All 27 modified E2E suites in isolation: 26 green; **1 stale pin fixed** — `test_source_chip_quality.py` durable-record order pin pre-dated the D1 re-rank (`aliases` stem sub-component name-hits `ssh_aliases.txt`, deterministically lifting `backups.md` over `kubernetes.md`; probe-verified 0.016277 vs 0.016036, 4/4 stable) — re-pinned with the phase-119 rationale; suite green ×2. - Gates: `uv run pytest --cov=app --cov-report=term-missing` → **2547 passed, app coverage 99%** (>90%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Completion criteria: 1 ✅ (battery, recorded), 2 ✅ (folder lines; block/LOW byte-identical pins green), 3 ✅ (read-only chips, zero-read chips nothing, related row + durable record untouched — unit+E2E agree), 4 ✅ (all green), 5 → commit/phase-move left to the harness per pass rules (nothing committed). - Deviations: battery output + real-model telemetry recorded in `.agents/reports/119_name_signal_read_chips/task06_battery_and_e2e.md` and `TOOL_CALLING_TESTING.md` §11 (task files in `complete/` are immutable to this pass); gateway canonical doc at #4 vs overview's #3 was already documented at task 06 (containment gate met). - Next pending phase: **none** — `todo/` holds only phase 119.
240 lines
10 KiB
Python
240 lines
10 KiB
Python
"""Phase 13 E2E (Playwright): the viewer's back button returns to the
|
|
page the document was opened from.
|
|
|
|
Story: ``.agents/user_stories/document-back-navigation.md``
|
|
Run in isolation (DB must be up: ``podman compose up -d db``):
|
|
|
|
uv run pytest tests/e2e/test_document_back_navigation.py -v --no-cov
|
|
|
|
The viewer can be reached directly (no browser history to go back to),
|
|
so the return target is carried in the viewer URL: chat chips append
|
|
``&back=%2F`` (resolves to "Chat"), Sources links omit the param (the
|
|
viewer's default ``/sources.html`` applies → "Sources"). The viewer only
|
|
honors same-origin relative ``back`` values; everything else falls back
|
|
to ``/sources.html``.
|
|
|
|
Phase 26 adaptation: the chip/row-link LEFT click now opens the
|
|
document in the same-page modal — no new tab is spawned. The encoded
|
|
viewer URL survives as each link's ``href`` (the no-JS / context-menu
|
|
"open in new tab" escape hatch), so the back contract is asserted on
|
|
that exact href and verified by navigating to it directly.
|
|
|
|
Phase 119 re-target (LOCKED A1): chips cite READ docs only — a plain
|
|
question chips nothing, so test 1 drives the mock's scripted
|
|
summary-read flow (``SUMMARY_SEED_READ_TRIGGER``): the turn ``read``s
|
|
the kubernetes fixture, and its READ-doc chip is what carries the
|
|
``&back=%2F`` href the story asserts.
|
|
|
|
Test → story mapping (Playwright Mapping Rule):
|
|
1. ``test_back_from_chat_returns_to_chat`` — scripted-read question →
|
|
the read doc's source chip href (carries ``&back=%2F``) → viewer
|
|
back link href ``/`` labeled "Chat" → click → the chat page.
|
|
2. ``test_back_from_sources_returns_to_sources`` — Sources table link
|
|
href (no ``back`` param) → back link href ``/sources.html`` labeled
|
|
"Sources" → click → the Sources page.
|
|
3. ``test_malicious_back_param_is_rejected`` — absolute,
|
|
protocol-relative, and ``javascript:`` ``back`` values all fall back
|
|
to ``/sources.html`` (labeled "Sources", navigable).
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import re
|
|
from pathlib import Path
|
|
from threading import Thread
|
|
from typing import Any
|
|
|
|
from playwright.sync_api import Page, expect
|
|
from sqlalchemy import text
|
|
|
|
from app.config import Settings
|
|
from app.db import SessionLocal
|
|
from app.rag.importer import ImportSummary, import_sources
|
|
from app.rag.llm import LLMClient
|
|
from e2e.auth_helpers import login
|
|
|
|
REPO = Path(__file__).resolve().parents[2]
|
|
FIXTURES = REPO / "tests" / "fixtures" / "docs"
|
|
# Phase 119 (LOCKED A1): the chip is the turn's READ doc — the scripted
|
|
# summary-read flow (mock_llm.SUMMARY_SEED_READ_TRIGGER) reads the
|
|
# kubernetes fixture so the story's chip (with its back=%2F href)
|
|
# exists (a zero-read turn would chip nothing — the retired phase-118
|
|
# A4 suggested-chip is gone).
|
|
QUESTION = (
|
|
"Read the suggested document: read docs/homelab/kubernetes.md — "
|
|
"how is my Kubernetes cluster set up?"
|
|
)
|
|
# Seeded fixture doc (source=docs) shared by every test in this file.
|
|
DOC_SOURCE = "docs"
|
|
DOC_PATH = "homelab%2Fkubernetes.md"
|
|
DOC_TITLE = "Kubernetes Homelab Cluster"
|
|
|
|
|
|
async def _import_fixtures(mock_port: int) -> ImportSummary:
|
|
kwargs: dict[str, Any] = {"_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1"}
|
|
settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
|
|
return await import_sources([FIXTURES], LLMClient(settings))
|
|
|
|
|
|
def _run_in_thread(coro: Any) -> Any:
|
|
"""Run a coroutine on a worker thread.
|
|
|
|
Playwright's sync API keeps an asyncio loop running on the test thread,
|
|
so ``asyncio.run`` cannot be called directly from a test body.
|
|
"""
|
|
box: dict[str, Any] = {}
|
|
|
|
def runner() -> None:
|
|
try:
|
|
box["value"] = asyncio.run(coro)
|
|
except BaseException as e: # noqa: BLE001 — re-raised on the test thread
|
|
box["error"] = e
|
|
|
|
t = Thread(target=runner)
|
|
t.start()
|
|
t.join()
|
|
if "error" in box:
|
|
raise box["error"]
|
|
return box["value"]
|
|
|
|
|
|
def _reset_db(mock_port: int, seed: bool) -> ImportSummary | None:
|
|
"""Truncate the KB (and query log), then optionally re-import fixtures."""
|
|
with SessionLocal() as db:
|
|
db.execute(text("TRUNCATE chunks, documents, query_log"))
|
|
db.commit()
|
|
if not seed:
|
|
return None
|
|
return _run_in_thread(_import_fixtures(mock_port))
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1. Chat source chip → viewer with back=/ → back returns to the chat
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_back_from_chat_returns_to_chat(
|
|
page: Page, app_url: str, mock_llm: int, db_ready: None
|
|
) -> None:
|
|
_reset_db(mock_llm, seed=True)
|
|
page.set_default_timeout(30_000)
|
|
login(page, app_url, next="/") # phase 79: chat AND the viewer content are gated
|
|
|
|
page.fill("#message-input", QUESTION)
|
|
page.click("#send-btn")
|
|
|
|
chip = page.locator(".msg.brain .source-chip", has_text="kubernetes.md")
|
|
expect(chip).to_have_count(1, timeout=30_000)
|
|
# Phase 119 A1: the read doc is the turn's ONLY chip.
|
|
expect(page.locator(".msg.brain .source-chip")).to_have_count(1)
|
|
# Chat chips carry back=/ (encoded %2F) so the viewer knows where
|
|
# home is. Phase 26: the left click opens the same-page modal (no
|
|
# target=_blank); this href is what the no-JS / context-menu "open
|
|
# in a new tab" path reaches, so the back contract rides on it.
|
|
expect(chip.first).to_have_attribute(
|
|
"href", f"/document.html?source={DOC_SOURCE}&path={DOC_PATH}&back=%2F"
|
|
)
|
|
expect(chip.first).not_to_have_attribute("target") # phase 26: modal, not a new tab
|
|
|
|
# The exact href asserted above (the no-JS / new-tab escape hatch).
|
|
page.goto(f"{app_url}/document.html?source={DOC_SOURCE}&path={DOC_PATH}&back=%2F")
|
|
expect(page).to_have_url(
|
|
re.compile(
|
|
re.escape(f"{app_url}/document.html?source={DOC_SOURCE}&path={DOC_PATH}&back=%2F")
|
|
)
|
|
)
|
|
# The cited document actually rendered (this is the viewer, not an error).
|
|
expect(page.locator("#doc-title")).to_have_text(DOC_TITLE)
|
|
# Back link resolved to the chat page, labeled "Chat".
|
|
back = page.locator("#doc-back")
|
|
expect(back).to_have_attribute("href", "/")
|
|
expect(back).to_have_text("Chat")
|
|
|
|
# Click: deterministic anchor navigation back to the chat page.
|
|
back.click()
|
|
expect(page).to_have_url(f"{app_url}/")
|
|
expect(page.locator("#composer")).to_be_visible()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2. Sources table link → viewer without back param → back returns to Sources
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_back_from_sources_returns_to_sources(
|
|
page: Page, app_url: str, mock_llm: int, db_ready: None
|
|
) -> None:
|
|
_reset_db(mock_llm, seed=True)
|
|
login(page, app_url) # phase 16: the Sources table is admin-only
|
|
# Phase 97: the catalog is the drill-down tree — the kubernetes.md
|
|
# row lives at the homelab level (the drill is the only change).
|
|
for name in ("docs", "homelab"):
|
|
page.click(f'#folders-tbody a.folder-link:text-is("{name}")')
|
|
row = page.locator("#docs-tbody tr", has_text="kubernetes.md")
|
|
expect(row).to_have_count(1)
|
|
link = row.locator("td:nth-child(2) a.doc-link")
|
|
expect(link).to_have_count(1)
|
|
# Sources links carry NO back param — the viewer's default target
|
|
# (/sources.html) applies. Phase 26: left click opens the modal;
|
|
# the href (no back param) is the no-JS / new-tab escape hatch.
|
|
expect(link).to_have_attribute(
|
|
"href", f"/document.html?source={DOC_SOURCE}&path={DOC_PATH}"
|
|
)
|
|
expect(link).not_to_have_attribute("target") # phase 26: modal, not a new tab
|
|
|
|
# The exact href asserted above — no back param in the URL.
|
|
page.goto(f"{app_url}/document.html?source={DOC_SOURCE}&path={DOC_PATH}")
|
|
assert "back=" not in page.url, f"unexpected back param: {page.url}"
|
|
expect(page.locator("#doc-title")).to_have_text(DOC_TITLE)
|
|
# Back link kept the default target, labeled "Sources".
|
|
back = page.locator("#doc-back")
|
|
expect(back).to_have_attribute("href", "/sources.html")
|
|
expect(back).to_have_text("Sources")
|
|
|
|
back.click()
|
|
expect(page).to_have_url(f"{app_url}/sources.html")
|
|
# Phase 97: the re-mount lands on the tree's top level (the sources
|
|
# list — the file table is per-level, hidden at the top); the
|
|
# source row is the catalog-rendered signal.
|
|
expect(page.locator("#folders-tbody tr").first).to_be_visible(timeout=15_000)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 3. Hostile back values (absolute, protocol-relative, pseudo-protocol)
|
|
# are all rejected in favor of the same-origin default
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_malicious_back_param_is_rejected(
|
|
page: Page, app_url: str, mock_llm: int, db_ready: None
|
|
) -> None:
|
|
_reset_db(mock_llm, seed=True)
|
|
login(page, app_url, next="/") # phase 79: the viewer content is gated
|
|
errors: list[str] = []
|
|
dialogs: list[str] = []
|
|
page.on("pageerror", lambda e: errors.append(str(e)))
|
|
|
|
def _catch_dialog(d) -> None: # a fired dialog == executed script
|
|
dialogs.append(d.message)
|
|
d.dismiss()
|
|
|
|
page.on("dialog", _catch_dialog)
|
|
|
|
viewer_base = f"{app_url}/document.html?source={DOC_SOURCE}&path={DOC_PATH}"
|
|
# Anything that is not a same-origin relative URL must be rejected:
|
|
# an absolute https URL, a protocol-relative URL, and a javascript:
|
|
# pseudo-protocol.
|
|
for evil in ("https%3A%2F%2Fevil.com", "%2F%2Fevil.com", "javascript%3Aalert(1)"):
|
|
page.goto(f"{viewer_base}&back={evil}")
|
|
expect(page.locator("#doc-title")).to_have_text(DOC_TITLE)
|
|
back = page.locator("#doc-back")
|
|
expect(back).to_have_attribute("href", "/sources.html")
|
|
expect(back).to_have_text("Sources")
|
|
|
|
# And the fallback is really navigable: clicking lands on Sources.
|
|
page.click("#doc-back")
|
|
expect(page).to_have_url(f"{app_url}/sources.html")
|
|
|
|
assert dialogs == [], f"dialog fired — a back param escaped validation: {dialogs}"
|
|
assert errors == [], f"console crashes: {errors}"
|