"""Phase 145 task 04 E2E (Playwright, mock-only): the git-source-links' dedicated story suite — the owner's scenario, end to end. The browser-visible proof (TODO.md L7: "there always needs to be a link to the original git source if possible … point to the skills repo in the answer so I can go download the skills"): a question about a git-sourced document produces (1) an answer that contains the repository link (the model-side teaching, through the real read result that carries the ``origin:`` line) and (2) a document modal whose meta row carries the "Original" external anchor. A local-sourced document produces neither (the byte-identity guard). Run in isolation (DB must be up: ``podman compose up -d db``): uv run pytest tests/e2e/test_git_source_links.py -v --no-cov MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the real ``turbo`` does whatever it does with the tools, while this story's gate is the deterministic SCRIPTED flow in ``tests/e2e/mock_llm.py`` (the mock is the only E2E lens on the LLM's context): * ``ORIGIN_LINK_TRIGGER`` — a user message containing ``original source link`` **and** the system prompt carrying the HIGH prompt's ```` section. The question carries its own scripted call after the colon — ``original source link: read source/path`` (the question-carried-call convention, ``_READ_CAP_CALL_RE``). Request 1 is the scripted ``read``; request 2 (the read result present) is the deterministic FINAL answer — the repository URL VERBATIM, parsed from the read result's ``origin:`` line BY THE MOCK (it cannot fabricate it — the real server's resolver put the line in the result, so the URL in the answer is the server-side half's assertion) — followed by the house scripted-turn echo (the read result VERBATIM — the rendered output is the lens on the ``origin:``/``branch:`` lines' presence; the local-source control's read result carries none, and the mock answers with its deterministic no-origin sentence instead). KB fixture — TWO local directories (the house per-module import — the ``test_document_viewer.py`` importer-in-a-thread pattern), each holding ONE readable markdown document: * ``skills/phased-execution.md`` — the owner's scenario document (the phased execution process for pi.dev). The source's name is ``skills`` — it MATCHES the seeded git row's URL basename (``https://example.com/reese/skills`` → ``skills``: the path component, trailing ``/`` stripped, no ``.git`` suffix to strip, last segment — the resolver's name mapping). The fixture document is locally imported, but the mapping is name-based — which is the point of the seam; * ``control/notes.md`` — a SECOND plain local source with NO git row: the byte-identity control (its read result and its modal meta row must be exactly the pre-phase shapes). The git row is seeded in the suite's OWN DB session (a direct ``SessionLocal`` write, per test — ``branch=None`` for tests 1–2, ``branch="main"`` for the pinned-branch test, and ABSENT for the control test) — never through the API, and never during the import (a ``kind='git'`` row present at import time would be cloned — ``example.com`` is not a git host; the resolver reads the table LIVE at read/content time, so seeding after the import is the whole trick). The app under test is the conftest session app (mock LLM + the mock-calibrated gate: threshold 0.30 + lexical floor 0.15): both questions share the subject words with their own fixture document (the mock's token-overlap distribution), so every scripted turn stays GROUNDED (the ```` section is present) and the read document is the turn's cited chip (phase 119 A1 — the done frame's ``sources`` is the READ docs). Test → observable mapping (Playwright Mapping Rule; the story is the owner's TODO item — one Playwright file per story, A16): 1. ``test_answer_carries_origin_link`` — the ``original source link`` question on the git-matched source's document → the rendered tool output for the ``read`` (the mock's verbatim echo in the answer — the house scripted-turn lens) carries ``origin: https://example.com/reese/skills`` (the server put the line in the result the model saw) and the final ``done`` answer contains the repository URL (the scripted echo — the model-side contract, deterministic). 2. ``test_modal_shows_original_anchor`` — same setup → click the source chip → ``#doc-modal`` rendered → the meta row contains the ``.doc-origin`` anchor with ``href`` = the URL, ``target="_blank"``, ``rel="noopener"``, ``textContent`` = the URL verbatim (and NO branch in the text — ``branch=None``: the ``title`` is the bare URL). 3. ``test_pinned_branch_in_title`` — the ``branch="main"`` row → the read result's surface also carries ``branch: main``, the done answer still carries the repository URL, and the same anchor's ``title`` is ``https://example.com/reese/skills — branch: main`` (the text stays the bare URL — the copyable-link contract). 4. ``test_local_source_has_no_origin`` — the control source's document (no git rows at all — the pre-phase world) → the read's rendered tool output carries NO ``origin:`` line (the byte-identical result shape — the mock's no-origin sentence answers) and the modal's meta row has NO ``.doc-origin`` anchor (the pre-phase badge row). """ from __future__ import annotations import asyncio import json import time from collections.abc import Iterator from pathlib import Path from threading import Thread from typing import Any import pytest from playwright.sync_api import Page, expect from sqlalchemy import delete, func, select, text from app.config import Settings from app.db import SessionLocal from app.models import Document, GitSource from app.rag.importer import import_sources from app.rag.llm import LLMClient from e2e.auth_helpers import login from e2e.mock_llm import ORIGIN_LINK_ANSWER, ORIGIN_LINK_NO_ORIGIN REPO = Path(__file__).resolve().parents[2] SOURCE = "skills" # the local directory's basename = the source name SKILLS_REL = "phased-execution.md" SKILLS_SP = f"{SOURCE}/{SKILLS_REL}" # the combined read identity CONTROL = "control" # the plain local control source (NO git row) CONTROL_REL = "notes.md" CONTROL_SP = f"{CONTROL}/{CONTROL_REL}" #: The seeded git row's public URL (no credential — the public form; #: the phase-121 sanitizer is a byte-identical no-op for it). Its #: URL basename — ``skills`` — MATCHES the fixture source's name #: (the resolver's name mapping — the seam). ORIGIN_URL = "https://example.com/reese/skills" PINNED_BRANCH = "main" SKILLS_DOC = """# Phased execution for pi.dev How does the phased execution process work for pi.dev? It runs one task at a time: each task file executes in a separate pi subprocess, and the harness manages phase and task files, retries, and final validation. ## Task files Every task file is a unit of work. The pipeline runs the todo phases in numeric order, and a completed phase moves to the complete directory with its own test suite green. ## Validation A phase is complete only when the unit and integration tests pass, the coverage gate is green, and the Playwright story suite passes in isolation. The harness then moves the task file and commits the phase atomically. """ CONTROL_DOC = """# Control notes This is a plain local directory source. It is walked directly by the importer and never cloned from a repository. The control note exists so the byte-identity guard has a document: local documents read, render, and cite exactly as before, with nothing new in the read result or the meta row. """ # The suite's own content pins — the mock's echo assertion keys off # the read result's ``origin:``/``branch:`` lines, so neither # fixture may contain a line with either prefix (a fixture defect, # not a mock defect — see the mock's ``_origin_link_url``). assert "origin:" not in SKILLS_DOC and "branch:" not in SKILLS_DOC assert "origin:" not in CONTROL_DOC and "branch:" not in CONTROL_DOC # The scripted questions — the mock's ``ORIGIN_LINK_TRIGGER`` + the # question-carried call (``_ORIGIN_LINK_CALL_RE``) + the subject # words (the mock-calibrated gate: both questions stay GROUNDED on # their own fixture document). Q_SKILLS = ( f"Original source link: read {SKILLS_SP} — " "how does the phased execution process work for pi.dev?" ) Q_CONTROL = ( f"Original source link: read {CONTROL_SP} — " "what does the control note say about plain local sources?" ) #: The final answer the mock scripts for the git-matched document #: (the ``ORIGIN_LINK_ANSWER`` constant with the seeded URL — the #: model-side contract, imported so the mock can never drift). ANSWER_SENTENCE = ORIGIN_LINK_ANSWER.format(url=ORIGIN_URL) # -------------------------------------------------------------------------- # Fixtures # -------------------------------------------------------------------------- @pytest.fixture(scope="module") def kb_dirs(tmp_path_factory: pytest.TempPathFactory) -> tuple[Path, Path]: """The story's two local-directory sources (host temp dirs — the app server runs on the same host, so the paths are visible to it): the ``skills`` source (its basename MATCHES the seeded git row's URL basename — the resolver's name mapping) and the ``control`` source (no git row — the byte-identity control).""" skills = tmp_path_factory.mktemp("bor_origin_skills") / SOURCE skills.mkdir() (skills / SKILLS_REL).write_text(SKILLS_DOC, encoding="utf-8") control = tmp_path_factory.mktemp("bor_origin_control") / CONTROL control.mkdir() (control / CONTROL_REL).write_text(CONTROL_DOC, encoding="utf-8") return skills, control def _run_in_thread(coro: Any) -> Any: """Run a coroutine on a worker thread (the ``test_document_viewer`` pattern — Playwright's sync API keeps an asyncio loop on the test thread, so ``asyncio.run`` cannot run directly in a test body).""" box: dict[str, Any] = {} def runner() -> None: try: box["value"] = asyncio.run(coro) except BaseException as e: # noqa: BLE001 — re-raised in the test box["error"] = e t = Thread(target=runner) t.start() t.join() if "error" in box: raise box["error"] return box["value"] async def _import_sources(roots: list[Path], mock_port: int) -> None: """The house per-module import (the ``test_document_viewer`` pattern): the REAL importer against the fixture roots under the deterministic mock embeddings — the mock LLM supplies the embeds, and the owner's local ``.env`` is NOT read (the ``_env_file=None`` isolation).""" kwargs: dict[str, Any] = { "_env_file": None, "llm_base_url": f"http://127.0.0.1:{mock_port}/v1", } settings = Settings(**kwargs) # pyright: ignore[reportCallIssue] await import_sources(roots, LLMClient(settings)) def _truncate_all() -> None: """Fresh registry + KB (the E2E isolation pattern): the E2E suites share one Postgres, so a leftover git_sources row or document would pollute the resolver and the retrieval the scripted turns run against (the two-source KB's gate margins are pinned against EXACTLY these two documents).""" with SessionLocal() as db: db.execute( text( "TRUNCATE chunks, documents, query_log, steering_notes, " "kb_overview, git_sources, folder_summaries" ) ) db.commit() @pytest.fixture(scope="module") def seeded_kb(mock_llm: int, kb_dirs: tuple[Path, Path]) -> None: """The story's precondition: the two-source KB (one readable markdown doc each) imported under the deterministic mock, the stored content byte-pinned. The git row is NOT part of the import (a git row present at import time would be cloned — the row is seeded in the suite's own DB session per test, AFTER the import; the resolver reads the table live at read/content time).""" _truncate_all() _run_in_thread(_import_sources(list(kb_dirs), mock_llm)) with SessionLocal() as db: docs = { (d.source, d.path): d for d in db.scalars(select(Document)).all() } assert set(docs) == {(SOURCE, SKILLS_REL), (CONTROL, CONTROL_REL)}, ( docs.keys() ) assert docs[(SOURCE, SKILLS_REL)].content == SKILLS_DOC assert docs[(CONTROL, CONTROL_REL)].content == CONTROL_DOC # The import ran with NO git rows (the local-only KB): the # seeded row lands after, per test. assert db.scalar(select(func.count()).select_from(GitSource)) == 0 def _seed_git_row(branch: str | None, exist: bool = True) -> None: """The git row in the suite's OWN DB session (the task's seam): delete every row, then — when *exist* — insert the one ``kind='git'`` row whose URL basename matches the ``skills`` fixture source (public URL, no credential; *branch* pinned or NULL). The resolver reads the table LIVE — the row never touches the importer.""" with SessionLocal() as db: db.execute(delete(GitSource)) if exist: db.add(GitSource(url=ORIGIN_URL, kind="git", branch=branch)) db.commit() @pytest.fixture(autouse=True) def _clean(db_ready: None) -> Iterator[None]: """Per-test query_log isolation (the KB itself is module-scoped — the scripted turns never change it, so the documents persist across the tests of this module).""" with SessionLocal() as db: db.execute(text("TRUNCATE query_log")) db.commit() yield with SessionLocal() as db: db.execute(text("TRUNCATE query_log")) db.commit() # -------------------------------------------------------------------------- # Page helpers (the test_read_line_ranges / test_document_viewer # house patterns) # -------------------------------------------------------------------------- #: Captures the raw SSE ``data:`` payloads of the /api/chat stream #: (a response clone read in the background) — wire-level #: assertions for the ``tool`` / ``done`` frames, independent of #: the UI rendering. SSE_HOOK = """ () => { if (window.__sseInstalled) return; window.__sseInstalled = true; window.__sseFrames = []; const origFetch = window.fetch; window.fetch = async function (...args) { const res = await origFetch.apply(this, args); try { const url = typeof args[0] === 'string' ? args[0] : args[0].url; if (url.includes('/api/chat')) { res.clone().text().then((bodyText) => { for (const block of bodyText.split('\\n\\n')) { const line = block.trim(); if (line.startsWith('data: ')) { window.__sseFrames.push(line.slice(6)); } } }); } } catch (e) { /* non-clonable responses: ignored */ } return res; }; } """ def _install_page_hooks(page: Page) -> None: page.evaluate(SSE_HOOK) def _frames(page: Page, deadline_s: float = 120.0) -> list[dict]: """The SSE frames captured since the last submit (``_submit`` clears the buffer), once the hook's background read settles.""" deadline = time.monotonic() + deadline_s while True: raw = page.evaluate("() => window.__sseFrames || []") parsed = [json.loads(line) for line in raw if line] if any(f.get("type") == "done" for f in parsed): return parsed if time.monotonic() > deadline: raise AssertionError( f"SSE hook captured no `done` frame (frames so far: " f"{len(parsed)}) — hook install failed?" ) time.sleep(0.05) def _tool_frames(frames: list[dict]) -> list[dict]: return [f for f in frames if f.get("type") == "tool"] def _submit(page: Page, question: str) -> None: page.evaluate("window.__sseFrames = []") page.fill("#message-input", question) page.click("#send-btn") # The user bubble lands synchronously with the submit handler. expect(page.locator(".msg.user .bubble").last).to_contain_text(question) def _wait_settled(page: Page, timeout_ms: int = 120_000) -> None: """The turn is complete: answer text in the bubble, button recovered (the phase-48 settle wait).""" expect(page.locator(".msg.brain .bubble").last).not_to_have_text( "", timeout=timeout_ms ) expect(page.locator("#send-btn")).to_be_enabled(timeout=timeout_ms) expect(page.locator("#send-label")).to_have_text("Send", timeout=timeout_ms) def _bubble_text(page: Page) -> str: """The turn's rendered answer (the brain bubble's text content — the mock's scripted echo lands here: the house scripted-turn lens on the read result's ``origin:``/``branch:`` lines).""" return page.locator(".msg.brain .bubble").last.text_content() or "" def _open_modal_for_chip(page: Page, chip_text: str) -> None: """Drive the story's modal surface: the turn's cited chip (phase 119 A1 — done.sources is the READ docs, so the read document is the turn's ONLY chip) opens the same-page document modal (the ``test_document_viewer`` contract).""" chip = page.locator(".msg.brain .source-chip", has_text=chip_text) expect(chip).to_have_count(1, timeout=30_000) chip.first.click() expect(page.locator("#doc-modal")).not_to_have_attribute("hidden") expect(page.locator("#doc-modal")).to_be_visible() # -------------------------------------------------------------------------- # 1. The answer carries the repository link (the model-side # contract, through the real read result's ``origin:`` line) # -------------------------------------------------------------------------- def test_answer_carries_origin_link( page: Page, app_url: str, seeded_kb: None, db_ready: None ) -> None: _seed_git_row(branch=None) page.set_default_timeout(60_000) login(page, app_url, next="/") _install_page_hooks(page) _submit(page, Q_SKILLS) _wait_settled(page) frames = _frames(page) # The scripted read on the wire — the turn's only tool call. assert _tool_frames(frames) == [ {"type": "tool", "name": "read", "argument": SKILLS_SP} ], _tool_frames(frames) done = next(f for f in frames if f.get("type") == "done") assert done["deflected"] is False, done # The read document is the turn's cited source (the chip the # modal tests click — phase 119 A1). assert any( s["source"] == SOURCE and s["path"] == SKILLS_REL for s in done["sources"] ), done["sources"] # The read result's surface (the mock's verbatim echo in the # answer — the house scripted-turn lens): the sanitized # ``origin:`` line the SERVER put in the result the model saw # (the task-01 resolver's output — the git row's URL, # credential-free). text = _bubble_text(page) assert f"origin: {ORIGIN_URL}" in text, text # The done answer carries the repository URL (the scripted # echo — the model-side contract, deterministic): the mock # parsed the URL from the origin line — it cannot appear in # the answer unless the server sent it. assert ANSWER_SENTENCE in text, text # -------------------------------------------------------------------------- # 2. The modal's meta row shows the "Original" external anchor # -------------------------------------------------------------------------- def test_modal_shows_original_anchor( page: Page, app_url: str, seeded_kb: None, db_ready: None ) -> None: _seed_git_row(branch=None) page.set_default_timeout(60_000) login(page, app_url, next="/") _install_page_hooks(page) _submit(page, Q_SKILLS) _wait_settled(page) _open_modal_for_chip(page, SKILLS_REL) expect(page.locator("#doc-modal-title")).to_have_text( "Phased execution for pi.dev" ) # The meta row's "Original" anchor — the task-03 contract # (ONE shared renderDocument site: the modal AND # /document.html show it). anchor = page.locator("#doc-modal-meta .doc-origin") expect(anchor).to_have_count(1) expect(anchor).to_have_attribute("href", ORIGIN_URL) expect(anchor).to_have_attribute("target", "_blank") expect(anchor).to_have_attribute("rel", "noopener") # The copyable-link contract: the full URL VERBATIM as the # visible text — NO branch in the text (branch=None), and the # title is the bare URL (the pathCode precision idiom, no # branch suffix). expect(anchor).to_have_text(ORIGIN_URL) expect(anchor).to_have_attribute("title", ORIGIN_URL) # -------------------------------------------------------------------------- # 3. The pinned branch rides the read result and the anchor's title # -------------------------------------------------------------------------- def test_pinned_branch_in_title( page: Page, app_url: str, seeded_kb: None, db_ready: None ) -> None: _seed_git_row(branch=PINNED_BRANCH) page.set_default_timeout(60_000) login(page, app_url, next="/") _install_page_hooks(page) _submit(page, Q_SKILLS) _wait_settled(page) # The read result's surface (the mock's verbatim echo): the # origin line AND the pinned branch line (the task-01 second # line, only when the row pins a branch). text = _bubble_text(page) assert f"origin: {ORIGIN_URL}" in text, text assert f"branch: {PINNED_BRANCH}" in text, text # The done answer still carries the repository URL (tests # 1–3's model-side contract — the branch never joins the # URL). assert ANSWER_SENTENCE in text, text _open_modal_for_chip(page, SKILLS_REL) anchor = page.locator("#doc-modal-meta .doc-origin") expect(anchor).to_have_count(1) expect(anchor).to_have_attribute("href", ORIGIN_URL) # The text stays the bare URL (copyable — the branch never # joins the anchor text)… expect(anchor).to_have_text(ORIGIN_URL) # …the branch rides the title (the task-03 label contract). expect(anchor).to_have_attribute( "title", f"{ORIGIN_URL} — branch: {PINNED_BRANCH}" ) # -------------------------------------------------------------------------- # 4. The local-source control is byte-identical (no ``origin:`` # line in the read result, no anchor in the meta row) # -------------------------------------------------------------------------- def test_local_source_has_no_origin( page: Page, app_url: str, seeded_kb: None, db_ready: None ) -> None: _seed_git_row(branch=None, exist=False) # the pre-phase world: NO git rows at all page.set_default_timeout(60_000) login(page, app_url, next="/") _install_page_hooks(page) _submit(page, Q_CONTROL) _wait_settled(page) frames = _frames(page) assert _tool_frames(frames) == [ {"type": "tool", "name": "read", "argument": CONTROL_SP} ], _tool_frames(frames) done = next(f for f in frames if f.get("type") == "done") assert done["deflected"] is False, done # The byte-identity guard on the read result's surface (the # mock's verbatim echo): the result carried NO origin line — # the local source's result is byte-identical to pre-phase, # so the mock's deterministic no-origin sentence answers. text = _bubble_text(page) assert ORIGIN_LINK_NO_ORIGIN in text, text assert "origin:" not in text, text assert "branch:" not in text, text # …and the modal's meta row is the pre-phase badge row: NO # .doc-origin anchor (the content payload's origin fields are # absent on the wire — never null). _open_modal_for_chip(page, CONTROL_REL) expect(page.locator("#doc-modal-title")).to_have_text("Control notes") expect(page.locator("#doc-modal-meta .doc-origin")).to_have_count(0) # The pre-phase badges are untouched (the row still shows # source · format · path · created · indexed · chunks). expect(page.locator("#doc-modal-meta .doc-source-badge")).to_have_text( CONTROL ) expect(page.locator("#doc-modal-meta .doc-path")).to_have_text( CONTROL_REL ) expect(page.locator("#doc-modal-meta .doc-created")).not_to_be_empty()