phase: 145_git_source_links
**Phase 145 final verification pass — all green, no code changes needed** - **Verified (no defects found):** resolver `origin_for_source` + `_url_basename` (scp/`.git`/trailing-slash, sanitize, ambiguity WARNING); `origin:`/`branch:` lines in all four read shapes via shared `_origin_lines`; HIGH `<tools>` origin rule (verbatim + only-byte-change pins); `DocContent.origin_url/origin_branch` (absent-never-null serializer); `renderDocument` "Original" anchor (copyable verbatim URL, `_blank`/`noopener`, branch in title); mock scripted origin-link flow (disjoint trigger — grep-verified vs all other E2E files) - **Tests:** `uv run pytest --cov=app --cov-report=term-missing` → 3402 passed, `app/` coverage **99%** (>90% gate); `uv run pytest tests/e2e/test_git_source_links.py -v --no-cov` → **4 passed** in isolation; `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings - **Completion criteria:** (1) git read results carry origin/branch, local byte-identical — ✅ (unit pins + E2E control); (2) HIGH `<tools>` rule pinned, LOW byte-identical — ✅; (3) modal AND /document.html show Original link, branch in label when pinned, nothing for local — ✅; (4) full suite green + coverage + E2E + lint/types — ✅; (5) no completed-phase behavior change (read header line 1, mock prefix, done frame/SourceRef untouched, LOW sha) — ✅ - **Extra regression check:** re-ran shared-mock E2E suites `test_read_line_ranges.py` (2 passed) and `test_document_viewer.py` (8 passed) — no cross-suite regressions from the `mock_llm.py` addition - **Documentation:** README.md reviewed in full (env table ⊇ `.env.example`, pages/routes, config defaults, scripts, 17-color palette, 300 s guard, 2 s sync poll, phase-145 provenance section matches implementation) and AGENTS.md (72rem container, no-CDN test, smoke test, gitignore, gates) — **verified consistent, no changes needed** - **Notable:** phase-116 screenshot PNG was regenerated during this phase's E2E runs (byte re-encode only, visually identical, correct no-anchor local-source meta row) — left in tree for the harness commit - **Next pending phase:** `146_search_tool_scope` (then 147, 148, 149)
This commit is contained in:
@@ -714,6 +714,55 @@ Implements just enough of the aipi surface:
|
||||
deflected turns never hit it); verified 2026-10-01: no existing
|
||||
E2E question or fixture file contains the phrase, so every
|
||||
other suite is unaffected.
|
||||
- user message containing ``original source link``
|
||||
(``ORIGIN_LINK_TRIGGER``, phase 145 task 04 — the git-source-
|
||||
links' dedicated story suite
|
||||
``tests/e2e/test_git_source_links.py``) **and** the system
|
||||
prompt carries the ``<tools>`` section -> the deterministic
|
||||
SCRIPTED ORIGIN-LINK flow (the owner's scenario, TODO.md L7:
|
||||
the answer grounded in a git-sourced document must carry the
|
||||
way to the complete original in ONE step): the question
|
||||
carries its own tool call after the colon — ``original source
|
||||
link: read source/path`` — the question-carried-call
|
||||
convention (``_READ_CAP_CALL_RE``), then discriminated
|
||||
statelessly from the tool results (streaming only):
|
||||
* request 1 (``tools`` offered, no tool results yet): a
|
||||
``tool_calls`` delta — ``read`` on the parsed target
|
||||
(synthetic id ``call_0``);
|
||||
* a ``tool``-role READ result is in the messages (content
|
||||
starting with the agent's ``"Document "`` prefix —
|
||||
``_READ_RESULT_PREFIX``; the phase-145 ``origin:`` /
|
||||
``branch:`` lines sit BELOW the header, so the first-line
|
||||
parsing is untouched): the deterministic FINAL answer —
|
||||
the repository URL VERBATIM, parsed from the read
|
||||
result's ``origin:`` line BY THE MOCK ITSELF (the mock
|
||||
cannot fabricate it — the real server's resolver put the
|
||||
line in the result, so the URL in the answer is the
|
||||
server-side half's assertion): ``The full source lives at
|
||||
<url> — see the repo for the complete original.``
|
||||
(``ORIGIN_LINK_ANSWER``) followed by the house
|
||||
scripted-turn echo — the read result VERBATIM under
|
||||
``Here's what the read returned:`` (the rendered output
|
||||
is the E2E's lens on the ``origin:``/``branch:`` lines'
|
||||
presence);
|
||||
* a tool result is in the messages that carries NO
|
||||
``origin:`` line (a local-sourced document — the byte-
|
||||
identity control): the deterministic NO-ORIGIN answer —
|
||||
``No origin line in the read result — the document is not
|
||||
from a git source.`` (``ORIGIN_LINK_NO_ORIGIN``) + the
|
||||
same verbatim echo (the suite asserts the line's ABSENCE
|
||||
through it);
|
||||
* a tool result is in the messages that is NOT a read
|
||||
result (a refusal — the premise broke): the
|
||||
deterministic broken-premise echo — the offending result
|
||||
VERBATIM under ``Premise broke:`` (the two-ranges
|
||||
convention — the suite fails loudly on it).
|
||||
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint
|
||||
trigger phrases — the phase-71/72/94/118/131/139/141 ordering
|
||||
convention; the trigger needs the ``<tools>`` section, so
|
||||
deflected turns never hit it); verified 2026-10-01: no
|
||||
existing E2E question or fixture file contains the phrase, so
|
||||
every other suite is unaffected.
|
||||
|
||||
Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic
|
||||
dead-endpoint behavior for the retry E2E suite (``tests/e2e/
|
||||
@@ -2667,6 +2716,148 @@ def _two_ranges_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
|
||||
return ("read1", target)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 145 (task 04): the git-source-links' dedicated story suite — the
|
||||
# owner's "the answer points at the original git source" scenario. See
|
||||
# the module docstring.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: A user message containing this substring (case-insensitive) —
|
||||
#: combined with the ``<tools>`` section in the system prompt —
|
||||
#: drives the deterministic SCRIPTED ORIGIN-LINK flow (the
|
||||
#: git-source-links' story suite,
|
||||
#: ``tests/e2e/test_git_source_links.py`` — the owner's scenario,
|
||||
#: TODO.md L7: the answer grounded in a git-sourced document must
|
||||
#: carry the way to the complete original in ONE step). Checked
|
||||
#: BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
|
||||
#: phrases — the phase-71/72/94/118/131/139/141 ordering
|
||||
#: convention; the trigger needs the ``<tools>`` section, so
|
||||
#: deflected turns never hit it); verified 2026-10-01: no
|
||||
#: existing E2E question or fixture file contains the phrase, so
|
||||
#: every other suite is unaffected.
|
||||
ORIGIN_LINK_TRIGGER = "original source link"
|
||||
|
||||
#: The scripted call in the origin-link question (case-insensitive
|
||||
#: — the suite's questions capitalize the trigger's first
|
||||
#: letter): the verb (``read``) plus the target — a combined
|
||||
#: ``source/path``, parsed from the RAW user message so the target
|
||||
#: keeps its case. The target is a ``[a-z0-9_./-]`` run (case-
|
||||
#: insensitively), so the suite's `` — `` flavor separator (em
|
||||
#: dash) can never bleed into it (the read-cap / drill-down
|
||||
#: convention, ``_READ_CAP_CALL_RE`` / ``_DRILL_CALL_RE``).
|
||||
_ORIGIN_LINK_CALL_RE = re.compile(
|
||||
r"original source link:\s*read\s+(?P<arg>[a-z0-9_./-]+)",
|
||||
re.I,
|
||||
)
|
||||
|
||||
#: The flow's FINAL answer for a read result that carries the
|
||||
#: phase-145 ``origin:`` line — the DETERMINISTIC string carrying
|
||||
#: the repository URL VERBATIM (the model-side contract, owner-
|
||||
#: confirmed 2026-09-30: the answer includes the original source).
|
||||
#: ``{url}`` is parsed from the read result's ``origin:`` line BY
|
||||
#: THE MOCK ITSELF — the mock cannot fabricate it, so the URL in
|
||||
#: the answer is the server-side half's assertion (the real
|
||||
#: server's resolver put the line in the result the model saw).
|
||||
ORIGIN_LINK_ANSWER = (
|
||||
"The full source lives at {url} — see the repo for the complete original."
|
||||
)
|
||||
|
||||
#: The flow's FINAL answer for a read result that carries NO
|
||||
#: ``origin:`` line (a local-sourced document — the byte-identity
|
||||
#: control): the deterministic no-origin sentence (its wording
|
||||
#: carries NO ``origin:`` substring — the suite's absence
|
||||
#: assertion keys off the line's exact prefix).
|
||||
ORIGIN_LINK_NO_ORIGIN = (
|
||||
"No origin line in the read result — the document is not from a git source."
|
||||
)
|
||||
|
||||
#: The house scripted-turn echo header (the ``READ_CAP_TRIGGER``
|
||||
#: flow's convention): the read result rides VERBATIM into the
|
||||
#: answer, so the rendered output is the E2E's lens on the
|
||||
#: ``origin:``/``branch:`` lines' presence (and their absence in
|
||||
#: the local-source control).
|
||||
_ORIGIN_LINK_ECHO_HEADER = "Here's what the read returned:"
|
||||
|
||||
|
||||
def _origin_link_url(result: str) -> str | None:
|
||||
"""The read result's ``origin:`` line's URL (the phase-145
|
||||
task-01 line — the sanitized repository URL the real server's
|
||||
resolver put in the result) — or ``None`` when the result
|
||||
carries no origin line (a local-sourced document, no git row,
|
||||
or an ambiguous basename).
|
||||
|
||||
The search is position-independent (any line with the exact
|
||||
``origin: `` prefix): the phase-145 lines sit between the
|
||||
``date:`` line and the content, but the parse never drifts if
|
||||
the result's header shape changes. A document BODY line that
|
||||
merely starts with ``origin: `` would be a fixture defect (the
|
||||
suite's fixtures are pinned content — no such line), never a
|
||||
mock defect.
|
||||
"""
|
||||
for line in result.splitlines():
|
||||
if line.startswith("origin: "):
|
||||
return line[len("origin: "):]
|
||||
return None
|
||||
|
||||
|
||||
def _origin_link_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
|
||||
"""Classify a phase-145 scripted origin-link request (see the
|
||||
module docstring). The question carries the scripted call
|
||||
(``original source link: read source/path``); the step is then
|
||||
discriminated statelessly from the tool results, like the
|
||||
other marker flows:
|
||||
|
||||
* ``("call", target, "call_0")`` — ``tools`` are offered and
|
||||
no ``tool``-role result is in the messages yet: the scripted
|
||||
``read`` on the parsed target (synthetic id ``call_0``).
|
||||
* ``("answer", url, result)`` — a ``tool``-role READ result
|
||||
(content starting with the agent's ``"Document "`` prefix —
|
||||
``_READ_RESULT_PREFIX``; the phase-145 ``origin:`` /
|
||||
``branch:`` lines sit BELOW the header, so the first-line
|
||||
parsing is untouched) is in the messages: the deterministic
|
||||
FINAL answer — ``url`` the result's ``origin:`` line's URL
|
||||
(or ``""`` — the no-origin sentence, the local-source
|
||||
control; the empty-string sentinel keeps the return
|
||||
homogeneous ``tuple[str, …]``, the ``_read_lines_flow``
|
||||
convention) and ``result`` the read result VERBATIM (the
|
||||
house scripted-turn echo — the rendered output is the lens
|
||||
on the origin lines).
|
||||
* ``("bad", result)`` — a tool result is in the messages that
|
||||
is NOT a read result (a refusal — the premise broke): the
|
||||
deterministic broken-premise echo — the offending result
|
||||
VERBATIM under ``Premise broke:`` (the two-ranges convention
|
||||
— the suite fails loudly on it).
|
||||
* ``None`` — not the flow: the trigger is absent, the
|
||||
``<tools>`` section is missing (deflected turns never carry
|
||||
it), the scripted call is unparseable, or ``tools`` are not
|
||||
offered and no tool results are in the messages yet (e.g.
|
||||
``agent_max_rounds=0``).
|
||||
"""
|
||||
user = _user(body)
|
||||
if ORIGIN_LINK_TRIGGER not in user.lower():
|
||||
return None
|
||||
if "<tools>" not in _system(body):
|
||||
return None
|
||||
match = _ORIGIN_LINK_CALL_RE.search(user)
|
||||
if match is None:
|
||||
return None
|
||||
target = match.group("arg")
|
||||
results = _tool_results(body)
|
||||
if results:
|
||||
last = results[-1]
|
||||
if not last.startswith(_READ_RESULT_PREFIX):
|
||||
return ("bad", last)
|
||||
# The empty-string sentinel (never a real URL — a URL is
|
||||
# always non-empty: the resolver returns the stored URL's
|
||||
# sanitized form, which is a non-empty string): keeps the
|
||||
# return homogeneous ``tuple[str, …]`` (pyright), the
|
||||
# ``_read_lines_flow`` convention.
|
||||
return ("answer", _origin_link_url(last) or "", last)
|
||||
if not body.get("tools"):
|
||||
return None
|
||||
return ("call", target, "call_0")
|
||||
|
||||
|
||||
def long_answer() -> str:
|
||||
"""~900-word deterministic walkthrough (phase 11): numbered steps plus
|
||||
a unique final line that must survive the stream untruncated."""
|
||||
@@ -4044,6 +4235,57 @@ def chat_completions(body: dict[str, Any]) -> Any:
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
# Phase 145 (task 04): the owner's "the answer points at the
|
||||
# original git source" scenario (the git-source-links' story
|
||||
# suite, tests/e2e/test_git_source_links.py — TODO.md L7):
|
||||
# the scripted read, then the final answer that carries the
|
||||
# repository URL parsed from the read result's ``origin:``
|
||||
# line (the server-side half — the mock cannot fabricate it)
|
||||
# + the result VERBATIM (the lens on the origin lines'
|
||||
# presence/absence). Checked BEFORE the plain TOOLS_TRIGGER
|
||||
# flow (disjoint trigger phrases — the phase-71/72/94/118/
|
||||
# 131/139/141 ordering convention; the trigger needs the
|
||||
# ``<tools>`` section, so deflected turns never hit it).
|
||||
origin_link = _origin_link_flow(body)
|
||||
if origin_link is not None:
|
||||
if origin_link[0] == "call":
|
||||
stream = _tool_call_stream(
|
||||
"read", {"path": origin_link[1]}, origin_link[2]
|
||||
)
|
||||
elif origin_link[0] == "bad":
|
||||
# The premise broke (a refusal — the scripted read
|
||||
# was refused): echo the offending result verbatim
|
||||
# so a suite fails loudly on it (the two-ranges
|
||||
# convention).
|
||||
stream = _sse_stream(
|
||||
_apply_max_tokens(
|
||||
f"Premise broke:\n{origin_link[1]}",
|
||||
body.get("max_tokens"),
|
||||
),
|
||||
0.0,
|
||||
)
|
||||
else: # "answer" — the final answer (tests 1–3's model-
|
||||
# side contract) + the verbatim echo (the lens on
|
||||
# the origin lines' presence/absence). ``url`` is
|
||||
# the ``origin:`` line's URL or the "" sentinel
|
||||
# (the no-origin sentence — the local-source
|
||||
# control).
|
||||
url = origin_link[1]
|
||||
sentence = (
|
||||
ORIGIN_LINK_ANSWER.format(url=url) if url else ORIGIN_LINK_NO_ORIGIN
|
||||
)
|
||||
stream = _sse_stream(
|
||||
_apply_max_tokens(
|
||||
f"{sentence}\n{_ORIGIN_LINK_ECHO_HEADER}\n{origin_link[2]}",
|
||||
body.get("max_tokens"),
|
||||
),
|
||||
0.0,
|
||||
)
|
||||
return StreamingResponse(
|
||||
stream,
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
# Phase 124 (task 07, image chips and answers, TODO L3): the
|
||||
# scripted SHOW_IMAGE flow — the mock shows the seeded image
|
||||
# document (the ``<documents>`` block with the image-
|
||||
|
||||
@@ -0,0 +1,585 @@
|
||||
"""Phase 145 task 04 E2E (Playwright, mock-only): the git-source-links'
|
||||
dedicated story suite — the owner's scenario, end to end.
|
||||
|
||||
The browser-visible proof (TODO.md L7: "there always needs to be a
|
||||
link to the original git source if possible … point to the skills
|
||||
repo in the answer so I can go download the skills"): a question
|
||||
about a git-sourced document produces (1) an answer that contains
|
||||
the repository link (the model-side teaching, through the real read
|
||||
result that carries the ``origin:`` line) and (2) a document modal
|
||||
whose meta row carries the "Original" external anchor. A
|
||||
local-sourced document produces neither (the byte-identity guard).
|
||||
|
||||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
uv run pytest tests/e2e/test_git_source_links.py -v --no-cov
|
||||
|
||||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the real
|
||||
``turbo`` does whatever it does with the tools, while this story's
|
||||
gate is the deterministic SCRIPTED flow in ``tests/e2e/mock_llm.py``
|
||||
(the mock is the only E2E lens on the LLM's context):
|
||||
|
||||
* ``ORIGIN_LINK_TRIGGER`` — a user message containing ``original
|
||||
source link`` **and** the system prompt carrying the HIGH
|
||||
prompt's ``<tools>`` section. The question carries its own
|
||||
scripted call after the colon — ``original source link: read
|
||||
source/path`` (the question-carried-call convention,
|
||||
``_READ_CAP_CALL_RE``). Request 1 is the scripted ``read``;
|
||||
request 2 (the read result present) is the deterministic FINAL
|
||||
answer — the repository URL VERBATIM, parsed from the read
|
||||
result's ``origin:`` line BY THE MOCK (it cannot fabricate it —
|
||||
the real server's resolver put the line in the result, so the
|
||||
URL in the answer is the server-side half's assertion) —
|
||||
followed by the house scripted-turn echo (the read result
|
||||
VERBATIM — the rendered output is the lens on the
|
||||
``origin:``/``branch:`` lines' presence; the local-source
|
||||
control's read result carries none, and the mock answers with
|
||||
its deterministic no-origin sentence instead).
|
||||
|
||||
KB fixture — TWO local directories (the house per-module import —
|
||||
the ``test_document_viewer.py`` importer-in-a-thread pattern), each
|
||||
holding ONE readable markdown document:
|
||||
|
||||
* ``skills/phased-execution.md`` — the owner's scenario document
|
||||
(the phased execution process for pi.dev). The source's name is
|
||||
``skills`` — it MATCHES the seeded git row's URL basename
|
||||
(``https://example.com/reese/skills`` → ``skills``: the path
|
||||
component, trailing ``/`` stripped, no ``.git`` suffix to strip,
|
||||
last segment — the resolver's name mapping). The fixture document
|
||||
is locally imported, but the mapping is name-based — which is the
|
||||
point of the seam;
|
||||
* ``control/notes.md`` — a SECOND plain local source with NO git
|
||||
row: the byte-identity control (its read result and its modal
|
||||
meta row must be exactly the pre-phase shapes).
|
||||
|
||||
The git row is seeded in the suite's OWN DB session (a direct
|
||||
``SessionLocal`` write, per test — ``branch=None`` for tests 1–2,
|
||||
``branch="main"`` for the pinned-branch test, and ABSENT for the
|
||||
control test) — never through the API, and never during the import
|
||||
(a ``kind='git'`` row present at import time would be cloned —
|
||||
``example.com`` is not a git host; the resolver reads the table
|
||||
LIVE at read/content time, so seeding after the import is the
|
||||
whole trick).
|
||||
|
||||
The app under test is the conftest session app (mock LLM + the
|
||||
mock-calibrated gate: threshold 0.30 + lexical floor 0.15): both
|
||||
questions share the subject words with their own fixture document
|
||||
(the mock's token-overlap distribution), so every scripted turn
|
||||
stays GROUNDED (the ``<tools>`` section is present) and the read
|
||||
document is the turn's cited chip (phase 119 A1 — the done frame's
|
||||
``sources`` is the READ docs).
|
||||
|
||||
Test → observable mapping (Playwright Mapping Rule; the story is
|
||||
the owner's TODO item — one Playwright file per story, A16):
|
||||
1. ``test_answer_carries_origin_link`` — the ``original source
|
||||
link`` question on the git-matched source's document → the
|
||||
rendered tool output for the ``read`` (the mock's verbatim echo
|
||||
in the answer — the house scripted-turn lens) carries
|
||||
``origin: https://example.com/reese/skills`` (the server put
|
||||
the line in the result the model saw) and the final ``done``
|
||||
answer contains the repository URL (the scripted echo — the
|
||||
model-side contract, deterministic).
|
||||
2. ``test_modal_shows_original_anchor`` — same setup → click the
|
||||
source chip → ``#doc-modal`` rendered → the meta row contains
|
||||
the ``.doc-origin`` anchor with ``href`` = the URL,
|
||||
``target="_blank"``, ``rel="noopener"``, ``textContent`` = the
|
||||
URL verbatim (and NO branch in the text — ``branch=None``: the
|
||||
``title`` is the bare URL).
|
||||
3. ``test_pinned_branch_in_title`` — the ``branch="main"`` row →
|
||||
the read result's surface also carries ``branch: main``, the
|
||||
done answer still carries the repository URL, and the same
|
||||
anchor's ``title`` is
|
||||
``https://example.com/reese/skills — branch: main`` (the text
|
||||
stays the bare URL — the copyable-link contract).
|
||||
4. ``test_local_source_has_no_origin`` — the control source's
|
||||
document (no git rows at all — the pre-phase world) → the
|
||||
read's rendered tool output carries NO ``origin:`` line (the
|
||||
byte-identical result shape — the mock's no-origin sentence
|
||||
answers) and the modal's meta row has NO ``.doc-origin``
|
||||
anchor (the pre-phase badge row).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
from threading import Thread
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from playwright.sync_api import Page, expect
|
||||
from sqlalchemy import delete, func, select, text
|
||||
|
||||
from app.config import Settings
|
||||
from app.db import SessionLocal
|
||||
from app.models import Document, GitSource
|
||||
from app.rag.importer import import_sources
|
||||
from app.rag.llm import LLMClient
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.mock_llm import ORIGIN_LINK_ANSWER, ORIGIN_LINK_NO_ORIGIN
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
SOURCE = "skills" # the local directory's basename = the source name
|
||||
SKILLS_REL = "phased-execution.md"
|
||||
SKILLS_SP = f"{SOURCE}/{SKILLS_REL}" # the combined read identity
|
||||
CONTROL = "control" # the plain local control source (NO git row)
|
||||
CONTROL_REL = "notes.md"
|
||||
CONTROL_SP = f"{CONTROL}/{CONTROL_REL}"
|
||||
|
||||
#: The seeded git row's public URL (no credential — the public form;
|
||||
#: the phase-121 sanitizer is a byte-identical no-op for it). Its
|
||||
#: URL basename — ``skills`` — MATCHES the fixture source's name
|
||||
#: (the resolver's name mapping — the seam).
|
||||
ORIGIN_URL = "https://example.com/reese/skills"
|
||||
PINNED_BRANCH = "main"
|
||||
|
||||
SKILLS_DOC = """# Phased execution for pi.dev
|
||||
|
||||
How does the phased execution process work for pi.dev? It runs one
|
||||
task at a time: each task file executes in a separate pi subprocess,
|
||||
and the harness manages phase and task files, retries, and final
|
||||
validation.
|
||||
|
||||
## Task files
|
||||
|
||||
Every task file is a unit of work. The pipeline runs the todo phases
|
||||
in numeric order, and a completed phase moves to the complete
|
||||
directory with its own test suite green.
|
||||
|
||||
## Validation
|
||||
|
||||
A phase is complete only when the unit and integration tests pass,
|
||||
the coverage gate is green, and the Playwright story suite passes
|
||||
in isolation. The harness then moves the task file and commits the
|
||||
phase atomically.
|
||||
"""
|
||||
|
||||
CONTROL_DOC = """# Control notes
|
||||
|
||||
This is a plain local directory source. It is walked directly by
|
||||
the importer and never cloned from a repository. The control note
|
||||
exists so the byte-identity guard has a document: local documents
|
||||
read, render, and cite exactly as before, with nothing new in the
|
||||
read result or the meta row.
|
||||
"""
|
||||
|
||||
# The suite's own content pins — the mock's echo assertion keys off
|
||||
# the read result's ``origin:``/``branch:`` lines, so neither
|
||||
# fixture may contain a line with either prefix (a fixture defect,
|
||||
# not a mock defect — see the mock's ``_origin_link_url``).
|
||||
assert "origin:" not in SKILLS_DOC and "branch:" not in SKILLS_DOC
|
||||
assert "origin:" not in CONTROL_DOC and "branch:" not in CONTROL_DOC
|
||||
|
||||
# The scripted questions — the mock's ``ORIGIN_LINK_TRIGGER`` + the
|
||||
# question-carried call (``_ORIGIN_LINK_CALL_RE``) + the subject
|
||||
# words (the mock-calibrated gate: both questions stay GROUNDED on
|
||||
# their own fixture document).
|
||||
Q_SKILLS = (
|
||||
f"Original source link: read {SKILLS_SP} — "
|
||||
"how does the phased execution process work for pi.dev?"
|
||||
)
|
||||
Q_CONTROL = (
|
||||
f"Original source link: read {CONTROL_SP} — "
|
||||
"what does the control note say about plain local sources?"
|
||||
)
|
||||
|
||||
#: The final answer the mock scripts for the git-matched document
|
||||
#: (the ``ORIGIN_LINK_ANSWER`` constant with the seeded URL — the
|
||||
#: model-side contract, imported so the mock can never drift).
|
||||
ANSWER_SENTENCE = ORIGIN_LINK_ANSWER.format(url=ORIGIN_URL)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Fixtures
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def kb_dirs(tmp_path_factory: pytest.TempPathFactory) -> tuple[Path, Path]:
|
||||
"""The story's two local-directory sources (host temp dirs — the
|
||||
app server runs on the same host, so the paths are visible to
|
||||
it): the ``skills`` source (its basename MATCHES the seeded git
|
||||
row's URL basename — the resolver's name mapping) and the
|
||||
``control`` source (no git row — the byte-identity control)."""
|
||||
skills = tmp_path_factory.mktemp("bor_origin_skills") / SOURCE
|
||||
skills.mkdir()
|
||||
(skills / SKILLS_REL).write_text(SKILLS_DOC, encoding="utf-8")
|
||||
control = tmp_path_factory.mktemp("bor_origin_control") / CONTROL
|
||||
control.mkdir()
|
||||
(control / CONTROL_REL).write_text(CONTROL_DOC, encoding="utf-8")
|
||||
return skills, control
|
||||
|
||||
|
||||
def _run_in_thread(coro: Any) -> Any:
|
||||
"""Run a coroutine on a worker thread (the
|
||||
``test_document_viewer`` pattern — Playwright's sync API keeps
|
||||
an asyncio loop on the test thread, so ``asyncio.run`` cannot
|
||||
run directly in a test body)."""
|
||||
box: dict[str, Any] = {}
|
||||
|
||||
def runner() -> None:
|
||||
try:
|
||||
box["value"] = asyncio.run(coro)
|
||||
except BaseException as e: # noqa: BLE001 — re-raised in the test
|
||||
box["error"] = e
|
||||
|
||||
t = Thread(target=runner)
|
||||
t.start()
|
||||
t.join()
|
||||
if "error" in box:
|
||||
raise box["error"]
|
||||
return box["value"]
|
||||
|
||||
|
||||
async def _import_sources(roots: list[Path], mock_port: int) -> None:
|
||||
"""The house per-module import (the ``test_document_viewer``
|
||||
pattern): the REAL importer against the fixture roots under the
|
||||
deterministic mock embeddings — the mock LLM supplies the
|
||||
embeds, and the owner's local ``.env`` is NOT read (the
|
||||
``_env_file=None`` isolation)."""
|
||||
kwargs: dict[str, Any] = {
|
||||
"_env_file": None,
|
||||
"llm_base_url": f"http://127.0.0.1:{mock_port}/v1",
|
||||
}
|
||||
settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
|
||||
await import_sources(roots, LLMClient(settings))
|
||||
|
||||
|
||||
def _truncate_all() -> None:
|
||||
"""Fresh registry + KB (the E2E isolation pattern): the E2E
|
||||
suites share one Postgres, so a leftover git_sources row or
|
||||
document would pollute the resolver and the retrieval the
|
||||
scripted turns run against (the two-source KB's gate margins
|
||||
are pinned against EXACTLY these two documents)."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(
|
||||
text(
|
||||
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
||||
"kb_overview, git_sources, folder_summaries"
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def seeded_kb(mock_llm: int, kb_dirs: tuple[Path, Path]) -> None:
|
||||
"""The story's precondition: the two-source KB (one readable
|
||||
markdown doc each) imported under the deterministic mock, the
|
||||
stored content byte-pinned. The git row is NOT part of the
|
||||
import (a git row present at import time would be cloned — the
|
||||
row is seeded in the suite's own DB session per test, AFTER
|
||||
the import; the resolver reads the table live at
|
||||
read/content time)."""
|
||||
_truncate_all()
|
||||
_run_in_thread(_import_sources(list(kb_dirs), mock_llm))
|
||||
with SessionLocal() as db:
|
||||
docs = {
|
||||
(d.source, d.path): d for d in db.scalars(select(Document)).all()
|
||||
}
|
||||
assert set(docs) == {(SOURCE, SKILLS_REL), (CONTROL, CONTROL_REL)}, (
|
||||
docs.keys()
|
||||
)
|
||||
assert docs[(SOURCE, SKILLS_REL)].content == SKILLS_DOC
|
||||
assert docs[(CONTROL, CONTROL_REL)].content == CONTROL_DOC
|
||||
# The import ran with NO git rows (the local-only KB): the
|
||||
# seeded row lands after, per test.
|
||||
assert db.scalar(select(func.count()).select_from(GitSource)) == 0
|
||||
|
||||
|
||||
def _seed_git_row(branch: str | None, exist: bool = True) -> None:
|
||||
"""The git row in the suite's OWN DB session (the task's seam):
|
||||
delete every row, then — when *exist* — insert the one
|
||||
``kind='git'`` row whose URL basename matches the ``skills``
|
||||
fixture source (public URL, no credential; *branch* pinned or
|
||||
NULL). The resolver reads the table LIVE — the row never
|
||||
touches the importer."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(delete(GitSource))
|
||||
if exist:
|
||||
db.add(GitSource(url=ORIGIN_URL, kind="git", branch=branch))
|
||||
db.commit()
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean(db_ready: None) -> Iterator[None]:
|
||||
"""Per-test query_log isolation (the KB itself is module-scoped
|
||||
— the scripted turns never change it, so the documents persist
|
||||
across the tests of this module)."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE query_log"))
|
||||
db.commit()
|
||||
yield
|
||||
with SessionLocal() as db:
|
||||
db.execute(text("TRUNCATE query_log"))
|
||||
db.commit()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Page helpers (the test_read_line_ranges / test_document_viewer
|
||||
# house patterns)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
#: Captures the raw SSE ``data:`` payloads of the /api/chat stream
|
||||
#: (a response clone read in the background) — wire-level
|
||||
#: assertions for the ``tool`` / ``done`` frames, independent of
|
||||
#: the UI rendering.
|
||||
SSE_HOOK = """
|
||||
() => {
|
||||
if (window.__sseInstalled) return;
|
||||
window.__sseInstalled = true;
|
||||
window.__sseFrames = [];
|
||||
const origFetch = window.fetch;
|
||||
window.fetch = async function (...args) {
|
||||
const res = await origFetch.apply(this, args);
|
||||
try {
|
||||
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
|
||||
if (url.includes('/api/chat')) {
|
||||
res.clone().text().then((bodyText) => {
|
||||
for (const block of bodyText.split('\\n\\n')) {
|
||||
const line = block.trim();
|
||||
if (line.startsWith('data: ')) {
|
||||
window.__sseFrames.push(line.slice(6));
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
} catch (e) { /* non-clonable responses: ignored */ }
|
||||
return res;
|
||||
};
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def _install_page_hooks(page: Page) -> None:
|
||||
page.evaluate(SSE_HOOK)
|
||||
|
||||
|
||||
def _frames(page: Page, deadline_s: float = 120.0) -> list[dict]:
|
||||
"""The SSE frames captured since the last submit (``_submit``
|
||||
clears the buffer), once the hook's background read settles."""
|
||||
deadline = time.monotonic() + deadline_s
|
||||
while True:
|
||||
raw = page.evaluate("() => window.__sseFrames || []")
|
||||
parsed = [json.loads(line) for line in raw if line]
|
||||
if any(f.get("type") == "done" for f in parsed):
|
||||
return parsed
|
||||
if time.monotonic() > deadline:
|
||||
raise AssertionError(
|
||||
f"SSE hook captured no `done` frame (frames so far: "
|
||||
f"{len(parsed)}) — hook install failed?"
|
||||
)
|
||||
time.sleep(0.05)
|
||||
|
||||
|
||||
def _tool_frames(frames: list[dict]) -> list[dict]:
|
||||
return [f for f in frames if f.get("type") == "tool"]
|
||||
|
||||
|
||||
def _submit(page: Page, question: str) -> None:
|
||||
page.evaluate("window.__sseFrames = []")
|
||||
page.fill("#message-input", question)
|
||||
page.click("#send-btn")
|
||||
# The user bubble lands synchronously with the submit handler.
|
||||
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
|
||||
|
||||
|
||||
def _wait_settled(page: Page, timeout_ms: int = 120_000) -> None:
|
||||
"""The turn is complete: answer text in the bubble, button
|
||||
recovered (the phase-48 settle wait)."""
|
||||
expect(page.locator(".msg.brain .bubble").last).not_to_have_text(
|
||||
"", timeout=timeout_ms
|
||||
)
|
||||
expect(page.locator("#send-btn")).to_be_enabled(timeout=timeout_ms)
|
||||
expect(page.locator("#send-label")).to_have_text("Send", timeout=timeout_ms)
|
||||
|
||||
|
||||
def _bubble_text(page: Page) -> str:
|
||||
"""The turn's rendered answer (the brain bubble's text content
|
||||
— the mock's scripted echo lands here: the house scripted-turn
|
||||
lens on the read result's ``origin:``/``branch:`` lines)."""
|
||||
return page.locator(".msg.brain .bubble").last.text_content() or ""
|
||||
|
||||
|
||||
def _open_modal_for_chip(page: Page, chip_text: str) -> None:
|
||||
"""Drive the story's modal surface: the turn's cited chip
|
||||
(phase 119 A1 — done.sources is the READ docs, so the read
|
||||
document is the turn's ONLY chip) opens the same-page document
|
||||
modal (the ``test_document_viewer`` contract)."""
|
||||
chip = page.locator(".msg.brain .source-chip", has_text=chip_text)
|
||||
expect(chip).to_have_count(1, timeout=30_000)
|
||||
chip.first.click()
|
||||
expect(page.locator("#doc-modal")).not_to_have_attribute("hidden")
|
||||
expect(page.locator("#doc-modal")).to_be_visible()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 1. The answer carries the repository link (the model-side
|
||||
# contract, through the real read result's ``origin:`` line)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_answer_carries_origin_link(
|
||||
page: Page, app_url: str, seeded_kb: None, db_ready: None
|
||||
) -> None:
|
||||
_seed_git_row(branch=None)
|
||||
page.set_default_timeout(60_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_page_hooks(page)
|
||||
|
||||
_submit(page, Q_SKILLS)
|
||||
_wait_settled(page)
|
||||
|
||||
frames = _frames(page)
|
||||
# The scripted read on the wire — the turn's only tool call.
|
||||
assert _tool_frames(frames) == [
|
||||
{"type": "tool", "name": "read", "argument": SKILLS_SP}
|
||||
], _tool_frames(frames)
|
||||
done = next(f for f in frames if f.get("type") == "done")
|
||||
assert done["deflected"] is False, done
|
||||
# The read document is the turn's cited source (the chip the
|
||||
# modal tests click — phase 119 A1).
|
||||
assert any(
|
||||
s["source"] == SOURCE and s["path"] == SKILLS_REL for s in done["sources"]
|
||||
), done["sources"]
|
||||
|
||||
# The read result's surface (the mock's verbatim echo in the
|
||||
# answer — the house scripted-turn lens): the sanitized
|
||||
# ``origin:`` line the SERVER put in the result the model saw
|
||||
# (the task-01 resolver's output — the git row's URL,
|
||||
# credential-free).
|
||||
text = _bubble_text(page)
|
||||
assert f"origin: {ORIGIN_URL}" in text, text
|
||||
# The done answer carries the repository URL (the scripted
|
||||
# echo — the model-side contract, deterministic): the mock
|
||||
# parsed the URL from the origin line — it cannot appear in
|
||||
# the answer unless the server sent it.
|
||||
assert ANSWER_SENTENCE in text, text
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 2. The modal's meta row shows the "Original" external anchor
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_modal_shows_original_anchor(
|
||||
page: Page, app_url: str, seeded_kb: None, db_ready: None
|
||||
) -> None:
|
||||
_seed_git_row(branch=None)
|
||||
page.set_default_timeout(60_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_page_hooks(page)
|
||||
|
||||
_submit(page, Q_SKILLS)
|
||||
_wait_settled(page)
|
||||
_open_modal_for_chip(page, SKILLS_REL)
|
||||
expect(page.locator("#doc-modal-title")).to_have_text(
|
||||
"Phased execution for pi.dev"
|
||||
)
|
||||
|
||||
# The meta row's "Original" anchor — the task-03 contract
|
||||
# (ONE shared renderDocument site: the modal AND
|
||||
# /document.html show it).
|
||||
anchor = page.locator("#doc-modal-meta .doc-origin")
|
||||
expect(anchor).to_have_count(1)
|
||||
expect(anchor).to_have_attribute("href", ORIGIN_URL)
|
||||
expect(anchor).to_have_attribute("target", "_blank")
|
||||
expect(anchor).to_have_attribute("rel", "noopener")
|
||||
# The copyable-link contract: the full URL VERBATIM as the
|
||||
# visible text — NO branch in the text (branch=None), and the
|
||||
# title is the bare URL (the pathCode precision idiom, no
|
||||
# branch suffix).
|
||||
expect(anchor).to_have_text(ORIGIN_URL)
|
||||
expect(anchor).to_have_attribute("title", ORIGIN_URL)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 3. The pinned branch rides the read result and the anchor's title
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_pinned_branch_in_title(
|
||||
page: Page, app_url: str, seeded_kb: None, db_ready: None
|
||||
) -> None:
|
||||
_seed_git_row(branch=PINNED_BRANCH)
|
||||
page.set_default_timeout(60_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_page_hooks(page)
|
||||
|
||||
_submit(page, Q_SKILLS)
|
||||
_wait_settled(page)
|
||||
|
||||
# The read result's surface (the mock's verbatim echo): the
|
||||
# origin line AND the pinned branch line (the task-01 second
|
||||
# line, only when the row pins a branch).
|
||||
text = _bubble_text(page)
|
||||
assert f"origin: {ORIGIN_URL}" in text, text
|
||||
assert f"branch: {PINNED_BRANCH}" in text, text
|
||||
# The done answer still carries the repository URL (tests
|
||||
# 1–3's model-side contract — the branch never joins the
|
||||
# URL).
|
||||
assert ANSWER_SENTENCE in text, text
|
||||
|
||||
_open_modal_for_chip(page, SKILLS_REL)
|
||||
anchor = page.locator("#doc-modal-meta .doc-origin")
|
||||
expect(anchor).to_have_count(1)
|
||||
expect(anchor).to_have_attribute("href", ORIGIN_URL)
|
||||
# The text stays the bare URL (copyable — the branch never
|
||||
# joins the anchor text)…
|
||||
expect(anchor).to_have_text(ORIGIN_URL)
|
||||
# …the branch rides the title (the task-03 label contract).
|
||||
expect(anchor).to_have_attribute(
|
||||
"title", f"{ORIGIN_URL} — branch: {PINNED_BRANCH}"
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 4. The local-source control is byte-identical (no ``origin:``
|
||||
# line in the read result, no anchor in the meta row)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_local_source_has_no_origin(
|
||||
page: Page, app_url: str, seeded_kb: None, db_ready: None
|
||||
) -> None:
|
||||
_seed_git_row(branch=None, exist=False) # the pre-phase world: NO git rows at all
|
||||
page.set_default_timeout(60_000)
|
||||
login(page, app_url, next="/")
|
||||
_install_page_hooks(page)
|
||||
|
||||
_submit(page, Q_CONTROL)
|
||||
_wait_settled(page)
|
||||
|
||||
frames = _frames(page)
|
||||
assert _tool_frames(frames) == [
|
||||
{"type": "tool", "name": "read", "argument": CONTROL_SP}
|
||||
], _tool_frames(frames)
|
||||
done = next(f for f in frames if f.get("type") == "done")
|
||||
assert done["deflected"] is False, done
|
||||
|
||||
# The byte-identity guard on the read result's surface (the
|
||||
# mock's verbatim echo): the result carried NO origin line —
|
||||
# the local source's result is byte-identical to pre-phase,
|
||||
# so the mock's deterministic no-origin sentence answers.
|
||||
text = _bubble_text(page)
|
||||
assert ORIGIN_LINK_NO_ORIGIN in text, text
|
||||
assert "origin:" not in text, text
|
||||
assert "branch:" not in text, text
|
||||
|
||||
# …and the modal's meta row is the pre-phase badge row: NO
|
||||
# .doc-origin anchor (the content payload's origin fields are
|
||||
# absent on the wire — never null).
|
||||
_open_modal_for_chip(page, CONTROL_REL)
|
||||
expect(page.locator("#doc-modal-title")).to_have_text("Control notes")
|
||||
expect(page.locator("#doc-modal-meta .doc-origin")).to_have_count(0)
|
||||
# The pre-phase badges are untouched (the row still shows
|
||||
# source · format · path · created · indexed · chunks).
|
||||
expect(page.locator("#doc-modal-meta .doc-source-badge")).to_have_text(
|
||||
CONTROL
|
||||
)
|
||||
expect(page.locator("#doc-modal-meta .doc-path")).to_have_text(
|
||||
CONTROL_REL
|
||||
)
|
||||
expect(page.locator("#doc-modal-meta .doc-created")).not_to_be_empty()
|
||||
Reference in New Issue
Block a user