phase: 145_git_source_links
Build and Push Containers / build-and-push-app (push) Successful in 2m23s
Build and Push Containers / build-and-push-db (push) Successful in 13s

**Phase 145 final verification pass — all green, no code changes needed**

- **Verified (no defects found):** resolver `origin_for_source` + `_url_basename` (scp/`.git`/trailing-slash, sanitize, ambiguity WARNING); `origin:`/`branch:` lines in all four read shapes via shared `_origin_lines`; HIGH `<tools>` origin rule (verbatim + only-byte-change pins); `DocContent.origin_url/origin_branch` (absent-never-null serializer); `renderDocument` "Original" anchor (copyable verbatim URL, `_blank`/`noopener`, branch in title); mock scripted origin-link flow (disjoint trigger — grep-verified vs all other E2E files)
- **Tests:** `uv run pytest --cov=app --cov-report=term-missing` → 3402 passed, `app/` coverage **99%** (>90% gate); `uv run pytest tests/e2e/test_git_source_links.py -v --no-cov` → **4 passed** in isolation; `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
- **Completion criteria:** (1) git read results carry origin/branch, local byte-identical — ✅ (unit pins + E2E control); (2) HIGH `<tools>` rule pinned, LOW byte-identical — ✅; (3) modal AND /document.html show Original link, branch in label when pinned, nothing for local — ✅; (4) full suite green + coverage + E2E + lint/types — ✅; (5) no completed-phase behavior change (read header line 1, mock prefix, done frame/SourceRef untouched, LOW sha) — ✅
- **Extra regression check:** re-ran shared-mock E2E suites `test_read_line_ranges.py` (2 passed) and `test_document_viewer.py` (8 passed) — no cross-suite regressions from the `mock_llm.py` addition
- **Documentation:** README.md reviewed in full (env table ⊇ `.env.example`, pages/routes, config defaults, scripts, 17-color palette, 300 s guard, 2 s sync poll, phase-145 provenance section matches implementation) and AGENTS.md (72rem container, no-CDN test, smoke test, gitignore, gates) — **verified consistent, no changes needed**
- **Notable:** phase-116 screenshot PNG was regenerated during this phase's E2E runs (byte re-encode only, visually identical, correct no-anchor local-source meta row) — left in tree for the harness commit
- **Next pending phase:** `146_search_tool_scope` (then 147, 148, 149)
This commit is contained in:
2026-10-01 11:13:43 -04:00
parent 486ceb4019
commit 32a3ec4f5e
37 changed files with 2856 additions and 22 deletions
+330 -2
View File
@@ -269,8 +269,11 @@ def test_agent_tools_names_and_parameters() -> None:
"and `limit` (number of lines) to read exactly that "
"part. Do not re-read a document you have already read "
"in full — its full text is already in your prompt. "
"Call one tool at a time — wait for this result before "
"your next call."
"For a document from a git source, the result also "
"carries an `origin` line (and `branch` when pinned) — "
"include that original source in your answer (see your "
"tools instructions). Call one tool at a time — wait "
"for this result before your next call."
)
# The byte-preserved contracts inside the rewrite, pinned as
# substrings: the combined source/path identity, the
@@ -293,6 +296,17 @@ def test_agent_tools_names_and_parameters() -> None:
"Call one tool at a time — wait for this result before "
"your next call."
) in read["description"]
# Phase 145 (task 02): the origin clause mirrors the TOOLS_SECTION
# rule (the description points, the section teaches — the
# phase-124 show_image "see the instructions" shape): a git-
# sourced document's read result carries an `origin` line (and
# `branch` when pinned); include that original source in the
# answer.
assert (
"For a document from a git source, the result also carries "
"an `origin` line (and `branch` when pinned) — include that "
"original source in your answer (see your tools instructions)."
) in read["description"]
read_params = read["parameters"]
assert read_params["type"] == "object"
# Phase 131 (task 01, LOCKED A2): offset/limit are OPTIONAL —
@@ -3649,6 +3663,320 @@ def test_unknown_tool_name_refused(monkeypatch: pytest.MonkeyPatch) -> None:
assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered
# ---------- phase 145 (task 01): the origin/branch lines in the read result ----------
#: The fixture git row's URL (basename ``skills`` = the fixture source
#: name) and the four origin outcomes the read-branch matrix runs:
#: git row matched (branch NULL), git row + pinned branch, local source
#: (no git row), two rows sharing the basename (ambiguous — the source
#: name is ``repo`` for the last one).
_GIT_URL = "https://github.com/owner/skills"
_GIT_ROWS = [GitSource(url=_GIT_URL, kind="git")]
_GIT_PINNED_ROWS = [GitSource(url=_GIT_URL, kind="git", branch="release/1.2")]
_LOCAL_ROWS = [GitSource(url="/srv/skills", kind="local", path="/srv/skills")]
_AMBIGUOUS_ROWS = [
GitSource(url="https://a.example/org/repo.git", kind="git"),
GitSource(url="git@b.example:other/repo.git", kind="git"),
]
def _git_rows_factory(rows: list[GitSource]) -> Callable[[], Session]:
"""A db factory whose mock session answers the resolver's query with
the git rows of *rows* (like the real ``kind = 'git'`` WHERE clause
— the seeded local rows are filtered out, proving they are never
candidates): the REAL ``origin_for_source`` runs in the read branch,
no database."""
git_rows = [row for row in rows if row.kind == "git"]
def _factory() -> Session:
mock = cast("Session", MagicMock())
mock.__enter__ = MagicMock(return_value=mock)
mock.__exit__ = MagicMock(return_value=False)
mock.scalars = MagicMock(return_value=MagicMock(all=MagicMock(return_value=git_rows)))
return mock
return _factory
def _one_read(
monkeypatch: pytest.MonkeyPatch,
doc: Document,
rows: list[GitSource],
arguments: dict[str, Any],
read_max_chars: int | None = None,
) -> str:
"""One scripted read turn against the REAL resolver; returns the
exact tool-result string (the successful-read bookkeeping is
asserted — the matrix asserts the bytes)."""
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="read", arguments=arguments)],
[StreamPiece("content", "ans")],
)
kwargs: dict[str, Any] = {"read_max_chars": read_max_chars} if read_max_chars else {}
asyncio.run(_run(llm, holder, _settings(**kwargs), db_factory=_git_rows_factory(rows)))
assert holder.read_docs == [doc] and holder.tool_calls == 1
return llm.requests[1][0][3]["content"]
def test_origin_lines_plain_read_under_cap(
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""Shape 1 (plain read at or under the cap): the origin block rides
IMMEDIATELY AFTER the ``date:`` line — ``Document …: / date: /
origin: / branch: / content`` — in all four origin outcomes; the
first line (the mock's ``_READ_RESULT_PREFIX`` contract) is
untouched, and the local/ambiguous results are byte-identical to
pre-phase (the ambiguity still warns)."""
doc = _doc("skills", "docs/a.md", "A", "A-CONTENT")
plain = {"path": "skills/docs/a.md"}
result = _one_read(monkeypatch, doc, _GIT_ROWS, plain)
assert result == (
"Document skills/docs/a.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"A-CONTENT"
)
# The E2E mock's header contract: first line, byte-identical.
assert result.startswith("Document ")
assert result.splitlines()[0] == "Document skills/docs/a.md:"
result = _one_read(monkeypatch, doc, _GIT_PINNED_ROWS, plain)
assert result == (
"Document skills/docs/a.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"branch: release/1.2\n"
"A-CONTENT"
)
result = _one_read(monkeypatch, doc, _LOCAL_ROWS, plain)
assert result == "Document skills/docs/a.md:\ndate: 2024-06-15\nA-CONTENT"
repo_doc = _doc("repo", "docs/a.md", "A", "A-CONTENT")
caplog.clear()
result = _one_read(monkeypatch, repo_doc, _AMBIGUOUS_ROWS, {"path": "repo/docs/a.md"})
assert result == "Document repo/docs/a.md:\ndate: 2024-06-15\nA-CONTENT"
# The ambiguity is logged at WARNING (the agent's INFO per-turn log
# lines are captured too — filter to the resolver's records).
warnings = [
r for r in caplog.records if r.levelname == "WARNING" and r.name == "app.rag.git_sources"
]
assert len(warnings) == 1
assert "repo" in warnings[0].message
def test_origin_lines_truncated_plain_read(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Shape 2 (plain read cut at the cap): the origin block sits at the
same position (after ``date:``, before the cut content), the
truncation marker + the pinned notice unchanged, the holder
truncation tuple unchanged (the ``read_truncations`` signal is
untouched)."""
cap = 20
content = "x" * (cap + 1)
doc = _doc("skills", "big.md", "Big", content)
plain = {"path": "skills/big.md"}
result = _one_read(monkeypatch, doc, _GIT_ROWS, plain, read_max_chars=cap)
assert result == (
"Document skills/big.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
+ content[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(content))
)
result = _one_read(monkeypatch, doc, _GIT_PINNED_ROWS, plain, read_max_chars=cap)
assert result == (
"Document skills/big.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"branch: release/1.2\n"
+ content[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(content))
)
result = _one_read(monkeypatch, doc, _LOCAL_ROWS, plain, read_max_chars=cap)
assert result == (
"Document skills/big.md:\n"
"date: 2024-06-15\n"
+ content[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(content))
)
repo_doc = _doc("repo", "big.md", "Big", content)
result = _one_read(
monkeypatch, repo_doc, _AMBIGUOUS_ROWS, {"path": "repo/big.md"}, read_max_chars=cap
)
assert result == (
"Document repo/big.md:\n"
"date: 2024-06-15\n"
+ content[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(content))
)
def test_origin_lines_range_read_under_cap(
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""Shape 3 (range read at or under the cap): the origin block rides
between the ``date:`` line and the ``lines:`` header (the same
position as in the plain shapes); the local/ambiguous results stay
byte-identical to pre-phase."""
doc = _doc("skills", "b.md", "B", "one\ntwo\nthree\nfour\nfive")
ranged = {"path": "skills/b.md", "offset": 2, "limit": 2}
result = _one_read(monkeypatch, doc, _GIT_ROWS, ranged)
assert result == (
"Document skills/b.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"lines: 2-3 of 5\n"
"two\nthree"
)
result = _one_read(monkeypatch, doc, _GIT_PINNED_ROWS, ranged)
assert result == (
"Document skills/b.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"branch: release/1.2\n"
"lines: 2-3 of 5\n"
"two\nthree"
)
result = _one_read(monkeypatch, doc, _LOCAL_ROWS, ranged)
assert result == "Document skills/b.md:\ndate: 2024-06-15\nlines: 2-3 of 5\ntwo\nthree"
repo_doc = _doc("repo", "b.md", "B", "one\ntwo\nthree\nfour\nfive")
caplog.clear()
result = _one_read(
monkeypatch, repo_doc, _AMBIGUOUS_ROWS, {"path": "repo/b.md", "offset": 2, "limit": 2}
)
assert result == "Document repo/b.md:\ndate: 2024-06-15\nlines: 2-3 of 5\ntwo\nthree"
# The ambiguity is logged at WARNING (the agent's INFO per-turn log
# lines are captured too — filter to the resolver's records).
warnings = [
r for r in caplog.records if r.levelname == "WARNING" and r.name == "app.rag.git_sources"
]
assert len(warnings) == 1
assert "repo" in warnings[0].message
def test_origin_lines_truncated_range_read(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Shape 4 (range read whose SLICE exceeds the cap): the origin block
at the same position; the cap on the slice + the shared marker + the
pinned notice (``{total}`` = the SLICE's length) all unchanged."""
cap = 40
doc = _doc("skills", "c.md", "C", "a" * 30 + "\n" + "b" * 30 + "\n" + "c" * 30)
slice_text = "a" * 30 + "\n" + "b" * 30 # the requested lines 1-2 (61 chars)
ranged = {"path": "skills/c.md", "offset": 1, "limit": 2}
result = _one_read(monkeypatch, doc, _GIT_ROWS, ranged, read_max_chars=cap)
assert result == (
"Document skills/c.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"lines: 1-2 of 3\n"
+ slice_text[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
)
result = _one_read(monkeypatch, doc, _GIT_PINNED_ROWS, ranged, read_max_chars=cap)
assert result == (
"Document skills/c.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"branch: release/1.2\n"
"lines: 1-2 of 3\n"
+ slice_text[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
)
result = _one_read(monkeypatch, doc, _LOCAL_ROWS, ranged, read_max_chars=cap)
assert result == (
"Document skills/c.md:\n"
"date: 2024-06-15\n"
"lines: 1-2 of 3\n"
+ slice_text[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
)
repo_doc = _doc("repo", "c.md", "C", "a" * 30 + "\n" + "b" * 30 + "\n" + "c" * 30)
result = _one_read(
monkeypatch, repo_doc, _AMBIGUOUS_ROWS, {"path": "repo/c.md", "offset": 1, "limit": 2},
read_max_chars=cap,
)
assert result == (
"Document repo/c.md:\n"
"date: 2024-06-15\n"
"lines: 1-2 of 3\n"
+ slice_text[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
)
def test_origin_lines_image_doc_keeps_marker_position(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The image-doc shape keeps its phase-122 pin: the marker stays
IMMEDIATELY ABOVE the content, and the origin lines ride ABOVE the
marker (``Document …: / date: / origin: / branch: / marker /
content``); a local image doc is byte-identical to pre-phase."""
doc = _doc("skills", "img.png", "Img", "IMG-DESC")
doc.is_image = True
plain = {"path": "skills/img.png"}
result = _one_read(monkeypatch, doc, _GIT_PINNED_ROWS, plain)
assert result == (
"Document skills/img.png:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"branch: release/1.2\n"
f"{agent.IMAGE_DOC_MARKER}\n"
"IMG-DESC"
)
doc = _doc("skills", "img.png", "Img", "IMG-DESC")
doc.is_image = True
result = _one_read(monkeypatch, doc, _LOCAL_ROWS, plain)
assert result == (
"Document skills/img.png:\ndate: 2024-06-15\n"
f"{agent.IMAGE_DOC_MARKER}\n"
"IMG-DESC"
)
# ---------- grep (the phase-68 A5 contract under the new name) ----------
+362 -8
View File
@@ -20,9 +20,29 @@ Frontend side:
* phase 26 — the shared ``renderDocument`` export in ``document.js``, the
import-safe page guard, the modal skeleton on chat + Sources, and the
modal module's close/focus/URL contract.
Phase 145 (task 03) — the content payload's git origin fields + the
``renderDocument`` "Original" anchor (modal AND /document.html, one
site):
* the ``DocContent`` omission rule — both origin keys ABSENT on the
wire when ``None`` (never ``null``; non-git payload byte-identical);
* the content endpoint (stubbed session) — a matching ``kind='git'``
row adds ``origin_url`` (+ ``origin_branch`` for a pinned row), a
legacy embedded-credential row is served sanitized, and local /
ambiguous sources keep the pre-phase key set;
* the anchor itself (source pins + a node behavioral check over a
minimal fake DOM) — ``createElement("a")``, the full URL verbatim as
``href`` AND visible text (the copyable-link contract),
``target="_blank" rel="noopener"``, the ``title`` = full URL (+
`` — branch: <name>`` when pinned), inserted AFTER the path code and
BEFORE the Created badge (absent → the pre-phase six children), no
innerHTML/outerHTML (the module's ONE innerHTML survives), and the
``.doc-origin`` CSS mirroring the ``.doc-path`` ellipsis idiom with
the ``.doc-link`` affordance (+ the ≤640px squeeze).
"""
from __future__ import annotations
import json
import re
import shutil
import subprocess
@@ -35,7 +55,8 @@ from fastapi.testclient import TestClient
from app.api.docs import doc_format
from app.db import get_db
from app.main import create_app
from app.models import Document
from app.models import Document, GitSource
from app.schemas import DocContent
from tests.conftest import ADMIN_PASSWORD
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
@@ -126,22 +147,39 @@ class _FakeResult:
return self._row
class _FakeScalars:
"""The resolver's ``db.scalars(...).all()`` — only the git rows
ever reach the caller (the ``kind = 'git'`` WHERE clause is
emulated, the task-01 ``_GitOnlySession`` pattern)."""
def __init__(self, rows: tuple[GitSource, ...]) -> None:
self._rows = [row for row in rows if row.kind == "git"]
def all(self) -> list[GitSource]:
return self._rows
class _FakeSession:
def __init__(self, row: object) -> None:
def __init__(self, row: object, git_rows: tuple[GitSource, ...] = ()) -> None:
self._row = row
self._git_rows = git_rows
def execute(self, _stmt: object) -> _FakeResult:
return _FakeResult(self._row)
def scalars(self, _stmt: object) -> _FakeScalars:
return _FakeScalars(self._git_rows)
def _client_with_row(row: object) -> TestClient:
def _client_with_row(row: object, git_rows: tuple[GitSource, ...] = ()) -> TestClient:
"""Fresh app whose ``get_db`` dependency is a stub returning ``row``
(``None`` → no matching document row). Phase 79: the endpoint is
user-gated, so the client signs in as the admin first — these tests
pin the CONTENT mapping (200/404/422), not the auth contract (which
``test_auth_api.py`` pins)."""
(``None`` → no matching document row) and, for the phase-145
origin lookup, *git_rows* (default: no rows — the non-git corner).
Phase 79: the endpoint is user-gated, so the client signs in as
the admin first — these tests pin the CONTENT mapping (200/404/
422), not the auth contract (which ``test_auth_api.py`` pins)."""
app = create_app()
app.dependency_overrides[get_db] = lambda: _FakeSession(row)
app.dependency_overrides[get_db] = lambda: _FakeSession(row, git_rows)
client = TestClient(app)
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login failed: {r.status_code} {r.text}"
@@ -217,6 +255,322 @@ def test_content_requires_both_params() -> None:
assert client.get("/api/documents/content", params={"source": "Homelab"}).status_code == 422
# ---------------------------------------------------------------------------
# Phase 145 (task 03) — the content payload's git origin fields + the
# renderDocument "Original" anchor (modal AND /document.html, one site)
# ---------------------------------------------------------------------------
def _text_doc() -> Document:
"""The content-endpoint stub's document row (a markdown doc — no
summary, the pre-phase shape)."""
doc = Document(
source="Homelab",
path="notes/deep mark.md",
full_path="/tmp/deep mark.md",
title="Deep Mark",
content="# Deep Mark\n\nbody",
content_hash="f" * 64,
is_image=False, # phase 122: the stub session never applies defaults
)
doc.indexed_at = datetime(2026, 8, 22, 1, 2, 3, tzinfo=UTC)
doc.created_at = datetime(2026, 8, 20, 9, 0, 0, tzinfo=UTC)
return doc
def _text_doc_content(
origin_url: str | None = None, origin_branch: str | None = None
) -> DocContent:
"""One text doc's content payload (the pre-phase shape) with the
phase-145 origin fields optionally set."""
return DocContent(
source="S",
path="a.md",
title="A",
format="md",
created_at="2026-01-01T00:00:00+00:00",
content="body",
indexed_at="2026-01-01T00:00:00+00:00",
chunks=1,
origin_url=origin_url,
origin_branch=origin_branch,
)
def test_doc_content_origin_fields_absent_on_the_wire_when_unset() -> None:
"""The model-level omission rule (the :func:`_drop_absent_share_url`
precedent, the ``image_url`` shape): a non-git document (no
``origin_url``) serializes with BOTH origin keys ABSENT — never
``null`` — so the payload is byte-identical to pre-phase. A
sanitized-URL-only payload (an unpinned row) drops
``origin_branch``; a pinned row keeps both keys."""
dumped = _text_doc_content().model_dump()
assert "origin_url" not in dumped # absent — never null
assert "origin_branch" not in dumped # absent — never null
dumped = _text_doc_content(origin_url="https://host/x.git").model_dump()
assert dumped["origin_url"] == "https://host/x.git"
assert "origin_branch" not in dumped # NULL = the default branch — nothing to show
dumped = _text_doc_content(
origin_url="https://host/x.git", origin_branch="docs"
).model_dump()
assert dumped["origin_branch"] == "docs"
def test_content_origin_url_carried_for_a_matching_git_row() -> None:
"""The endpoint (stubbed session, no database): a ``kind='git'``
row whose URL basename matches the document's source adds
``origin_url`` — the sanitized clone URL — to the payload, and
every pre-phase key stands untouched. The row pins no branch →
``origin_branch`` is ABSENT (never ``null``)."""
row = GitSource(url="https://github.com/reese/Homelab.git", kind="git")
with _client_with_row((_text_doc(), 3), git_rows=(row,)) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "notes/deep mark.md"},
)
assert r.status_code == 200
body = r.json()
assert set(body) == {
"source", "path", "title", "format", "summary", "created_at",
"content", "indexed_at", "chunks", "is_image", "origin_url",
}
assert body["origin_url"] == "https://github.com/reese/Homelab.git"
assert "origin_branch" not in body # absent — never null (unpinned row)
def test_content_origin_branch_carried_for_a_pinned_row() -> None:
"""A pinned row (phase 135 ``branch``) also carries
``origin_branch`` — the branch the title attribute will show."""
row = GitSource(
url="https://github.com/reese/Homelab.git", kind="git", branch="docs"
)
with _client_with_row((_text_doc(), 3), git_rows=(row,)) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "notes/deep mark.md"},
)
body = r.json()
assert body["origin_url"] == "https://github.com/reese/Homelab.git"
assert body["origin_branch"] == "docs"
def test_content_origin_url_sanitizes_a_legacy_embedded_credential_row() -> None:
"""End-to-end (stubbed): a legacy row whose credential is still
embedded in the stored URL never leaks it — the payload carries
the phase-121 sanitized URL (the resolver's guarantee, one mapping
for the whole app)."""
row = GitSource(url="https://user:tok@example.com/reese/Homelab.git", kind="git")
with _client_with_row((_text_doc(), 3), git_rows=(row,)) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "notes/deep mark.md"},
)
body = r.json()
assert body["origin_url"] == "https://example.com/reese/Homelab.git"
assert "user:tok" not in r.text # the credential stays out of the wire
def test_content_origin_fields_absent_for_local_and_ambiguous_sources() -> None:
"""No git row (a plain local source) OR two rows sharing the
basename (ambiguous — never guess) → both origin keys ABSENT from
the JSON (not ``null`` — the omission pinned on the wire; the
payload is byte-identical to pre-phase, the ``test_content_known_
pair_maps_to_doc_content`` key-set pin)."""
local_only = (_text_doc(), 3)
with _client_with_row(local_only) as client: # no git rows at all
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "notes/deep mark.md"},
)
assert r.status_code == 200
assert "origin_url" not in r.json()
assert "origin_branch" not in r.json()
ambiguous = (
GitSource(url="https://a.example/org/Homelab.git", kind="git"),
GitSource(url="git@b.example:other/Homelab.git", kind="git"),
)
with _client_with_row(local_only, git_rows=ambiguous) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "notes/deep mark.md"},
)
assert r.status_code == 200
assert "origin_url" not in r.json()
assert "origin_branch" not in r.json()
# ---------- the renderDocument "Original" anchor (the shared core) ----------
def _fn(js: str, name: str) -> str:
"""The source of a top-level function via balanced-brace counting
(the ``test_date_editor.py`` pattern)."""
start = js.find(f"function {name}(")
assert start != -1, f"{name}() must exist in document.js"
i = js.find("(", start)
depth = 0
close = i
while i < len(js):
if js[i] == "(":
depth += 1
elif js[i] == ")":
depth -= 1
if depth == 0:
close = i
break
i += 1
brace = js.find("{", close)
depth = 0
for j in range(brace, len(js)):
if js[j] == "{":
depth += 1
elif js[j] == "}":
depth -= 1
if depth == 0:
return js[start : j + 1]
raise AssertionError(f"unbalanced braces in {name}()")
def test_origin_anchor_is_a_copyable_external_link() -> None:
"""The anchor (house-style source contract): ``createElement("a")``
(the never-build-HTML rule), the ``doc-origin`` class, ``href =
doc.origin_url``, ``target="_blank" rel="noopener"`` (the modal's
"Full page" escape-hatch idiom), the full URL VERBATIM as the
visible text (the copyable-link contract — no decorative label),
and the ``title`` = the full URL, plus `` — branch: <name>`` when
the payload pins a branch (the pathCode precision idiom)."""
body = _fn(_read(DOCUMENT_JS), "originAnchor")
assert 'document.createElement("a")' in body
assert 'a.className = "doc-origin"' in body
assert "a.href = doc.origin_url" in body
assert 'a.target = "_blank"' in body
assert 'a.rel = "noopener"' in body
assert "a.textContent = doc.origin_url" in body, (
"the visible text is the full URL verbatim (copy-pasteable)"
)
branch_title = 'doc.origin_url + " — branch: " + doc.origin_branch'
assert 'doc.origin_branch ? ' + branch_title + " : doc.origin_url" in body, (
"the title carries the full URL (+ the pinned branch when present)"
)
# The never-build-HTML shape: no innerHTML/outerHTML in the new code.
code = re.sub(r"//.*?$|/\*.*?\*/", "", body, flags=re.S | re.M)
assert "innerHTML" not in code and "outerHTML" not in code
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_origin_anchor_behaves_for_branch_and_no_branch_variants() -> None:
"""Behavioral check (node, minimal fake DOM) of the real anchor
builder: the exact ``href``/``target``/``rel``/``textContent``/
``title`` for a git doc, branch and no-branch variants."""
fn = _fn(_read(DOCUMENT_JS), "originAnchor")
fake_dom = (
"globalThis.document = { createElement: (tag) => {"
' const el = { tag, className: "", href: "", target: "", rel: "",'
' textContent: "", setAttribute(k, v) { this[k] = v; } };'
" return el; } };"
)
out = _run_node(
fake_dom
+ "\n" + fn
+ '\nconst g1 = originAnchor({ origin_url: "https://github.com/reese/skills.git",'
' origin_branch: "main" });' + "\n"
+ '\nconst g2 = originAnchor({ origin_url: "https://github.com/reese/skills.git" });' + "\n"
+ "console.log(JSON.stringify(g1));\nconsole.log(JSON.stringify(g2));"
)
j1, j2 = (json.loads(line) for line in out.splitlines())
for j in (j1, j2):
assert j["tag"] == "a"
assert j["className"] == "doc-origin"
assert j["href"] == "https://github.com/reese/skills.git"
assert j["target"] == "_blank"
assert j["rel"] == "noopener"
# The full URL VERBATIM as the visible text (copy-pasteable).
assert j["textContent"] == "https://github.com/reese/skills.git"
# The title: the full URL, + the pinned branch when present.
assert j1["title"] == "https://github.com/reese/skills.git — branch: main"
assert j2["title"] == "https://github.com/reese/skills.git"
def test_origin_anchor_sits_after_the_path_code_before_the_created_badge() -> None:
"""renderDocument (the ONE shared core — modal AND /document.html):
the anchor joins the meta row AFTER the path ``code`` and BEFORE
the Created badge — and ONLY when the payload carries
``origin_url``. Absent (a non-git document — the field simply
missing on the wire) → the gate builds the empty array, the
spread adds nothing, and the meta row's children are EXACTLY the
pre-phase six (the existing badge-order pins pass unchanged)."""
render = _fn(_read(DOCUMENT_JS), "renderDocument")
gate_i = render.find("const originItems = doc.origin_url ? [originAnchor(doc)] : [];")
call_i = render.find("metaEl.replaceChildren(")
path_i = render.find("pathCode,", call_i)
spread_i = render.find("...originItems,", call_i)
created_i = render.find('metaBadge("doc-created"', call_i)
assert -1 < gate_i < call_i < path_i < spread_i < created_i, (
"the anchor gate precedes the row build and lands AFTER the path code, "
"BEFORE the Created badge"
)
# The gate: present only on a truthy origin_url — an ABSENT field
# (undefined) is falsy → [] → the pre-phase children stand.
assert "doc.origin_url ? [originAnchor(doc)] : []" in render
# The spread is the meta row's ONLY conditional child (the six
# pre-phase children are the fixed ones).
assert render[call_i:].count("...") == 1
def test_origin_anchor_keeps_the_module_xss_contract() -> None:
"""The module's ONE innerHTML (the escape-first markdown render)
survives — the new anchor adds no HTML-building of its own
(createElement + properties/textContent only)."""
code = re.sub(r"//.*", "", re.sub(r"/\*.*?\*/", "", _read(DOCUMENT_JS), flags=re.S))
assert code.count("innerHTML") == 1
assert "outerHTML" not in code
def test_doc_origin_css_mirrors_the_doc_path_ellipsis_idiom() -> None:
"""styles.css: the ``.doc-origin`` rule sits next to the
``.doc-path`` family (mono, 0.75rem, the same 26rem ellipsis
budget + 6rem floor) with the ``.doc-link`` affordance
(brand-ink, underline on hover/focus — the existing link
tokens), provenance-cited, and the ≤640px block squeezes it like
the path (16rem — the row's own wrap/clip rules are unchanged:
the anchor is one more inline item)."""
css = _read(FRONTEND / "assets" / "styles.css")
rule_i = css.find(".doc-origin {")
assert rule_i != -1, "styles.css must style .doc-origin"
# Provenance comment directly above the rule (house style).
comment_start = css.rfind("/*", 0, rule_i)
assert -1 < comment_start < rule_i and "Phase 145 (task 03)" in css[comment_start:rule_i]
# Next to the .doc-path family (before .doc-chunks).
assert css.find(".doc-path {") < rule_i < css.find(".doc-chunks {")
rule = css[rule_i : css.find("}", rule_i)]
for decl in (
"font-family: var(--mono)",
"font-size: 0.75rem",
"max-width: 26rem",
"white-space: nowrap",
"overflow: hidden",
"text-overflow: ellipsis",
"min-width: 6rem",
"color: var(--brand-ink)",
"text-decoration: none",
):
assert decl in rule, f".doc-origin must carry {decl!r}"
# The .doc-link hover/focus affordance (the existing link idiom).
assert ".doc-origin:hover, .doc-origin:focus-visible { text-decoration: underline; }" in css
# The ≤640px squeeze mirrors .doc-path's 16rem (same media block).
mobile = css.find("@media (max-width: 640px)")
path_sq = css.find(".doc-path { max-width: 16rem; }", mobile)
origin_sq = css.find(".doc-origin { max-width: 16rem; }", mobile)
close_640 = css.find("@media", mobile + 1)
assert -1 < path_sq < origin_sq < (close_640 if close_640 != -1 else len(css)), (
"the mobile block squeezes the anchor like the path"
)
assert "url(http" not in css and "@import url(" not in css, (
"no CDN (AGENTS.md rule 6)"
)
# ---------------------------------------------------------------------------
# Viewer URL builder — encoded query (spaces/slashes in real paths)
# ---------------------------------------------------------------------------
+121
View File
@@ -179,3 +179,124 @@ def test_alias_env_fallback_unchanged(monkeypatch: pytest.MonkeyPatch) -> None:
assert urls == ["https://env.example/one.git"]
assert origin == "env"
# --- phase 145 (task 01): origin_for_source (the read result's origin) ---
class _GitOnlySession:
"""The origin resolver's session (phase 145, task 01): emulates the
resolver's ``kind = 'git'`` WHERE clause — only the git rows ever
reach the caller, exactly like the real query; the seeded local
rows prove they are never candidates."""
def __init__(self, rows: list[GitSource]) -> None:
self._rows = [row for row in rows if row.kind == "git"]
self.statements: list[Any] = []
def scalars(self, stmt: Any) -> _FakeScalars:
self.statements.append(stmt)
return _FakeScalars(self._rows)
# --- the basename mapping --------------------------------------------------
def test_origin_basename_mapping() -> None:
"""The URL's path component — after the ``://``, or after the ``:``
for scp-style ``git@host:owner/repo`` — trailing ``/`` stripped,
a single trailing ``.git`` suffix stripped, last segment taken. An
``@`` inside the PATH is not userinfo and stays (no split, no
strip)."""
cases = {
"https://github.com/owner/myrepo.git": "myrepo", # the .git suffix
"https://github.com/owner/myrepo": "myrepo", # no suffix, same dir
"https://github.com/owner/myrepo/": "myrepo", # trailing slash
"https://github.com/owner/myrepo.git/": "myrepo", # both
"git@github.com:owner/myrepo.git": "myrepo", # scp-style + suffix
"git@github.com:owner/myrepo": "myrepo", # scp-style, no suffix
"ssh://git@host/owner/myrepo.git": "myrepo", # ssh scheme, @ in the host part
"https://host/a@b/myrepo.git": "myrepo", # @ inside the PATH — not userinfo
}
for url, expected in cases.items():
assert resolver._url_basename(url) == expected, url
# --- the match: sanitize + branch ------------------------------------------
def test_origin_exact_match_is_sanitized() -> None:
"""EXACTLY ONE match → the phase-121 masked URL + the branch — a
legacy embedded-credential row never leaks its secret (the stored
value is untouched; the OUTPUT mask does the work)."""
session = _GitOnlySession(
[GitSource(url="https://user:tok@host/x.git", kind="git", branch=None)]
)
assert resolver.origin_for_source(session, "x") == ("https://host/x.git", None) # pyright: ignore[reportArgumentType]
def test_origin_branch_passthrough() -> None:
"""The branch rides through as-is: pinned → the name (the read
result's ``branch:`` line), NULL → ``None`` (the default branch —
no ``branch:`` line)."""
session = _GitOnlySession(
[GitSource(url="https://host/x.git", kind="git", branch="release/1.2")]
)
assert resolver.origin_for_source(session, "x") == ("https://host/x.git", "release/1.2") # pyright: ignore[reportArgumentType]
def test_origin_local_rows_never_origins() -> None:
"""Local rows are never origins (nothing to link to from the user's
browser): a source that exists ONLY as a local row gets no origin
(zero git candidates); a local + git pair resolves to the git row."""
local = GitSource(url="/srv/repo", kind="local", path="/srv/repo")
session = _GitOnlySession([local]) # the WHERE clause drops it
assert resolver.origin_for_source(session, "repo") is None # pyright: ignore[reportArgumentType]
session = _GitOnlySession(
[local, GitSource(url="https://host/repo.git", kind="git", branch=None)]
)
assert resolver.origin_for_source(session, "repo") == ("https://host/repo.git", None) # pyright: ignore[reportArgumentType]
# --- the no-guess rows: None + WARNING --------------------------------------
def test_origin_zero_match_is_none_with_warning(
caplog: pytest.LogCaptureFixture,
) -> None:
"""No git row's basename matches (a plain local source) → ``None``
+ a WARNING naming the source — the read result stays
byte-identical to pre-phase."""
session = _GitOnlySession([GitSource(url="https://host/other.git", kind="git")])
assert resolver.origin_for_source(session, "repo") is None # pyright: ignore[reportArgumentType]
warnings = [record for record in caplog.records if record.levelname == "WARNING"]
assert len(warnings) == 1
assert "repo" in warnings[0].message
def test_origin_ambiguous_is_none_with_warning(
caplog: pytest.LogCaptureFixture,
) -> None:
"""TWO-OR-MORE rows share the basename → ``None`` + a WARNING naming
the sanitized candidates (never guess — a wrong origin would ride
into every answer about the document). The candidates are logged
SANITIZED: a legacy embedded credential stays out of the log too."""
session = _GitOnlySession(
[
GitSource(url="https://user:tok@a.example/org/repo.git", kind="git"),
GitSource(url="git@b.example:other/repo.git", kind="git"),
]
)
assert resolver.origin_for_source(session, "repo") is None # pyright: ignore[reportArgumentType]
warnings = [record for record in caplog.records if record.levelname == "WARNING"]
assert len(warnings) == 1
assert "https://a.example/org/repo.git" in warnings[0].message
assert "git@b.example:other/repo.git" in warnings[0].message
assert "tok" not in warnings[0].message
+75 -6
View File
@@ -15,6 +15,7 @@ from __future__ import annotations
import hashlib
from app.rag.prompts import (
ORIGIN_RULE,
PERSONA,
SUGGEST_INTRO,
TOOLS_SECTION,
@@ -77,12 +78,18 @@ def test_high_and_low_bases_byte_locked() -> None:
#: form — do not re-read a document read IN FULL (answer from it);
#: a PARTIAL document (a range read, or a truncated read) is not
#: complete in the context until read all — read the other parts with
#: ``offset``/``limit``. Only that sentence moved — the prefix (the
#: ``ls``-clause opening) and the suffix (the discipline-rules
#: ending) survived byte-identical, so they are the same anchors as
#: pre-phase-141.
TOOLS_SECTION_SHA256 = "3671680bbf4e04efe1186690c18d24869e4974ceb29485cedc3e34a9286bf3b2"
TOOLS_SECTION_LEN = 2656
#: ``offset``/``limit``. RE-CUT again for phase 145 (task 02): the
#: ``read`` clause gains the original-source rule — :data:`ORIGIN_RULE`,
#: the one sentence-group appended at the ``read`` clause's end (the
#: truncation workflow sentence) and BEFORE the ``grep`` clause begins
#: (the ``origin:`` line is a read-result fact; the rule binds the
#: ANSWER that uses what the read added). Only that sentence-group was
#: inserted — the prefix (the ``ls``-clause opening) and the suffix
#: (the discipline-rules ending) survived byte-identical, so they are
#: the same anchors as pre-phase-145; the pre-phase-145 sha/length are
#: kept below for the diff-style pin.
TOOLS_SECTION_SHA256 = "41a84885be98d4e15b95205694d59d364296a30e27758d39ae737881d5bacaa7"
TOOLS_SECTION_LEN = 3279
TOOLS_SECTION_PREFIX = (
"<tools>\n"
"You may extend your context with three tools. `ls` lists the "
@@ -94,6 +101,33 @@ TOOLS_SECTION_SUFFIX = (
"you need.\n</tools>"
)
#: The pre-phase-145 anchors — the section BEFORE :data:`ORIGIN_RULE` was
#: inserted. Kept for the diff-style pin: the ONLY byte change is the
#: rule's insertion, so removing the rule reproduces these exact bytes.
TOOLS_SECTION_PRE_PHASE_145_SHA256 = (
"3671680bbf4e04efe1186690c18d24869e4974ceb29485cedc3e34a9286bf3b2"
)
TOOLS_SECTION_PRE_PHASE_145_LEN = 2656
#: Phase 145 (task 02) — the original-source rule the ``read`` teaching
#: carries is :data:`app.rag.prompts.ORIGIN_RULE` (imported above, LOCKED
#: verbatim). :data:`TOOLS_SECTION` embeds that exact sentence-group at
#: the ``read`` clause's end (the truncation workflow sentence) and
#: BEFORE the ``grep`` clause begins. A read result for a git-sourced
#: document carries an ``origin:`` line (and a ``branch:`` line when a
#: branch is pinned) — the read contract's fact — so when the answer uses
#: such a document it includes the original source in the answer: a link
#: to the repository (the ``origin`` line) with the document's path (the
#: ``Document …`` header) named and the pinned branch when one is shown,
#: as the repository's file-view URL where the model can tell it
#: (``<repo-url>/blob/<branch>/<path>``), or the repository URL plus the
#: path as plain text otherwise. The rule is CONDITIONAL on the ``origin``
#: line's presence — a local-sourced document's read result carries none,
#: so the rule adds nothing to those answers (the model has no URL to
#: cite; it must not invent one — the sentence's conditioning does that
#: job).
def test_tools_section_byte_locked() -> None:
"""The ``<tools>`` teaching is LOCKED verbatim too (the E2E mock keys
@@ -105,6 +139,41 @@ def test_tools_section_byte_locked() -> None:
assert TOOLS_SECTION.endswith(TOOLS_SECTION_SUFFIX)
def test_tools_section_origin_rule_verbatim_at_its_position() -> None:
"""Phase 145 (task 02): the original-source rule is LOCKED verbatim
and sits at its EXACT position — the ``read`` clause's end (the
truncation workflow sentence) and BEFORE the ``grep`` clause begins.
The sentence-group is pinned byte-for-byte, and the surrounding
boundary (the truncation sentence's tail → the rule → the ``grep``
clause's opening) is pinned as one contiguous substring, so a move
or re-spelling of ANY byte (the rule's or the boundary's) fails."""
# Verbatim presence.
assert ORIGIN_RULE in TOOLS_SECTION
assert TOOLS_SECTION.count(ORIGIN_RULE) == 1 # exactly once
# Exact position: after the truncation sentence, before the grep
# clause — one contiguous span, no drift at either seam.
assert (
"it searches the whole document. "
+ ORIGIN_RULE
+ " `grep` locates an exact string"
) in TOOLS_SECTION
def test_tools_section_origin_rule_is_the_only_byte_change() -> None:
"""Phase 145 (task 02) diff-style pin: the section's OTHER bytes are
byte-identical to pre-phase — the pre-phase section with ONLY the new
sentence-group inserted. Removing the rule (and the single space
that now separates it from the ``grep`` clause) reproduces the
pre-phase-145 section byte-for-byte (sha + length), and the length
delta is exactly the rule's length plus that one inserted space."""
assert len(TOOLS_SECTION) == (
TOOLS_SECTION_PRE_PHASE_145_LEN + len(ORIGIN_RULE) + 1
)
pre_phase = TOOLS_SECTION.replace(ORIGIN_RULE + " ", "")
assert len(pre_phase) == TOOLS_SECTION_PRE_PHASE_145_LEN
assert _sha256(pre_phase) == TOOLS_SECTION_PRE_PHASE_145_SHA256
# ---------- the LOW prompt's locked DEFLECT_MODE body ----------
#: The ``DEFLECT_MODE`` body exactly as it was pre-phase-112 (the
+38
View File
@@ -48,6 +48,7 @@ import pytest
from app.config import Settings
from app.models import Document
from app.rag.prompts import (
ORIGIN_RULE,
PERSONA,
SHOW_IMAGE_CLAUSE,
SUGGEST_INTRO,
@@ -463,6 +464,43 @@ def test_tools_section_phase72_clauses_in_high_prompt_not_low() -> None:
assert "including the source name" not in low
def test_tools_section_phase145_origin_rule() -> None:
"""Phase 145 (task 02): the ``<tools>`` ``read`` clause carries the
original-source rule (LOCKED verbatim in :data:`ORIGIN_RULE`) — when
the answer uses a document whose read result carries an ``origin:``
line, it includes the original source (a link to the repository
from the origin line, with the document's path and the pinned branch
where shown) so the user opens the complete original in one step.
The rule rides the HIGH prompt (``<tools>`` is HIGH-only) and never
the LOW/deflection prompt. The citation discipline is UNCHANGED —
the rule extends the ANSWERS of cited documents; it does not widen
the cite set (the "cite only what you read" sentence stays
byte-identical — pinned in ``test_prompt_lock``)."""
# The rule is present verbatim in the constant, exactly once, and at
# its exact position (the read clause's end, before the grep clause).
assert TOOLS_SECTION.count(ORIGIN_RULE) == 1
assert (
"it searches the whole document. "
+ ORIGIN_RULE
+ " `grep` locates an exact string"
) in TOOLS_SECTION
# It rides the built HIGH prompt with the rest of the section.
doc = _doc("kubernetes.md", "Talos Linux on three nodes.", "Kubernetes")
high = build_high_prompt([doc])
assert ORIGIN_RULE in high
# ...and never the LOW/deflection prompt (``<tools>`` is HIGH-only).
for low in (build_deflect_prompt(["T1"]), build_deflect_prompt([])):
assert ORIGIN_RULE not in low
# The citation discipline is UNCHANGED: the rule does not widen the
# cite set (the "cite only what you read" sentence is untouched).
assert (
"Cite only the document(s) you read — or, if you answered from a "
"suggested summary without reading it, cite that suggested "
"document — never a document you neither read nor used."
) in high
assert "Cite only the document(s) you read" in SUGGEST_INTRO
def test_documents_section_leads_with_the_suggest_intro() -> None:
"""Phase 118, task 03: the ``<documents>`` section leads with the
start-here :data:`SUGGEST_INTRO` line BEFORE the first block (the