Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 145_git_source_links
Build and Push Containers / build-and-push-app (push) Successful in 2m23s
Details
Build and Push Containers / build-and-push-db (push) Successful in 13s
Details

Browse Source 
**Phase 145 final verification pass — all green, no code changes needed**

- **Verified (no defects found):** resolver `origin_for_source` + `_url_basename` (scp/`.git`/trailing-slash, sanitize, ambiguity WARNING); `origin:`/`branch:` lines in all four read shapes via shared `_origin_lines`; HIGH `<tools>` origin rule (verbatim + only-byte-change pins); `DocContent.origin_url/origin_branch` (absent-never-null serializer); `renderDocument` "Original" anchor (copyable verbatim URL, `_blank`/`noopener`, branch in title); mock scripted origin-link flow (disjoint trigger — grep-verified vs all other E2E files)
- **Tests:** `uv run pytest --cov=app --cov-report=term-missing` → 3402 passed, `app/` coverage **99%** (>90% gate); `uv run pytest tests/e2e/test_git_source_links.py -v --no-cov` → **4 passed** in isolation; `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
- **Completion criteria:** (1) git read results carry origin/branch, local byte-identical — ✅ (unit pins + E2E control); (2) HIGH `<tools>` rule pinned, LOW byte-identical — ✅; (3) modal AND /document.html show Original link, branch in label when pinned, nothing for local — ✅; (4) full suite green + coverage + E2E + lint/types — ✅; (5) no completed-phase behavior change (read header line 1, mock prefix, done frame/SourceRef untouched, LOW sha) — ✅
- **Extra regression check:** re-ran shared-mock E2E suites `test_read_line_ranges.py` (2 passed) and `test_document_viewer.py` (8 passed) — no cross-suite regressions from the `mock_llm.py` addition
- **Documentation:** README.md reviewed in full (env table ⊇ `.env.example`, pages/routes, config defaults, scripts, 17-color palette, 300 s guard, 2 s sync poll, phase-145 provenance section matches implementation) and AGENTS.md (72rem container, no-CDN test, smoke test, gitignore, gates) — **verified consistent, no changes needed**
- **Notable:** phase-116 screenshot PNG was regenerated during this phase's E2E runs (byte re-encode only, visually identical, correct no-anchor local-source meta row) — left in tree for the harness commit
- **Next pending phase:** `146_search_tool_scope` (then 147, 148, 149)
This commit is contained in:
ducoterra
2026-10-01 11:13:43 -04:00
parent 486ceb4019
commit 32a3ec4f5e
37 changed files with 2856 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/todo/145_git_source_links/00_phase.md → .agents/phases/complete/145_git_source_links/00_phase.md
View File
.agents/phases/todo/145_git_source_links/01_origin_resolution.md → .agents/phases/complete/145_git_source_links/01_origin_resolution.md
View File
.agents/phases/todo/145_git_source_links/02_prompt_teaching.md → .agents/phases/complete/145_git_source_links/02_prompt_teaching.md
View File
.agents/phases/todo/145_git_source_links/03_content_payload_render_link.md → .agents/phases/complete/145_git_source_links/03_content_payload_render_link.md
View File
.agents/phases/todo/145_git_source_links/04_e2e_source_link.md → .agents/phases/complete/145_git_source_links/04_e2e_source_link.md
View File
.agents/reports/145_git_source_links/145_git_source_links__00_phase.a1.err
View File
.agents/reports/145_git_source_links/145_git_source_links__00_phase.a1.md
+9
View File
@@ -0,0 +1,9 @@
**Phase 145 final verification pass — all green, no code changes needed**
- **Verified (no defects found):** resolver `origin_for_source` + `_url_basename` (scp/`.git`/trailing-slash, sanitize, ambiguity WARNING); `origin:`/`branch:` lines in all four read shapes via shared `_origin_lines`; HIGH `<tools>` origin rule (verbatim + only-byte-change pins); `DocContent.origin_url/origin_branch` (absent-never-null serializer); `renderDocument` "Original" anchor (copyable verbatim URL, `_blank`/`noopener`, branch in title); mock scripted origin-link flow (disjoint trigger — grep-verified vs all other E2E files)
- **Tests:** `uv run pytest --cov=app --cov-report=term-missing` → 3402 passed, `app/` coverage **99%** (>90% gate); `uv run pytest tests/e2e/test_git_source_links.py -v --no-cov` → **4 passed** in isolation; `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings
- **Completion criteria:** (1) git read results carry origin/branch, local byte-identical — ✅ (unit pins + E2E control); (2) HIGH `<tools>` rule pinned, LOW byte-identical — ✅; (3) modal AND /document.html show Original link, branch in label when pinned, nothing for local — ✅; (4) full suite green + coverage + E2E + lint/types — ✅; (5) no completed-phase behavior change (read header line 1, mock prefix, done frame/SourceRef untouched, LOW sha) — ✅
- **Extra regression check:** re-ran shared-mock E2E suites `test_read_line_ranges.py` (2 passed) and `test_document_viewer.py` (8 passed) — no cross-suite regressions from the `mock_llm.py` addition
- **Documentation:** README.md reviewed in full (env table ⊇ `.env.example`, pages/routes, config defaults, scripts, 17-color palette, 300 s guard, 2 s sync poll, phase-145 provenance section matches implementation) and AGENTS.md (72rem container, no-CDN test, smoke test, gitignore, gates) — **verified consistent, no changes needed**
- **Notable:** phase-116 screenshot PNG was regenerated during this phase's E2E runs (byte re-encode only, visually identical, correct no-anchor local-source meta row) — left in tree for the harness commit
- **Next pending phase:** `146_search_tool_scope` (then 147, 148, 149)
.agents/reports/145_git_source_links/145_git_source_links__00_phase.a1.validate
+119
View File
@@ -0,0 +1,119 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
............... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 181 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 431 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 155 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 430 0 100%
--------------------------------------------------
TOTAL 5537 22 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/145_git_source_links/145_git_source_links__01_origin_resolution.a1.err
View File
.agents/reports/145_git_source_links/145_git_source_links__01_origin_resolution.a1.md
+17
View File
@@ -0,0 +1,17 @@
Task complete — all green.
**Implemented**
- `app/rag/git_sources.py`: `_url_basename` (path component after `://` or scp-style `:`, trailing `/` + single `.git` stripped, last segment) and `origin_for_source(db, source_name)` beside `sanitize_url` — `kind='git'` rows only, exact-match → `(sanitize_url(url), branch)`, zero/two-or-more → `None` + WARNING (sanitized candidates), pure-DB (no git subprocess).
- `app/rag/agent.py`: shared `_origin_lines` helper; origin computed ONCE per successful read before the ranged/truncated branching; all four result builders insert `origin:`/`branch:` immediately after `date:` (before the image marker / `lines:` header); first line and all non-git results byte-identical.
- Tests: 4-shape × 4-origin read-result matrix in `tests/unit/test_agent.py` (exact strings, first-line `_READ_RESULT_PREFIX` contract, phase-122 marker pin, caplog ambiguity WARNING, real resolver driven via a git-only mock session) + resolver units in `tests/unit/test_git_sources.py` (basename mapping incl. scp-style/path-`@`, legacy `user:tok@` masked, zero/ambiguous `None`+WARNING, branch passthrough, local rows never origins).
**Results**
- `uv run pytest --cov=app --cov-report=term-missing`: **3382 passed**, `app/` coverage **99%** (`git_sources.py` 100%, `agent.py` 99%)
- `uv run ruff check .` clean; `uv run pyright`: 0 errors
- Bonus: `uv run pytest tests/e2e/test_smoke.py -v --no-cov`: 3 passed (mock parse + SSE unaffected)
**Docs**: no doc impact — model-internal read-result lines only; user-visible "Original" link arrives in task 03.
**Decisions**: zero-match case also logs a WARNING (per task spec's "zero matches … → None + WARNING"); `origin:` sits above the `lines:` header in range shapes (strictly "immediately after the `date:` line").
**Next pending task**: `02_prompt_teaching.md` (the `<tools>` original-source rule + prompt pins).
.agents/reports/145_git_source_links/145_git_source_links__01_origin_resolution.a1.validate
+118
View File
@@ -0,0 +1,118 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
...................................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 179 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 431 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 154 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 424 0 100%
--------------------------------------------------
TOTAL 5528 22 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/145_git_source_links/145_git_source_links__02_prompt_teaching.a1.err
View File
.agents/reports/145_git_source_links/145_git_source_links__02_prompt_teaching.a1.md
+21
View File
@@ -0,0 +1,21 @@
Task 02 complete. All gates 100% green.
## Report
**Implemented**
- `app/rag/prompts.py`: new canonical `ORIGIN_RULE` constant; inserted the locked verbatim original-source rule into `TOOLS_SECTION` at the `read` clause's end (after the truncation workflow sentence, before the `grep` clause).
- `app/rag/agent.py`: `AGENT_TOOLS` `read` description gains the one-clause pointer ("the result also carries an `origin` line (and `branch` when pinned) — include that original source in your answer (see your tools instructions)").
- Pins: re-cut `TOOLS_SECTION` sha/len + kept pre-phase-145 anchors for a diff-style pin (rule is the *only* byte change); added verbatim+position, HIGH-only/LOW-absent, read-description-clause, and citation-discipline-unchanged tests.
- `README.md`: concise git-source-provenance note (origin/branch line + the answer link).
**Results**
- `uv run pytest` → **3385 passed, 0 failed**
- `uv run pytest --cov=app` → **app/ 99%** (>90%); `prompts.py` 100%, `agent.py` 99%
- `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors/warnings
- `uv run pytest tests/e2e/test_smoke.py -v --no-cov` → 3 passed (mock regression check)
**Docs**: README updated; AGENTS.md no impact (no command/config/convention change).
**Decisions**: `ORIGIN_RULE` made the single source of truth, cross-pinned to the `TOOLS_SECTION` literal (removing it reproduces pre-phase bytes byte-for-byte); kept the LOW prompt and the mock's `<tools>` marker keying untouched; README describes only the prompt teaching + origin line (the UI "Original" anchor is task 03, so not over-claimed).
**Next pending task**: `03_content_payload_render_link.md`.
.agents/reports/145_git_source_links/145_git_source_links__02_prompt_teaching.a1.validate
+119
View File
@@ -0,0 +1,119 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 179 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 431 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 155 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 424 0 100%
--------------------------------------------------
TOTAL 5529 22 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/145_git_source_links/145_git_source_links__03_content_payload_render_link.a1.err
View File
.agents/reports/145_git_source_links/145_git_source_links__03_content_payload_render_link.a1.md
+19
View File
@@ -0,0 +1,19 @@
**Task 03 complete — report**
**Implemented**
- `app/schemas.py`: `DocContent` gains `origin_url`/`origin_branch` (optional metadata, `image_url` style); serializer drops both when `None` — never `null` (non-git payload byte-identical)
- `app/api/docs.py`: `get_document_content` calls task-01's `origin_for_source(db, doc.source)` (same resolver as the read branch) and sets the two fields
- `frontend/assets/document.js`: new `originAnchor()` helper (`createElement("a")`, full URL verbatim as href+text, `target="_blank" rel="noopener"`, title = URL + ` — branch: <name>` when pinned); `renderDocument` inserts it after the path `code`, before the Created badge — one shared site for modal AND /document.html; absent → pre-phase meta row
- `frontend/assets/styles.css`: `.doc-origin` (`.doc-path` ellipsis idiom + `.doc-link` brand-ink affordance) + ≤640px 16rem squeeze
- Tests: 10 unit (model omission rule, endpoint stubbed git/pinned/sanitized/local/ambiguous, anchor source pins + node behavioral branch/no-branch variants, position pin, XSS contract, CSS pin) + 4 integration (seeded git row, pinned row, legacy credential sanitized end-to-end, plain local source → keys ABSENT)
**Results**
- `uv run pytest --cov=app --cov-report=term-missing`: **3399 passed**, `app/` coverage **99%** (>90%)
- `uv run ruff check .`: clean; `uv run pyright`: **0 errors**
- E2E spot-runs in isolation: `tests/e2e/test_document_viewer.py` 8 passed, `tests/e2e/test_git_source_dates.py` 4 passed (screenshot visually identical — no anchor on the local fixture)
**Docs**: README phase-145 section extended with the viewer's "Original" link; AGENTS.md — no doc impact (no command/config/convention change)
**Notable decisions**: meta-row children built with a conditional spread (`...originItems`) so all pre-phase badge pins (sources_dates/date_editor/image_documents) pass untouched; existing E2E meta pins are selector-based and unaffected. Dev DB confirmed an empty test sandbox (integration truncation is the established pattern).
**Next pending task**: `.agents/phases/todo/145_git_source_links/04_e2e_source_link.md`
.agents/reports/145_git_source_links/145_git_source_links__03_content_payload_render_link.a1.validate
+119
View File
@@ -0,0 +1,119 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
............... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 181 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 431 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 155 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 430 0 100%
--------------------------------------------------
TOTAL 5537 22 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/145_git_source_links/145_git_source_links__04_e2e_source_link.a1.err
View File
.agents/reports/145_git_source_links/145_git_source_links__04_e2e_source_link.a1.md
+17
View File
@@ -0,0 +1,17 @@
Task 04 complete — all gates 100% green.
**Implemented**
- `tests/e2e/mock_llm.py`: new scripted `ORIGIN_LINK_TRIGGER` flow ("original source link", house style — docstring entry + constants + `_origin_link_url` + `_origin_link_flow` + dispatch before the plain tools flow). Request 1: scripted `read` (question-carried call); request 2: deterministic final answer — repo-URL sentence parsed by the mock FROM the read result's `origin:` line (the server-side half — the mock cannot fabricate it) + the read result VERBATIM (the rendered lens on the `origin:`/`branch:` lines); no-origin sentence for local docs; `Premise broke:` echo for refusals
- `tests/e2e/test_git_source_links.py`: dedicated Playwright suite, mock-only, isolated — 2-source KB (`skills` local dir whose name matches the seeded git row's URL basename `https://example.com/reese/skills` + `control` byte-identity dir), importer-in-thread, git row seeded per-test in the suite's own DB session (never at import time, so no clone); 4 tests: origin line in tool output + repo URL in done answer / modal `.doc-origin` anchor (href, `_blank`, `noopener`, verbatim URL text, bare title) / pinned-branch `title` (`… — branch: main`, text stays bare) + `branch: main` in the result / local control with NO `origin:` line and NO anchor
**Results**
- `uv run pytest tests/e2e/test_git_source_links.py -v --no-cov` → **4 passed** (in isolation)
- Regression, in isolation: `tests/e2e/test_read_line_ranges.py` → 2 passed; `tests/e2e/test_document_viewer.py` → 8 passed
- `uv run pytest --cov=app --cov-report=term-missing` → **3399 passed**, `app/` coverage **99%** (>90%)
- `uv run ruff check .` → clean; `uv run pyright` → **0 errors**
**Docs**: no doc impact (test-only task; README's phase-145 section was already accurate from tasks 02/03)
**Decisions**: the phase-117 compact tool surface renders the *call* (not result text), so the origin-line assertion rides the mock's verbatim echo in the answer — the house scripted-turn lens; mock URL sentinel is `""` (homogeneous `tuple[str, …]`, pyright)
**Next pending task**: none — 04 is phase 145's last task (harness performs the phase's final pass).
.agents/reports/145_git_source_links/145_git_source_links__04_e2e_source_link.a1.validate
+119
View File
@@ -0,0 +1,119 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 33%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 48%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 63%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 69%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 91%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 99%]
............... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 181 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 431 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 155 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 430 0 100%
--------------------------------------------------
TOTAL 5537 22 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/screenshots/doc_modal_scrollbar_1280.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 42 KiB

After

Width:  |  Height:  |  Size: 40 KiB

README.md
+15
View File
@@ -374,6 +374,21 @@ server-side tools — on **grounded** (high-relevance) turns only:
(See [Answer images](#answer-images-phase-124) for the `show_image` tool.)
**Git-source provenance (phase 145).** When a document comes from a git
source, its `read` result also carries an `origin:` line (the repository's
URL, credentials stripped) and, when the source pins one, a `branch:` line.
The prompt teaches the model to include that original source in its answer —
a link to the repository with the document's path named (and the pinned
branch when one is shown) — so you can open the complete original in one
step rather than asking again for the source. Local (non-git) sources carry
no `origin:` line, so nothing is added and no link is invented. The same
origin rides the document-content payload (`origin_url` + `origin_branch`
when pinned), and the document viewer — the chat's doc modal AND the full
`/document.html` page, one render site — shows it as an **Original**
external link in the document's meta row (the full repository URL as the
link text, opening in a new tab; the pinned branch rides the link's
title). Non-git documents show nothing new.
Each call is executed against Postgres only (no extra LLM round trip) and
streamed as an SSE `tool` frame. In the chat, each call shows a transient
**calling-tool status** alongside "thinking" (the send button keeps its
app/api/docs.py
+18
View File
@@ -69,6 +69,7 @@ from app.models import Chunk, Document, FolderSummary
from app.rag.agent import list_source_names
from app.rag.doc_dates import normalize_doc_date
from app.rag.folder_summaries import MIN_DOCS_PER_FOLDER, folder_of
from app.rag.git_sources import origin_for_source
from app.rag.importer import match_extension
from app.rag.llm import EmbeddingError, LLMClient
from app.rag.summarizer import IMAGE_FALLBACK_MIME, IMAGE_MIMES
@@ -181,6 +182,16 @@ def get_document_content(
documents and a signed-in user (admin or token holder) opens a cited
document by direct URL. The ONLY anonymous content left is the
shared chats.
Phase 145 (task 03): the payload carries the document's git origin
when one resolves — ``origin_url`` (the sanitized repository URL)
+ ``origin_branch`` (only when the row pins one), computed by the
SAME :func:`app.rag.git_sources.origin_for_source` resolver the
agent's read branch uses (one mapping for the whole app: the
source's directory basename → the ``kind='git'`` row whose URL
basename matches). ``None`` — a non-git source, no row, or an
ambiguous basename — keeps the payload byte-identical to pre-phase
(both fields are dropped by the serializer, never ``null``).
"""
row = db.execute(
select(Document, func.count(Chunk.id).label("chunks"))
@@ -191,6 +202,11 @@ def get_document_content(
if row is None:
raise HTTPException(status_code=404, detail="document not found")
doc, chunks = row
# Phase 145 (task 03): the document's git origin — the SAME resolver
# the read branch uses (one mapping for the whole app). ``None``
# (non-git source, no row, ambiguous basename) → both fields are
# dropped on serialization, never ``null``.
origin = origin_for_source(db, doc.source)
# Phase 122 (task 04): the image affordance — ``is_image`` is ALWAYS
# present on the wire (text docs: false — the one new key); for an
# image doc, ``image_url`` is the bytes route's path (absent for
@@ -210,6 +226,8 @@ def get_document_content(
image_url=(
f"/api/documents/{doc.id}/image" if doc.is_image and doc.image_path else None
),
origin_url=origin[0] if origin else None,
origin_branch=origin[1] if origin else None,
)
app/rag/agent.py
+44 -3
View File
@@ -280,7 +280,7 @@ from sqlalchemy.orm import Session
from app.config import Settings
from app.models import Document, FolderSummary
from app.rag.folder_summaries import folder_of
from app.rag.git_sources import effective_sources
from app.rag.git_sources import effective_sources, origin_for_source
from app.rag.llm import (
LLMClient,
LLMError,
@@ -362,8 +362,11 @@ AGENT_TOOLS: list[dict[str, Any]] = [
"first line) and `limit` (number of lines) to read "
"exactly that part. Do not re-read a document you have "
"already read in full — its full text is already in "
"your prompt. Call one tool at a time — wait for this "
"result before your next call."
"your prompt. For a document from a git source, the "
"result also carries an `origin` line (and `branch` "
"when pinned) — include that original source in your "
"answer (see your tools instructions). Call one tool at "
"a time — wait for this result before your next call."
),
"parameters": {
"type": "object",
@@ -1272,6 +1275,32 @@ def _resolve_path(db: Session, combined: str) -> tuple[Document | None, str, str
return find_document(db, source, path), source, path
def _origin_lines(origin: tuple[str, str | None] | None) -> str:
"""Phase 145 (task 01) — the read result's origin lines, shared by
the four result builders (the lines are identical in every shape).
``None`` (the document's source has no unambiguous ``kind='git'``
row — a local source, zero matches, or an ambiguous basename) →
``""``: the result stays byte-identical to pre-phase. Otherwise
one ``origin: {url}`` line (the sanitized repository URL — the
phase-121 mask, so a legacy embedded-credential row's secret never
reaches the prompt) and, when a branch is pinned (non-NULL), one
``branch: {branch}`` line. The builders insert the block
IMMEDIATELY AFTER the ``date:`` line and BEFORE the image marker
(the marker keeps its phase-122 position relative to the content,
the origin lines ride above it); the FIRST line
(``Document {source}/{path}:``) is untouched — the E2E mock's
``_READ_RESULT_PREFIX`` header contract.
"""
if origin is None:
return ""
url, branch = origin
lines = f"origin: {url}\n"
if branch is not None:
lines += f"branch: {branch}\n"
return lines
def all_documents(db: Session) -> list[Document]:
"""Every indexed document (full rows), ordered by ``(source, path)``
— catalog order.
@@ -1728,6 +1757,14 @@ def _execute_tool(
if (doc.source, doc.path) not in known:
holder.read_docs.append(doc)
holder.tool_calls += 1
# Phase 145 (task 01): the git-sourced document's repository
# origin — computed ONCE per successful read, BEFORE the
# ranged/truncated branching (the lines are identical in all
# four result shapes, which each render the shared block via
# :func:`_origin_lines`). ``None`` (local source, no git row,
# ambiguous basename) → no lines: the result stays
# byte-identical to pre-phase.
origin = origin_for_source(db, doc.source)
# Phase 122 (task 05): an image document's content IS the vision
# description — the marker line (a third line between the
# byte-identical header/date lines and the text) tells the model
@@ -1775,6 +1812,7 @@ def _execute_tool(
return (
f"Document {doc.source}/{doc.path}:\n"
f"date: {doc.created_at:%Y-%m-%d}\n"
f"{_origin_lines(origin)}"
f"{marker}"
f"{doc.content[:cap]}\n"
f"{TRUNCATION_MARKER}\n"
@@ -1792,6 +1830,7 @@ def _execute_tool(
return (
f"Document {doc.source}/{doc.path}:\n"
f"date: {doc.created_at:%Y-%m-%d}\n"
f"{_origin_lines(origin)}"
f"{marker}"
f"{doc.content}"
)
@@ -1829,6 +1868,7 @@ def _execute_tool(
return (
f"Document {doc.source}/{doc.path}:\n"
f"date: {doc.created_at:%Y-%m-%d}\n"
f"{_origin_lines(origin)}"
f"lines: {first}-{end} of {total_lines}\n"
f"{marker}{slice_text[:cap]}\n"
f"{TRUNCATION_MARKER}\n"
@@ -1842,6 +1882,7 @@ def _execute_tool(
return (
f"Document {doc.source}/{doc.path}:\n"
f"date: {doc.created_at:%Y-%m-%d}\n"
f"{_origin_lines(origin)}"
f"lines: {first}-{end} of {total_lines}\n"
f"{marker}{slice_text}"
)
app/rag/git_sources.py
+77
View File
@@ -86,6 +86,83 @@ def sanitize_url(url: str) -> str:
return _USERINFO_RE.sub(r"\1", url, count=1)
def _url_basename(url: str) -> str:
"""Phase 145 (task 01) — the repository directory a git row's URL
imports into.
The URL's path component — after the ``://``, or after the ``:``
for scp-style ``git@host:owner/repo`` (the port-carrying
``ssh://user@host:22/…`` form already fell into the ``://``
branch) — with trailing ``/`` stripped, a single trailing ``.git``
suffix stripped, and the last segment taken. The importer clones
into (and names the source after) exactly this basename
(``source = root.name``), so it is the join key between a
document's ``source`` name and the ``git_sources`` row whose clone
produced it. An ``@`` inside the *path* is not userinfo and is
never treated as one (no split, no strip).
"""
rest = url
if "://" in rest:
rest = rest.partition("://")[2]
elif ":" in rest:
# scp-style ``git@host:owner/repo`` — the path component starts
# after the colon (``host:port`` has no other form here: the
# only colon-carrying scheme without ``//`` is the scp one).
rest = rest.partition(":")[2]
rest = rest.rstrip("/")
if rest.endswith(".git"):
rest = rest[: -len(".git")]
return rest.rsplit("/", 1)[-1]
def origin_for_source(db: Session, source_name: str) -> tuple[str, str | None] | None:
"""Phase 145 (task 01) — the origin a read result of *source_name*
documents carries: the sanitized repository URL (and the pinned
branch, when one exists), or ``None``.
The candidate rows are the ``kind='git'`` rows (local rows are
never origins — nothing to link to from the user's browser); a row
matches when its URL basename (:func:`_url_basename` — the clone
directory the importer names the source after) equals
``source_name``.
* EXACTLY ONE match → ``(sanitize_url(row.url), row.branch)`` —
the phase-121 output mask keeps a legacy embedded-credential
row's secret out of the prompt (the stored value is never
modified — the :func:`clone_url_for` lesson), and ``branch`` is
the pinned branch or ``None`` (NULL = the default branch);
* zero matches, or TWO-OR-MORE (two repos sharing a basename) →
``None`` + a WARNING log (the ambiguity names the sanitized
candidates) — never guess: the read result would otherwise
carry a wrong origin into every answer about the document, and
a ``None`` result is byte-identical to pre-phase.
Pure-DB helper — one query, no git subprocess
(``scripts/git_sync.py`` stays the only git-invocation site, PLAN
§13), no caching (the table is a handful of rows; a read's extra
query is trivial).
"""
rows = db.scalars(select(GitSource).where(GitSource.kind == "git")).all()
candidates = [row for row in rows if _url_basename(row.url) == source_name]
if len(candidates) == 1:
row = candidates[0]
return sanitize_url(row.url), row.branch
if not candidates:
logger.warning(
"no git origin for source %r: no kind='git' row's URL basename matches",
source_name,
)
return None
logger.warning(
"ambiguous git origin for source %r: %d rows share the URL basename (%s) — "
"no origin line is added to its read results",
source_name,
len(candidates),
", ".join(sanitize_url(row.url) for row in candidates),
)
return None
def clone_url_for(row: GitSource) -> str:
"""Phase 121, LOCKED A2 — the URL git actually clones, with the
row's credential injected ONLY here.
app/rag/prompts.py
+45 -1
View File
@@ -231,6 +231,40 @@ SUGGEST_INTRO = (
#: summaries are the starting points, not the content; ``grep`` with
#: only ``pattern`` searches the whole knowledge base, and a source
#: name is not a document.
#: Phase 145 (task 02) — the original-source rule the ``read`` teaching
#: carries (LOCKED verbatim — pinned in :mod:`tests.unit.test_prompt_lock`
#: and :mod:`tests.unit.test_prompts`). :data:`TOOLS_SECTION` embeds this
#: exact sentence-group at the ``read`` clause's end (the truncation
#: workflow sentence) and BEFORE the ``grep`` clause begins; the
#: cross-pins (``TOOLS_SECTION`` minus this rule reproduces the
#: pre-phase-145 bytes; ``count == 1`` + the boundary substring) keep
#: the embedded copy and this constant byte-identical. A read result for
#: a git-sourced document carries an ``origin:`` line (and a ``branch:``
#: line when a branch is pinned) — the read contract's fact — so when
#: the answer uses such a document it includes the original source: a
#: link to the repository (the ``origin`` line) with the document's path
#: (the ``Document …`` header) named and the pinned branch when one is
#: shown, as the repository's file-view URL where the model can tell it
#: (``<repo-url>/blob/<branch>/<path>``), or the repository URL plus the
#: path as plain text otherwise — the user opens the complete original
#: from the answer in one step, not a second question. The rule is
#: CONDITIONAL on the ``origin`` line's presence — a local-sourced
#: document's read result carries none, so the rule adds nothing to
#: those answers (the model has no URL to cite; it must not invent one —
#: the sentence's conditioning does that job).
ORIGIN_RULE: str = (
"A read result for a document from a git source carries an origin line "
"(and a branch line when a branch is pinned): when your answer uses "
"such a document, include the original source in the answer — a link "
"to the repository from the origin line, with the document's path "
"(from the Document header) named and the pinned branch when one is "
"shown — as the repository's file-view URL where you can tell it "
"(for example <repo-url>/blob/<branch>/<path>), or the repository "
"URL plus the path as plain text otherwise. The user should be able "
"to open the complete original from your answer in one step, not ask "
"again for the source."
)
TOOLS_SECTION: str = (
"<tools>\n"
"You may extend your context with three tools. `ls` lists the "
@@ -260,7 +294,17 @@ TOOLS_SECTION: str = (
"are capped: a "
"cut read returns the first part plus a TRUNCATED notice — the "
"document did not end where it stopped; use `grep` (pattern) to "
"find the rest, it searches the whole document. `grep` locates an exact string "
"find the rest, it searches the whole document. A read result for a "
"document from a git source carries an origin line (and a branch line "
"when a branch is pinned): when your answer uses such a document, "
"include the original source in the answer — a link to the "
"repository from the origin line, with the document's path (from the "
"Document header) named and the pinned branch when one is shown — as "
"the repository's file-view URL where you can tell it (for example "
"<repo-url>/blob/<branch>/<path>), or the repository URL plus the "
"path as plain text otherwise. The user should be able to open the "
"complete original from your answer in one step, not ask again for "
"the source. `grep` locates an exact string "
"(case-insensitive) in the indexed documents and returns up to 20 "
"matching `source/path:line: text` lines — a locator, not a "
"context-adder: read the winner with `read`. A grep pattern is a "
app/schemas.py
+18
View File
@@ -561,12 +561,30 @@ class DocContent(BaseModel):
#: omission precedent); also absent for an image row whose
#: ``image_path`` is NULL (the viewer's onerror fallback covers it).
image_url: str | None = None
#: Phase 145 (task 03) — the document's git origin: the sanitized
#: repository URL (the task-01 resolver's output — the phase-121
#: credential-free mask; the document's ``source`` is the directory
#: the clone landed in, so no unambiguous git row means no origin).
#: ABSENT from the wire for non-git documents (``None`` → dropped by
#: the serializer — never ``null``, the :func:`_drop_absent_share_url`
#: omission precedent), so a non-git document's payload is
#: byte-identical to pre-phase.
origin_url: str | None = None
#: Phase 145 (task 03) — the row's PINNED branch (phase 135).
#: ABSENT from the wire when the row pins none (NULL = the
#: repository's default branch — nothing to show); also dropped by
#: the serializer (never ``null``).
origin_branch: str | None = None
@model_serializer(mode="wrap")
def _serialize(self, handler: SerializerFunctionWrapHandler) -> Any:
data = handler(self)
if self.image_url is None:
data.pop("image_url", None)
if self.origin_url is None:
data.pop("origin_url", None)
if self.origin_branch is None:
data.pop("origin_branch", None)
return data
frontend/assets/document.js
+52
View File
@@ -118,6 +118,22 @@
* the row exists but the copy was lost) swaps the block for a small
* "Image unavailable" note (role=status): the page still shows the
* description. ONE shared core — page + modal both get it.
*
* Phase 145 (task 03): the meta row's "Original" anchor — when the
* content payload carries origin_url (a git-sourced document — the
* task-01 resolver's sanitized repository URL; ABSENT on the wire
* for non-git documents, whose meta row stays byte-identical to
* pre-phase), an <a class="doc-origin"> joins the .doc-meta row
* AFTER the path code and BEFORE the Created badge: href + visible
* text = the full URL VERBATIM (the copyable-link contract — no
* decorative label inside the anchor text; the .doc-origin ellipsis
* clips the display), target=_blank + rel=noopener (the modal's
* "Full page" escape-hatch idiom), and the title carries the full
* URL plus ` — branch: <name>` when the payload pins a branch (the
* pathCode ellipsis-precision idiom). ONE shared core — the modal
* AND /document.html render it (no per-surface copy); createElement
* + properties/textContent only (the never-build-HTML rule — no
* innerHTML).
*/
import { bindSharedHeaderControls, fetchIsAdmin, initSharedHeader } from "./header.js";
@@ -158,6 +174,32 @@ function metaBadge(cls, text, title) {
return el;
}
/* Phase 145 (task 03): the meta row's "Original" anchor — the git
* origin as a copyable external link (called only when the payload
* carries origin_url — a non-git document never gets one, so its
* meta row stays the pre-phase shape). The FULL URL is the anchor's
* visible text (the copyable-link contract — no decorative label
* inside the anchor text: a " (branch)" suffix would break
* copy-paste; the .doc-origin CSS ellipsis clips the display
* instead), target=_blank + rel=noopener (the modal's "Full page"
* escape-hatch idiom), and the title carries the full URL — plus
* ` — branch: <name>` when the payload pins a branch (the pathCode
* ellipsis-precision idiom). Properties + textContent only — never
* innerHTML (the XSS contract). */
function originAnchor(doc) {
const a = document.createElement("a");
a.className = "doc-origin";
a.href = doc.origin_url;
a.target = "_blank";
a.rel = "noopener";
a.textContent = doc.origin_url;
a.setAttribute(
"title",
doc.origin_branch ? doc.origin_url + " — branch: " + doc.origin_branch : doc.origin_url,
);
return a;
}
/* ---------- shared renderer (phase 26, task 02) ----------
* Populates the three elements every render surface provides: a title,
* a .doc-meta badge row (source · format · mono path · created ·
@@ -178,10 +220,20 @@ export function renderDocument(doc, { titleEl, metaEl, contentEl }) {
const pathCode = document.createElement("code");
pathCode.className = "doc-path";
pathCode.textContent = doc.path;
// Phase 145 (task 03): the "Original" anchor — the git origin as a
// copyable external link, present ONLY when the content payload
// carries origin_url (a git-sourced document's sanitized repository
// URL — the task-01 resolver's output; ABSENT on the wire for
// non-git documents, so their meta row stays byte-identical to
// pre-phase). It lands AFTER the path code and BEFORE the Created
// badge — one more inline item in the row (the modal's wrap rule
// and the page's clip rule both keep it on their shapes).
const originItems = doc.origin_url ? [originAnchor(doc)] : [];
metaEl.replaceChildren(
metaBadge("doc-source-badge", doc.source),
metaBadge("format-badge", doc.format),
pathCode,
...originItems,
// Phase 106 (task 08, D8): the Created badge BEFORE the Indexed
// one (the owner's verbatim position — the date at the top of a
// clicked document). textContent = the locale date (fmtDate — the
frontend/assets/styles.css
+23
View File
@@ -4400,6 +4400,28 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
keeps a visible box on narrow screens while the trailing badges clip. */
min-width: 6rem;
}
/* Phase 145 (task 03): the "Original" anchor — the git origin as a
copyable external link in the meta row (rendered by the ONE
renderDocument core — the modal AND /document.html; a non-git
document never gets one). The .doc-path code family (mono, 0.75rem,
the SAME 26rem ellipsis budget + 6rem floor — the row clips, the
title carries the full URL) with the .doc-link affordance (brand-ink
8.7:1 on --surface — the recorded .doc-link pair; underline on
hover/focus, the .doc-link idiom). The anchor's visible text IS the
full URL (the copyable-link contract — the ellipsis clips the
display). */
.doc-origin {
font-family: var(--mono);
font-size: 0.75rem;
max-width: 26rem;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
min-width: 6rem;
color: var(--brand-ink); /* 8.7:1 on --surface (the .doc-link pair) */
text-decoration: none;
}
.doc-origin:hover, .doc-origin:focus-visible { text-decoration: underline; }
.doc-chunks { font-family: var(--mono); }
/* Phase 106 (task 08, D8): the Created badge — the SAME family as the
Indexed one (no own look: the meta row's text, var(--ink-soft) on
@@ -5447,6 +5469,7 @@ a.answer-image:hover { text-decoration: underline; } /* flat — the caption und
.doc-titlebar .container { gap: 0.5rem; } /* phase 34: row 2 squeezes like row 1 at mobile widths */
#doc-title { font-size: 1.1rem; }
.doc-path { max-width: 16rem; }
.doc-origin { max-width: 16rem; } /* phase 145 (task 03): the Original anchor squeezes like the path */
.doc-md { padding: 1.1rem 1rem; }
.doc-summary { padding: 0.75rem 0.9rem; }
.doc-raw { padding: 1rem; font-size: 0.8rem; }
tests/e2e/mock_llm.py
+242
View File
@@ -714,6 +714,55 @@ Implements just enough of the aipi surface:
deflected turns never hit it); verified 2026-10-01: no existing
E2E question or fixture file contains the phrase, so every
other suite is unaffected.
- user message containing ``original source link``
(``ORIGIN_LINK_TRIGGER``, phase 145 task 04 — the git-source-
links' dedicated story suite
``tests/e2e/test_git_source_links.py``) **and** the system
prompt carries the ``<tools>`` section -> the deterministic
SCRIPTED ORIGIN-LINK flow (the owner's scenario, TODO.md L7:
the answer grounded in a git-sourced document must carry the
way to the complete original in ONE step): the question
carries its own tool call after the colon — ``original source
link: read source/path`` — the question-carried-call
convention (``_READ_CAP_CALL_RE``), then discriminated
statelessly from the tool results (streaming only):
* request 1 (``tools`` offered, no tool results yet): a
``tool_calls`` delta — ``read`` on the parsed target
(synthetic id ``call_0``);
* a ``tool``-role READ result is in the messages (content
starting with the agent's ``"Document "`` prefix —
``_READ_RESULT_PREFIX``; the phase-145 ``origin:`` /
``branch:`` lines sit BELOW the header, so the first-line
parsing is untouched): the deterministic FINAL answer —
the repository URL VERBATIM, parsed from the read
result's ``origin:`` line BY THE MOCK ITSELF (the mock
cannot fabricate it — the real server's resolver put the
line in the result, so the URL in the answer is the
server-side half's assertion): ``The full source lives at
<url> — see the repo for the complete original.``
(``ORIGIN_LINK_ANSWER``) followed by the house
scripted-turn echo — the read result VERBATIM under
``Here's what the read returned:`` (the rendered output
is the E2E's lens on the ``origin:``/``branch:`` lines'
presence);
* a tool result is in the messages that carries NO
``origin:`` line (a local-sourced document — the byte-
identity control): the deterministic NO-ORIGIN answer —
``No origin line in the read result — the document is not
from a git source.`` (``ORIGIN_LINK_NO_ORIGIN``) + the
same verbatim echo (the suite asserts the line's ABSENCE
through it);
* a tool result is in the messages that is NOT a read
result (a refusal — the premise broke): the
deterministic broken-premise echo — the offending result
VERBATIM under ``Premise broke:`` (the two-ranges
convention — the suite fails loudly on it).
Checked BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint
trigger phrases — the phase-71/72/94/118/131/139/141 ordering
convention; the trigger needs the ``<tools>`` section, so
deflected turns never hit it); verified 2026-10-01: no
existing E2E question or fixture file contains the phrase, so
every other suite is unaffected.
Failure injection (phase 67, LLM retry, TODO.md L3) — deterministic
dead-endpoint behavior for the retry E2E suite (``tests/e2e/
@@ -2667,6 +2716,148 @@ def _two_ranges_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
return ("read1", target)
# ---------------------------------------------------------------------------
# Phase 145 (task 04): the git-source-links' dedicated story suite — the
# owner's "the answer points at the original git source" scenario. See
# the module docstring.
# ---------------------------------------------------------------------------
#: A user message containing this substring (case-insensitive) —
#: combined with the ``<tools>`` section in the system prompt —
#: drives the deterministic SCRIPTED ORIGIN-LINK flow (the
#: git-source-links' story suite,
#: ``tests/e2e/test_git_source_links.py`` — the owner's scenario,
#: TODO.md L7: the answer grounded in a git-sourced document must
#: carry the way to the complete original in ONE step). Checked
#: BEFORE the plain ``TOOLS_TRIGGER`` flow (disjoint trigger
#: phrases — the phase-71/72/94/118/131/139/141 ordering
#: convention; the trigger needs the ``<tools>`` section, so
#: deflected turns never hit it); verified 2026-10-01: no
#: existing E2E question or fixture file contains the phrase, so
#: every other suite is unaffected.
ORIGIN_LINK_TRIGGER = "original source link"
#: The scripted call in the origin-link question (case-insensitive
#: — the suite's questions capitalize the trigger's first
#: letter): the verb (``read``) plus the target — a combined
#: ``source/path``, parsed from the RAW user message so the target
#: keeps its case. The target is a ``[a-z0-9_./-]`` run (case-
#: insensitively), so the suite's `` — `` flavor separator (em
#: dash) can never bleed into it (the read-cap / drill-down
#: convention, ``_READ_CAP_CALL_RE`` / ``_DRILL_CALL_RE``).
_ORIGIN_LINK_CALL_RE = re.compile(
r"original source link:\s*read\s+(?P<arg>[a-z0-9_./-]+)",
re.I,
)
#: The flow's FINAL answer for a read result that carries the
#: phase-145 ``origin:`` line — the DETERMINISTIC string carrying
#: the repository URL VERBATIM (the model-side contract, owner-
#: confirmed 2026-09-30: the answer includes the original source).
#: ``{url}`` is parsed from the read result's ``origin:`` line BY
#: THE MOCK ITSELF — the mock cannot fabricate it, so the URL in
#: the answer is the server-side half's assertion (the real
#: server's resolver put the line in the result the model saw).
ORIGIN_LINK_ANSWER = (
"The full source lives at {url} — see the repo for the complete original."
)
#: The flow's FINAL answer for a read result that carries NO
#: ``origin:`` line (a local-sourced document — the byte-identity
#: control): the deterministic no-origin sentence (its wording
#: carries NO ``origin:`` substring — the suite's absence
#: assertion keys off the line's exact prefix).
ORIGIN_LINK_NO_ORIGIN = (
"No origin line in the read result — the document is not from a git source."
)
#: The house scripted-turn echo header (the ``READ_CAP_TRIGGER``
#: flow's convention): the read result rides VERBATIM into the
#: answer, so the rendered output is the E2E's lens on the
#: ``origin:``/``branch:`` lines' presence (and their absence in
#: the local-source control).
_ORIGIN_LINK_ECHO_HEADER = "Here's what the read returned:"
def _origin_link_url(result: str) -> str | None:
"""The read result's ``origin:`` line's URL (the phase-145
task-01 line — the sanitized repository URL the real server's
resolver put in the result) — or ``None`` when the result
carries no origin line (a local-sourced document, no git row,
or an ambiguous basename).
The search is position-independent (any line with the exact
``origin: `` prefix): the phase-145 lines sit between the
``date:`` line and the content, but the parse never drifts if
the result's header shape changes. A document BODY line that
merely starts with ``origin: `` would be a fixture defect (the
suite's fixtures are pinned content — no such line), never a
mock defect.
"""
for line in result.splitlines():
if line.startswith("origin: "):
return line[len("origin: "):]
return None
def _origin_link_flow(body: dict[str, Any]) -> tuple[str, ...] | None:
"""Classify a phase-145 scripted origin-link request (see the
module docstring). The question carries the scripted call
(``original source link: read source/path``); the step is then
discriminated statelessly from the tool results, like the
other marker flows:
* ``("call", target, "call_0")`` — ``tools`` are offered and
no ``tool``-role result is in the messages yet: the scripted
``read`` on the parsed target (synthetic id ``call_0``).
* ``("answer", url, result)`` — a ``tool``-role READ result
(content starting with the agent's ``"Document "`` prefix —
``_READ_RESULT_PREFIX``; the phase-145 ``origin:`` /
``branch:`` lines sit BELOW the header, so the first-line
parsing is untouched) is in the messages: the deterministic
FINAL answer — ``url`` the result's ``origin:`` line's URL
(or ``""`` — the no-origin sentence, the local-source
control; the empty-string sentinel keeps the return
homogeneous ``tuple[str, …]``, the ``_read_lines_flow``
convention) and ``result`` the read result VERBATIM (the
house scripted-turn echo — the rendered output is the lens
on the origin lines).
* ``("bad", result)`` — a tool result is in the messages that
is NOT a read result (a refusal — the premise broke): the
deterministic broken-premise echo — the offending result
VERBATIM under ``Premise broke:`` (the two-ranges convention
— the suite fails loudly on it).
* ``None`` — not the flow: the trigger is absent, the
``<tools>`` section is missing (deflected turns never carry
it), the scripted call is unparseable, or ``tools`` are not
offered and no tool results are in the messages yet (e.g.
``agent_max_rounds=0``).
"""
user = _user(body)
if ORIGIN_LINK_TRIGGER not in user.lower():
return None
if "<tools>" not in _system(body):
return None
match = _ORIGIN_LINK_CALL_RE.search(user)
if match is None:
return None
target = match.group("arg")
results = _tool_results(body)
if results:
last = results[-1]
if not last.startswith(_READ_RESULT_PREFIX):
return ("bad", last)
# The empty-string sentinel (never a real URL — a URL is
# always non-empty: the resolver returns the stored URL's
# sanitized form, which is a non-empty string): keeps the
# return homogeneous ``tuple[str, …]`` (pyright), the
# ``_read_lines_flow`` convention.
return ("answer", _origin_link_url(last) or "", last)
if not body.get("tools"):
return None
return ("call", target, "call_0")
def long_answer() -> str:
"""~900-word deterministic walkthrough (phase 11): numbered steps plus
a unique final line that must survive the stream untruncated."""
@@ -4044,6 +4235,57 @@ def chat_completions(body: dict[str, Any]) -> Any:
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 145 (task 04): the owner's "the answer points at the
# original git source" scenario (the git-source-links' story
# suite, tests/e2e/test_git_source_links.py — TODO.md L7):
# the scripted read, then the final answer that carries the
# repository URL parsed from the read result's ``origin:``
# line (the server-side half — the mock cannot fabricate it)
# + the result VERBATIM (the lens on the origin lines'
# presence/absence). Checked BEFORE the plain TOOLS_TRIGGER
# flow (disjoint trigger phrases — the phase-71/72/94/118/
# 131/139/141 ordering convention; the trigger needs the
# ``<tools>`` section, so deflected turns never hit it).
origin_link = _origin_link_flow(body)
if origin_link is not None:
if origin_link[0] == "call":
stream = _tool_call_stream(
"read", {"path": origin_link[1]}, origin_link[2]
)
elif origin_link[0] == "bad":
# The premise broke (a refusal — the scripted read
# was refused): echo the offending result verbatim
# so a suite fails loudly on it (the two-ranges
# convention).
stream = _sse_stream(
_apply_max_tokens(
f"Premise broke:\n{origin_link[1]}",
body.get("max_tokens"),
),
0.0,
)
else: # "answer" — the final answer (tests 1–3's model-
# side contract) + the verbatim echo (the lens on
# the origin lines' presence/absence). ``url`` is
# the ``origin:`` line's URL or the "" sentinel
# (the no-origin sentence — the local-source
# control).
url = origin_link[1]
sentence = (
ORIGIN_LINK_ANSWER.format(url=url) if url else ORIGIN_LINK_NO_ORIGIN
)
stream = _sse_stream(
_apply_max_tokens(
f"{sentence}\n{_ORIGIN_LINK_ECHO_HEADER}\n{origin_link[2]}",
body.get("max_tokens"),
),
0.0,
)
return StreamingResponse(
stream,
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
# Phase 124 (task 07, image chips and answers, TODO L3): the
# scripted SHOW_IMAGE flow — the mock shows the seeded image
# document (the ``<documents>`` block with the image-
tests/e2e/test_git_source_links.py
+585
View File
@@ -0,0 +1,585 @@
"""Phase 145 task 04 E2E (Playwright, mock-only): the git-source-links'
dedicated story suite — the owner's scenario, end to end.
The browser-visible proof (TODO.md L7: "there always needs to be a
link to the original git source if possible … point to the skills
repo in the answer so I can go download the skills"): a question
about a git-sourced document produces (1) an answer that contains
the repository link (the model-side teaching, through the real read
result that carries the ``origin:`` line) and (2) a document modal
whose meta row carries the "Original" external anchor. A
local-sourced document produces neither (the byte-identity guard).
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_git_source_links.py -v --no-cov
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the real
``turbo`` does whatever it does with the tools, while this story's
gate is the deterministic SCRIPTED flow in ``tests/e2e/mock_llm.py``
(the mock is the only E2E lens on the LLM's context):
* ``ORIGIN_LINK_TRIGGER`` — a user message containing ``original
source link`` **and** the system prompt carrying the HIGH
prompt's ``<tools>`` section. The question carries its own
scripted call after the colon — ``original source link: read
source/path`` (the question-carried-call convention,
``_READ_CAP_CALL_RE``). Request 1 is the scripted ``read``;
request 2 (the read result present) is the deterministic FINAL
answer — the repository URL VERBATIM, parsed from the read
result's ``origin:`` line BY THE MOCK (it cannot fabricate it —
the real server's resolver put the line in the result, so the
URL in the answer is the server-side half's assertion) —
followed by the house scripted-turn echo (the read result
VERBATIM — the rendered output is the lens on the
``origin:``/``branch:`` lines' presence; the local-source
control's read result carries none, and the mock answers with
its deterministic no-origin sentence instead).
KB fixture — TWO local directories (the house per-module import —
the ``test_document_viewer.py`` importer-in-a-thread pattern), each
holding ONE readable markdown document:
* ``skills/phased-execution.md`` — the owner's scenario document
(the phased execution process for pi.dev). The source's name is
``skills`` — it MATCHES the seeded git row's URL basename
(``https://example.com/reese/skills`` → ``skills``: the path
component, trailing ``/`` stripped, no ``.git`` suffix to strip,
last segment — the resolver's name mapping). The fixture document
is locally imported, but the mapping is name-based — which is the
point of the seam;
* ``control/notes.md`` — a SECOND plain local source with NO git
row: the byte-identity control (its read result and its modal
meta row must be exactly the pre-phase shapes).
The git row is seeded in the suite's OWN DB session (a direct
``SessionLocal`` write, per test — ``branch=None`` for tests 1–2,
``branch="main"`` for the pinned-branch test, and ABSENT for the
control test) — never through the API, and never during the import
(a ``kind='git'`` row present at import time would be cloned —
``example.com`` is not a git host; the resolver reads the table
LIVE at read/content time, so seeding after the import is the
whole trick).
The app under test is the conftest session app (mock LLM + the
mock-calibrated gate: threshold 0.30 + lexical floor 0.15): both
questions share the subject words with their own fixture document
(the mock's token-overlap distribution), so every scripted turn
stays GROUNDED (the ``<tools>`` section is present) and the read
document is the turn's cited chip (phase 119 A1 — the done frame's
``sources`` is the READ docs).
Test → observable mapping (Playwright Mapping Rule; the story is
the owner's TODO item — one Playwright file per story, A16):
1. ``test_answer_carries_origin_link`` — the ``original source
link`` question on the git-matched source's document → the
rendered tool output for the ``read`` (the mock's verbatim echo
in the answer — the house scripted-turn lens) carries
``origin: https://example.com/reese/skills`` (the server put
the line in the result the model saw) and the final ``done``
answer contains the repository URL (the scripted echo — the
model-side contract, deterministic).
2. ``test_modal_shows_original_anchor`` — same setup → click the
source chip → ``#doc-modal`` rendered → the meta row contains
the ``.doc-origin`` anchor with ``href`` = the URL,
``target="_blank"``, ``rel="noopener"``, ``textContent`` = the
URL verbatim (and NO branch in the text — ``branch=None``: the
``title`` is the bare URL).
3. ``test_pinned_branch_in_title`` — the ``branch="main"`` row →
the read result's surface also carries ``branch: main``, the
done answer still carries the repository URL, and the same
anchor's ``title`` is
``https://example.com/reese/skills — branch: main`` (the text
stays the bare URL — the copyable-link contract).
4. ``test_local_source_has_no_origin`` — the control source's
document (no git rows at all — the pre-phase world) → the
read's rendered tool output carries NO ``origin:`` line (the
byte-identical result shape — the mock's no-origin sentence
answers) and the modal's meta row has NO ``.doc-origin``
anchor (the pre-phase badge row).
"""
from __future__ import annotations
import asyncio
import json
import time
from collections.abc import Iterator
from pathlib import Path
from threading import Thread
from typing import Any
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import delete, func, select, text
from app.config import Settings
from app.db import SessionLocal
from app.models import Document, GitSource
from app.rag.importer import import_sources
from app.rag.llm import LLMClient
from e2e.auth_helpers import login
from e2e.mock_llm import ORIGIN_LINK_ANSWER, ORIGIN_LINK_NO_ORIGIN
REPO = Path(__file__).resolve().parents[2]
SOURCE = "skills" # the local directory's basename = the source name
SKILLS_REL = "phased-execution.md"
SKILLS_SP = f"{SOURCE}/{SKILLS_REL}" # the combined read identity
CONTROL = "control" # the plain local control source (NO git row)
CONTROL_REL = "notes.md"
CONTROL_SP = f"{CONTROL}/{CONTROL_REL}"
#: The seeded git row's public URL (no credential — the public form;
#: the phase-121 sanitizer is a byte-identical no-op for it). Its
#: URL basename — ``skills`` — MATCHES the fixture source's name
#: (the resolver's name mapping — the seam).
ORIGIN_URL = "https://example.com/reese/skills"
PINNED_BRANCH = "main"
SKILLS_DOC = """# Phased execution for pi.dev
How does the phased execution process work for pi.dev? It runs one
task at a time: each task file executes in a separate pi subprocess,
and the harness manages phase and task files, retries, and final
validation.
## Task files
Every task file is a unit of work. The pipeline runs the todo phases
in numeric order, and a completed phase moves to the complete
directory with its own test suite green.
## Validation
A phase is complete only when the unit and integration tests pass,
the coverage gate is green, and the Playwright story suite passes
in isolation. The harness then moves the task file and commits the
phase atomically.
"""
CONTROL_DOC = """# Control notes
This is a plain local directory source. It is walked directly by
the importer and never cloned from a repository. The control note
exists so the byte-identity guard has a document: local documents
read, render, and cite exactly as before, with nothing new in the
read result or the meta row.
"""
# The suite's own content pins — the mock's echo assertion keys off
# the read result's ``origin:``/``branch:`` lines, so neither
# fixture may contain a line with either prefix (a fixture defect,
# not a mock defect — see the mock's ``_origin_link_url``).
assert "origin:" not in SKILLS_DOC and "branch:" not in SKILLS_DOC
assert "origin:" not in CONTROL_DOC and "branch:" not in CONTROL_DOC
# The scripted questions — the mock's ``ORIGIN_LINK_TRIGGER`` + the
# question-carried call (``_ORIGIN_LINK_CALL_RE``) + the subject
# words (the mock-calibrated gate: both questions stay GROUNDED on
# their own fixture document).
Q_SKILLS = (
f"Original source link: read {SKILLS_SP} — "
"how does the phased execution process work for pi.dev?"
)
Q_CONTROL = (
f"Original source link: read {CONTROL_SP} — "
"what does the control note say about plain local sources?"
)
#: The final answer the mock scripts for the git-matched document
#: (the ``ORIGIN_LINK_ANSWER`` constant with the seeded URL — the
#: model-side contract, imported so the mock can never drift).
ANSWER_SENTENCE = ORIGIN_LINK_ANSWER.format(url=ORIGIN_URL)
# --------------------------------------------------------------------------
# Fixtures
# --------------------------------------------------------------------------
@pytest.fixture(scope="module")
def kb_dirs(tmp_path_factory: pytest.TempPathFactory) -> tuple[Path, Path]:
"""The story's two local-directory sources (host temp dirs — the
app server runs on the same host, so the paths are visible to
it): the ``skills`` source (its basename MATCHES the seeded git
row's URL basename — the resolver's name mapping) and the
``control`` source (no git row — the byte-identity control)."""
skills = tmp_path_factory.mktemp("bor_origin_skills") / SOURCE
skills.mkdir()
(skills / SKILLS_REL).write_text(SKILLS_DOC, encoding="utf-8")
control = tmp_path_factory.mktemp("bor_origin_control") / CONTROL
control.mkdir()
(control / CONTROL_REL).write_text(CONTROL_DOC, encoding="utf-8")
return skills, control
def _run_in_thread(coro: Any) -> Any:
"""Run a coroutine on a worker thread (the
``test_document_viewer`` pattern — Playwright's sync API keeps
an asyncio loop on the test thread, so ``asyncio.run`` cannot
run directly in a test body)."""
box: dict[str, Any] = {}
def runner() -> None:
try:
box["value"] = asyncio.run(coro)
except BaseException as e: # noqa: BLE001 — re-raised in the test
box["error"] = e
t = Thread(target=runner)
t.start()
t.join()
if "error" in box:
raise box["error"]
return box["value"]
async def _import_sources(roots: list[Path], mock_port: int) -> None:
"""The house per-module import (the ``test_document_viewer``
pattern): the REAL importer against the fixture roots under the
deterministic mock embeddings — the mock LLM supplies the
embeds, and the owner's local ``.env`` is NOT read (the
``_env_file=None`` isolation)."""
kwargs: dict[str, Any] = {
"_env_file": None,
"llm_base_url": f"http://127.0.0.1:{mock_port}/v1",
}
settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
await import_sources(roots, LLMClient(settings))
def _truncate_all() -> None:
"""Fresh registry + KB (the E2E isolation pattern): the E2E
suites share one Postgres, so a leftover git_sources row or
document would pollute the resolver and the retrieval the
scripted turns run against (the two-source KB's gate margins
are pinned against EXACTLY these two documents)."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, steering_notes, "
"kb_overview, git_sources, folder_summaries"
)
)
db.commit()
@pytest.fixture(scope="module")
def seeded_kb(mock_llm: int, kb_dirs: tuple[Path, Path]) -> None:
"""The story's precondition: the two-source KB (one readable
markdown doc each) imported under the deterministic mock, the
stored content byte-pinned. The git row is NOT part of the
import (a git row present at import time would be cloned — the
row is seeded in the suite's own DB session per test, AFTER
the import; the resolver reads the table live at
read/content time)."""
_truncate_all()
_run_in_thread(_import_sources(list(kb_dirs), mock_llm))
with SessionLocal() as db:
docs = {
(d.source, d.path): d for d in db.scalars(select(Document)).all()
}
assert set(docs) == {(SOURCE, SKILLS_REL), (CONTROL, CONTROL_REL)}, (
docs.keys()
)
assert docs[(SOURCE, SKILLS_REL)].content == SKILLS_DOC
assert docs[(CONTROL, CONTROL_REL)].content == CONTROL_DOC
# The import ran with NO git rows (the local-only KB): the
# seeded row lands after, per test.
assert db.scalar(select(func.count()).select_from(GitSource)) == 0
def _seed_git_row(branch: str | None, exist: bool = True) -> None:
"""The git row in the suite's OWN DB session (the task's seam):
delete every row, then — when *exist* — insert the one
``kind='git'`` row whose URL basename matches the ``skills``
fixture source (public URL, no credential; *branch* pinned or
NULL). The resolver reads the table LIVE — the row never
touches the importer."""
with SessionLocal() as db:
db.execute(delete(GitSource))
if exist:
db.add(GitSource(url=ORIGIN_URL, kind="git", branch=branch))
db.commit()
@pytest.fixture(autouse=True)
def _clean(db_ready: None) -> Iterator[None]:
"""Per-test query_log isolation (the KB itself is module-scoped
— the scripted turns never change it, so the documents persist
across the tests of this module)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
yield
with SessionLocal() as db:
db.execute(text("TRUNCATE query_log"))
db.commit()
# --------------------------------------------------------------------------
# Page helpers (the test_read_line_ranges / test_document_viewer
# house patterns)
# --------------------------------------------------------------------------
#: Captures the raw SSE ``data:`` payloads of the /api/chat stream
#: (a response clone read in the background) — wire-level
#: assertions for the ``tool`` / ``done`` frames, independent of
#: the UI rendering.
SSE_HOOK = """
() => {
if (window.__sseInstalled) return;
window.__sseInstalled = true;
window.__sseFrames = [];
const origFetch = window.fetch;
window.fetch = async function (...args) {
const res = await origFetch.apply(this, args);
try {
const url = typeof args[0] === 'string' ? args[0] : args[0].url;
if (url.includes('/api/chat')) {
res.clone().text().then((bodyText) => {
for (const block of bodyText.split('\\n\\n')) {
const line = block.trim();
if (line.startsWith('data: ')) {
window.__sseFrames.push(line.slice(6));
}
}
});
}
} catch (e) { /* non-clonable responses: ignored */ }
return res;
};
}
"""
def _install_page_hooks(page: Page) -> None:
page.evaluate(SSE_HOOK)
def _frames(page: Page, deadline_s: float = 120.0) -> list[dict]:
"""The SSE frames captured since the last submit (``_submit``
clears the buffer), once the hook's background read settles."""
deadline = time.monotonic() + deadline_s
while True:
raw = page.evaluate("() => window.__sseFrames || []")
parsed = [json.loads(line) for line in raw if line]
if any(f.get("type") == "done" for f in parsed):
return parsed
if time.monotonic() > deadline:
raise AssertionError(
f"SSE hook captured no `done` frame (frames so far: "
f"{len(parsed)}) — hook install failed?"
)
time.sleep(0.05)
def _tool_frames(frames: list[dict]) -> list[dict]:
return [f for f in frames if f.get("type") == "tool"]
def _submit(page: Page, question: str) -> None:
page.evaluate("window.__sseFrames = []")
page.fill("#message-input", question)
page.click("#send-btn")
# The user bubble lands synchronously with the submit handler.
expect(page.locator(".msg.user .bubble").last).to_contain_text(question)
def _wait_settled(page: Page, timeout_ms: int = 120_000) -> None:
"""The turn is complete: answer text in the bubble, button
recovered (the phase-48 settle wait)."""
expect(page.locator(".msg.brain .bubble").last).not_to_have_text(
"", timeout=timeout_ms
)
expect(page.locator("#send-btn")).to_be_enabled(timeout=timeout_ms)
expect(page.locator("#send-label")).to_have_text("Send", timeout=timeout_ms)
def _bubble_text(page: Page) -> str:
"""The turn's rendered answer (the brain bubble's text content
— the mock's scripted echo lands here: the house scripted-turn
lens on the read result's ``origin:``/``branch:`` lines)."""
return page.locator(".msg.brain .bubble").last.text_content() or ""
def _open_modal_for_chip(page: Page, chip_text: str) -> None:
"""Drive the story's modal surface: the turn's cited chip
(phase 119 A1 — done.sources is the READ docs, so the read
document is the turn's ONLY chip) opens the same-page document
modal (the ``test_document_viewer`` contract)."""
chip = page.locator(".msg.brain .source-chip", has_text=chip_text)
expect(chip).to_have_count(1, timeout=30_000)
chip.first.click()
expect(page.locator("#doc-modal")).not_to_have_attribute("hidden")
expect(page.locator("#doc-modal")).to_be_visible()
# --------------------------------------------------------------------------
# 1. The answer carries the repository link (the model-side
# contract, through the real read result's ``origin:`` line)
# --------------------------------------------------------------------------
def test_answer_carries_origin_link(
page: Page, app_url: str, seeded_kb: None, db_ready: None
) -> None:
_seed_git_row(branch=None)
page.set_default_timeout(60_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, Q_SKILLS)
_wait_settled(page)
frames = _frames(page)
# The scripted read on the wire — the turn's only tool call.
assert _tool_frames(frames) == [
{"type": "tool", "name": "read", "argument": SKILLS_SP}
], _tool_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The read document is the turn's cited source (the chip the
# modal tests click — phase 119 A1).
assert any(
s["source"] == SOURCE and s["path"] == SKILLS_REL for s in done["sources"]
), done["sources"]
# The read result's surface (the mock's verbatim echo in the
# answer — the house scripted-turn lens): the sanitized
# ``origin:`` line the SERVER put in the result the model saw
# (the task-01 resolver's output — the git row's URL,
# credential-free).
text = _bubble_text(page)
assert f"origin: {ORIGIN_URL}" in text, text
# The done answer carries the repository URL (the scripted
# echo — the model-side contract, deterministic): the mock
# parsed the URL from the origin line — it cannot appear in
# the answer unless the server sent it.
assert ANSWER_SENTENCE in text, text
# --------------------------------------------------------------------------
# 2. The modal's meta row shows the "Original" external anchor
# --------------------------------------------------------------------------
def test_modal_shows_original_anchor(
page: Page, app_url: str, seeded_kb: None, db_ready: None
) -> None:
_seed_git_row(branch=None)
page.set_default_timeout(60_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, Q_SKILLS)
_wait_settled(page)
_open_modal_for_chip(page, SKILLS_REL)
expect(page.locator("#doc-modal-title")).to_have_text(
"Phased execution for pi.dev"
)
# The meta row's "Original" anchor — the task-03 contract
# (ONE shared renderDocument site: the modal AND
# /document.html show it).
anchor = page.locator("#doc-modal-meta .doc-origin")
expect(anchor).to_have_count(1)
expect(anchor).to_have_attribute("href", ORIGIN_URL)
expect(anchor).to_have_attribute("target", "_blank")
expect(anchor).to_have_attribute("rel", "noopener")
# The copyable-link contract: the full URL VERBATIM as the
# visible text — NO branch in the text (branch=None), and the
# title is the bare URL (the pathCode precision idiom, no
# branch suffix).
expect(anchor).to_have_text(ORIGIN_URL)
expect(anchor).to_have_attribute("title", ORIGIN_URL)
# --------------------------------------------------------------------------
# 3. The pinned branch rides the read result and the anchor's title
# --------------------------------------------------------------------------
def test_pinned_branch_in_title(
page: Page, app_url: str, seeded_kb: None, db_ready: None
) -> None:
_seed_git_row(branch=PINNED_BRANCH)
page.set_default_timeout(60_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, Q_SKILLS)
_wait_settled(page)
# The read result's surface (the mock's verbatim echo): the
# origin line AND the pinned branch line (the task-01 second
# line, only when the row pins a branch).
text = _bubble_text(page)
assert f"origin: {ORIGIN_URL}" in text, text
assert f"branch: {PINNED_BRANCH}" in text, text
# The done answer still carries the repository URL (tests
# 1–3's model-side contract — the branch never joins the
# URL).
assert ANSWER_SENTENCE in text, text
_open_modal_for_chip(page, SKILLS_REL)
anchor = page.locator("#doc-modal-meta .doc-origin")
expect(anchor).to_have_count(1)
expect(anchor).to_have_attribute("href", ORIGIN_URL)
# The text stays the bare URL (copyable — the branch never
# joins the anchor text)…
expect(anchor).to_have_text(ORIGIN_URL)
# …the branch rides the title (the task-03 label contract).
expect(anchor).to_have_attribute(
"title", f"{ORIGIN_URL} — branch: {PINNED_BRANCH}"
)
# --------------------------------------------------------------------------
# 4. The local-source control is byte-identical (no ``origin:``
# line in the read result, no anchor in the meta row)
# --------------------------------------------------------------------------
def test_local_source_has_no_origin(
page: Page, app_url: str, seeded_kb: None, db_ready: None
) -> None:
_seed_git_row(branch=None, exist=False) # the pre-phase world: NO git rows at all
page.set_default_timeout(60_000)
login(page, app_url, next="/")
_install_page_hooks(page)
_submit(page, Q_CONTROL)
_wait_settled(page)
frames = _frames(page)
assert _tool_frames(frames) == [
{"type": "tool", "name": "read", "argument": CONTROL_SP}
], _tool_frames(frames)
done = next(f for f in frames if f.get("type") == "done")
assert done["deflected"] is False, done
# The byte-identity guard on the read result's surface (the
# mock's verbatim echo): the result carried NO origin line —
# the local source's result is byte-identical to pre-phase,
# so the mock's deterministic no-origin sentence answers.
text = _bubble_text(page)
assert ORIGIN_LINK_NO_ORIGIN in text, text
assert "origin:" not in text, text
assert "branch:" not in text, text
# …and the modal's meta row is the pre-phase badge row: NO
# .doc-origin anchor (the content payload's origin fields are
# absent on the wire — never null).
_open_modal_for_chip(page, CONTROL_REL)
expect(page.locator("#doc-modal-title")).to_have_text("Control notes")
expect(page.locator("#doc-modal-meta .doc-origin")).to_have_count(0)
# The pre-phase badges are untouched (the row still shows
# source · format · path · created · indexed · chunks).
expect(page.locator("#doc-modal-meta .doc-source-badge")).to_have_text(
CONTROL
)
expect(page.locator("#doc-modal-meta .doc-path")).to_have_text(
CONTROL_REL
)
expect(page.locator("#doc-modal-meta .doc-created")).not_to_be_empty()
tests/integration/test_document_content.py
+134 -2
View File
@@ -10,7 +10,13 @@ Uses the real compose Postgres (``db`` fixture) and FastAPI's TestClient:
tests and the anonymous 403/401 pins build their own client;
* GET: 404 for an unknown (source, path) pair;
* GET: 404 for traversal-style ``path`` values (no filesystem access → no
leak).
leak);
* GET: phase 145 (task 03) — the payload's git origin fields: a seeded
``kind='git'`` row (URL basename = the fixture source) adds
``origin_url`` (+ ``origin_branch`` for a pinned row); a legacy
embedded-credential row is served sanitized (end-to-end); a plain
local source keeps the pre-phase key set (both keys ABSENT — never
``null``).
PATCH /api/documents/summary (phase 57, task 01) — the admin summary
editor, on the same DB-backed fixtures:
@@ -27,6 +33,7 @@ editor, on the same DB-backed fixtures:
"""
from __future__ import annotations
from collections.abc import Iterator
from datetime import UTC, datetime
import pytest
@@ -37,7 +44,7 @@ from sqlalchemy.orm import Session
import app.api.docs as docs_api
from app.config import get_settings
from app.main import app as fastapi_app
from app.models import Chunk, Document
from app.models import Chunk, Document, GitSource
from app.rag.llm import EmbeddingError
from tests.conftest import ADMIN_PASSWORD
from tests.fakes import FakeEmbedder
@@ -571,6 +578,131 @@ def test_content_format_from_suffix(client, db) -> None:
db.commit()
# ---------------------------------------------------------------------------
# Phase 145 (task 03) — the content payload's git origin fields
# (the task-01 resolver's output, end-to-end over the real registry)
# ---------------------------------------------------------------------------
@pytest.fixture()
def origin_registry(db) -> Iterator[None]:
"""A FRESH ``git_sources`` table around the origin tests: the
resolver reads EVERY ``kind='git'`` row, so a controlled registry
is required (truncated before and after — the
``test_agent_tools.py`` registry pattern)."""
db.execute(text("TRUNCATE git_sources"))
db.commit()
yield
db.execute(text("TRUNCATE git_sources"))
db.commit()
def test_content_carries_origin_url_for_a_seeded_git_row(
client, db, origin_registry
) -> None:
"""A ``kind='git'`` row whose URL basename matches the document's
source → the payload carries ``origin_url`` (the sanitized clone
URL). The row pins no branch (NULL = the default branch) →
``origin_branch`` is ABSENT from the JSON (not ``null`` — the
omission pinned on the wire); every pre-phase key stands."""
_seed_doc(db)
db.add(GitSource(url="https://github.com/reese/Homelab.git", kind="git"))
db.commit()
try:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "kubernetes.md"},
)
assert r.status_code == 200
body = r.json()
assert set(body) == {
"source", "path", "title", "format", "summary", "created_at",
"content", "indexed_at", "chunks", "is_image", "origin_url",
}
assert body["origin_url"] == "https://github.com/reese/Homelab.git"
assert "origin_branch" not in body # absent — never null
finally:
db.execute(text("TRUNCATE chunks, documents"))
db.commit()
def test_content_carries_origin_branch_for_a_pinned_row(
client, db, origin_registry
) -> None:
"""A pinned row (phase 135 ``branch``) also carries
``origin_branch`` — the value the viewer's title attribute shows."""
_seed_doc(db)
db.add(
GitSource(
url="https://github.com/reese/Homelab.git", kind="git", branch="docs"
)
)
db.commit()
try:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "kubernetes.md"},
)
body = r.json()
assert body["origin_url"] == "https://github.com/reese/Homelab.git"
assert body["origin_branch"] == "docs"
finally:
db.execute(text("TRUNCATE chunks, documents"))
db.commit()
def test_content_origin_is_sanitized_end_to_end(client, db, origin_registry) -> None:
"""A legacy row whose credential is still embedded in the stored
URL never leaks it: the payload carries the phase-121 sanitized
URL (the resolver's guarantee, end-to-end — the stored row value
is untouched, the output mask does the work)."""
_seed_doc(db)
db.add(
GitSource(url="https://user:tok@example.com/reese/Homelab.git", kind="git")
)
db.commit()
try:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "kubernetes.md"},
)
body = r.json()
assert body["origin_url"] == "https://example.com/reese/Homelab.git"
assert "user:tok" not in r.text # the credential stays out of the wire
finally:
db.execute(text("TRUNCATE chunks, documents"))
db.commit()
def test_content_origin_absent_for_a_plain_local_source(
client, db, origin_registry
) -> None:
"""No unambiguous git row (a local row is never an origin, and a
non-matching git row does not count) → BOTH origin keys ABSENT
from the JSON (not ``null`` — the omission pinned on the wire; the
payload is byte-identical to pre-phase)."""
_seed_doc(db)
db.add(GitSource(url="/srv/Homelab", kind="local", path="/srv/Homelab"))
db.add(GitSource(url="https://github.com/reese/OtherRepo.git", kind="git"))
db.commit()
try:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "kubernetes.md"},
)
assert r.status_code == 200
body = r.json()
assert set(body) == {
"source", "path", "title", "format", "summary", "created_at",
"content", "indexed_at", "chunks", "is_image",
}
assert "origin_url" not in body
assert "origin_branch" not in body
finally:
db.execute(text("TRUNCATE chunks, documents"))
db.commit()
def test_content_format_name_token_extensionless(
client, db, monkeypatch
) -> None:
tests/unit/test_agent.py
+330 -2
View File
@@ -269,8 +269,11 @@ def test_agent_tools_names_and_parameters() -> None:
"and `limit` (number of lines) to read exactly that "
"part. Do not re-read a document you have already read "
"in full — its full text is already in your prompt. "
"Call one tool at a time — wait for this result before "
"your next call."
"For a document from a git source, the result also "
"carries an `origin` line (and `branch` when pinned) — "
"include that original source in your answer (see your "
"tools instructions). Call one tool at a time — wait "
"for this result before your next call."
)
# The byte-preserved contracts inside the rewrite, pinned as
# substrings: the combined source/path identity, the
@@ -293,6 +296,17 @@ def test_agent_tools_names_and_parameters() -> None:
"Call one tool at a time — wait for this result before "
"your next call."
) in read["description"]
# Phase 145 (task 02): the origin clause mirrors the TOOLS_SECTION
# rule (the description points, the section teaches — the
# phase-124 show_image "see the instructions" shape): a git-
# sourced document's read result carries an `origin` line (and
# `branch` when pinned); include that original source in the
# answer.
assert (
"For a document from a git source, the result also carries "
"an `origin` line (and `branch` when pinned) — include that "
"original source in your answer (see your tools instructions)."
) in read["description"]
read_params = read["parameters"]
assert read_params["type"] == "object"
# Phase 131 (task 01, LOCKED A2): offset/limit are OPTIONAL —
@@ -3649,6 +3663,320 @@ def test_unknown_tool_name_refused(monkeypatch: pytest.MonkeyPatch) -> None:
assert llm.requests[1][1] == AGENT_TOOLS # rejected → tools stay offered
# ---------- phase 145 (task 01): the origin/branch lines in the read result ----------
#: The fixture git row's URL (basename ``skills`` = the fixture source
#: name) and the four origin outcomes the read-branch matrix runs:
#: git row matched (branch NULL), git row + pinned branch, local source
#: (no git row), two rows sharing the basename (ambiguous — the source
#: name is ``repo`` for the last one).
_GIT_URL = "https://github.com/owner/skills"
_GIT_ROWS = [GitSource(url=_GIT_URL, kind="git")]
_GIT_PINNED_ROWS = [GitSource(url=_GIT_URL, kind="git", branch="release/1.2")]
_LOCAL_ROWS = [GitSource(url="/srv/skills", kind="local", path="/srv/skills")]
_AMBIGUOUS_ROWS = [
GitSource(url="https://a.example/org/repo.git", kind="git"),
GitSource(url="git@b.example:other/repo.git", kind="git"),
]
def _git_rows_factory(rows: list[GitSource]) -> Callable[[], Session]:
"""A db factory whose mock session answers the resolver's query with
the git rows of *rows* (like the real ``kind = 'git'`` WHERE clause
— the seeded local rows are filtered out, proving they are never
candidates): the REAL ``origin_for_source`` runs in the read branch,
no database."""
git_rows = [row for row in rows if row.kind == "git"]
def _factory() -> Session:
mock = cast("Session", MagicMock())
mock.__enter__ = MagicMock(return_value=mock)
mock.__exit__ = MagicMock(return_value=False)
mock.scalars = MagicMock(return_value=MagicMock(all=MagicMock(return_value=git_rows)))
return mock
return _factory
def _one_read(
monkeypatch: pytest.MonkeyPatch,
doc: Document,
rows: list[GitSource],
arguments: dict[str, Any],
read_max_chars: int | None = None,
) -> str:
"""One scripted read turn against the REAL resolver; returns the
exact tool-result string (the successful-read bookkeeping is
asserted — the matrix asserts the bytes)."""
monkeypatch.setattr(agent, "find_document", lambda db, source, path: doc)
holder = AgentHolder()
llm = ScriptedLLM(
[ToolCallPiece(id="c1", name="read", arguments=arguments)],
[StreamPiece("content", "ans")],
)
kwargs: dict[str, Any] = {"read_max_chars": read_max_chars} if read_max_chars else {}
asyncio.run(_run(llm, holder, _settings(**kwargs), db_factory=_git_rows_factory(rows)))
assert holder.read_docs == [doc] and holder.tool_calls == 1
return llm.requests[1][0][3]["content"]
def test_origin_lines_plain_read_under_cap(
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""Shape 1 (plain read at or under the cap): the origin block rides
IMMEDIATELY AFTER the ``date:`` line — ``Document …: / date: /
origin: / branch: / content`` — in all four origin outcomes; the
first line (the mock's ``_READ_RESULT_PREFIX`` contract) is
untouched, and the local/ambiguous results are byte-identical to
pre-phase (the ambiguity still warns)."""
doc = _doc("skills", "docs/a.md", "A", "A-CONTENT")
plain = {"path": "skills/docs/a.md"}
result = _one_read(monkeypatch, doc, _GIT_ROWS, plain)
assert result == (
"Document skills/docs/a.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"A-CONTENT"
)
# The E2E mock's header contract: first line, byte-identical.
assert result.startswith("Document ")
assert result.splitlines()[0] == "Document skills/docs/a.md:"
result = _one_read(monkeypatch, doc, _GIT_PINNED_ROWS, plain)
assert result == (
"Document skills/docs/a.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"branch: release/1.2\n"
"A-CONTENT"
)
result = _one_read(monkeypatch, doc, _LOCAL_ROWS, plain)
assert result == "Document skills/docs/a.md:\ndate: 2024-06-15\nA-CONTENT"
repo_doc = _doc("repo", "docs/a.md", "A", "A-CONTENT")
caplog.clear()
result = _one_read(monkeypatch, repo_doc, _AMBIGUOUS_ROWS, {"path": "repo/docs/a.md"})
assert result == "Document repo/docs/a.md:\ndate: 2024-06-15\nA-CONTENT"
# The ambiguity is logged at WARNING (the agent's INFO per-turn log
# lines are captured too — filter to the resolver's records).
warnings = [
r for r in caplog.records if r.levelname == "WARNING" and r.name == "app.rag.git_sources"
]
assert len(warnings) == 1
assert "repo" in warnings[0].message
def test_origin_lines_truncated_plain_read(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Shape 2 (plain read cut at the cap): the origin block sits at the
same position (after ``date:``, before the cut content), the
truncation marker + the pinned notice unchanged, the holder
truncation tuple unchanged (the ``read_truncations`` signal is
untouched)."""
cap = 20
content = "x" * (cap + 1)
doc = _doc("skills", "big.md", "Big", content)
plain = {"path": "skills/big.md"}
result = _one_read(monkeypatch, doc, _GIT_ROWS, plain, read_max_chars=cap)
assert result == (
"Document skills/big.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
+ content[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(content))
)
result = _one_read(monkeypatch, doc, _GIT_PINNED_ROWS, plain, read_max_chars=cap)
assert result == (
"Document skills/big.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"branch: release/1.2\n"
+ content[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(content))
)
result = _one_read(monkeypatch, doc, _LOCAL_ROWS, plain, read_max_chars=cap)
assert result == (
"Document skills/big.md:\n"
"date: 2024-06-15\n"
+ content[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(content))
)
repo_doc = _doc("repo", "big.md", "Big", content)
result = _one_read(
monkeypatch, repo_doc, _AMBIGUOUS_ROWS, {"path": "repo/big.md"}, read_max_chars=cap
)
assert result == (
"Document repo/big.md:\n"
"date: 2024-06-15\n"
+ content[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(content))
)
def test_origin_lines_range_read_under_cap(
monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""Shape 3 (range read at or under the cap): the origin block rides
between the ``date:`` line and the ``lines:`` header (the same
position as in the plain shapes); the local/ambiguous results stay
byte-identical to pre-phase."""
doc = _doc("skills", "b.md", "B", "one\ntwo\nthree\nfour\nfive")
ranged = {"path": "skills/b.md", "offset": 2, "limit": 2}
result = _one_read(monkeypatch, doc, _GIT_ROWS, ranged)
assert result == (
"Document skills/b.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"lines: 2-3 of 5\n"
"two\nthree"
)
result = _one_read(monkeypatch, doc, _GIT_PINNED_ROWS, ranged)
assert result == (
"Document skills/b.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"branch: release/1.2\n"
"lines: 2-3 of 5\n"
"two\nthree"
)
result = _one_read(monkeypatch, doc, _LOCAL_ROWS, ranged)
assert result == "Document skills/b.md:\ndate: 2024-06-15\nlines: 2-3 of 5\ntwo\nthree"
repo_doc = _doc("repo", "b.md", "B", "one\ntwo\nthree\nfour\nfive")
caplog.clear()
result = _one_read(
monkeypatch, repo_doc, _AMBIGUOUS_ROWS, {"path": "repo/b.md", "offset": 2, "limit": 2}
)
assert result == "Document repo/b.md:\ndate: 2024-06-15\nlines: 2-3 of 5\ntwo\nthree"
# The ambiguity is logged at WARNING (the agent's INFO per-turn log
# lines are captured too — filter to the resolver's records).
warnings = [
r for r in caplog.records if r.levelname == "WARNING" and r.name == "app.rag.git_sources"
]
assert len(warnings) == 1
assert "repo" in warnings[0].message
def test_origin_lines_truncated_range_read(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Shape 4 (range read whose SLICE exceeds the cap): the origin block
at the same position; the cap on the slice + the shared marker + the
pinned notice (``{total}`` = the SLICE's length) all unchanged."""
cap = 40
doc = _doc("skills", "c.md", "C", "a" * 30 + "\n" + "b" * 30 + "\n" + "c" * 30)
slice_text = "a" * 30 + "\n" + "b" * 30 # the requested lines 1-2 (61 chars)
ranged = {"path": "skills/c.md", "offset": 1, "limit": 2}
result = _one_read(monkeypatch, doc, _GIT_ROWS, ranged, read_max_chars=cap)
assert result == (
"Document skills/c.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"lines: 1-2 of 3\n"
+ slice_text[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
)
result = _one_read(monkeypatch, doc, _GIT_PINNED_ROWS, ranged, read_max_chars=cap)
assert result == (
"Document skills/c.md:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"branch: release/1.2\n"
"lines: 1-2 of 3\n"
+ slice_text[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
)
result = _one_read(monkeypatch, doc, _LOCAL_ROWS, ranged, read_max_chars=cap)
assert result == (
"Document skills/c.md:\n"
"date: 2024-06-15\n"
"lines: 1-2 of 3\n"
+ slice_text[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
)
repo_doc = _doc("repo", "c.md", "C", "a" * 30 + "\n" + "b" * 30 + "\n" + "c" * 30)
result = _one_read(
monkeypatch, repo_doc, _AMBIGUOUS_ROWS, {"path": "repo/c.md", "offset": 1, "limit": 2},
read_max_chars=cap,
)
assert result == (
"Document repo/c.md:\n"
"date: 2024-06-15\n"
"lines: 1-2 of 3\n"
+ slice_text[:cap]
+ "\n"
+ TRUNCATION_MARKER
+ "\n"
+ READ_TRUNCATION_NOTICE.format(shown=cap, total=len(slice_text))
)
def test_origin_lines_image_doc_keeps_marker_position(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The image-doc shape keeps its phase-122 pin: the marker stays
IMMEDIATELY ABOVE the content, and the origin lines ride ABOVE the
marker (``Document …: / date: / origin: / branch: / marker /
content``); a local image doc is byte-identical to pre-phase."""
doc = _doc("skills", "img.png", "Img", "IMG-DESC")
doc.is_image = True
plain = {"path": "skills/img.png"}
result = _one_read(monkeypatch, doc, _GIT_PINNED_ROWS, plain)
assert result == (
"Document skills/img.png:\n"
"date: 2024-06-15\n"
f"origin: {_GIT_URL}\n"
"branch: release/1.2\n"
f"{agent.IMAGE_DOC_MARKER}\n"
"IMG-DESC"
)
doc = _doc("skills", "img.png", "Img", "IMG-DESC")
doc.is_image = True
result = _one_read(monkeypatch, doc, _LOCAL_ROWS, plain)
assert result == (
"Document skills/img.png:\ndate: 2024-06-15\n"
f"{agent.IMAGE_DOC_MARKER}\n"
"IMG-DESC"
)
# ---------- grep (the phase-68 A5 contract under the new name) ----------
tests/unit/test_document_viewer.py
+362 -8
View File
@@ -20,9 +20,29 @@ Frontend side:
* phase 26 — the shared ``renderDocument`` export in ``document.js``, the
import-safe page guard, the modal skeleton on chat + Sources, and the
modal module's close/focus/URL contract.
Phase 145 (task 03) — the content payload's git origin fields + the
``renderDocument`` "Original" anchor (modal AND /document.html, one
site):
* the ``DocContent`` omission rule — both origin keys ABSENT on the
wire when ``None`` (never ``null``; non-git payload byte-identical);
* the content endpoint (stubbed session) — a matching ``kind='git'``
row adds ``origin_url`` (+ ``origin_branch`` for a pinned row), a
legacy embedded-credential row is served sanitized, and local /
ambiguous sources keep the pre-phase key set;
* the anchor itself (source pins + a node behavioral check over a
minimal fake DOM) — ``createElement("a")``, the full URL verbatim as
``href`` AND visible text (the copyable-link contract),
``target="_blank" rel="noopener"``, the ``title`` = full URL (+
`` — branch: <name>`` when pinned), inserted AFTER the path code and
BEFORE the Created badge (absent → the pre-phase six children), no
innerHTML/outerHTML (the module's ONE innerHTML survives), and the
``.doc-origin`` CSS mirroring the ``.doc-path`` ellipsis idiom with
the ``.doc-link`` affordance (+ the ≤640px squeeze).
"""
from __future__ import annotations
import json
import re
import shutil
import subprocess
@@ -35,7 +55,8 @@ from fastapi.testclient import TestClient
from app.api.docs import doc_format
from app.db import get_db
from app.main import create_app
from app.models import Document
from app.models import Document, GitSource
from app.schemas import DocContent
from tests.conftest import ADMIN_PASSWORD
FRONTEND = Path(__file__).resolve().parents[2] / "frontend"
@@ -126,22 +147,39 @@ class _FakeResult:
return self._row
class _FakeScalars:
"""The resolver's ``db.scalars(...).all()`` — only the git rows
ever reach the caller (the ``kind = 'git'`` WHERE clause is
emulated, the task-01 ``_GitOnlySession`` pattern)."""
def __init__(self, rows: tuple[GitSource, ...]) -> None:
self._rows = [row for row in rows if row.kind == "git"]
def all(self) -> list[GitSource]:
return self._rows
class _FakeSession:
def __init__(self, row: object) -> None:
def __init__(self, row: object, git_rows: tuple[GitSource, ...] = ()) -> None:
self._row = row
self._git_rows = git_rows
def execute(self, _stmt: object) -> _FakeResult:
return _FakeResult(self._row)
def scalars(self, _stmt: object) -> _FakeScalars:
return _FakeScalars(self._git_rows)
def _client_with_row(row: object) -> TestClient:
def _client_with_row(row: object, git_rows: tuple[GitSource, ...] = ()) -> TestClient:
"""Fresh app whose ``get_db`` dependency is a stub returning ``row``
(``None`` → no matching document row). Phase 79: the endpoint is
user-gated, so the client signs in as the admin first — these tests
pin the CONTENT mapping (200/404/422), not the auth contract (which
``test_auth_api.py`` pins)."""
(``None`` → no matching document row) and, for the phase-145
origin lookup, *git_rows* (default: no rows — the non-git corner).
Phase 79: the endpoint is user-gated, so the client signs in as
the admin first — these tests pin the CONTENT mapping (200/404/
422), not the auth contract (which ``test_auth_api.py`` pins)."""
app = create_app()
app.dependency_overrides[get_db] = lambda: _FakeSession(row)
app.dependency_overrides[get_db] = lambda: _FakeSession(row, git_rows)
client = TestClient(app)
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, f"admin login failed: {r.status_code} {r.text}"
@@ -217,6 +255,322 @@ def test_content_requires_both_params() -> None:
assert client.get("/api/documents/content", params={"source": "Homelab"}).status_code == 422
# ---------------------------------------------------------------------------
# Phase 145 (task 03) — the content payload's git origin fields + the
# renderDocument "Original" anchor (modal AND /document.html, one site)
# ---------------------------------------------------------------------------
def _text_doc() -> Document:
"""The content-endpoint stub's document row (a markdown doc — no
summary, the pre-phase shape)."""
doc = Document(
source="Homelab",
path="notes/deep mark.md",
full_path="/tmp/deep mark.md",
title="Deep Mark",
content="# Deep Mark\n\nbody",
content_hash="f" * 64,
is_image=False, # phase 122: the stub session never applies defaults
)
doc.indexed_at = datetime(2026, 8, 22, 1, 2, 3, tzinfo=UTC)
doc.created_at = datetime(2026, 8, 20, 9, 0, 0, tzinfo=UTC)
return doc
def _text_doc_content(
origin_url: str | None = None, origin_branch: str | None = None
) -> DocContent:
"""One text doc's content payload (the pre-phase shape) with the
phase-145 origin fields optionally set."""
return DocContent(
source="S",
path="a.md",
title="A",
format="md",
created_at="2026-01-01T00:00:00+00:00",
content="body",
indexed_at="2026-01-01T00:00:00+00:00",
chunks=1,
origin_url=origin_url,
origin_branch=origin_branch,
)
def test_doc_content_origin_fields_absent_on_the_wire_when_unset() -> None:
"""The model-level omission rule (the :func:`_drop_absent_share_url`
precedent, the ``image_url`` shape): a non-git document (no
``origin_url``) serializes with BOTH origin keys ABSENT — never
``null`` — so the payload is byte-identical to pre-phase. A
sanitized-URL-only payload (an unpinned row) drops
``origin_branch``; a pinned row keeps both keys."""
dumped = _text_doc_content().model_dump()
assert "origin_url" not in dumped # absent — never null
assert "origin_branch" not in dumped # absent — never null
dumped = _text_doc_content(origin_url="https://host/x.git").model_dump()
assert dumped["origin_url"] == "https://host/x.git"
assert "origin_branch" not in dumped # NULL = the default branch — nothing to show
dumped = _text_doc_content(
origin_url="https://host/x.git", origin_branch="docs"
).model_dump()
assert dumped["origin_branch"] == "docs"
def test_content_origin_url_carried_for_a_matching_git_row() -> None:
"""The endpoint (stubbed session, no database): a ``kind='git'``
row whose URL basename matches the document's source adds
``origin_url`` — the sanitized clone URL — to the payload, and
every pre-phase key stands untouched. The row pins no branch →
``origin_branch`` is ABSENT (never ``null``)."""
row = GitSource(url="https://github.com/reese/Homelab.git", kind="git")
with _client_with_row((_text_doc(), 3), git_rows=(row,)) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "notes/deep mark.md"},
)
assert r.status_code == 200
body = r.json()
assert set(body) == {
"source", "path", "title", "format", "summary", "created_at",
"content", "indexed_at", "chunks", "is_image", "origin_url",
}
assert body["origin_url"] == "https://github.com/reese/Homelab.git"
assert "origin_branch" not in body # absent — never null (unpinned row)
def test_content_origin_branch_carried_for_a_pinned_row() -> None:
"""A pinned row (phase 135 ``branch``) also carries
``origin_branch`` — the branch the title attribute will show."""
row = GitSource(
url="https://github.com/reese/Homelab.git", kind="git", branch="docs"
)
with _client_with_row((_text_doc(), 3), git_rows=(row,)) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "notes/deep mark.md"},
)
body = r.json()
assert body["origin_url"] == "https://github.com/reese/Homelab.git"
assert body["origin_branch"] == "docs"
def test_content_origin_url_sanitizes_a_legacy_embedded_credential_row() -> None:
"""End-to-end (stubbed): a legacy row whose credential is still
embedded in the stored URL never leaks it — the payload carries
the phase-121 sanitized URL (the resolver's guarantee, one mapping
for the whole app)."""
row = GitSource(url="https://user:tok@example.com/reese/Homelab.git", kind="git")
with _client_with_row((_text_doc(), 3), git_rows=(row,)) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "notes/deep mark.md"},
)
body = r.json()
assert body["origin_url"] == "https://example.com/reese/Homelab.git"
assert "user:tok" not in r.text # the credential stays out of the wire
def test_content_origin_fields_absent_for_local_and_ambiguous_sources() -> None:
"""No git row (a plain local source) OR two rows sharing the
basename (ambiguous — never guess) → both origin keys ABSENT from
the JSON (not ``null`` — the omission pinned on the wire; the
payload is byte-identical to pre-phase, the ``test_content_known_
pair_maps_to_doc_content`` key-set pin)."""
local_only = (_text_doc(), 3)
with _client_with_row(local_only) as client: # no git rows at all
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "notes/deep mark.md"},
)
assert r.status_code == 200
assert "origin_url" not in r.json()
assert "origin_branch" not in r.json()
ambiguous = (
GitSource(url="https://a.example/org/Homelab.git", kind="git"),
GitSource(url="git@b.example:other/Homelab.git", kind="git"),
)
with _client_with_row(local_only, git_rows=ambiguous) as client:
r = client.get(
"/api/documents/content",
params={"source": "Homelab", "path": "notes/deep mark.md"},
)
assert r.status_code == 200
assert "origin_url" not in r.json()
assert "origin_branch" not in r.json()
# ---------- the renderDocument "Original" anchor (the shared core) ----------
def _fn(js: str, name: str) -> str:
"""The source of a top-level function via balanced-brace counting
(the ``test_date_editor.py`` pattern)."""
start = js.find(f"function {name}(")
assert start != -1, f"{name}() must exist in document.js"
i = js.find("(", start)
depth = 0
close = i
while i < len(js):
if js[i] == "(":
depth += 1
elif js[i] == ")":
depth -= 1
if depth == 0:
close = i
break
i += 1
brace = js.find("{", close)
depth = 0
for j in range(brace, len(js)):
if js[j] == "{":
depth += 1
elif js[j] == "}":
depth -= 1
if depth == 0:
return js[start : j + 1]
raise AssertionError(f"unbalanced braces in {name}()")
def test_origin_anchor_is_a_copyable_external_link() -> None:
"""The anchor (house-style source contract): ``createElement("a")``
(the never-build-HTML rule), the ``doc-origin`` class, ``href =
doc.origin_url``, ``target="_blank" rel="noopener"`` (the modal's
"Full page" escape-hatch idiom), the full URL VERBATIM as the
visible text (the copyable-link contract — no decorative label),
and the ``title`` = the full URL, plus `` — branch: <name>`` when
the payload pins a branch (the pathCode precision idiom)."""
body = _fn(_read(DOCUMENT_JS), "originAnchor")
assert 'document.createElement("a")' in body
assert 'a.className = "doc-origin"' in body
assert "a.href = doc.origin_url" in body
assert 'a.target = "_blank"' in body
assert 'a.rel = "noopener"' in body
assert "a.textContent = doc.origin_url" in body, (
"the visible text is the full URL verbatim (copy-pasteable)"
)
branch_title = 'doc.origin_url + " — branch: " + doc.origin_branch'
assert 'doc.origin_branch ? ' + branch_title + " : doc.origin_url" in body, (
"the title carries the full URL (+ the pinned branch when present)"
)
# The never-build-HTML shape: no innerHTML/outerHTML in the new code.
code = re.sub(r"//.*?$|/\*.*?\*/", "", body, flags=re.S | re.M)
assert "innerHTML" not in code and "outerHTML" not in code
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_origin_anchor_behaves_for_branch_and_no_branch_variants() -> None:
"""Behavioral check (node, minimal fake DOM) of the real anchor
builder: the exact ``href``/``target``/``rel``/``textContent``/
``title`` for a git doc, branch and no-branch variants."""
fn = _fn(_read(DOCUMENT_JS), "originAnchor")
fake_dom = (
"globalThis.document = { createElement: (tag) => {"
' const el = { tag, className: "", href: "", target: "", rel: "",'
' textContent: "", setAttribute(k, v) { this[k] = v; } };'
" return el; } };"
)
out = _run_node(
fake_dom
+ "\n" + fn
+ '\nconst g1 = originAnchor({ origin_url: "https://github.com/reese/skills.git",'
' origin_branch: "main" });' + "\n"
+ '\nconst g2 = originAnchor({ origin_url: "https://github.com/reese/skills.git" });' + "\n"
+ "console.log(JSON.stringify(g1));\nconsole.log(JSON.stringify(g2));"
)
j1, j2 = (json.loads(line) for line in out.splitlines())
for j in (j1, j2):
assert j["tag"] == "a"
assert j["className"] == "doc-origin"
assert j["href"] == "https://github.com/reese/skills.git"
assert j["target"] == "_blank"
assert j["rel"] == "noopener"
# The full URL VERBATIM as the visible text (copy-pasteable).
assert j["textContent"] == "https://github.com/reese/skills.git"
# The title: the full URL, + the pinned branch when present.
assert j1["title"] == "https://github.com/reese/skills.git — branch: main"
assert j2["title"] == "https://github.com/reese/skills.git"
def test_origin_anchor_sits_after_the_path_code_before_the_created_badge() -> None:
"""renderDocument (the ONE shared core — modal AND /document.html):
the anchor joins the meta row AFTER the path ``code`` and BEFORE
the Created badge — and ONLY when the payload carries
``origin_url``. Absent (a non-git document — the field simply
missing on the wire) → the gate builds the empty array, the
spread adds nothing, and the meta row's children are EXACTLY the
pre-phase six (the existing badge-order pins pass unchanged)."""
render = _fn(_read(DOCUMENT_JS), "renderDocument")
gate_i = render.find("const originItems = doc.origin_url ? [originAnchor(doc)] : [];")
call_i = render.find("metaEl.replaceChildren(")
path_i = render.find("pathCode,", call_i)
spread_i = render.find("...originItems,", call_i)
created_i = render.find('metaBadge("doc-created"', call_i)
assert -1 < gate_i < call_i < path_i < spread_i < created_i, (
"the anchor gate precedes the row build and lands AFTER the path code, "
"BEFORE the Created badge"
)
# The gate: present only on a truthy origin_url — an ABSENT field
# (undefined) is falsy → [] → the pre-phase children stand.
assert "doc.origin_url ? [originAnchor(doc)] : []" in render
# The spread is the meta row's ONLY conditional child (the six
# pre-phase children are the fixed ones).
assert render[call_i:].count("...") == 1
def test_origin_anchor_keeps_the_module_xss_contract() -> None:
"""The module's ONE innerHTML (the escape-first markdown render)
survives — the new anchor adds no HTML-building of its own
(createElement + properties/textContent only)."""
code = re.sub(r"//.*", "", re.sub(r"/\*.*?\*/", "", _read(DOCUMENT_JS), flags=re.S))
assert code.count("innerHTML") == 1
assert "outerHTML" not in code
def test_doc_origin_css_mirrors_the_doc_path_ellipsis_idiom() -> None:
"""styles.css: the ``.doc-origin`` rule sits next to the
``.doc-path`` family (mono, 0.75rem, the same 26rem ellipsis
budget + 6rem floor) with the ``.doc-link`` affordance
(brand-ink, underline on hover/focus — the existing link
tokens), provenance-cited, and the ≤640px block squeezes it like
the path (16rem — the row's own wrap/clip rules are unchanged:
the anchor is one more inline item)."""
css = _read(FRONTEND / "assets" / "styles.css")
rule_i = css.find(".doc-origin {")
assert rule_i != -1, "styles.css must style .doc-origin"
# Provenance comment directly above the rule (house style).
comment_start = css.rfind("/*", 0, rule_i)
assert -1 < comment_start < rule_i and "Phase 145 (task 03)" in css[comment_start:rule_i]
# Next to the .doc-path family (before .doc-chunks).
assert css.find(".doc-path {") < rule_i < css.find(".doc-chunks {")
rule = css[rule_i : css.find("}", rule_i)]
for decl in (
"font-family: var(--mono)",
"font-size: 0.75rem",
"max-width: 26rem",
"white-space: nowrap",
"overflow: hidden",
"text-overflow: ellipsis",
"min-width: 6rem",
"color: var(--brand-ink)",
"text-decoration: none",
):
assert decl in rule, f".doc-origin must carry {decl!r}"
# The .doc-link hover/focus affordance (the existing link idiom).
assert ".doc-origin:hover, .doc-origin:focus-visible { text-decoration: underline; }" in css
# The ≤640px squeeze mirrors .doc-path's 16rem (same media block).
mobile = css.find("@media (max-width: 640px)")
path_sq = css.find(".doc-path { max-width: 16rem; }", mobile)
origin_sq = css.find(".doc-origin { max-width: 16rem; }", mobile)
close_640 = css.find("@media", mobile + 1)
assert -1 < path_sq < origin_sq < (close_640 if close_640 != -1 else len(css)), (
"the mobile block squeezes the anchor like the path"
)
assert "url(http" not in css and "@import url(" not in css, (
"no CDN (AGENTS.md rule 6)"
)
# ---------------------------------------------------------------------------
# Viewer URL builder — encoded query (spaces/slashes in real paths)
# ---------------------------------------------------------------------------
tests/unit/test_git_sources.py
+121
View File
@@ -179,3 +179,124 @@ def test_alias_env_fallback_unchanged(monkeypatch: pytest.MonkeyPatch) -> None:
assert urls == ["https://env.example/one.git"]
assert origin == "env"
# --- phase 145 (task 01): origin_for_source (the read result's origin) ---
class _GitOnlySession:
"""The origin resolver's session (phase 145, task 01): emulates the
resolver's ``kind = 'git'`` WHERE clause — only the git rows ever
reach the caller, exactly like the real query; the seeded local
rows prove they are never candidates."""
def __init__(self, rows: list[GitSource]) -> None:
self._rows = [row for row in rows if row.kind == "git"]
self.statements: list[Any] = []
def scalars(self, stmt: Any) -> _FakeScalars:
self.statements.append(stmt)
return _FakeScalars(self._rows)
# --- the basename mapping --------------------------------------------------
def test_origin_basename_mapping() -> None:
"""The URL's path component — after the ``://``, or after the ``:``
for scp-style ``git@host:owner/repo`` — trailing ``/`` stripped,
a single trailing ``.git`` suffix stripped, last segment taken. An
``@`` inside the PATH is not userinfo and stays (no split, no
strip)."""
cases = {
"https://github.com/owner/myrepo.git": "myrepo", # the .git suffix
"https://github.com/owner/myrepo": "myrepo", # no suffix, same dir
"https://github.com/owner/myrepo/": "myrepo", # trailing slash
"https://github.com/owner/myrepo.git/": "myrepo", # both
"git@github.com:owner/myrepo.git": "myrepo", # scp-style + suffix
"git@github.com:owner/myrepo": "myrepo", # scp-style, no suffix
"ssh://git@host/owner/myrepo.git": "myrepo", # ssh scheme, @ in the host part
"https://host/a@b/myrepo.git": "myrepo", # @ inside the PATH — not userinfo
}
for url, expected in cases.items():
assert resolver._url_basename(url) == expected, url
# --- the match: sanitize + branch ------------------------------------------
def test_origin_exact_match_is_sanitized() -> None:
"""EXACTLY ONE match → the phase-121 masked URL + the branch — a
legacy embedded-credential row never leaks its secret (the stored
value is untouched; the OUTPUT mask does the work)."""
session = _GitOnlySession(
[GitSource(url="https://user:tok@host/x.git", kind="git", branch=None)]
)
assert resolver.origin_for_source(session, "x") == ("https://host/x.git", None) # pyright: ignore[reportArgumentType]
def test_origin_branch_passthrough() -> None:
"""The branch rides through as-is: pinned → the name (the read
result's ``branch:`` line), NULL → ``None`` (the default branch —
no ``branch:`` line)."""
session = _GitOnlySession(
[GitSource(url="https://host/x.git", kind="git", branch="release/1.2")]
)
assert resolver.origin_for_source(session, "x") == ("https://host/x.git", "release/1.2") # pyright: ignore[reportArgumentType]
def test_origin_local_rows_never_origins() -> None:
"""Local rows are never origins (nothing to link to from the user's
browser): a source that exists ONLY as a local row gets no origin
(zero git candidates); a local + git pair resolves to the git row."""
local = GitSource(url="/srv/repo", kind="local", path="/srv/repo")
session = _GitOnlySession([local]) # the WHERE clause drops it
assert resolver.origin_for_source(session, "repo") is None # pyright: ignore[reportArgumentType]
session = _GitOnlySession(
[local, GitSource(url="https://host/repo.git", kind="git", branch=None)]
)
assert resolver.origin_for_source(session, "repo") == ("https://host/repo.git", None) # pyright: ignore[reportArgumentType]
# --- the no-guess rows: None + WARNING --------------------------------------
def test_origin_zero_match_is_none_with_warning(
caplog: pytest.LogCaptureFixture,
) -> None:
"""No git row's basename matches (a plain local source) → ``None``
+ a WARNING naming the source — the read result stays
byte-identical to pre-phase."""
session = _GitOnlySession([GitSource(url="https://host/other.git", kind="git")])
assert resolver.origin_for_source(session, "repo") is None # pyright: ignore[reportArgumentType]
warnings = [record for record in caplog.records if record.levelname == "WARNING"]
assert len(warnings) == 1
assert "repo" in warnings[0].message
def test_origin_ambiguous_is_none_with_warning(
caplog: pytest.LogCaptureFixture,
) -> None:
"""TWO-OR-MORE rows share the basename → ``None`` + a WARNING naming
the sanitized candidates (never guess — a wrong origin would ride
into every answer about the document). The candidates are logged
SANITIZED: a legacy embedded credential stays out of the log too."""
session = _GitOnlySession(
[
GitSource(url="https://user:tok@a.example/org/repo.git", kind="git"),
GitSource(url="git@b.example:other/repo.git", kind="git"),
]
)
assert resolver.origin_for_source(session, "repo") is None # pyright: ignore[reportArgumentType]
warnings = [record for record in caplog.records if record.levelname == "WARNING"]
assert len(warnings) == 1
assert "https://a.example/org/repo.git" in warnings[0].message
assert "git@b.example:other/repo.git" in warnings[0].message
assert "tok" not in warnings[0].message
tests/unit/test_prompt_lock.py
+75 -6
View File
@@ -15,6 +15,7 @@ from __future__ import annotations
import hashlib
from app.rag.prompts import (
ORIGIN_RULE,
PERSONA,
SUGGEST_INTRO,
TOOLS_SECTION,
@@ -77,12 +78,18 @@ def test_high_and_low_bases_byte_locked() -> None:
#: form — do not re-read a document read IN FULL (answer from it);
#: a PARTIAL document (a range read, or a truncated read) is not
#: complete in the context until read all — read the other parts with
#: ``offset``/``limit``. Only that sentence moved — the prefix (the
#: ``ls``-clause opening) and the suffix (the discipline-rules
#: ending) survived byte-identical, so they are the same anchors as
#: pre-phase-141.
TOOLS_SECTION_SHA256 = "3671680bbf4e04efe1186690c18d24869e4974ceb29485cedc3e34a9286bf3b2"
TOOLS_SECTION_LEN = 2656
#: ``offset``/``limit``. RE-CUT again for phase 145 (task 02): the
#: ``read`` clause gains the original-source rule — :data:`ORIGIN_RULE`,
#: the one sentence-group appended at the ``read`` clause's end (the
#: truncation workflow sentence) and BEFORE the ``grep`` clause begins
#: (the ``origin:`` line is a read-result fact; the rule binds the
#: ANSWER that uses what the read added). Only that sentence-group was
#: inserted — the prefix (the ``ls``-clause opening) and the suffix
#: (the discipline-rules ending) survived byte-identical, so they are
#: the same anchors as pre-phase-145; the pre-phase-145 sha/length are
#: kept below for the diff-style pin.
TOOLS_SECTION_SHA256 = "41a84885be98d4e15b95205694d59d364296a30e27758d39ae737881d5bacaa7"
TOOLS_SECTION_LEN = 3279
TOOLS_SECTION_PREFIX = (
"<tools>\n"
"You may extend your context with three tools. `ls` lists the "
@@ -94,6 +101,33 @@ TOOLS_SECTION_SUFFIX = (
"you need.\n</tools>"
)
#: The pre-phase-145 anchors — the section BEFORE :data:`ORIGIN_RULE` was
#: inserted. Kept for the diff-style pin: the ONLY byte change is the
#: rule's insertion, so removing the rule reproduces these exact bytes.
TOOLS_SECTION_PRE_PHASE_145_SHA256 = (
"3671680bbf4e04efe1186690c18d24869e4974ceb29485cedc3e34a9286bf3b2"
)
TOOLS_SECTION_PRE_PHASE_145_LEN = 2656
#: Phase 145 (task 02) — the original-source rule the ``read`` teaching
#: carries is :data:`app.rag.prompts.ORIGIN_RULE` (imported above, LOCKED
#: verbatim). :data:`TOOLS_SECTION` embeds that exact sentence-group at
#: the ``read`` clause's end (the truncation workflow sentence) and
#: BEFORE the ``grep`` clause begins. A read result for a git-sourced
#: document carries an ``origin:`` line (and a ``branch:`` line when a
#: branch is pinned) — the read contract's fact — so when the answer uses
#: such a document it includes the original source in the answer: a link
#: to the repository (the ``origin`` line) with the document's path (the
#: ``Document …`` header) named and the pinned branch when one is shown,
#: as the repository's file-view URL where the model can tell it
#: (``<repo-url>/blob/<branch>/<path>``), or the repository URL plus the
#: path as plain text otherwise. The rule is CONDITIONAL on the ``origin``
#: line's presence — a local-sourced document's read result carries none,
#: so the rule adds nothing to those answers (the model has no URL to
#: cite; it must not invent one — the sentence's conditioning does that
#: job).
def test_tools_section_byte_locked() -> None:
"""The ``<tools>`` teaching is LOCKED verbatim too (the E2E mock keys
@@ -105,6 +139,41 @@ def test_tools_section_byte_locked() -> None:
assert TOOLS_SECTION.endswith(TOOLS_SECTION_SUFFIX)
def test_tools_section_origin_rule_verbatim_at_its_position() -> None:
"""Phase 145 (task 02): the original-source rule is LOCKED verbatim
and sits at its EXACT position — the ``read`` clause's end (the
truncation workflow sentence) and BEFORE the ``grep`` clause begins.
The sentence-group is pinned byte-for-byte, and the surrounding
boundary (the truncation sentence's tail → the rule → the ``grep``
clause's opening) is pinned as one contiguous substring, so a move
or re-spelling of ANY byte (the rule's or the boundary's) fails."""
# Verbatim presence.
assert ORIGIN_RULE in TOOLS_SECTION
assert TOOLS_SECTION.count(ORIGIN_RULE) == 1 # exactly once
# Exact position: after the truncation sentence, before the grep
# clause — one contiguous span, no drift at either seam.
assert (
"it searches the whole document. "
+ ORIGIN_RULE
+ " `grep` locates an exact string"
) in TOOLS_SECTION
def test_tools_section_origin_rule_is_the_only_byte_change() -> None:
"""Phase 145 (task 02) diff-style pin: the section's OTHER bytes are
byte-identical to pre-phase — the pre-phase section with ONLY the new
sentence-group inserted. Removing the rule (and the single space
that now separates it from the ``grep`` clause) reproduces the
pre-phase-145 section byte-for-byte (sha + length), and the length
delta is exactly the rule's length plus that one inserted space."""
assert len(TOOLS_SECTION) == (
TOOLS_SECTION_PRE_PHASE_145_LEN + len(ORIGIN_RULE) + 1
)
pre_phase = TOOLS_SECTION.replace(ORIGIN_RULE + " ", "")
assert len(pre_phase) == TOOLS_SECTION_PRE_PHASE_145_LEN
assert _sha256(pre_phase) == TOOLS_SECTION_PRE_PHASE_145_SHA256
# ---------- the LOW prompt's locked DEFLECT_MODE body ----------
#: The ``DEFLECT_MODE`` body exactly as it was pre-phase-112 (the
tests/unit/test_prompts.py
+38
View File
@@ -48,6 +48,7 @@ import pytest
from app.config import Settings
from app.models import Document
from app.rag.prompts import (
ORIGIN_RULE,
PERSONA,
SHOW_IMAGE_CLAUSE,
SUGGEST_INTRO,
@@ -463,6 +464,43 @@ def test_tools_section_phase72_clauses_in_high_prompt_not_low() -> None:
assert "including the source name" not in low
def test_tools_section_phase145_origin_rule() -> None:
"""Phase 145 (task 02): the ``<tools>`` ``read`` clause carries the
original-source rule (LOCKED verbatim in :data:`ORIGIN_RULE`) — when
the answer uses a document whose read result carries an ``origin:``
line, it includes the original source (a link to the repository
from the origin line, with the document's path and the pinned branch
where shown) so the user opens the complete original in one step.
The rule rides the HIGH prompt (``<tools>`` is HIGH-only) and never
the LOW/deflection prompt. The citation discipline is UNCHANGED —
the rule extends the ANSWERS of cited documents; it does not widen
the cite set (the "cite only what you read" sentence stays
byte-identical — pinned in ``test_prompt_lock``)."""
# The rule is present verbatim in the constant, exactly once, and at
# its exact position (the read clause's end, before the grep clause).
assert TOOLS_SECTION.count(ORIGIN_RULE) == 1
assert (
"it searches the whole document. "
+ ORIGIN_RULE
+ " `grep` locates an exact string"
) in TOOLS_SECTION
# It rides the built HIGH prompt with the rest of the section.
doc = _doc("kubernetes.md", "Talos Linux on three nodes.", "Kubernetes")
high = build_high_prompt([doc])
assert ORIGIN_RULE in high
# ...and never the LOW/deflection prompt (``<tools>`` is HIGH-only).
for low in (build_deflect_prompt(["T1"]), build_deflect_prompt([])):
assert ORIGIN_RULE not in low
# The citation discipline is UNCHANGED: the rule does not widen the
# cite set (the "cite only what you read" sentence is untouched).
assert (
"Cite only the document(s) you read — or, if you answered from a "
"suggested summary without reading it, cite that suggested "
"document — never a document you neither read nor used."
) in high
assert "Cite only the document(s) you read" in SUGGEST_INTRO
def test_documents_section_leads_with_the_suggest_intro() -> None:
"""Phase 118, task 03: the ``<documents>`` section leads with the
start-here :data:`SUGGEST_INTRO` line BEFORE the first block (the