phase: 187_sync_peak_memory_line
**Phase 187 — final verification pass: all criteria met, repo 100% green.**
Verified (all four tasks already in `complete/`)
- `app/core/memory.py`: `rss_mb` / `rss_peak_mb` / `cgroup_peak_mb` / `fmt_mb`, pure stdlib, never-raise, **100%** coverage, 24 branch tests.
- Lifecycle lines append-only: `sync: started … rss_mb=`, all three terminals `… rss_mb= rss_peak_mb=` (failed = sanitized one-liner beside the surviving `logger.exception`), `import: summary … rss_mb=`; prefixes byte-identical.
- `GET /api/sync/status`: additive `rss_mb`/`rss_peak_mb` (`int|null`), keyed off the ONE `_sample_rss` (run start, phase transitions, forced terminals, 5 s-throttled hook), `peak >= current` invariant, anonymous 403.
- `sources.js` `fmtSyncPeakClause`: ` · peak <N> MB` trailing the counts line / "Sync cancelled", nothing when null, no new control/live region.
Gates (exact commands)
- `uv run pytest --cov=app --cov-report=term-missing` → **4570 passed**, TOTAL **99%**, `app/api/sync.py` **100%**, `app/core/memory.py` **100%**
- `uv run ruff check .` → clean · `uv run pyright` → **0 errors**
- `uv run pytest tests/e2e/test_sync_peak_memory_visible.py -v --no-cov` → **5 passed in isolation** (ran twice)
- All 27 touched E2E families re-run **in isolation**, all green (progress_box 8, cancel 3, status_freshness 4, summary_visibility 3, sync_button 3, session_reset 4, git_freeze 2, git_log_stream_dates 5, PDF/image suites, …)
Completion criteria: every checkbox ✅ except the commit + phase-dir move, which belong to the harness.
Documentation
- README reviewed in full (pages, commands, config tables cross-checked against `app/config.py` — 85 env rows, all defaults match; phase-187 ops section, greps, process-vs-cgroup caveat): consistent, no changes needed.
- `AGENTS.md`: **fixed one pre-existing inconsistency** — rule 6 claimed the no-CDN test covers "the index page"; it actually parametrizes 10 pages (all but `/doc-edit.html`).
Notable
- Two failures early in the pass were self-inflicted by running the E2E module concurrently with the full suite on the shared Postgres (TRUNCATE deadlock + one order-dependent `test_overview` flake); both vanish under the documented serial isolation — no code defect, no change made.
- **PLAN §9 still needs the owner's reconciliation** (phase 187 adds the sync memory fields + two status keys; PLAN is never agent-edited — AGENTS rule 10 carries the spec). `TODO.md` holds an unrelated owner note, left untouched.
Next pending phase: `188_chunks_hnsw_index`.
This commit is contained in:
@@ -0,0 +1,61 @@
|
||||
"""Shared sync-result-line helpers (phase 187, task 03).
|
||||
|
||||
The Sources page's sync result line — the ``fmtSyncResult`` counts line
|
||||
and the neutral ``"Sync cancelled"`` line the ``#sync-progress`` box
|
||||
speaks — now trails the run's PEAK-MEMORY clause (`` · peak 1936 MB``),
|
||||
sourced from ``GET /api/sync/status``'s additive ``rss_peak_mb`` /
|
||||
``rss_mb`` pair (the 2026-10-15 incident: "how much RAM did that sync
|
||||
use?" used to mean reading ``systemd`` unit-teardown lines off three
|
||||
boots; now it is on the page that started the run and in the log).
|
||||
|
||||
The clause is per-run DATA, so the suites that pin the OWNER-VISIBLE
|
||||
counts pin the counts plus an OPTIONAL clause:
|
||||
|
||||
* :func:`counts_line` — a Playwright-ready pattern: the exact expected
|
||||
line, with `` · peak <N> MB`` allowed at the end (and the whole line
|
||||
anchored, so a mutated counts line still fails);
|
||||
* :func:`strip_peak_clause` — for the suites that compare two runs' or
|
||||
two apps' lines byte-for-byte: the memory number belongs to the
|
||||
process, not to the KB, so it is not part of a byte-identity contract.
|
||||
|
||||
A NULL pair renders NOTHING (``fmtSyncPeakClause``), which is why the
|
||||
clause is optional rather than required: an idle app, a status that
|
||||
predates phase 187, and the upload job all settle without it.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
#: The clause sources.js appends (``fmtSyncPeakClause``), as a regex —
|
||||
#: optional, because a null peak/current pair appends the empty string.
|
||||
PEAK_CLAUSE = r"(?: · peak \d+ MB)?"
|
||||
|
||||
#: The same clause, for a substitution (``strip_peak_clause``): anchored
|
||||
#: at end of line, so nothing else is ever stripped.
|
||||
_PEAK_CLAUSE_TAIL = re.compile(r" · peak \d+ MB$")
|
||||
|
||||
|
||||
def counts_line(expected: str | re.Pattern[str]) -> re.Pattern[str]:
|
||||
"""Match the EXACT ``expected`` sync result line, clause included or
|
||||
not — the pin a suite writes when it cares about the counts, not
|
||||
about the number of megabytes the run happened to reach.
|
||||
|
||||
``expected`` is either the literal line (escaped, so the ``·`` and
|
||||
the digits mean exactly what they say) or a suite's own shape
|
||||
pattern (its anchors are dropped and re-applied around the optional
|
||||
clause, so ``^3 added$`` becomes ``^3 added(?: · peak \\d+ MB)?$``).
|
||||
"""
|
||||
if isinstance(expected, re.Pattern):
|
||||
body = expected.pattern
|
||||
if body.startswith("^"):
|
||||
body = body[1:]
|
||||
if body.endswith("$"):
|
||||
body = body[:-1]
|
||||
return re.compile(rf"^{body}{PEAK_CLAUSE}$")
|
||||
return re.compile(rf"^{re.escape(expected)}{PEAK_CLAUSE}$")
|
||||
|
||||
|
||||
def strip_peak_clause(text: str) -> str:
|
||||
"""``text`` without its trailing peak clause (if any) — the
|
||||
byte-identity comparison's view of the line."""
|
||||
return _PEAK_CLAUSE_TAIL.sub("", text)
|
||||
@@ -90,6 +90,7 @@ from app.models import Document
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
||||
from e2e.mock_llm import IMAGE_DESCRIPTION_ANSWER
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
GIT_SOURCES_URL = "/git-sources.html"
|
||||
@@ -381,7 +382,7 @@ def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
|
||||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(expected_result)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(expected_result))
|
||||
|
||||
|
||||
def _seed_image_doc(
|
||||
|
||||
@@ -102,6 +102,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -447,7 +448,7 @@ def _expect_counts_line(page: Page, expected: str) -> None:
|
||||
"""The counts line — the owner-visible sync report on the single sync
|
||||
text surface (phase 150; sources.js fmtSyncResult; the zero "updated"
|
||||
part is omitted)."""
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(expected, timeout=15_000)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(expected), timeout=15_000)
|
||||
|
||||
|
||||
def _goto_sources(page: Page, base_url: str) -> None:
|
||||
|
||||
@@ -104,6 +104,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -483,7 +484,7 @@ def test_sync_survives_a_force_push(
|
||||
assert body["state"] == "success", body
|
||||
# The fixture doc (one A9-format file) is the only change — the
|
||||
# counts line rides the box (phase 150).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("1 added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("1 added"))
|
||||
# The REAL clone was imported: the fixture path is in the Sources
|
||||
# table (the drill-down tree, the test_sync_button pattern).
|
||||
_expect_fixture_row(page)
|
||||
@@ -511,7 +512,7 @@ def test_sync_survives_a_force_push(
|
||||
# …and the digest delta re-indexed the file as updated (the counts
|
||||
# are the secondary signal — the content assertion below is
|
||||
# primary). The counts line rides the box (phase 150).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("0 added · 1 updated")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("0 added · 1 updated"))
|
||||
# The KB holds the NEW head: the document modal serves the V2
|
||||
# sentinel — the reset's content flowed through the import.
|
||||
_expect_fixture_row(page)
|
||||
@@ -533,7 +534,7 @@ def test_sync_survives_a_force_push(
|
||||
# "1 unchanged" means the content hash did not move — drift back
|
||||
# to V1 would have re-reported as "1 updated" (the no-drift pin).
|
||||
# The counts line rides the box (phase 150).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("0 added · 1 unchanged")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("0 added · 1 unchanged"))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -137,6 +137,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
from scripts.git_sync import file_commit_dates
|
||||
from scripts.import_docs import repo_name
|
||||
|
||||
@@ -488,7 +489,7 @@ def _run_sync(page: Page, expected_line: str | re.Pattern[str]) -> dict[str, Any
|
||||
expect(btn).to_be_enabled() # never stale at the terminal state
|
||||
expect(page.locator("#sync-error-banner")).to_be_hidden()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
expected_line, timeout=UI_SETTLE_TIMEOUT_MS
|
||||
counts_line(expected_line), timeout=UI_SETTLE_TIMEOUT_MS
|
||||
)
|
||||
return dict(body["detail"] or {})
|
||||
|
||||
|
||||
@@ -103,6 +103,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -515,7 +516,7 @@ def test_add_with_branch_syncs_that_branch(
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=15_000)
|
||||
# Both branch docs are fresh after the per-test truncate — the
|
||||
# counts line rides the box (phase 150).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("2 added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("2 added"))
|
||||
|
||||
# --- the branch's content is what got indexed ----------------------
|
||||
expect(catalog_has_doc(page, FIXTURE_NAME, "release-only.md")).to_have_count(1)
|
||||
@@ -539,7 +540,7 @@ def test_add_with_branch_syncs_that_branch(
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=15_000)
|
||||
# Nothing re-embeds (sha256 delta) — the no-op run announces the
|
||||
# unchanged count (release's two files) on the box (phase 150).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("0 added · 2 unchanged")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("0 added · 2 unchanged"))
|
||||
# The branch's doc survived the prune re-import.
|
||||
expect(catalog_has_doc(page, FIXTURE_NAME, "release-only.md")).to_have_count(1)
|
||||
|
||||
@@ -577,7 +578,7 @@ def test_default_branch_source_unaffected(
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=15_000)
|
||||
# The default branch's two docs are fresh — the counts line rides
|
||||
# the box (phase 150).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("2 added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("2 added"))
|
||||
|
||||
# The default branch (main) is what landed — the pre-phase
|
||||
# byte-identical guarantee, observable through the UI…
|
||||
@@ -684,7 +685,7 @@ def test_row_editor_branch_change_fails_loud(
|
||||
# The KB is still empty (the failed sync aborted before the
|
||||
# import) — the restored branch's two docs are fresh additions;
|
||||
# the counts line rides the box (phase 150).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("2 added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("2 added"))
|
||||
# And the branch's doc is what is indexed again.
|
||||
expect(catalog_has_doc(page, FIXTURE_NAME, "release-only.md")).to_have_count(1)
|
||||
|
||||
|
||||
@@ -99,6 +99,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -330,7 +331,7 @@ def _sync_in_app(page: Page, app_url: str) -> None:
|
||||
expect(btn).to_be_enabled() # never stale — re-enabled at the terminal state
|
||||
# Both fixture docs are fresh after the per-test truncate — the
|
||||
# counts line rides the box (phase 150).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("2 added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("2 added"))
|
||||
|
||||
|
||||
def _source_node(tree_json: dict[str, Any]) -> dict[str, Any]:
|
||||
|
||||
@@ -59,6 +59,7 @@ from app.models import Chunk, Document
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
||||
from e2e.mock_llm import IMAGE_DESCRIPTION_ANSWER
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
GIT_SOURCES_URL = "/git-sources.html"
|
||||
@@ -342,7 +343,7 @@ def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
|
||||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(expected_result)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(expected_result))
|
||||
|
||||
|
||||
def _captured_systems(mock_llm: int) -> list[str]:
|
||||
|
||||
@@ -51,6 +51,7 @@ from app.rag.agent import IMAGE_DOC_MARKER
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
||||
from e2e.mock_llm import IMAGE_DESCRIPTION_ANSWER
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
GIT_SOURCES_URL = "/git-sources.html"
|
||||
@@ -280,7 +281,7 @@ def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
|
||||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(expected_result)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(expected_result))
|
||||
|
||||
|
||||
def _drill_to_source(page: Page, name: str) -> None:
|
||||
|
||||
@@ -70,6 +70,7 @@ from app.models import Document
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
||||
from e2e.mock_llm import IMAGE_DESCRIPTION_ANSWER, IMAGE_DESCRIPTION_CONTEXT_ANSWER
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -366,7 +367,7 @@ def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
|
||||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(expected_result)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(expected_result))
|
||||
|
||||
|
||||
def _ask(page: Page, app_url: str, question: str) -> None:
|
||||
|
||||
@@ -88,6 +88,7 @@ from app.rag.pdf import is_blank_image
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
||||
from e2e.mock_llm import IMAGE_DESCRIPTION_CONTEXT_ANSWER
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
SOURCE_ON = "e2e-pdf151" # the story app's archive stem (archive_source_name)
|
||||
@@ -427,7 +428,7 @@ def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
|
||||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(expected_result)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(expected_result))
|
||||
|
||||
|
||||
def _drill_to_source(page: Page, name: str) -> None:
|
||||
|
||||
@@ -59,6 +59,7 @@ from app.models import Document
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
||||
from e2e.mock_llm import IMAGE_DESCRIPTION_CONTEXT_ANSWER
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
SOURCE_NAME = "e2e-pdf" # the archive stem (archive_source_name)
|
||||
@@ -324,7 +325,7 @@ def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
|
||||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(expected_result)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(expected_result))
|
||||
|
||||
|
||||
def _drill_to_source(page: Page, name: str) -> None:
|
||||
|
||||
@@ -147,6 +147,7 @@ from app.rag.pdf import (
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import ADMIN_PASSWORD, SESSION_SECRET, USE_REAL_LLM, _wait_http
|
||||
from e2e.mock_llm import PICTORIAL_PAGE_ANSWER
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
SOURCE_NAME = "e2e-pictorial" # the archive stem (archive_source_name)
|
||||
@@ -479,7 +480,7 @@ def _sync_via_ui(page: Page, app_url: str, expected_result: str) -> None:
|
||||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(expected_result)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(expected_result))
|
||||
|
||||
|
||||
def _drill_to_source(page: Page, name: str) -> None:
|
||||
|
||||
@@ -101,6 +101,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -403,7 +404,7 @@ def _sync_from_sources_page(page: Page, app_url: str) -> dict[str, Any]:
|
||||
def _expect_counts_line(page: Page, expected: str) -> None:
|
||||
"""The Sources page's counts line reads exactly ``expected`` —
|
||||
"9 added" / "0 added · 3 unchanged · 6 pruned"."""
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(expected, timeout=15_000)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(expected), timeout=15_000)
|
||||
|
||||
|
||||
def _add_local_source(page: Page, app_url: str, path: Path) -> str:
|
||||
|
||||
@@ -52,6 +52,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -273,7 +274,7 @@ def test_admin_sync_lifecycle(page: Page, app_url: str, db_ready: None) -> None:
|
||||
# The fixture doc (one A9-format file) is the only change — the
|
||||
# settle's counts line rides the box (phase 150: the box is the
|
||||
# sole sync text surface).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("1 added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("1 added"))
|
||||
|
||||
# The REAL clone was imported: the fixture path is in the Sources
|
||||
# table (the sentinel lives inside it) — via the drill-down tree
|
||||
@@ -297,7 +298,7 @@ def test_admin_sync_lifecycle(page: Page, app_url: str, db_ready: None) -> None:
|
||||
# Nothing re-embedded (sha256 delta) — the no-op run announces the
|
||||
# unchanged count instead of an empty live region (the box keeps
|
||||
# its result line — phase 150).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("0 added · 1 unchanged")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("0 added · 1 unchanged"))
|
||||
# The doc survived the prune re-import (its file is still in the
|
||||
# repo) — via the drill-down tree (phase 97).
|
||||
_expect_fixture_row(page)
|
||||
|
||||
@@ -93,6 +93,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import PEAK_CLAUSE, counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -135,8 +136,9 @@ SYNCED_LABEL = re.compile(r"Synced \d{1,2}:\d{2}")
|
||||
|
||||
#: The settled SUCCESS result line — "N added" or "N added · M
|
||||
#: unchanged" (sources.js's fmtSyncResult — updated/pruned are zero on
|
||||
#: the re-sync: the repo is unchanged and nothing was pruned).
|
||||
RESULT_LINE = re.compile(r"^(\d+) added(?: · (\d+) unchanged)?$")
|
||||
#: the re-sync: the repo is unchanged and nothing was pruned) with the
|
||||
#: phase-187 peak clause optional at the end (null pair → no clause).
|
||||
RESULT_LINE = re.compile(rf"^(\d+) added(?: · (\d+) unchanged)?{PEAK_CLAUSE}$")
|
||||
|
||||
#: Settle budgets: the cancel terminal lands ~1 s after the click and
|
||||
#: the UI's 2 s poll settles at most one tick after the terminal
|
||||
@@ -547,7 +549,7 @@ def test_admin_cancel_lifecycle(page: Page, app_url: str, db_ready: None) -> Non
|
||||
# Phase 150: the neutral result line rides the box (the sole sync
|
||||
# text surface — the retired announcer span is gone).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
"Sync cancelled", timeout=SYNC_SETTLE_TIMEOUT_MS
|
||||
counts_line("Sync cancelled"), timeout=SYNC_SETTLE_TIMEOUT_MS
|
||||
)
|
||||
# A1: the cancel is never rendered as a failure — no error banner,
|
||||
# no error modal in the DOM.
|
||||
|
||||
@@ -69,6 +69,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -267,7 +268,7 @@ def _expect_settle_line(page: Page, expected: str) -> None:
|
||||
Phase 150: the box IS the result line's surface (the retired
|
||||
announcer span is gone)."""
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
expected, timeout=SYNC_TIMEOUT_MS
|
||||
counts_line(expected), timeout=SYNC_TIMEOUT_MS
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -145,6 +145,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -948,5 +949,5 @@ def test_label_shows_the_chunk_part(
|
||||
expect(page.locator(".sync-label-chunk")).to_be_hidden()
|
||||
expect(page.locator(".sync-label-chunk")).to_have_text("")
|
||||
# The counts line rides the box (phase 150 — the sole sync surface).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(f"{N_FILES} added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(f"{N_FILES} added"))
|
||||
expect(page.locator("#sync-error-banner")).to_be_hidden()
|
||||
|
||||
@@ -129,6 +129,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import PEAK_CLAUSE, counts_line
|
||||
from scripts.import_docs import repo_name
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
@@ -187,9 +188,10 @@ RUN_SETTLE_TIMEOUT_MS = 90_000
|
||||
UI_SETTLE_TIMEOUT_MS = 15_000
|
||||
|
||||
#: "Synced HH:MM" (``sources.js`` ``fmtSyncTime``) and the settled
|
||||
#: SUCCESS result line (``fmtSyncResult``) — updated/pruned are 0 here.
|
||||
#: SUCCESS result line (``fmtSyncResult``) — updated/pruned are 0 here,
|
||||
#: and the phase-187 peak clause is optional (null pair → no clause).
|
||||
SYNCED_LABEL = re.compile(r"Synced \d{1,2}:\d{2}")
|
||||
RESULT_LINE = re.compile(r"^(\d+) added(?: · (\d+) unchanged)?$")
|
||||
RESULT_LINE = re.compile(rf"^(\d+) added(?: · (\d+) unchanged)?{PEAK_CLAUSE}$")
|
||||
|
||||
#: The console noise the anonymous probes legitimately produce (this
|
||||
#: module's own tests are admin — the collector stays as the uncaught-JS
|
||||
@@ -616,7 +618,7 @@ def test_status_polls_responsive_during_long_clone(
|
||||
|
||||
# And it stayed in its live state: no error banner, no modal, the
|
||||
# neutral running copy, the Cancel control still there.
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("Syncing…"))
|
||||
expect(btn).to_be_disabled()
|
||||
expect(cancel).to_be_visible()
|
||||
expect(page.locator("#sync-error-banner")).to_be_hidden()
|
||||
@@ -784,7 +786,7 @@ def test_cancel_during_clone_reaches_terminal(
|
||||
|
||||
# The neutral UI settle: "Sync cancelled", retry-ready, no error.
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
"Sync cancelled", timeout=UI_SETTLE_TIMEOUT_MS
|
||||
counts_line("Sync cancelled"), timeout=UI_SETTLE_TIMEOUT_MS
|
||||
)
|
||||
expect(page.locator("#sync-label")).to_have_text(
|
||||
"Sync sources", timeout=UI_SETTLE_TIMEOUT_MS
|
||||
|
||||
@@ -141,6 +141,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -710,13 +711,15 @@ def test_label_advances_through_the_image_span(
|
||||
if snap["box"] != last:
|
||||
seen.append(snap["box"])
|
||||
last = snap["box"]
|
||||
if snap["box"] == SETTLE_TEXT:
|
||||
if counts_line(SETTLE_TEXT).fullmatch(snap["box"]):
|
||||
break
|
||||
time.sleep(0.2)
|
||||
|
||||
# --- the settle (the UI's 2 s poll renders the terminal at most one
|
||||
# tick after it lands; the endpoint is the source of truth) ---
|
||||
assert last == SETTLE_TEXT, f"the run never settled to the counts line: {seen[-5:]}"
|
||||
assert counts_line(SETTLE_TEXT).fullmatch(str(last)), (
|
||||
f"the run never settled to the counts line: {seen[-5:]}"
|
||||
)
|
||||
terminal = _wait_terminal(app_url)
|
||||
# The counts line's ground truth: the PDF + its 4 embedded-image
|
||||
# docs — one file, 5 added.
|
||||
@@ -806,7 +809,7 @@ def test_label_advances_through_the_image_span(
|
||||
# surface): the image part is not in it, not in the title, not in
|
||||
# the aria (the settle restores the ship aria value).
|
||||
box = page.locator("#sync-progress-text").inner_text()
|
||||
assert box == SETTLE_TEXT, box
|
||||
assert counts_line(SETTLE_TEXT).fullmatch(box), box
|
||||
assert "· image" not in box and "· image" not in (btn.get_attribute("title") or "")
|
||||
assert btn.get_attribute("aria-label") == "Sync sources", btn.get_attribute("aria-label")
|
||||
expect(page.locator("#sync-error-banner")).to_be_hidden()
|
||||
@@ -869,7 +872,7 @@ def test_mid_run_reload_re_attaches_with_image_position(
|
||||
# terminal clears the position, test 1 pins the endpoint half),
|
||||
# the terminal label, the parts cleared + empty.
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
SETTLE_TEXT, timeout=SETTLE_TIMEOUT_MS
|
||||
counts_line(SETTLE_TEXT), timeout=SETTLE_TIMEOUT_MS
|
||||
)
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=30_000)
|
||||
btn = page.locator("#sync-btn")
|
||||
|
||||
@@ -157,6 +157,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -688,7 +689,7 @@ def _settle(page: Page, app_url: str) -> None:
|
||||
# retired announcer spoke, byte-identical: the fresh 4-file
|
||||
# fixture's "4 added"), hiding only when its text is empty.
|
||||
expect(page.locator("#sync-progress")).to_be_visible()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(f"{N_FILES} added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(f"{N_FILES} added"))
|
||||
expect(page.locator("#sync-error-banner")).to_be_hidden()
|
||||
|
||||
|
||||
|
||||
@@ -69,6 +69,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -454,7 +455,7 @@ def test_healthy_sync_still_succeeds(
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||
expect(btn).to_be_enabled() # never stale — re-enabled at the terminal state
|
||||
# The counts line rides the box (phase 150 — the sole sync surface).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("1 added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("1 added"))
|
||||
|
||||
# The REAL clone was imported: the fixture path is in the Sources
|
||||
# table (the sentinel lives inside it) — via the drill-down tree
|
||||
@@ -475,7 +476,7 @@ def test_healthy_sync_still_succeeds(
|
||||
expect(page.locator("#sync-label")).to_have_text("Syncing…")
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS)
|
||||
expect(btn).to_be_enabled()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("0 added · 1 unchanged")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line("0 added · 1 unchanged"))
|
||||
# The doc survived the prune re-import — via the drill-down tree
|
||||
# (phase 97).
|
||||
_expect_fixture_row(page)
|
||||
|
||||
@@ -0,0 +1,844 @@
|
||||
"""Phase 187 (task 04) E2E (Playwright, mock-only): the sync's PEAK MEMORY
|
||||
is visible on the page that started the run — and it is the truth.
|
||||
|
||||
Source: owner incident 2026-10-15. Reconstructing "how much RAM did that
|
||||
sync use?" meant reading ``systemd`` unit-teardown lines from three boots
|
||||
(``brain-of-code-app.service: … 3.3G memory peak``), and boot −1 — the run
|
||||
that killed the box — had NO teardown record at all, because the VM never
|
||||
shut down cleanly. The app had no opinion about its own memory: no log
|
||||
line, no status field, nothing on the Sources page. Phase 187 made it a
|
||||
number (``app/core/memory.py``, the lifecycle lines, the status keys) and
|
||||
this module is the browser proof of the last mile.
|
||||
|
||||
What the phase put on the wire, in the operator's words:
|
||||
|
||||
* ``GET /api/sync/status`` carries the ADDITIVE pair ``rss_mb`` (the
|
||||
process's CURRENT resident set, MB) and ``rss_peak_mb`` (its HIGH-WATER
|
||||
mark) — ``int | null``, where ``null`` means "no reading" and NEVER 0;
|
||||
sampled at the run start, at every ``phase`` transition, throttled to
|
||||
one read per 5 s through the import's per-file hook, and FORCED in every
|
||||
terminal, which KEEPS the final pair next to the final counts;
|
||||
* the Sources page appends `` · peak <N> MB`` to the result line it
|
||||
already spoke (the ``fmtSyncResult`` counts line, "Sync cancelled") —
|
||||
trailing text on a line that already exists, never a new control, never
|
||||
a new live region, and NOTHING rendered when the pair is null.
|
||||
|
||||
**Which number this is (read it before comparing it to a dashboard).**
|
||||
Both fields are PROCESS readings — ``VmRSS`` from ``/proc/self/status`` and
|
||||
``getrusage(RUSAGE_SELF).ru_maxrss`` — NOT the host's cgroup figure. A
|
||||
cgroup counts PAGE CACHE too: the incident box's app cgroup showed ~2.16 GB
|
||||
of ``file`` against ~211 MB of ``anon`` while idle, and ``systemd`` printed
|
||||
``3.3G memory peak`` for the unit while the uvicorn process was a fraction
|
||||
of that. Neither number is wrong; LOCKED A1 pins the app to the process
|
||||
one, and `README.md` (*Operational notes: the sync's memory (phase 187)*)
|
||||
owns the full caveat and the operator grep
|
||||
(``grep ' :: ' server.log | grep ' rss_peak_mb='``). This module never
|
||||
asserts a MAGNITUDE — only that the number is present, typed, terminal,
|
||||
honest (``peak >= current``, with the one exception the sampler owns: a
|
||||
kernel high-water refreshed lazily is lifted to the current reading, never
|
||||
invented from it), and equal on both surfaces.
|
||||
|
||||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||||
|
||||
uv run pytest tests/e2e/test_sync_peak_memory_visible.py -v --no-cov
|
||||
|
||||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the deterministic
|
||||
mock plus the phase-64 ``tests/e2e/slow_llm.py`` delay proxy is what makes
|
||||
the RUNNING observation a fact instead of a race: the run must outlive the
|
||||
UI's 2 s status poll (and the tests' ~100 ms one) so the "no clause while
|
||||
running" pin and the cancel window are both deterministic. Two files →
|
||||
≈ 10 LLM calls (the two model probes, the per-file embed batches + the
|
||||
summaries, the KB overview, the source-root folder summary) ≈ 5 s at
|
||||
``SLOW_DELAY_S = 0.5``.
|
||||
|
||||
Isolation (the house shape): ONE module app on its own port (the conftest
|
||||
session app owns 8123 in a combined run), the mock LLM on the conftest
|
||||
port behind the proxy, a dedicated ``BOR_SOURCES_DIR`` /
|
||||
``BOR_PDF_PAGE_DIR`` so no run writes into the operator's, the shared E2E
|
||||
Postgres truncated BEFORE **and** AFTER every test (after waiting for the
|
||||
app to hold no running sync), and the admin in through the real form login
|
||||
(``tests/e2e/auth_helpers.py``). Each test seeds the same two-file
|
||||
``kind=local`` source directory and registers it through the authenticated
|
||||
API, so any single test re-runs alone with a fresh KB and the counts line
|
||||
says ``2 added``.
|
||||
|
||||
Test → observable mapping (Playwright Mapping Rule — the rendered DOM is
|
||||
the observable; the DOM number and the payload number are read as one
|
||||
comparison, so "the page lies" cannot slip through):
|
||||
|
||||
1. ``test_result_line_shows_the_peak`` — a full run from the Sources page:
|
||||
the box (``#sync-progress-text``) settles the counts line WITH the
|
||||
trailing clause (``^2 added · peak \\d+ MB$``), the clause matches
|
||||
``/peak \\d+ MB/``, the aria-live region IS the box (``role=status`` +
|
||||
``aria-live=polite``, no ``aria-hidden``) and carries the SAME text,
|
||||
the clause is TEXT and not an element (the box still holds exactly one
|
||||
child span — no new control), and the megabytes in the DOM EQUAL the
|
||||
terminal payload's ``rss_peak_mb``. Where the platform cannot know the
|
||||
pair, the mirror contract is pinned instead: NO clause at all.
|
||||
2. ``test_status_payload_reports_rss`` — the endpoint end to end: a tick
|
||||
during ``running`` (a) reports the pair with the documented types
|
||||
(``int | null``), (b) carries the pre-phase key set UNCHANGED (the
|
||||
additive pin: exactly the pre-187 keys plus the memory pair) and (c)
|
||||
shows NO ``peak`` clause on the page — the clause is terminal-only;
|
||||
after the terminal ``rss_peak_mb`` is a POSITIVE int with
|
||||
``peak >= current`` (the invariant the ONE sampler owns: the kernel
|
||||
refreshes its high-water lazily, so ``_sample_rss`` lifts a lagging peak
|
||||
to the current reading — a peak under the run's own footprint would be a
|
||||
false number), and the same pair survives a later poll (the page
|
||||
reloaded an hour later still answers).
|
||||
3. ``test_anonymous_cannot_read_memory`` — the A4 exposure guard: a
|
||||
logged-out ``GET /api/sync/status`` is 403 with the pre-phase body
|
||||
``{"detail": "admin only"}`` — no memory number on an anonymous
|
||||
surface, before or after a run — and the page shows the sign-in gate
|
||||
with no sync button and no peak clause anywhere in the DOM.
|
||||
4. ``test_cancelled_terminal_keeps_the_peak`` — the ``test_sync_cancel.py``
|
||||
shape: cancel mid-run and the NEUTRAL "Sync cancelled" state still
|
||||
renders (no banner, no modal, the button retry-ready) with the clause
|
||||
present and equal to the cancelled payload's ``rss_peak_mb`` — the
|
||||
run that stopped early is exactly the run whose memory matters.
|
||||
5. ``test_peak_clause_fits_at_375`` — PLAN §7 at phone width: after a run
|
||||
settled at 375 px, the clause is on screen, the box does not clip it
|
||||
(``scrollWidth <= clientWidth`` for the box and its text), the box
|
||||
stays inside the viewport and the PAGE has no horizontal overflow.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from playwright.sync_api import Browser, ConsoleMessage, Page, ViewportSize, expect
|
||||
from sqlalchemy import text
|
||||
|
||||
from app.config import Settings as _Settings
|
||||
from app.core.memory import rss_mb, rss_peak_mb
|
||||
from app.db import SessionLocal
|
||||
from e2e.auth_helpers import login
|
||||
from e2e.conftest import (
|
||||
ADMIN_PASSWORD,
|
||||
MOCK_PORT,
|
||||
SESSION_SECRET,
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
#: Dedicated module ports (the house env-overridable pattern): the conftest
|
||||
#: session app owns 8123 and the mock owns 8901 in a combined run.
|
||||
APP_PORT = int(os.environ.get("E2E_APP_PORT_PEAKMEM", "8204"))
|
||||
APP_URL = f"http://127.0.0.1:{APP_PORT}"
|
||||
|
||||
#: The slow-LLM proxy's port (the conftest's mock stays on MOCK_PORT — the
|
||||
#: proxy fronts it).
|
||||
SLOW_PORT = int(os.environ.get("E2E_SLOW_LLM_PORT_PEAKMEM", "8910"))
|
||||
SLOW_URL = f"http://127.0.0.1:{SLOW_PORT}"
|
||||
|
||||
#: Per-LLM-request delay (the phase-64 timing convention): the run spans
|
||||
#: ≈ 10 calls ≈ 5 s, i.e. ≫ the UI's 2 s status poll — the running window
|
||||
#: and the cancel window are both observable, not races.
|
||||
SLOW_DELAY_S = "0.5"
|
||||
|
||||
SOURCES_URL = "/sources.html"
|
||||
|
||||
#: The fixture directory's basename — the ``source`` a ``kind=local`` row
|
||||
#: records (``Path(path).name``), stable whatever temp root pytest picked.
|
||||
#:
|
||||
#: **Nothing here is allowed to contain the word "peak"** — not the source
|
||||
#: name, not a path, no marker. Several pins below ask whether the page
|
||||
#: says ANYTHING about memory (the running label must be clause-free, the
|
||||
#: anonymous shell must be silent), and those asks read the rendered text;
|
||||
#: a fixture path like ``peakmem/notes/peak-alpha.md`` rides the running
|
||||
#: label verbatim and would answer "yes" from the file name alone. So the
|
||||
#: fixture is ``memline-fixture/notes/mem-*.md`` and the clause stays the
|
||||
#: only thing on the page that can say the word.
|
||||
SOURCE_NAME = "memline-fixture"
|
||||
N_FILES = 2
|
||||
FIXTURE_FILES = {
|
||||
"notes/mem-alpha.md": (
|
||||
"# Memory line alpha\n"
|
||||
"\n"
|
||||
"Head marker RESE-MEMLINE-ALPHA-6d13 opens this note.\n"
|
||||
"\n"
|
||||
"## Alpha body\n"
|
||||
"\n"
|
||||
"A small note that exists so the phase-187 suite has a real run "
|
||||
"to watch: a real walk, real embeds, a real KB overview, and the "
|
||||
"run's resident-set clause on the line it settles.\n"
|
||||
),
|
||||
"notes/mem-beta.md": (
|
||||
"# Memory line beta\n"
|
||||
"\n"
|
||||
"Head marker RESE-MEMLINE-BETA-9a74 opens this note — the second "
|
||||
"file, the counts line's denominator.\n"
|
||||
),
|
||||
}
|
||||
|
||||
#: The clause sources.js appends (``fmtSyncPeakClause``): trailing text on
|
||||
#: a result line that already exists. REQUIRED here — this suite's whole
|
||||
#: point is that the number is there (the optional form, for the suites
|
||||
#: that only pin their own counts, lives in ``e2e.sync_helpers``).
|
||||
PEAK_CLAUSE = re.compile(r" · peak (\d+) MB$")
|
||||
|
||||
#: The clause SHAPE, for the "this surface says nothing about memory" pins
|
||||
#: (the running label, the anonymous shell). Matched case-insensitively on
|
||||
#: the word, because the clause is the only thing on these pages allowed to
|
||||
#: use it — which the fixture naming rule above is what keeps true.
|
||||
_ANY_PEAK = re.compile(r"peak", re.IGNORECASE)
|
||||
|
||||
#: The settled success line: the pre-phase ``fmtSyncResult`` counts line
|
||||
#: (a fresh KB of two files → "2 added", the zero "updated"/"unchanged"
|
||||
#: parts omitted) with the phase-187 clause trailing it.
|
||||
SUCCESS_LINE = re.compile(rf"^{N_FILES} added{PEAK_CLAUSE.pattern}$")
|
||||
|
||||
#: The neutral cancel line (phase 134/150 wording, byte-identical) with
|
||||
#: the clause trailing it.
|
||||
CANCELLED_LINE = re.compile(rf"^Sync cancelled{PEAK_CLAUSE.pattern}$")
|
||||
|
||||
#: "Synced HH:MM" — the local-time last-result label (sources.js's
|
||||
#: fmtSyncTime), any hour/minute (the test_sync_button.py pattern).
|
||||
SYNCED_LABEL = re.compile(r"Synced \d{1,2}:\d{2}$")
|
||||
|
||||
#: The status payload's key set BEFORE phase 187 — the additive pin's
|
||||
#: left-hand side. The payload must stay exactly this set plus the memory
|
||||
#: pair, forever (the same list the integration suite pins).
|
||||
PRE_187_STATUS_KEYS = frozenset(
|
||||
{
|
||||
"state",
|
||||
"started_at",
|
||||
"finished_at",
|
||||
"detail",
|
||||
"error",
|
||||
"current_file",
|
||||
"files_done",
|
||||
"files_total",
|
||||
"current_chunk",
|
||||
"chunks_total",
|
||||
"images_done",
|
||||
"images_total",
|
||||
"phase",
|
||||
"current_summary",
|
||||
"summaries_done",
|
||||
"summaries_total",
|
||||
}
|
||||
)
|
||||
MEMORY_KEYS = frozenset({"rss_mb", "rss_peak_mb"})
|
||||
|
||||
#: The pre-phase integer count slots (their types are part of the
|
||||
#: unchanged-shape pin in test 2 — never a float, never null).
|
||||
_COUNT_KEYS = frozenset(
|
||||
{
|
||||
"files_done",
|
||||
"files_total",
|
||||
"current_chunk",
|
||||
"chunks_total",
|
||||
"images_done",
|
||||
"images_total",
|
||||
"summaries_done",
|
||||
"summaries_total",
|
||||
}
|
||||
)
|
||||
|
||||
#: The invariant the page can rely on: ``peak >= current``. The raw kernel
|
||||
#: counters do NOT give that for free (``VmRSS`` and ``ru_maxrss`` are two
|
||||
#: different sources, both floored to whole MB, and the kernel refreshes the
|
||||
#: high-water lazily — measured at ``rss_mb=164`` against
|
||||
#: ``rss_peak_mb=162``), so the ONE sampling point
|
||||
#: (:func:`app.api.sync._sample_rss`) lifts a lagging peak to the current
|
||||
#: reading. Slack ZERO: a peak BELOW the run's own current footprint is a
|
||||
#: false number, and the page must never show one.
|
||||
MB_SLACK = 0
|
||||
|
||||
#: Does THIS platform answer the two helpers? Linux (the deploy target:
|
||||
#: ``/proc`` + ``resource``) always does. The app under test is a
|
||||
#: subprocess of this process's own OS, so this answer is the app's answer.
|
||||
#: Where it is False the value pins step aside and the ABSENCE contract is
|
||||
#: pinned instead (null renders nothing — never "peak ? MB", never 0).
|
||||
MEMORY_KNOWABLE = rss_mb() is not None and rss_peak_mb() is not None
|
||||
|
||||
#: The house phone viewport at which the clause must still fit (the
|
||||
#: PLAN §7 / AGENTS rule 5 width: no horizontal overflow at 375 px).
|
||||
MOBILE_VIEWPORT: ViewportSize = {"width": 375, "height": 720}
|
||||
|
||||
#: Settle budgets: a two-file run under the proxy is ≈ 5 s and the UI's 2 s
|
||||
#: poll settles at most one tick after the terminal — 90 s is an order of
|
||||
#: magnitude of head-room (the house sync budget, no client timeout).
|
||||
SETTLE_TIMEOUT_MS = 90_000
|
||||
|
||||
#: The browser logs "Failed to load resource: … 403" for the anonymous
|
||||
#: probes (the whoami gate + the admin-only status fetch) — expected
|
||||
#: noise, not a defect.
|
||||
_CONSOLE_NOISE = re.compile(r"Failed to load resource", re.IGNORECASE)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The fixture source
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _seed_source(root: Path) -> Path:
|
||||
"""(Re-)write the two fixture files under ``root / SOURCE_NAME``.
|
||||
|
||||
Idempotent on purpose: every test calls it, so the tree is at its
|
||||
ORIGINAL bytes whenever a test starts and a lone re-run of any single
|
||||
test still sees a fresh KB with ``2 added``.
|
||||
"""
|
||||
base = root / SOURCE_NAME
|
||||
for rel, body in FIXTURE_FILES.items():
|
||||
target = base / rel
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(body, encoding="utf-8")
|
||||
assert not (base / ".git").exists() # a plain directory — no git anywhere
|
||||
return base
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def source_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
|
||||
"""The fixture tree the module app walks (the app server runs on the
|
||||
same host, so the path is visible to it)."""
|
||||
return _seed_source(tmp_path_factory.mktemp("bor_peak_memory"))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The module app + the timing proxy
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def slow_llm(mock_llm: int) -> Iterator[int]:
|
||||
"""The delay-injecting reverse proxy in front of the mock LLM
|
||||
(``tests/e2e/slow_llm.py``) — the ONLY thing that makes the RUNNING
|
||||
observation and the cancel window deterministic (module docstring)."""
|
||||
env = dict(os.environ)
|
||||
env.pop("DEBUGPY", None)
|
||||
env["SLOW_LLM_DELAY_S"] = SLOW_DELAY_S
|
||||
env["E2E_MOCK_PORT"] = str(MOCK_PORT)
|
||||
proc = subprocess.Popen(
|
||||
[
|
||||
sys.executable, "-m", "uvicorn", "tests.e2e.slow_llm:app",
|
||||
"--host", "127.0.0.1", "--port", str(SLOW_PORT), "--log-level", "warning",
|
||||
],
|
||||
cwd=REPO,
|
||||
env=env,
|
||||
)
|
||||
try:
|
||||
_wait_http(f"{SLOW_URL}/v1/models")
|
||||
yield SLOW_PORT
|
||||
finally:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_server(
|
||||
mock_llm: int,
|
||||
slow_llm: int,
|
||||
source_dir: Path,
|
||||
tmp_path_factory: pytest.TempPathFactory,
|
||||
) -> Iterator[str]:
|
||||
"""The real app under test — per-module env (the conftest pattern):
|
||||
``BOR_LLM_BASE_URL`` is the SLOW proxy in front of the deterministic
|
||||
mock, ``BOR_GIT_SOURCES`` forced empty (each test registers its own
|
||||
local row; an operator's gitignored ``.env`` must never leak a real
|
||||
source into a run), and a dedicated ``BOR_SOURCES_DIR``.
|
||||
"""
|
||||
if USE_REAL_LLM:
|
||||
pytest.skip(
|
||||
"mock-only suite — the running window IS the slow proxy in "
|
||||
"front of the deterministic mock (E2E_REAL_LLM unsupported)"
|
||||
)
|
||||
assert mock_llm and source_dir
|
||||
scratch = tmp_path_factory.mktemp("bor_peak_memory_app")
|
||||
env = dict(os.environ)
|
||||
env.pop("DEBUGPY", None)
|
||||
env["BOR_ENVIRONMENT"] = "e2e"
|
||||
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
||||
env["BOR_LLM_BASE_URL"] = f"{SLOW_URL}/v1"
|
||||
# Mock-calibrated gate (the conftest pattern): both floors must stay
|
||||
# <= the threshold or the app refuses to boot (app/config.py).
|
||||
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
||||
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
|
||||
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
|
||||
# Instant retry waits + the code-default budget (the conftest leak
|
||||
# guard — this suite never needs a retry, and never waits 5 s for one).
|
||||
env["BOR_LLM_RETRY_DELAY"] = "0"
|
||||
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
|
||||
env.setdefault(
|
||||
"BOR_DATABASE_URL",
|
||||
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
||||
)
|
||||
# Phase 16: admin auth must be set or create_app() refuses to boot.
|
||||
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
||||
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
||||
env["BOR_GIT_SOURCES"] = ""
|
||||
env["BOR_SOURCES_DIR"] = str(scratch / "checkouts")
|
||||
env["BOR_PDF_PAGE_DIR"] = str(scratch / "pdf-pages")
|
||||
proc = subprocess.Popen(
|
||||
[
|
||||
sys.executable, "-m", "uvicorn", "app.main:app",
|
||||
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning",
|
||||
],
|
||||
cwd=REPO,
|
||||
env=env,
|
||||
)
|
||||
try:
|
||||
_wait_http(f"{APP_URL}/api/health")
|
||||
yield APP_URL
|
||||
finally:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def app_url(app_server: str) -> str:
|
||||
return app_server
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Isolation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _truncate_all() -> None:
|
||||
"""Fresh registry + KB (the E2E isolation pattern, the phase-184 list:
|
||||
``sources_meta`` and ``folder_summaries`` reset with the KB)."""
|
||||
with SessionLocal() as db:
|
||||
db.execute(
|
||||
text(
|
||||
"TRUNCATE chunks, documents, query_log, kb_overview, "
|
||||
"git_sources, sources_meta, folder_summaries"
|
||||
)
|
||||
)
|
||||
db.commit() # without the commit the TRUNCATE rolls back
|
||||
|
||||
|
||||
def _wait_sync_settled(base_url: str, timeout_s: float = 180.0) -> None:
|
||||
"""No run may leak across a truncate: wait for a non-running status with
|
||||
an own admin session (the endpoint is admin-only). Usually a no-op —
|
||||
every test settles before it ends."""
|
||||
deadline = time.monotonic() + timeout_s
|
||||
while time.monotonic() < deadline:
|
||||
with httpx.Client(base_url=base_url, timeout=5.0) as client:
|
||||
client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||||
r = client.get("/api/sync/status")
|
||||
assert r.status_code == 200, r.text
|
||||
if r.json()["state"] != "running":
|
||||
return
|
||||
time.sleep(0.2)
|
||||
raise AssertionError(f"a sync was still running at a test boundary: {base_url}")
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean(app_url: str, source_dir: Path, db_ready: None) -> Iterator[None]:
|
||||
"""Per-test KB isolation: the shared E2E Postgres is truncated before
|
||||
AND after every test, the app holding no running sync at both
|
||||
boundaries. The fixture tree is re-seeded first (idempotent) and the
|
||||
local row is registered through the authenticated API, so every test
|
||||
owns a fresh two-file source and its own ``2 added``."""
|
||||
_wait_sync_settled(app_url)
|
||||
_truncate_all()
|
||||
_seed_source(source_dir.parent)
|
||||
with httpx.Client(base_url=app_url, timeout=30.0) as client:
|
||||
assert client.post("/api/login", json={"password": ADMIN_PASSWORD}).status_code == 204
|
||||
r = client.post(
|
||||
"/api/git-sources", json={"kind": "local", "path": str(source_dir)}
|
||||
)
|
||||
assert r.status_code == 201, r.text
|
||||
yield
|
||||
_wait_sync_settled(app_url)
|
||||
_truncate_all()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def page(browser: Browser) -> Iterator[Page]:
|
||||
"""The conftest's page + the console-error collector: an uncaught JS
|
||||
exception (pageerror) or a console error that is not the browser's
|
||||
expected 403 "Failed to load resource" noise fails the owning test at
|
||||
teardown — a clause builder that threw inside ``setSyncProgress`` would
|
||||
otherwise settle a blank box and read as a passing test.
|
||||
|
||||
The conftest's ``page`` is NOT reused here because this suite's whole
|
||||
subject is the sync result line: a JS error inside the clause builder
|
||||
must fail the owning test, not leak (the ``test_sync_cancel.py``
|
||||
pattern)."""
|
||||
pg = browser.new_page(viewport={"width": 1280, "height": 800})
|
||||
errors: list[str] = []
|
||||
|
||||
def _on_console(msg: ConsoleMessage) -> None:
|
||||
if msg.type == "error" and not _CONSOLE_NOISE.search(msg.text):
|
||||
errors.append(msg.text)
|
||||
|
||||
pg.on("console", _on_console)
|
||||
pg.on("pageerror", lambda err: errors.append(str(err)))
|
||||
try:
|
||||
yield pg
|
||||
finally:
|
||||
pg.close()
|
||||
assert not errors, f"console/page errors: {errors}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _status(page: Page, base_url: str) -> dict[str, Any]:
|
||||
"""The status payload through the ADMIN browser context (the cookie
|
||||
rides ``page.request`` — the same surface the page's own poll uses)."""
|
||||
r = page.request.get(f"{base_url}/api/sync/status")
|
||||
assert r.status == 200, r.text
|
||||
body: dict[str, Any] = r.json()
|
||||
return body
|
||||
|
||||
|
||||
def _wait_sync_state(
|
||||
page: Page, base_url: str, states: set[str], timeout_s: float = 90.0
|
||||
) -> dict[str, Any]:
|
||||
"""Poll the status endpoint at ~100 ms until the run's state is in
|
||||
*states* (the deterministic layer: the UI's own poll ticks at 2 s)."""
|
||||
deadline = time.monotonic() + timeout_s
|
||||
body: dict[str, Any] = {}
|
||||
while time.monotonic() < deadline:
|
||||
body = _status(page, base_url)
|
||||
if body["state"] in states:
|
||||
return body
|
||||
time.sleep(0.1)
|
||||
raise AssertionError(f"sync did not reach {sorted(states)}: {body}")
|
||||
|
||||
|
||||
def _open_sources(page: Page, base_url: str) -> None:
|
||||
"""The real form login, landing on the Sources view (the sync button's
|
||||
home) with the module graph booted."""
|
||||
login(page, base_url) # lands on /sources.html
|
||||
btn = page.locator("#sync-btn")
|
||||
expect(btn).to_be_visible(timeout=30_000)
|
||||
expect(btn).to_be_enabled()
|
||||
|
||||
|
||||
def _click_sync(page: Page) -> None:
|
||||
"""Click **Sync sources** and pin the 202 moment: the button disables
|
||||
(the run is in flight), the box reveals with the bare running label —
|
||||
which carries NO peak clause (the clause is terminal-only)."""
|
||||
btn = page.locator("#sync-btn")
|
||||
btn.click()
|
||||
expect(btn).to_be_disabled()
|
||||
expect(page.locator("#sync-progress")).to_have_text("Syncing…")
|
||||
|
||||
|
||||
def _start_from_page(page: Page, base_url: str) -> None:
|
||||
"""Login on the Sources view, then the click (the two-step form of the
|
||||
same helper, so a test can change the viewport BETWEEN them)."""
|
||||
_open_sources(page, base_url)
|
||||
_click_sync(page)
|
||||
|
||||
|
||||
def _clause_number(text: str) -> int:
|
||||
"""The MB value the clause carries (loudly absent when there is none)."""
|
||||
m = PEAK_CLAUSE.search(text)
|
||||
assert m, f"no peak clause in {text!r}"
|
||||
return int(m.group(1))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. The success terminal: the clause on the line that already existed
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_result_line_shows_the_peak(page: Page, app_url: str, db_ready: None) -> None:
|
||||
"""A real sync started from the Sources page ends with the number on
|
||||
the line the page already spoke — ``2 added · peak <N> MB`` — inside
|
||||
the sole aria-live region, as TEXT (no new control), and equal to the
|
||||
terminal payload's ``rss_peak_mb`` (the page and the log can never
|
||||
disagree, because they read the one forced terminal sample)."""
|
||||
page.set_default_timeout(30_000)
|
||||
_start_from_page(page, app_url)
|
||||
|
||||
terminal = _wait_sync_state(page, app_url, {"success", "failed", "cancelled"})
|
||||
assert terminal["state"] == "success", terminal
|
||||
assert terminal["error"] is None, terminal
|
||||
|
||||
# The UI settle (the page's own 2 s poll, one tick after the terminal).
|
||||
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=SETTLE_TIMEOUT_MS)
|
||||
box = page.locator("#sync-progress")
|
||||
expect(box).to_be_visible()
|
||||
text_el = page.locator("#sync-progress-text")
|
||||
line = text_el.inner_text()
|
||||
|
||||
if not MEMORY_KNOWABLE:
|
||||
# The mirror contract: no reading → NO clause at all (never
|
||||
# "peak ? MB", never "peak 0 MB"), and the counts line alone.
|
||||
assert line == f"{N_FILES} added", line
|
||||
assert not _ANY_PEAK.search(box.inner_text()), box.inner_text()
|
||||
return
|
||||
|
||||
# The clause is there AND the pre-phase wording is intact (the counts
|
||||
# line keeps its exact bytes; the clause trails it).
|
||||
assert SUCCESS_LINE.fullmatch(line), line
|
||||
assert PEAK_CLAUSE.search(line), line
|
||||
assert line.startswith(f"{N_FILES} added"), line
|
||||
|
||||
# The clause is TEXT on a line that already exists — the box still
|
||||
# holds its ONE child span (no new control, no new live region).
|
||||
expect(box.locator("> *")).to_have_count(1)
|
||||
|
||||
# The box IS the sole aria-live region and it announces the clause
|
||||
# (phase 150: one live region, one surface — and the clause rides it).
|
||||
expect(box).to_have_attribute("role", "status")
|
||||
expect(box).to_have_attribute("aria-live", "polite")
|
||||
expect(box).not_to_have_attribute("aria-hidden")
|
||||
assert PEAK_CLAUSE.search(box.inner_text()), box.inner_text()
|
||||
assert box.inner_text().strip() == line.strip(), (box.inner_text(), line)
|
||||
|
||||
# HONESTY: the megabytes on the page ARE the terminal payload's
|
||||
# ``rss_peak_mb`` — one forced reading, two surfaces.
|
||||
assert terminal["rss_peak_mb"] is not None, terminal
|
||||
assert _clause_number(line) == terminal["rss_peak_mb"], (line, terminal)
|
||||
|
||||
# And the number survives the page: a reload re-attaches the terminal
|
||||
# (initSyncButton) and re-renders the SAME clause.
|
||||
page.reload()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
SUCCESS_LINE, timeout=SETTLE_TIMEOUT_MS
|
||||
)
|
||||
assert _clause_number(page.locator("#sync-progress-text").inner_text()) == (
|
||||
terminal["rss_peak_mb"]
|
||||
)
|
||||
|
||||
# No error surface on a success settle (unchanged contract).
|
||||
expect(page.locator("#sync-error-banner")).to_be_hidden()
|
||||
assert page.locator(".sync-modal-backdrop").count() == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. The payload: additive, typed, terminal-kept — and silent while running
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_status_payload_reports_rss(page: Page, app_url: str, db_ready: None) -> None:
|
||||
"""The additive pin end to end, through the browser's own authenticated
|
||||
request surface: while the run is in flight the payload already carries
|
||||
the pair (``int | null``) and the pre-phase keys are byte-for-byte
|
||||
unchanged, while the PAGE shows no clause at all; after the terminal the
|
||||
peak is a positive integer that stays with the run, and the pair keeps
|
||||
the invariant the page relies on (``peak >= current`` — the sampling
|
||||
point lifts the lazily-refreshed high-water, ``MB_SLACK`` is 0).
|
||||
"""
|
||||
page.set_default_timeout(30_000)
|
||||
_start_from_page(page, app_url)
|
||||
|
||||
# --- during `running` ------------------------------------------------
|
||||
running = _wait_sync_state(page, app_url, {"running"})
|
||||
assert running["state"] == "running", running
|
||||
# The additive contract on the wire: exactly the pre-phase keys plus
|
||||
# the memory pair — nothing renamed, nothing dropped.
|
||||
assert set(running) == PRE_187_STATUS_KEYS | MEMORY_KEYS, set(running)
|
||||
for key, value in running.items():
|
||||
if key in MEMORY_KEYS:
|
||||
# The documented type: int | null (null is "no reading").
|
||||
assert value is None or isinstance(value, int), (key, value)
|
||||
elif key == "detail":
|
||||
assert isinstance(value, dict), (key, value)
|
||||
elif key in _COUNT_KEYS:
|
||||
assert isinstance(value, int), (key, value)
|
||||
else:
|
||||
assert value is None or isinstance(value, (str, int, bool)), (key, value)
|
||||
# The types are ints on a platform that knows them (the run start
|
||||
# FORCES the first sample, so a running tick is never null here).
|
||||
if MEMORY_KNOWABLE:
|
||||
assert running["rss_mb"] is not None, running
|
||||
assert running["rss_peak_mb"] is not None, running
|
||||
assert running["rss_peak_mb"] >= running["rss_mb"] - MB_SLACK, running
|
||||
|
||||
# The RUNNING state shows NO clause: the box carries the live label and
|
||||
# says nothing about memory (the clause is terminal-only in the UI). The
|
||||
# word is the tell (the fixture names never carry it), so a clause that
|
||||
# leaked into a running label fails even when its number is null-ish.
|
||||
live_text = page.locator("#sync-progress-text").inner_text()
|
||||
assert live_text.startswith("Syncing…"), live_text
|
||||
assert not _ANY_PEAK.search(live_text), live_text
|
||||
assert not _ANY_PEAK.search(page.locator("#sync-progress").inner_text())
|
||||
|
||||
# --- after the terminal ---------------------------------------------
|
||||
terminal = _wait_sync_state(page, app_url, {"success", "failed", "cancelled"})
|
||||
assert terminal["state"] == "success", terminal
|
||||
assert set(terminal) == PRE_187_STATUS_KEYS | MEMORY_KEYS, set(terminal)
|
||||
current, peak = terminal["rss_mb"], terminal["rss_peak_mb"]
|
||||
if MEMORY_KNOWABLE:
|
||||
assert peak is not None and isinstance(peak, int), terminal
|
||||
assert peak > 0, terminal # a real reading, never a placeholder 0
|
||||
assert current is not None and current > 0, terminal
|
||||
assert peak >= current - MB_SLACK, (current, peak)
|
||||
else:
|
||||
assert peak is None and current is None, terminal
|
||||
|
||||
# The pair SURVIVES the run (no reset between the terminal and a later
|
||||
# poll — the page can be reloaded an hour later and still answers).
|
||||
again = _status(page, app_url)
|
||||
assert (again["rss_mb"], again["rss_peak_mb"]) == (current, peak), again
|
||||
# …and the pre-existing values rode the terminal unchanged too.
|
||||
assert again["state"] == "success"
|
||||
assert again["error"] is None
|
||||
assert isinstance(again["detail"], dict) and again["detail"]["added"] == N_FILES
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Anonymous: no memory number on any anonymous surface (A4)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_anonymous_cannot_read_memory(
|
||||
page: Page, browser: Browser, app_url: str, db_ready: None
|
||||
) -> None:
|
||||
"""The exposure guard is the status route's own admin gate: a logged-out
|
||||
visitor gets the pre-phase 403 with the pre-phase body, before or after
|
||||
a run — and the Sources shell shows the sign-in gate with no sync button
|
||||
and no memory clause anywhere in the DOM.
|
||||
|
||||
The probe runs in a SEPARATE browser context (``browser.new_context()``)
|
||||
— this test's own page is logged in by the run above, and
|
||||
``page.context.new_page()`` would SHARE that session's cookies, which
|
||||
would make the "anonymous" probe an admin one."""
|
||||
# A run FIRST, so the guard is proven with real numbers in existence.
|
||||
_start_from_page(page, app_url)
|
||||
terminal = _wait_sync_state(page, app_url, {"success"})
|
||||
if MEMORY_KNOWABLE:
|
||||
assert terminal["rss_peak_mb"] is not None, terminal
|
||||
|
||||
ctx = browser.new_context(viewport={"width": 1280, "height": 800})
|
||||
try:
|
||||
anon = ctx.new_page()
|
||||
r = anon.request.get(f"{app_url}/api/sync/status")
|
||||
assert r.status == 403, r.text
|
||||
assert r.json() == {"detail": "admin only"}
|
||||
|
||||
# The anonymous shell: the gate is the anonymity proof, the sync
|
||||
# controls are never revealed, and nothing about memory renders.
|
||||
anon.goto(f"{app_url}/sources.html")
|
||||
expect(anon.locator("#sources-gate")).to_be_visible(timeout=30_000)
|
||||
expect(anon.locator("#sync-btn")).to_be_hidden()
|
||||
expect(anon.locator("#sync-progress")).to_be_hidden()
|
||||
assert not _ANY_PEAK.search(anon.locator("body").inner_text())
|
||||
finally:
|
||||
ctx.close()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. The cancelled terminal keeps the peak (the test_sync_cancel shape)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_cancelled_terminal_keeps_the_peak(
|
||||
page: Page, app_url: str, db_ready: None
|
||||
) -> None:
|
||||
"""Cancel mid-run: the run lands in the NEUTRAL ``cancelled`` terminal
|
||||
(no banner, no modal, retry-ready) and the box still carries the
|
||||
clause — "Sync cancelled · peak <N> MB" — equal to the cancelled
|
||||
payload's ``rss_peak_mb``. A cancelled run's memory is still data: the
|
||||
phase-64 keep-final-values convention covers it like the counts."""
|
||||
page.set_default_timeout(30_000)
|
||||
_start_from_page(page, app_url)
|
||||
|
||||
_wait_sync_state(page, app_url, {"running"})
|
||||
cancel = page.locator("#sync-cancel-btn")
|
||||
expect(cancel).to_be_visible()
|
||||
expect(cancel).to_be_enabled()
|
||||
cancel.click()
|
||||
expect(cancel).to_be_disabled() # never stale, one request at a time
|
||||
|
||||
body = _wait_sync_state(page, app_url, {"cancelled"}, timeout_s=30.0)
|
||||
assert body["state"] == "cancelled"
|
||||
assert body["error"] is None # a cancel is intent, not failure
|
||||
assert body["detail"] == {}
|
||||
|
||||
# The neutral settle, clause and all (the 2 s poll, one tick later).
|
||||
expect(page.locator("#sync-label")).to_have_text("Sync sources", timeout=30_000)
|
||||
expect(page.locator("#sync-btn")).to_be_enabled()
|
||||
expect(cancel).to_be_hidden()
|
||||
expect(page.locator("#sync-error-banner")).to_be_hidden()
|
||||
assert page.locator(".sync-modal-backdrop").count() == 0
|
||||
|
||||
line = page.locator("#sync-progress-text").inner_text()
|
||||
if MEMORY_KNOWABLE:
|
||||
assert body["rss_peak_mb"] is not None, body
|
||||
assert body["rss_peak_mb"] > 0, body
|
||||
assert CANCELLED_LINE.fullmatch(line), line
|
||||
assert _clause_number(line) == body["rss_peak_mb"], (line, body)
|
||||
else:
|
||||
assert line == "Sync cancelled", line
|
||||
# The pair survives the cancel exactly like the counts do.
|
||||
again = _status(page, app_url)
|
||||
assert (again["rss_mb"], again["rss_peak_mb"]) == (
|
||||
body["rss_mb"],
|
||||
body["rss_peak_mb"],
|
||||
), again
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. PLAN §7 at 375 px: the clause fits, the page never scrolls sideways
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_peak_clause_fits_at_375(page: Page, app_url: str, db_ready: None) -> None:
|
||||
"""The clause at phone width: the result line wraps INSIDE the bordered
|
||||
box (``overflow-wrap: anywhere``) — nothing clipped, no horizontal page
|
||||
overflow, the box inside the viewport (AGENTS rule 5 / PLAN §7). The
|
||||
clause is a few characters of trailing text, so a layout shift or a
|
||||
clipped tail at 375 px is exactly what this pins out."""
|
||||
_open_sources(page, app_url)
|
||||
page.set_viewport_size(MOBILE_VIEWPORT) # the phone width, then the click
|
||||
page.wait_for_timeout(150) # the reflow settles before the run paints
|
||||
_click_sync(page)
|
||||
terminal = _wait_sync_state(page, app_url, {"success"})
|
||||
|
||||
text_el = page.locator("#sync-progress-text")
|
||||
expect(text_el).to_have_text(
|
||||
SUCCESS_LINE if MEMORY_KNOWABLE else f"{N_FILES} added",
|
||||
timeout=SETTLE_TIMEOUT_MS,
|
||||
)
|
||||
line = text_el.inner_text()
|
||||
if MEMORY_KNOWABLE:
|
||||
assert PEAK_CLAUSE.search(line), line
|
||||
assert _clause_number(line) == terminal["rss_peak_mb"], (line, terminal)
|
||||
|
||||
# The box + its text do not clip, the box stays in the viewport, and
|
||||
# the PAGE itself never scrolls sideways at 375 px.
|
||||
page.wait_for_timeout(150) # the reflow settles before the measure
|
||||
geometry = page.evaluate(
|
||||
"""() => {
|
||||
const box = document.querySelector('#sync-progress');
|
||||
const textEl = document.querySelector('#sync-progress-text');
|
||||
const b = box.getBoundingClientRect();
|
||||
return {
|
||||
boxScrollWidth: box.scrollWidth,
|
||||
boxClientWidth: box.clientWidth,
|
||||
textScrollWidth: textEl.scrollWidth,
|
||||
textClientWidth: textEl.clientWidth,
|
||||
left: b.left,
|
||||
right: b.right,
|
||||
height: b.height,
|
||||
docScroll: document.documentElement.scrollWidth,
|
||||
innerWidth: window.innerWidth,
|
||||
text: textEl.textContent,
|
||||
};
|
||||
}"""
|
||||
)
|
||||
assert geometry["height"] > 0, geometry
|
||||
assert geometry["boxScrollWidth"] <= geometry["boxClientWidth"], geometry
|
||||
assert geometry["textScrollWidth"] <= geometry["textClientWidth"], geometry
|
||||
assert geometry["left"] >= 0, geometry
|
||||
assert geometry["right"] <= MOBILE_VIEWPORT["width"] + 1, geometry
|
||||
assert geometry["docScroll"] <= geometry["innerWidth"] + 1, geometry
|
||||
assert PEAK_CLAUSE.search(geometry["text"]) or not MEMORY_KNOWABLE, geometry
|
||||
@@ -183,6 +183,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import PEAK_CLAUSE, counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -267,8 +268,12 @@ WALL_BUDGET_S = 600.0
|
||||
|
||||
#: The settle's counts line (sources.js's fmtSyncResult — the first
|
||||
#: part is always "N added"; the rest are conditional, " · "-joined —
|
||||
#: the shape the pre-phase announcer spoke, byte-identical).
|
||||
RESULT_LINE = re.compile(r"^\d+ added( · \d+ (updated|unchanged|pruned))*$")
|
||||
#: the shape the pre-phase announcer spoke, byte-identical) plus the
|
||||
#: optional phase-187 peak clause — the counts are still anchored, so
|
||||
#: a mutated counts line still fails this shape.
|
||||
RESULT_LINE = re.compile(
|
||||
rf"^\d+ added( · \d+ (updated|unchanged|pruned))*{PEAK_CLAUSE}$"
|
||||
)
|
||||
|
||||
#: A broken git source — the failure trigger (test 8): a ``file://``
|
||||
#: URL with no repository behind it (the URL shape passes add-time
|
||||
@@ -1146,7 +1151,7 @@ def test_settled_state_restores_the_pill(
|
||||
# counts line (the fmtSyncResult shape the pre-phase announcer
|
||||
# spoke, byte-identical).
|
||||
expect(page.locator("#sync-progress")).to_be_visible()
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(f"{N_FILES} added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(f"{N_FILES} added"))
|
||||
# The label parts are restored — the settle label carries no count
|
||||
# or chunk part.
|
||||
count = page.locator(".sync-label-count")
|
||||
@@ -1243,7 +1248,7 @@ def test_settled_cancel_shows_the_result_line(
|
||||
# line (byte-identical to the pre-phase announcer's string)…
|
||||
expect(page.locator("#sync-progress")).to_be_visible(timeout=30_000)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
"Sync cancelled", timeout=30_000
|
||||
counts_line("Sync cancelled"), timeout=30_000
|
||||
)
|
||||
# …the button settles retry-ready…
|
||||
expect(page.locator("#sync-label")).to_have_text("Sync sources", timeout=30_000)
|
||||
|
||||
@@ -130,6 +130,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line, strip_peak_clause
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -497,12 +498,18 @@ def _sync_from_sources_page(page: Page, base_url: str) -> dict[str, Any]:
|
||||
def _expect_counts_line(page: Page, expected: str) -> None:
|
||||
"""The counts line — the owner-visible sync report on the single sync text
|
||||
surface (phase 150; sources.js fmtSyncResult)."""
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(expected, timeout=15_000)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(expected), timeout=15_000)
|
||||
|
||||
|
||||
def _counts_line(page: Page) -> str:
|
||||
"""The counts line as text — the string test 4 compares across the apps."""
|
||||
return str(page.locator("#sync-progress-text").inner_text())
|
||||
"""The counts line as text — the string test 4 compares across the apps.
|
||||
|
||||
Phase 187 (task 03): the peak clause is stripped first. The byte-
|
||||
identity this test pins is the KB's (the same ten files report the same
|
||||
counts under both cadences); the megabytes belong to the process that
|
||||
ran them and differ between the two module apps by design.
|
||||
"""
|
||||
return strip_peak_clause(str(page.locator("#sync-progress-text").inner_text()))
|
||||
|
||||
|
||||
def _open_level(page: Page, *names: str) -> None:
|
||||
|
||||
@@ -98,6 +98,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -497,7 +498,7 @@ def test_fresh_sync_settles_same_day_shape(
|
||||
# the status is the source of truth for the settle; the UI's 2 s
|
||||
# poll renders it at most one tick later).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
f"{len(SYNC_FILES)} added", timeout=SYNC_TIMEOUT_MS
|
||||
counts_line(f"{len(SYNC_FILES)} added"), timeout=SYNC_TIMEOUT_MS
|
||||
)
|
||||
btn = page.locator("#sync-btn")
|
||||
expect(btn).to_be_enabled()
|
||||
|
||||
@@ -124,6 +124,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -807,7 +808,7 @@ def test_sync_button_names_the_summary_phase(
|
||||
expect(btn).to_be_enabled()
|
||||
expect(btn).not_to_have_attribute("aria-busy")
|
||||
# The counts line rides the box (phase 150 — the sole sync surface).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(f"{N_FILES} added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(f"{N_FILES} added"))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -132,6 +132,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -647,11 +648,11 @@ def test_upload_toast_then_navigate_away(
|
||||
# "(n/m)" count is IN the full string (the never-clipped pin),
|
||||
# and the button title carries the same full string (A4 — one
|
||||
# source, two surfaces).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(
|
||||
re.compile(
|
||||
rf"^Syncing… {re.escape(UPLOAD_NAME)}/.+\.md \(\d+/{N_FILES}\)"
|
||||
r"(?: · chunk \d+/\d+)?$"
|
||||
),
|
||||
)),
|
||||
timeout=SETTLE_TIMEOUT_MS,
|
||||
)
|
||||
# The box is the VISIBLE running surface (the count's old
|
||||
@@ -686,7 +687,7 @@ def test_upload_toast_then_navigate_away(
|
||||
expect(btn).to_be_enabled()
|
||||
expect(btn).not_to_have_attribute("aria-busy")
|
||||
# The counts line rides the box (phase 150 — the sole sync surface).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(f"{N_FILES} added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(f"{N_FILES} added"))
|
||||
# Phase 97: the catalog is the drill-down tree — drill source →
|
||||
# folder `docs` before the per-level row asserts (the flat total
|
||||
# no longer exists in one tbody).
|
||||
@@ -844,11 +845,11 @@ def test_sync_live_file_label(
|
||||
# "(n/m)" count is IN the full string (the never-clipped pin),
|
||||
# and the button title carries the same full string (A4 — one
|
||||
# source, two surfaces).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(
|
||||
re.compile(
|
||||
rf"^Syncing… {re.escape(SYNC_SOURCE_DIR)}/.+\.md \(\d+/{N_FILES}\)"
|
||||
r"(?: · chunk \d+/\d+)?$"
|
||||
),
|
||||
)),
|
||||
timeout=SETTLE_TIMEOUT_MS,
|
||||
)
|
||||
# The box is the VISIBLE running surface (the count's old
|
||||
@@ -885,7 +886,7 @@ def test_sync_live_file_label(
|
||||
expect(btn).to_be_enabled()
|
||||
expect(btn).not_to_have_attribute("aria-busy")
|
||||
# The counts line rides the box (phase 150 — the sole sync surface).
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(f"{N_FILES} added")
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(counts_line(f"{N_FILES} added"))
|
||||
# Phase 97: the catalog is the drill-down tree — drill source →
|
||||
# folder `notes` before the per-level row asserts (the flat total
|
||||
# no longer exists in one tbody).
|
||||
|
||||
@@ -111,6 +111,7 @@ from e2e.conftest import (
|
||||
USE_REAL_LLM,
|
||||
_wait_http,
|
||||
)
|
||||
from e2e.sync_helpers import counts_line
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
|
||||
@@ -574,7 +575,9 @@ def test_ignore_list_then_sync_scans(
|
||||
expect(page.locator("#sync-label")).to_have_text(
|
||||
SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS
|
||||
)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text("2 added", timeout=SYNC_TIMEOUT_MS)
|
||||
expect(page.locator("#sync-progress-text")).to_have_text(
|
||||
counts_line("2 added"), timeout=SYNC_TIMEOUT_MS
|
||||
)
|
||||
expect(page.locator("#sync-btn")).to_be_enabled()
|
||||
expect(page.locator("#sync-btn")).not_to_have_attribute("aria-busy")
|
||||
|
||||
|
||||
Reference in New Issue
Block a user