Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 152_sync_progress_image_pass
Build and Push Containers / build-and-push-app (push) Successful in 3m34s
Details
Build and Push Containers / build-and-push-db (push) Successful in 23s
Details

Browse Source 
All criteria verified. The CLI path is hook-less (only `app/api/sync.py` passes the hook), and the byte-identity is pinned by `test_on_image_progress_none_run_is_byte_identical`.

## Final Report — Phase 152 final verification pass

- **Verified (no new code needed):** `SyncStatus.images_done/images_total` + reset/clear rules in `app/api/sync.py`; `on_image_progress` threaded through the importer's three PDF vision spans with the continuous per-file counter; the `_image_hook` closure; `sources.js` `· image n/m` label part (box + title + aria).
- **C1** status climbs 1→N + UI shows `· image n/m`: ✅ unit 3/15→15/15, integration 1/15→7/15, E2E 1/4→4/4 (title/aria ride the full string)
- **C2** never frozen: ✅ E2E asserts ≥3 distinct positions, starting at 1, non-decreasing, ≤ total, reaching it
- **C3** cleared at overview/summaries + all terminals; per-file reset via `_hook`; no stale position for image-less files: ✅ unit pins + E2E settle (image part gone in box/title/aria, endpoint 0/0)
- **C4** `/api/sync/status` additive-only: ✅ full-dict pin passes (all pre-existing keys unchanged + the two new 0/0 keys)
- **C5** hook-less caller byte-identical: ✅ CLI passes no hook; `test_on_image_progress_none_run_is_byte_identical` green
- **C6** `uv run pytest`: **3526 passed, exit 0** (95s); dedicated E2E in isolation: **2 passed** (95s); `uv run ruff check . && uv run pyright`: **clean**; coverage: **99% on `app/` (>90% gate)**
- Regression: all five neighbouring E2E suites green in isolation (progress_box 8, chunk_progress 2, label_mobile 4, pdf_support 6, image_documents 4)
- **C7** atomic commit: left to the harness (no git index/history touched per rules)
- **Docs:** README.md — sync-label paragraph already updated this phase, accurately; all `.env.example` vars documented; routes/commands verified — **no changes needed**. AGENTS.md — all references (PLAN, user_stories, test_smoke.py, .env.example, gpg config, .gitignore) verified — **no changes needed**
- **Defects found in prior work:** none (one prior env-leak defect was already fixed inside this phase's task 04, conftest)
- No live-infrastructure touched; no ntfy used. **Next pending phase:** none — `todo/` held only this phase
This commit is contained in:
ducoterra
2026-10-03 01:52:41 -04:00
parent c7cfcd26b5
commit 12fa69b7ea
39 changed files with 3971 additions and 133 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/e2e/test_sync_image_progress.py
+885
View File
@@ -0,0 +1,885 @@
"""Phase 152 task 05 E2E (Playwright, mock-only): during a PDF sync whose
images take time to describe, the live label shows the image position
MOVING — the owner's "stuck for several minutes" (TODO.md:3: the label
sat on ``Syncing… … (2540/2540) · chunk 125/126`` through the PDF's
vision spans) is impossible on the fixture.
The dedicated story suite for ``152_sync_progress_image_pass`` (A16 —
one Playwright file per phase, run in isolation): the ``on_image_progress
hook (phase 152, tasks 02/03) fires immediately before each vision
describe call with ``(position, total)``, the runner's ``_image_hook``
closure lands it on ``SyncStatus.images_done/images_total``, and
``fmtSyncPhaseLabel`` renders ``· image n/m`` — the phase-149 box
(``#sync-progress``) is the surface that shows it, the button
``title``/``aria-label`` carry the UNTRUNCATED composed string (A4).
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_sync_image_progress.py -v --no-cov
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the gate is the
deterministic mock plus the phase-64 ``tests/e2e/slow_llm.py`` delay
proxy in front of it (``SLOW_DELAY_S = 1.5 s``) that stretches EVERY
LLM call (vision describes included) so the fixture's vision span
outlives the UI's 2 s status poll and each position outlives the
recorder's ~300 ms page-sampling cadence by ~10× — the observed
positions are dense samples of a real climb, not races.
Timing fixture (the phase-129/98 rationale, re-sized for vision spans):
the fixture's ONE-PDF run is ≈ 27 LLM calls ≈ 40 s: 2 model probes +
10 in-flight embed batches (the ~4 KB-per-page text packs into 20
content chunks at the app's code-default, pinned sizing — the token
budget groups them into 10 requests of 2) + 4 image pipelines (one
vision describe + 2 content-embed batches each — the image doc's
context + description content packs into 3 chunks) + 1 PDF summary +
1 overview + 1 folder summary. The image span itself is 4 × (1.5 s
describe + 2 × 1.5 s embed) = 18 s: each of the 4 positions holds
for 4.5 s — that outlives the UI's 2 s poll (at least one tick lands
in every 4.5 s window — every position is rendered, not a race) and
the test's ~300 ms page sampling sees every rendered value.
KB fixture (the ``test_local_directory_sources.py`` / phase-94 API
registration pattern — a host temp dir, the app runs on the same host;
no git anywhere): ``imgpass/`` with EXACTLY ONE file, a PyMuPDF-built
``imgpass.pdf`` (LOCKED A1) — 10 text pages (each ~4 KB of
deterministic filler, well over the 100-char bar — the phase-140
pictorial gate never fires, and the layer is clean — the phase-133
fallback never fires: the EMBEDDED-IMAGE pass is the file's one and
only vision span) + 4 GENUINELY DISTINCT non-blank images on pages
1–4 (different sizes + seeds → different stored bytes — the phase-151
dedupe keeps all four; colorful noise, sample mean ~127 — well under
the 235 near-white floor, so the blank gate keeps all four: the
span's total is the post-dedupe/post-gate count, the work actually
about to happen). The expected chunk count is computed IN THE TEST by
calling ``extract_pdf_text`` (the SAME parser the importer uses) and
``chunk_document`` (the SAME function + arguments — the importer's
``max(400, target)`` floor + the overlap setting — as
``_replace_content_chunks``), read back from the PDF on disk — the
ground truth never drifts from the app under test. The sizing knobs
are pinned to the code defaults in the app-under-test env (the
leak-guard pattern — an operator's gitignored ``.env`` tuning
``BOR_CHUNK_TARGET_CHARS`` must not change the suite's ground truth).
The suite runs ONE module app (it binds its own port — the conftest
session app is never started in an isolated run) over the shared E2E
Postgres: the SLOW app (``BOR_LLM_BASE_URL`` = the slow proxy in front
of the mock), ``BOR_IMAGES=true`` + the ``pdf`` import token pinned
EXPLICITLY (an operator's local gitignored ``.env`` cannot leak the
scope in either direction) + ``BOR_PDF_PICTORIAL`` pinned off (the
house E2E convention — this suite's span is the embedded pass; the
fixture's ≥100-char pages make the pictorial pass a no-op even if the
knob leaked on).
Autouse cleanup (the phase-96 pattern): before each test wait for no
running sync on the module app (the run state lives in its memory),
then truncate the shared registry + KB tables (documents/chunks/
query_log/steering_notes/kb_overview/git_sources/sources_meta/
folder_summaries) — each test seeds its own source row and runs its
own sync; after each test the same truncate.
Test → observable mapping (Playwright Mapping Rule):
1. ``test_label_advances_through_the_image_span`` (the anti-freeze
proof — the UI's own 2 s poll, densely sampled at ~300 ms): after
clicking **Sync sources**, the test collects the DISTINCT
``#sync-progress-text`` values seen while the run is in flight
(first-seen order) — and asserts: (a) at least THREE distinct
labels containing ``· image `` are observed (the freeze the owner
reported is a single repeated label — "≥3 distinct positions" IS
the bug's negation), the positions START at 1, are
NON-DECREASING in observation order, never EXCEED the total (4 —
the post-gate count), and REACH it; (b) the CHUNK part appeared
EARLIER in the run (the first ``· chunk n/20`` label precedes the
first ``· image `` label — the two positions are distinct) and
every image-span label carries the chunk part WITH IT (the last
chunk position persists through the file's image span —
"… (1/1) · chunk 19/20 · image 3/4"), its total == the file's
TRUE chunk count; (c) the button ``title`` + ``aria-label`` carry
the SAME full string as the box on every running sample (A4 —
asserted per sample, atomically); (d) at settle the box shows the
counts line (``5 added`` — the PDF + its 4 image docs), the label
reads the terminal ``Synced HH:MM``, the count + chunk parts are
hidden + empty, and the image part is GONE (no stale position in a
terminal state — the endpoint's terminal carries 0/0 + the cleared
chunk position while the file counts keep their final values).
2. ``test_mid_run_reload_re_attaches_with_image_position`` (the
never-stale contract): the API starts the sync, the page's own
poll renders the image span (the FIRST observed position is
``1/4`` — the whole span is ahead), the test reloads the page
mid-run, and the load-time re-attach (``initSyncButton`` — a
running sync re-enters with the WHOLE status, phase 98) renders
the CURRENT image position with the A4 title/aria — then the run
settles to the counts line.
"""
from __future__ import annotations
import json
import os
import re
import subprocess
import sys
import time
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import httpx
import pymupdf as fitz # PyMuPDF — the fixture-PDF builder (LOCKED A1)
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text
from app.config import Settings as _Settings
from app.db import SessionLocal
from app.rag.chunker import chunk_document
from app.rag.llm import LLMClient
from app.rag.pdf import extract_pdf_text, is_usable_pdf_text
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
MOCK_PORT,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
REPO = Path(__file__).resolve().parents[2]
# Phase 79 (task 04, full inventory): the conftest session app owns its
# port in a combined run — this module's app binds its own port
# instead (a same-port second uvicorn dies on bind and would drive the
# wrong server). Env-overridable.
APP_PORT = int(os.environ.get("E2E_APP_PORT_IMGPROG", "8171"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"
#: The slow-LLM proxy's port (the conftest's mock LLM stays on
#: MOCK_PORT — the proxy fronts it).
SLOW_PORT = int(os.environ.get("E2E_SLOW_LLM_PORT_IMGPROG", "8909"))
SLOW_URL = f"http://127.0.0.1:{SLOW_PORT}"
#: Per-LLM-request delay on the proxy — sized so each in-flight VISION
#: PIPELINE's position (the describe + the content embed) outlives the
#: UI's 2 s poll (see the module docstring's timing fixture paragraph).
SLOW_DELAY_S = "1.5"
# --------------------------------------------------------------------------
# Fixture constants (deterministic, token-controlled)
# --------------------------------------------------------------------------
#: The local source — the temp directory's basename (``kind=local`` →
#: the directory's basename is the source name, phase 38).
SOURCE = "imgpass"
#: The single fixture file (the source is one file — the count part
#: reads "(1/1)" throughout the import phase).
PDF_NAME = "imgpass.pdf"
N_FILES = 1
#: The fixture PDF's shape — 10 text pages (each ~4 KB of filler —
#: 20 content chunks at the code-default sizing, 10 embed batches) +
#: 4 distinct non-blank images on pages 1–4 (the span's total —
#: post-dedupe, post-blank-gate: the work actually about to happen).
N_PAGES = 10
N_IMAGES = 4
#: The settled counts line (phase 150 — the box is the sole sync text
#: surface): the PDF + its 4 embedded-image docs, a fresh KB (nothing
#: updated/unchanged/pruned).
SETTLE_TEXT = f"{1 + N_IMAGES} added"
#: The three sizing knobs, pinned to the CODE DEFAULTS in both the
#: app-under-test env (the leak guards below) and the test process's
#: expected-value computation — an operator's gitignored ``.env``
#: (which may legitimately tune ``BOR_CHUNK_TARGET_CHARS``) cannot
#: change the suite's ground truth. Derived from the class fields, so
#: they never drift from app/config.py (the conftest leak-guard
#: pattern).
CHUNK_TARGET = _Settings.model_fields["chunk_target_chars"].default
CHUNK_OVERLAP = _Settings.model_fields["chunk_overlap_chars"].default
EMBED_BATCH_SIZE = _Settings.model_fields["embed_batch_size"].default
#: The test process's mirror of the app-under-test sizing settings
#: (explicit kwargs beat any ``.env`` for the fields the expected
#: values depend on).
_EXPECTED_SETTINGS = _Settings(
chunk_target_chars=CHUNK_TARGET,
chunk_overlap_chars=CHUNK_OVERLAP,
embed_batch_size=EMBED_BATCH_SIZE,
)
#: "Synced HH:MM" — the local-time last-result label (sources.js's
#: fmtSyncTime), any hour/minute (the test_sync_button.py pattern).
SYNCED_LABEL = re.compile(r"Synced \d{1,2}:\d{2}$")
#: The settled boot label — idle ("Sync sources") or the RE-ATTACHED
#: terminal from the previous test's sync (the module app is shared,
#: its in-memory sync state survives the test boundary, and the
#: phase-32 boot re-attach renders the last result).
IDLE_OR_SETTLED = re.compile(r"^(Sync sources|Synced \d{1,2}:\d{2})$")
#: The chunk part (the phase-129 unit-pinned shape): "· chunk n/m" —
#: ``m`` is asserted against the expected chunk count by the test.
CHUNK_PART_RE = re.compile(r"· chunk (\d+)/(\d+)")
#: The image part (the phase-152 unit-pinned shape): "· image n/m".
IMAGE_PART_RE = re.compile(r"· image (\d+)/(\d+)")
#: Generous settle budgets: a changed sync against the slowed LLM is
#: ≈ 40 s; the UI's 2 s poll settles at most one tick after the
#: terminal state lands.
SETTLE_TIMEOUT_MS = 120_000
# ---------------------------------------------------------------------------
# The fixture PDF (built with PyMuPDF — LOCKED A1)
# ---------------------------------------------------------------------------
def _noise_png(w: int, h: int, seed: int) -> bytes:
"""A deterministic colorful-noise PNG (the "real image" stand-in).
Different ``(w, h, seed)`` → different stored bytes (the phase-151
byte-dedupe keeps all four); the formula's sample mean is ~127 —
well under the phase-151 blank gate's 235 near-white floor, and
the samples scatter far outside the ±17 band — every image
survives the gate with room to spare (see the module docstring's
KB fixture paragraph)."""
pix = fitz.Pixmap(fitz.csRGB, fitz.IRect(0, 0, w, h))
for y in range(h):
for x in range(w):
pix.set_pixel(
x,
y,
((x * 7 + y * 11 + seed * 13) % 256,
(x * 3 + y * 17 + seed * 29) % 256,
(x * 5 + y * 23 + seed * 41) % 256),
)
return pix.tobytes("png")
#: The four fixture images' shapes (distinct sizes — the distinct-
#: bytes guarantee even if two seeds collided) — one per page 1–4.
_IMAGE_SHAPES = [(24, 18), (32, 24), (40, 20), (28, 28)]
def _page_text(k: int) -> str:
"""One page's deterministic ~4 KB filler (no randomness — the
expected chunk count is stable run to run)."""
base = (
f"Page {k} notes for the imgpass fixture document carry "
f"deterministic filler number {k} of the ten-page pdf, written "
f"out to size so the chunker has real content to work with "
f"across the page. "
)
return base * 11
def _build_pdf(pdf_path: Path) -> None:
"""The fixture PDF: ``N_PAGES`` text pages (``insert_textbox`` —
the return value is checked — an overfull rect would silently
drop text and skew the ground truth) + the 4 images on pages 1–4
(distinct ``insert_image`` streams — distinct xrefs, distinct
bytes). The metadata title is the stem (no title — the stem
fallback, phase 126 A2)."""
doc = fitz.open()
for k in range(1, N_PAGES + 1):
page = doc.new_page()
rc = page.insert_textbox(
fitz.Rect(48, 90, 548, 812), _page_text(k), fontsize=10
)
assert rc >= 0, f"the page {k} text did not fit the rect (rc={rc})"
if k <= N_IMAGES:
w, h = _IMAGE_SHAPES[k - 1]
page.insert_image(
fitz.Rect(72, 820 - h * 2, 72 + w * 2, 860),
stream=_noise_png(w, h, k),
)
doc.save(pdf_path)
doc.close()
def _expected_chunks(pdf_path: Path) -> list[str]:
"""The PDF's expected content chunks — the SAME function and
arguments the importer's ``_replace_content_chunks`` uses
(``chunk_document(content, rel, max(400, target), overlap)`` — the
importer's token-cap floor rule; ``rel`` = the source-relative
path) over the SAME parser the importer reads
(``extract_pdf_text`` — the ``---PAGE-N---`` page markers): the
endpoint's ``chunks_total`` must equal
``len(chunk_document(...))`` — the file's TRUE chunk count, not a
batch count."""
content = extract_pdf_text(pdf_path)
return chunk_document(
content, PDF_NAME, max(400, CHUNK_TARGET), CHUNK_OVERLAP
)
def _expected_batch_count(chunks: list[str]) -> int:
"""The expected embed-batch count — the SAME
``LLMClient._batch_texts`` partition the app's embed loop walks
(the phase-129 unit pattern — call it directly for the boundaries).
The task's sizing bar: ≥ 2 batches (the chunk part's climb is a
real sequence, not a single point)."""
return len(LLMClient(_EXPECTED_SETTINGS)._batch_texts(chunks))
# ---------------------------------------------------------------------------
# The in-browser snapshot functions (Playwright Mapping Rule — the
# rendered DOM is the observable).
#
# ``_JS_SAMPLE`` — test 1's dense sampler: the box text + the button
# title/aria read in the SAME in-browser evaluation (a 2 s poll tick
# updates the parts + the title together in one JS task, so the atomic
# check can never straddle one — the phase-128 pattern); ``running``
# marks the in-flight labels (the A4 title/aria contract applies to
# them).
#
# ``_JS_IMAGE_SNAP`` — test 2's wait: null until the box carries the
# ``· image n/<arg>`` part, then the texts (read atomically with the
# title/aria).
# ---------------------------------------------------------------------------
_JS_SAMPLE = """
() => {
const boxTextEl = document.querySelector('#sync-progress-text');
const btn = document.querySelector('#sync-btn');
if (!boxTextEl || !btn) return null;
const box = boxTextEl.textContent;
return {
box: box,
title: btn.getAttribute('title') || '',
aria: btn.getAttribute('aria-label') || '',
running: box.startsWith('Syncing…'),
};
}
"""
_JS_IMAGE_SNAP = """
(n) => {
const boxTextEl = document.querySelector('#sync-progress-text');
const btn = document.querySelector('#sync-btn');
if (!boxTextEl || !btn) return null;
const box = boxTextEl.textContent;
const m = box.match(/^Syncing… .*· image (\\d+)\\/(\\d+)$/);
if (!m || Number(m[2]) !== n) return null;
return {
box: box,
posN: Number(m[1]),
title: btn.getAttribute('title') || '',
aria: btn.getAttribute('aria-label') || '',
};
}
"""
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture(scope="module")
def imgpass_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""The one-source temp tree (see the module docstring): the app
server runs on the same host, so the paths are visible to it.
The directory NAME is the source name (``kind=local``, phase 38).
EXACTLY ONE file: the PyMuPDF-built ``imgpass.pdf`` (10 text
pages + 4 distinct non-blank images — see :func:`_build_pdf`)."""
root = tmp_path_factory.mktemp("bor_imgprog")
src = root / SOURCE
src.mkdir()
_build_pdf(src / PDF_NAME)
assert len(list(src.rglob("*"))) == N_FILES
return src
def _app_env(base_url: str, tmp_path_factory: pytest.TempPathFactory) -> dict[str, str]:
"""The per-module app env (the conftest pattern, cf.
``test_oneshot_llm_retry.py`` / ``test_kb_tree.py``): the
``base_url`` picks the LLM (the slow proxy in front of the mock),
``BOR_GIT_SOURCES`` forced empty (the sync source is this suite's
own DB-registered local row), the sizing knobs pinned to the code
defaults (the leak guards — see the module constants), the
phase-126 knobs pinned EXPLICITLY (``BOR_IMPORT_EXTENSIONS`` with
the ``pdf`` token — without it the walk is blind to the fixture —
and ``BOR_IMAGES=true`` — the embedded pass is the span under
test), the E2E ``BOR_PDF_PICTORIAL`` off-pin (the house
convention — the suite's span is the embedded pass; the fixture's
≥100-char pages keep the pictorial pass a no-op even if the knob
leaked on), and the other leak-guarded code defaults."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1" if USE_REAL_LLM else base_url
)
# Mock-calibrated threshold (conftest pattern): this suite never
# asks the chat model anything — the gate is never on a path.
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
# Phase 67: instant retry waits + the code-default budget (the
# conftest leak-guard pattern).
env["BOR_LLM_RETRY_DELAY"] = "0"
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
# The repo's .env file carries the owner's BOR_GIT_SOURCES (the app
# reads it from cwd) — override it with an EMPTY value (the env var
# beats the .env file): the registry must hold EXACTLY the local
# directory this suite registers.
env["BOR_GIT_SOURCES"] = ""
# Leak guards (conftest pattern): an operator's local (gitignored)
# .env cannot leak corpus-specific settings into the app under
# test — including the chunking / batching knobs this suite's
# ground truth is computed from (see the module constants).
env["BOR_DOCS_REPO"] = ""
env["BOR_SUGGESTIONS"] = json.dumps(
_Settings.model_fields["suggestions"].default
)
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
env["BOR_CHUNK_TARGET_CHARS"] = str(CHUNK_TARGET)
env["BOR_CHUNK_OVERLAP_CHARS"] = str(CHUNK_OVERLAP)
env["BOR_EMBED_BATCH_SIZE"] = str(EMBED_BATCH_SIZE)
# Phase 126/122: the import scope WITH the ``pdf`` token + the
# embedded-image toggle ON (the span under test) + the E2E
# pictorial off-pin + the suite-private image home (the app under
# test must not write copies into the owner's real ~/bor-sources).
env["BOR_IMPORT_EXTENSIONS"] = "md,markdown,txt,pdf"
env["BOR_IMAGES"] = "true"
env["BOR_PDF_PICTORIAL"] = "0"
env["BOR_IMAGE_DIR"] = str(tmp_path_factory.mktemp("bor_imgprog_images"))
env["BOR_SOURCES_DIR"] = str(tmp_path_factory.mktemp("bor_checkouts"))
return env
@pytest.fixture(scope="module")
def slow_llm(mock_llm: int) -> Iterator[int]:
"""The delay-injecting reverse proxy in front of the mock LLM
(``tests/e2e/slow_llm.py``) — the timing fixture for the whole
suite: it delays EVERY LLM call, vision describes included, so
each in-flight image pipeline's position outlives the UI's 2 s
poll (see ``SLOW_DELAY_S`` + the module docstring's timing
fixture paragraph)."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["SLOW_LLM_DELAY_S"] = SLOW_DELAY_S
env["E2E_MOCK_PORT"] = str(MOCK_PORT)
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "tests.e2e.slow_llm:app",
"--host", "127.0.0.1", "--port", str(SLOW_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
try:
_wait_http(f"{SLOW_URL}/v1/models")
yield SLOW_PORT
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_server(
mock_llm: int,
slow_llm: int,
imgpass_dir: Path,
tmp_path_factory: pytest.TempPathFactory,
) -> Iterator[str]:
"""The SLOW app under test (the whole suite):
``BOR_LLM_BASE_URL`` is the slow proxy in front of the mock (the
timing fixture — see the module docstring). ``imgpass_dir`` is a
dependency only for the fixture ordering (the temp tree exists
before the first test). Skips the module (clear instructions)
when Postgres is down — the sync needs the KB tables."""
env = _app_env(f"{SLOW_URL}/v1", tmp_path_factory)
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
try:
_wait_http(f"{APP_URL}/api/health")
body = httpx.get(f"{APP_URL}/api/health", timeout=5).json()
if body["db"] != "up":
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
yield APP_URL
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
def _truncate_all() -> None:
"""Fresh registry + KB (the E2E isolation pattern): the E2E suites
share one Postgres, so a leftover git_sources row or document would
pollute the file counts + the counts line the syncs and the
assertions pin exactly. ``sources_meta`` (the KB generation counter)
resets with the KB — the sync re-creates the row."""
with SessionLocal() as db:
db.execute(
text(
"TRUNCATE chunks, documents, query_log, steering_notes, "
"kb_overview, git_sources, sources_meta, folder_summaries"
)
)
db.commit() # without the commit the TRUNCATE rolls back (the house pattern)
def _wait_sync_settled(base_url: str, timeout_s: float = 180.0) -> None:
"""No background sync may leak across tests (the run state lives in
the app's memory, and a still-running run would keep importing into
the NEXT test's truncated KB): wait for the app's sync status to
be non-running BEFORE the truncate. Own admin session (the
endpoint is admin-only) — usually a no-op: the tests settle only
after their run's terminal state."""
with httpx.Client(base_url=base_url, timeout=5.0) as client:
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, r.text
deadline = time.monotonic() + timeout_s
while time.monotonic() < deadline:
r = client.get("/api/sync/status")
assert r.status_code == 200, r.text
if r.json()["state"] != "running":
return
time.sleep(0.2)
raise AssertionError(f"a sync was still running at a test boundary: {base_url}")
@pytest.fixture(autouse=True)
def _clean(app_url: str) -> Iterator[None]:
"""Per-test KB isolation (the phase-96 cleanup pattern): the
shared E2E Postgres is truncated before AND after every test —
each test seeds its own source row and runs its own sync, so the
KB rows + folder_summaries rows of the temp source are exactly
this test's own. The pre-wait keeps a leaked run from the previous
test (the module app is shared) from racing the truncate."""
_wait_sync_settled(app_url)
_truncate_all()
yield
_truncate_all()
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _register_source(base_url: str, path: str) -> None:
"""Register the temp directory through the authenticated API (the
``test_local_directory_sources.py`` / phase-94 pattern — a plain
directory is an API/DB-only operation)."""
with httpx.Client(base_url=base_url, timeout=30.0) as client:
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, r.text
r = client.post(
"/api/git-sources", json={"kind": "local", "path": path}
)
assert r.status_code == 201, r.text
def _wait_terminal(app_url: str, timeout_s: float = 180.0) -> dict[str, Any]:
"""Poll the status endpoint (own admin session) until the
UI-/API-triggered run reaches a terminal state — the source of
truth for the settle (the UI's 2 s poll renders it at most one
tick later). A FAILED terminal fails the test with the terminal
body (the ``detail`` / error text) — never an opaque label
timeout."""
with httpx.Client(base_url=app_url, timeout=30.0) as client:
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
assert r.status_code == 204, r.text
deadline = time.monotonic() + timeout_s
body: dict[str, Any] = {}
while time.monotonic() < deadline:
r = client.get("/api/sync/status")
assert r.status_code == 200, r.text
body = r.json()
if body["state"] in ("success", "failed"):
assert body["state"] == "success", body
return body
time.sleep(0.2)
raise AssertionError(f"the run never reached a terminal state: {body}")
def _start_sync_click(page: Page, app_url: str, sync_dir: Path) -> None:
"""Register the source, land the admin on the sources view, wait
for the settled boot label (the phase-64 pattern — also proves the
phase-32 boot re-attach completed, so no later re-render can
clobber the running state), then click **Sync sources** and pin
the click's immediate 202 state (the bare "Syncing…" — the count
AND the chunk parts hidden: no file yet, no batch in flight — and
the image part is absent too: no PDF's vision span has started,
``images_total == 0``)."""
_register_source(app_url, str(sync_dir))
login(page, app_url) # lands on /sources.html (the button's home)
btn = page.locator("#sync-btn")
expect(btn).to_be_visible(timeout=30_000)
expect(page.locator("#sync-label")).to_have_text(IDLE_OR_SETTLED)
expect(btn).to_be_enabled()
btn.click()
# The click's immediate state (the 202 moment — the bare prefix,
# the phase-64 contract): disabled, aria-busy, spinning icon, and
# the label's MAIN part alone ("Syncing…" — the 202 click carries
# no status, so the pinned parts hide, phase 128/129 writers).
# Phase 149 (task 01): the running detail rides the box — the 202's
# bare label is in #sync-progress (the button is the spinner).
expect(btn).to_be_disabled()
expect(btn).to_have_attribute("aria-busy", "true")
expect(btn.locator(".sync-icon")).to_have_class(re.compile(r"\bis-spinning\b"))
expect(page.locator("#sync-progress")).to_have_text("Syncing…")
expect(page.locator(".sync-label-count")).to_be_hidden()
expect(page.locator(".sync-label-chunk")).to_be_hidden()
expect(page.locator("#sync-error-banner")).to_be_hidden()
# ---------------------------------------------------------------------------
# 1. The anti-freeze proof — the label advances through the PDF's
# image span (≥3 distinct positions, monotonic, chunk part earlier,
# the A4 title/aria, the settled clearing)
# ---------------------------------------------------------------------------
def test_label_advances_through_the_image_span(
page: Page, app_url: str, imgpass_dir: Path
) -> None:
"""A fresh admin page on ``/sources.html`` clicks **Sync sources**;
the test then densely samples (≈ every 300 ms) the rendered
``#sync-progress-text`` — NOT the API (the rendered DOM is the
observable; the UI's own 2 s poll is the writer) — collecting the
DISTINCT values in first-seen order until the settled counts line,
and asserts the phase-152 contract end to end (see the module
docstring's test mapping): the image part climbs (≥3 distinct
positions, starting at 1, non-decreasing, never above the total,
reaching it), the chunk part appeared earlier (and persists
through the image span with the file's TRUE chunk total), the
A4 title/aria ride the full string on every running sample, and
the settle clears the image position (the counts line, the
terminal label, the cleared endpoint position — no stale
position in a terminal state)."""
page.set_default_timeout(30_000)
# The ground truth — computed from the PDF ON DISK with the SAME
# parser + function + arguments the importer uses (see the helper
# docstrings): the TRUE chunk count + the batch count (the task's
# sizing bar — ≥2 embed batches, so the chunk part's climb is a
# real sequence).
pdf_path = imgpass_dir / PDF_NAME
chunks = _expected_chunks(pdf_path)
n_chunks = len(chunks)
assert n_chunks >= EMBED_BATCH_SIZE + 1, n_chunks # 2+ embed batches
assert _expected_batch_count(chunks) >= 2 # the sizing bar
# The happy path's premise: a clean text layer (the phase-133
# fallback never fires) — the embedded-image pass is the file's
# one and only vision span (the ≥100-char pages keep the phase-140
# gate candidate-free even under the app's pinned-off knob).
assert is_usable_pdf_text(extract_pdf_text(pdf_path))
_start_sync_click(page, app_url, imgpass_dir)
# The observation loop (the task's "poll the page, not the API"):
# dense samples of the rendered box text (≈ 300 ms — the evaluate
# round-trip + the sleep) in first-seen order, each sample read
# atomically with the button's title/aria (A4 checked per running
# sample), until the settled counts line lands (or the generous
# deadline — a failed run then fails LOUDLY in _wait_terminal
# with the terminal body, never an opaque timeout).
seen: list[str] = []
last: str | None = None
deadline = time.monotonic() + 150
while time.monotonic() < deadline:
snap = page.evaluate(_JS_SAMPLE)
assert snap is not None
if snap["running"]:
# A4 — while a run is in flight, the title + the
# aria-label carry the SAME untruncated full string as the
# box (the "· image" part included).
assert snap["title"] == snap["box"], snap
assert snap["aria"] == snap["box"], snap
if snap["box"] != last:
seen.append(snap["box"])
last = snap["box"]
if snap["box"] == SETTLE_TEXT:
break
time.sleep(0.2)
# --- the settle (the UI's 2 s poll renders the terminal at most one
# tick after it lands; the endpoint is the source of truth) ---
assert last == SETTLE_TEXT, f"the run never settled to the counts line: {seen[-5:]}"
terminal = _wait_terminal(app_url)
# The counts line's ground truth: the PDF + its 4 embedded-image
# docs — one file, 5 added.
assert terminal["detail"]["added"] == 1 + N_IMAGES, terminal
# The keep-final-file-counts split (the phase-64/129 convention,
# restated for the phase-152 sub-position): the FILE counts keep
# their final values, the sub-file chunk + image positions clear
# with the file (0/0 in the terminal — no stale position), the
# phase clears.
assert terminal["files_done"] == N_FILES and terminal["files_total"] == N_FILES, terminal
assert terminal["current_file"] is None, terminal
assert terminal["current_chunk"] == 0 and terminal["chunks_total"] == 0, terminal
assert terminal["images_done"] == 0 and terminal["images_total"] == 0, terminal
assert terminal["phase"] is None, terminal
# --- the run was real (the prelude + the import span were
# sampled) ---
assert any(t == "Syncing…" for t in seen), f"no bare prelude label: {seen[:6]}"
import_labels = [t for t in seen if t.startswith("Syncing… ")]
assert any(f"({N_FILES}/{N_FILES})" in t for t in import_labels), (
f"no file count part: {import_labels[:6]}"
)
# --- the chunk part appeared EARLIER in the run (the two
# positions are distinct) ---
chunk_idx = next(
(i for i, t in enumerate(seen) if CHUNK_PART_RE.search(t)), None
)
assert chunk_idx is not None, f"no chunk part observed: {seen}"
first_chunk = CHUNK_PART_RE.search(seen[chunk_idx])
assert first_chunk is not None, seen[chunk_idx]
# The chunk part's total is the file's TRUE chunk count (not a
# batch count — the phase-129 A6 assumption, the E2E lens).
assert int(first_chunk.group(2)) == n_chunks, (first_chunk.group(0), n_chunks)
# --- the image part: ≥3 distinct positions (the anti-freeze
# proof — the owner's freeze is a single repeated label), the
# positions START at 1, are NON-DECREASING in observation order,
# never EXCEED the total (the post-dedupe/post-gate count — the
# work actually about to happen), and REACH it ---
def _image_part(t: str) -> re.Match[str]:
m = IMAGE_PART_RE.search(t)
assert m is not None, t # the list comprehension's filter guarantees it
return m
image_items = [
(i, _image_part(t))
for i, t in enumerate(seen)
if IMAGE_PART_RE.search(t)
]
assert len(image_items) >= 3, (
f"fewer than 3 distinct image positions observed: "
f"{[m.group(0) for _, m in image_items]}"
)
first_image_idx = image_items[0][0]
assert chunk_idx < first_image_idx, (chunk_idx, first_image_idx)
positions = [(int(m.group(1)), int(m.group(2))) for _, m in image_items]
assert positions[0][0] == 1, f"the climb does not start at 1: {positions}"
assert all(m == N_IMAGES for _, m in positions), positions
assert all(n <= N_IMAGES for n, _ in positions), positions
ns = [n for n, _ in positions]
assert ns == sorted(ns), f"the climb is backwards somewhere: {positions}"
assert N_IMAGES in ns, f"the climb never reached the total: {positions}"
# The two positions COEXIST (distinct and both may show — "… (1/1)
# · chunk 19/20 · image 3/4"): every image-span label carries the
# chunk part with it (the last chunk position persists through the
# file's image span — the per-file reset + the phase transitions
# clear it, not the end of the chunk span).
for i, _ in image_items:
cm = CHUNK_PART_RE.search(seen[i])
assert cm is not None, f"an image-span label lost the chunk part: {seen[i]}"
assert int(cm.group(2)) == n_chunks, (cm.group(0), n_chunks)
# --- the settle: the counts line, the terminal label, the parts
# cleared + empty, and the IMAGE part GONE (no stale position in a
# terminal state — the endpoint pin above is the server-side
# half) ---
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=30_000)
btn = page.locator("#sync-btn")
expect(btn).to_be_enabled()
expect(btn).not_to_have_attribute("aria-busy")
expect(page.locator(".sync-label-count")).to_be_hidden()
expect(page.locator(".sync-label-count")).to_have_text("")
expect(page.locator(".sync-label-chunk")).to_be_hidden()
expect(page.locator(".sync-label-chunk")).to_have_text("")
# The counts line rides the box (phase 150 — the sole sync text
# surface): the image part is not in it, not in the title, not in
# the aria (the settle restores the ship aria value).
box = page.locator("#sync-progress-text").inner_text()
assert box == SETTLE_TEXT, box
assert "· image" not in box and "· image" not in (btn.get_attribute("title") or "")
assert btn.get_attribute("aria-label") == "Sync sources", btn.get_attribute("aria-label")
expect(page.locator("#sync-error-banner")).to_be_hidden()
# ---------------------------------------------------------------------------
# 2. The never-stale contract — a mid-run page reload re-attaches with
# the current image position visible
# ---------------------------------------------------------------------------
def test_mid_run_reload_re_attaches_with_image_position(
page: Page, app_url: str, imgpass_dir: Path
) -> None:
"""The admin clicks **Sync sources** (the run is started the way
the owner starts it — the click is what starts the page's 2 s
poll; the reload is the contract under test), the page's own 2 s
poll renders the image span (the FIRST observed position is
``1/4`` — the whole span is ahead, so the reload + the load-time
re-attach happen with the span in flight), the test reloads the
page mid-run, and the load-time re-attach (``initSyncButton`` —
a running sync re-enters with the WHOLE status, phase 98's
never-stale contract) renders the CURRENT ``· image n/4`` with
the A4 title/aria; the run then settles to the counts line (the
image part gone — no stale position)."""
page.set_default_timeout(30_000)
_start_sync_click(page, app_url, imgpass_dir)
# The image span, rendered by the page's own 2 s poll (the span
# starts after the prelude + the file's embed batches — ≈ 21 s
# under the slow proxy; each position holds 4.5 s — the describe
# + the image doc's 2 embed batches).
snap = page.wait_for_function(
_JS_IMAGE_SNAP, arg=N_IMAGES, timeout=60_000
).json_value()
# The FIRST observed position is 1 — the whole span is ahead (the
# reload's re-attach is guaranteed to catch the span in flight:
# ≈ 18 s remain, the local reload + the load-time status fetch
# land well inside it).
assert snap["posN"] == 1, snap
# A4 — the pre-reload running state is untruncated on the title +
# the aria (the "· image" part included).
assert snap["title"] == snap["box"] and "· image" in snap["title"], snap
assert snap["aria"] == snap["box"], snap
# The mid-run reload (the session cookie survives — the re-attach
# is the admin's, phase 98).
page.reload()
# The load-time re-attach: a running sync re-enters with the WHOLE
# status — the CURRENT image position is visible (the never-stale
# contract), the A4 title/aria ride it.
snap2 = page.wait_for_function(
_JS_IMAGE_SNAP, arg=N_IMAGES, timeout=30_000
).json_value()
assert snap2["title"] == snap2["box"] and "· image" in snap2["title"], snap2
assert snap2["aria"] == snap2["box"], snap2
# Then the settle: the counts line (the image part gone — the
# terminal clears the position, test 1 pins the endpoint half),
# the terminal label, the parts cleared + empty.
expect(page.locator("#sync-progress-text")).to_have_text(
SETTLE_TEXT, timeout=SETTLE_TIMEOUT_MS
)
expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=30_000)
btn = page.locator("#sync-btn")
expect(btn).to_be_enabled()
expect(btn).not_to_have_attribute("aria-busy")
expect(page.locator(".sync-label-count")).to_be_hidden()
expect(page.locator(".sync-label-chunk")).to_be_hidden()
assert "· image" not in (btn.get_attribute("title") or "")
expect(page.locator("#sync-error-banner")).to_be_hidden()
terminal = _wait_terminal(app_url)
assert terminal["detail"]["added"] == 1 + N_IMAGES, terminal
assert terminal["images_done"] == 0 and terminal["images_total"] == 0, terminal