Files
brain-of-reese/tests/e2e/test_upload_no_scan.py
T
ducoterra 3095c4c577 phase: 90_upload_no_scan
All verification is complete and green. Final report:

**Phase 90 (upload_no_scan) — final verification pass: all criteria verified, no fixes needed**

- Verified (not re-implemented): all 3 tasks already complete; working tree carries the phase's uncommitted changes
- Upload pipeline: `_run_upload` stops after row upsert; success = `{"message": "uploaded"}`, null/0/0 progress; `UploadOut` removed from `app/schemas.py`; gates/unpack/swap/failed states intact
- Copy: button "Upload", bare "Processing…", result line "Uploaded \<name\> — press Sync sources to import it."; hint + caption re-pointed at Sync; no "Upload &" remnants in `frontend/`
- Tests: `pytest tests/unit/test_git_sources.py tests/integration/test_git_sources_upload.py tests/unit/test_frontend_sync_upload.py` → 56 passed; E2E phase suite `tests/e2e/test_upload_no_scan.py` → 3 passed (zero docs after upload; ignore edit honored by Sync — 2 added, `notes/skipme.md` excluded; re-upload in-place, still zero docs); affected suites `test_archive_upload_sources.py` + `test_sync_upload_progress.py` + `test_source_removal_cleanup.py` → 15 passed
- Full suite: `uv run pytest --cov=app --cov-report=term-missing` → **1808 passed, 99% coverage** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors

**Completion criteria:**
1. Upload settles "ready for sync", zero docs, row + ignore editor — ✅ (E2E test 1)
2. Button "Upload", no "Upload &" copy — ✅ (`rg` empty in frontend/, app/, tests/)
3. RAG-page "Sync sources" imports upload honoring edited ignores — ✅ (isolated E2E test 2)
4. Suite green, >90% coverage, lint clean — ✅ (99%, ruff+pyright clean)
5. Atomic commit — ✅ left to harness per executor rules (no git add/commit run; tree staged-ready)

**Deviations:** `regenerate_overview` still appears once in `git_sources.py` — inside `delete_git_source` (phase-69 locked total-removal behavior, not the upload leg; upload leg is scan-free). `.agents/PLAN.md` does not exist in this repo; locked decisions A1–A4 were enforced from the phase files.

**Next pending phase:** `91_admin_theme_tab` (in `todo/`).
2026-09-09 14:08:18 -04:00

643 lines
28 KiB
Python

"""Phase 90 story E2E (Playwright): upload → (no scan) → edit ignores →
Sync sources.
Source: ``TODO.md`` L3 — "Uploading a source archive should not trigger a
scan — that should be left to the sync button on the RAG page. Right now
the sync starts right away which doesn't give the user time to edit the
ignore list."
Run in isolation (DB must be up: ``podman compose up -d db``):
uv run pytest tests/e2e/test_upload_no_scan.py -v --no-cov
The story gate for the whole **deferred-scan loop** end-to-end against the
real pipeline: an upload indexes **nothing** (phase 90 A1 — unpack +
register only, the button reads "Upload"), the owner edits the new
source's ignore list in the phase-89 per-row editor while the scan is
still owed, and the RAG page's **Sync sources** button performs the scan
and honors the edited ignores (phase 90 A4 — the sync's existing
``kind='local'`` + prune + ``ignore_paths`` path is the proof, no sync
change in this phase).
**Timing (the task's note):** the upload leg needs NO ``slow_llm`` proxy
— the upload makes **zero LLM calls** now (phase 90 removed the model
check + import), so its background run settles in milliseconds and the
2 s status poll settles one tick after the 202. The sync leg is short
(2 files + 1 ignored → a handful of mock-LLM requests), so instead of
racing a timer the suite polls the RAG page's settled sync UI
(``#sync-label`` "Synced HH:MM" + ``#sync-result`` counts) with generous
timeouts — the live "Syncing… <file> (n/m)" label is the
``test_sync_upload_progress.py`` suite's subject, not this one's.
The archive is **built in-test** with Python's ``tarfile``:
``e2e-upload-no-scan.tar.gz`` (source name ``e2e-upload-no-scan``) holds
``alpha.md`` + ``beta.md`` + ``notes/skipme.md`` with markdown sentinels
(``ALPHA-…`` / ``BETA-…`` / ``SKIPME-…``); the v2 archive (test 3, same
basename) modifies ``beta.md``, adds ``gamma.md`` (``GAMMA-…``), and
drops ``alpha.md`` — the in-place-replace subject. ``notes`` is the
phase-89 ignore entry: the pure prefix rule
(``is_ignored("notes/skipme.md", ("notes",))``) excludes the skipme file
from the sync's walk — and from its ``files_total`` (the pre-walk uses
the same ignore tuple), so a 3-file upload imports exactly 2 documents.
Per-module app env (the conftest pattern, module-scoped — mirroring
``test_archive_upload_sources.py``'s local helpers, NOT importing that
module): ``BOR_UPLOAD_DIR`` points at a scratch dir the host-side
assertions inspect (the app runs on the same machine), ``BOR_GIT_SOURCES``
is forced empty (the dev ``.env``'s fallback URL must never render as an
env row or become a second sync source), ``BOR_LLM_BASE_URL`` is the mock
LLM (the sync leg's model check + embeds + overview), and the autouse
``_clean`` truncates the shared Postgres (plus waits for no running
background job — this suite's sync must never leak into the next test).
Contract under test:
* **test 1 — upload does not scan**: the "Upload" button (phase 90 A3)
→ the archive → the 202 "Successfully uploaded — <source>" toast →
the settled result line "Uploaded <source> — press Sync sources to
import it." with the terminal no-count status payload
(``{"message": "uploaded"}``, null/0/0 progress — phase 90 A2); the
source row is present (Local badge + the phase-89 "Ignore paths"
control) and its folder exists on the host under ``BOR_UPLOAD_DIR``
with all three files — but **zero documents are indexed**:
``GET /api/docs`` is empty and the RAG catalog (``/sources.html``)
settles on its empty state;
* **test 2 — ignore list, then Sync scans**: fresh state → upload →
success line → the row's phase-89 "Ignore paths" editor: type
``notes``, Save → the row shows the "1 ignored" count tag (the A5
round-trip through ``GET``) and the stored row carries
``ignore_paths == ["notes"]``; navigate to the RAG page → click
**Sync sources** (``#sync-btn``) → the sync settles ("Synced HH:MM",
"2 added", counts on the status endpoint) and the catalog lists
``alpha.md`` + ``beta.md`` for the source and **not**
``notes/skipme.md`` — the edit made BEFORE the sync is honored;
* **test 3 — re-upload replaces without a scan**: fresh state → upload
v1, success → upload v2 (same basename: the in-place identity) →
success again with exactly ONE source row (phase-49 contract — one
folder, one row) whose on-disk contents are ONLY v2's files — and
still **zero** documents indexed from it.
Test → story mapping (Playwright Mapping Rule):
1. ``test_upload_does_not_scan``
2. ``test_ignore_list_then_sync_scans``
3. ``test_reupload_replaces_without_scan``
"""
from __future__ import annotations
import io
import os
import re
import subprocess
import sys
import tarfile
import time
from collections.abc import Iterator
from pathlib import Path
from typing import Any
import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import text
from app.db import SessionLocal
from e2e.auth_helpers import login
from e2e.conftest import (
ADMIN_PASSWORD,
SESSION_SECRET,
USE_REAL_LLM,
_wait_http,
)
REPO = Path(__file__).resolve().parents[2]
# Phase 79 (task 04, full inventory): in a combined session run the
# conftest session app already owns its port — a second uvicorn on it
# dies on bind and this suite would silently drive the wrong server.
# The module app binds its own port instead (env-overridable).
APP_PORT = int(os.environ.get("E2E_APP_PORT_UPLOADSCAN", "8135"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"
GIT_SOURCES_URL = "/git-sources.html"
SOURCES_URL = "/sources.html"
#: The archive basename (both versions) — the source/folder name is the
#: filename minus the archive suffix (the phase-49 naming rule).
SOURCE_NAME = "e2e-upload-no-scan"
#: v1: two sentinel docs + the notes/ file the phase-89 ignore entry
#: (``notes``) excludes from the sync. v2 (same basename): beta CHANGED,
#: alpha DROPPED, gamma ADDED — the in-place-replace subject (the
#: on-disk folder swap).
ALPHA_SENTINEL = "ALPHA-NO-SCAN-7f31"
BETA_SENTINEL_V1 = "BETA-NO-SCAN-v1-2c90"
BETA_SENTINEL_V2 = "BETA-NO-SCAN-v2-8b42"
SKIPME_SENTINEL = "SKIPME-NO-SCAN-5e44"
GAMMA_SENTINEL = "GAMMA-NO-SCAN-9d16"
V1_FILES: dict[str, str] = {
"alpha.md": (
"# Alpha note\n"
"\n"
"First version of the alpha note — v2 drops it.\n"
f"\nMarker: {ALPHA_SENTINEL}\n"
),
"beta.md": (
"# Beta note\n"
"\n"
"First version of the beta note — it changes in v2.\n"
f"\nMarker: {BETA_SENTINEL_V1}\n"
),
"notes/skipme.md": (
"# Skip me\n"
"\n"
"Lives under the notes/ dir the owner ignores before the sync.\n"
f"\nMarker: {SKIPME_SENTINEL}\n"
),
}
V2_FILES: dict[str, str] = {
"beta.md": (
"# Beta note\n"
"\n"
"Second version of the beta note — modified in place.\n"
f"\nMarker: {BETA_SENTINEL_V2}\n"
),
"gamma.md": (
"# Gamma note\n"
"\n"
"Brand new in v2 — the add subject of the re-upload.\n"
f"\nMarker: {GAMMA_SENTINEL}\n"
),
}
#: The phase-89 ignore entry typed into the editor — the pure prefix
#: rule excludes ``notes/skipme.md`` (and every other notes/ file).
IGNORE_ENTRY = "notes"
#: "Synced HH:MM" — the local-time last-result label (sources.js's
#: fmtSyncTime), any hour/minute (test_sync_button.py's pattern).
SYNCED_LABEL = re.compile(r"Synced \d{1,2}:\d{2}")
#: Generous settle budgets: the upload run settles in milliseconds
#: (phase 90) and the UI's 2 s poll lands one tick after; the sync leg
#: is short (2 files against the fast mock LLM) and may settle between
#: the 2 s poll ticks — the settled-state assertions retry until then.
UPLOAD_TIMEOUT_MS = 30_000
SYNC_TIMEOUT_MS = 45_000
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
def _build_targz(path: Path, files: dict[str, str]) -> Path:
"""A deterministic ``.tar.gz`` (mtime 0) over the given files."""
with tarfile.open(path, "w:gz") as tf:
for rel, content in files.items():
data = content.encode("utf-8")
info = tarfile.TarInfo(rel)
info.size = len(data)
info.mtime = 0
tf.addfile(info, io.BytesIO(data))
return path
@pytest.fixture(scope="module")
def upload_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""The app's ``BOR_UPLOAD_DIR`` for this suite — a scratch dir the
host-side assertions inspect (the app server runs on the same
machine). The app creates it on the first upload."""
return tmp_path_factory.mktemp("bor_uploads") / "uploads"
@pytest.fixture(scope="module")
def tarball_v1(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""v1 — in its OWN subdirectory so v2 can reuse the same basename
(``e2e-upload-no-scan.tar.gz``): the in-place-replace identity IS
the filename, and ``set_input_files`` sends the path's basename."""
root = tmp_path_factory.mktemp("bor_archive_v1")
return _build_targz(root / f"{SOURCE_NAME}.tar.gz", V1_FILES)
@pytest.fixture(scope="module")
def tarball_v2(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""v2 — same basename as v1 (a different parent dir)."""
root = tmp_path_factory.mktemp("bor_archive_v2")
return _build_targz(root / f"{SOURCE_NAME}.tar.gz", V2_FILES)
@pytest.fixture(scope="module")
def app_server(
mock_llm: int,
upload_dir: Path,
tmp_path_factory: pytest.TempPathFactory,
) -> Iterator[str]:
"""The real app under test — per-module env: uploads unpack into a
scratch dir, the env git list is forced empty (this suite's own
upload row is the only sync source), and the LLM base URL is the
mock (the upload makes zero LLM calls — phase 90; the mock serves
the SYNC leg's model check + embeds + overview). No ``slow_llm``
proxy: the sync leg is 2 files and the suite polls the settled sync
UI with generous timeouts instead of racing a live label."""
env = dict(os.environ)
env.pop("DEBUGPY", None)
env["BOR_ENVIRONMENT"] = "e2e"
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
env["BOR_LLM_BASE_URL"] = (
"https://aipi.reeseapps.com/v1"
if USE_REAL_LLM
else f"http://127.0.0.1:{mock_llm}/v1"
)
# Mock-calibrated threshold (conftest pattern) — no chat turn is
# ever sent in this suite, but the app boots with the same env shape.
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
env.setdefault(
"BOR_DATABASE_URL",
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
)
# Phase 16: admin auth must be set or create_app() refuses to boot.
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
env["BOR_SESSION_SECRET"] = SESSION_SECRET
env["BOR_GIT_SOURCES"] = ""
# Phase 49: unpack uploads into the suite's scratch dir (host-
# inspectable) and keep the (unused) git checkouts out of the dev
# location.
env["BOR_UPLOAD_DIR"] = str(upload_dir)
env["BOR_SOURCES_DIR"] = str(tmp_path_factory.mktemp("bor_checkouts"))
proc = subprocess.Popen(
[sys.executable, "-m", "uvicorn", "app.main:app",
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
cwd=REPO,
env=env,
)
try:
_wait_http(f"{APP_URL}/api/health")
yield APP_URL
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
return app_server
def _truncate_all() -> None:
"""Fresh registry + KB per test (the E2E isolation pattern): the
row-count and doc-list assertions must be this test's own doing.
The E2E suites share one Postgres, and a leftover git_sources row
or document would corrupt them (a leftover row would be a SECOND
sync source, skewing the sync's counts)."""
with SessionLocal() as db:
db.execute(text("TRUNCATE chunks, documents, query_log, kb_overview, git_sources"))
db.commit()
def _wait_no_running_jobs(app_url: str) -> None:
"""No background job may leak across tests (the run states live in
the app's memory, and a still-running sync would keep importing
into the NEXT test's truncated KB): wait for both status endpoints
to be non-running BEFORE the truncate. Own admin session (the
endpoints are admin-only) — usually a no-op: test 2 settles only
after its sync's terminal state, and the upload run is
sub-second."""
with httpx.Client(timeout=5.0) as client:
client.post(f"{app_url}/api/login", json={"password": ADMIN_PASSWORD})
for path in ("/api/sync/status", "/api/git-sources/upload/status"):
body: dict[str, Any] = {}
deadline = time.monotonic() + 90
while time.monotonic() < deadline:
r = client.get(f"{app_url}{path}")
if r.status_code == 200:
body = r.json()
if body["state"] != "running":
break
time.sleep(0.2)
assert body["state"] != "running", (
f"a background job was still running at test boundary: {path} {body}"
)
@pytest.fixture(autouse=True)
def _clean(app_url: str, db_ready: None) -> Iterator[None]:
_wait_no_running_jobs(app_url)
_truncate_all()
yield
_truncate_all()
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _admin_git_sources_page(page: Page, app_url: str) -> None:
"""Real form login landing on the git sources page (admin settled:
Sign out visible, the manager revealed by the page module)."""
login(page, app_url, next=GIT_SOURCES_URL)
expect(page).to_have_url(app_url + GIT_SOURCES_URL, timeout=30_000)
expect(page.locator("#sign-out-btn")).to_be_visible(timeout=15_000)
expect(page.locator("#git-sources-gate")).to_be_hidden()
expect(page.locator("#git-sources-content")).to_be_visible()
def _docs(page: Page, app_url: str) -> list[tuple[str, str]]:
"""``GET /api/docs`` as the signed-in page → sorted (source, path)
pairs (the admin cookie rides the browser context)."""
r = page.request.get(f"{app_url}/api/docs")
assert r.status == 200, r.text
return sorted((d["source"], d["path"]) for d in r.json()["documents"])
def _upload_via_page(page: Page, archive: Path) -> str:
"""Pick the archive, submit the form, and wait for the result line
(the phase-64 202 path: toast at the 202, then the button's status
polling renders the line from the run's ``success``) — returns its
text. Every upload in this suite succeeds (the failure path is
another suite's subject), so any non-result outcome here is a test
error."""
page.set_input_files("#archive-upload-file", str(archive))
page.click("#archive-upload-btn")
result = page.locator("#archive-upload-result")
expect(result).to_be_visible(timeout=UPLOAD_TIMEOUT_MS)
text = result.text_content()
assert text is not None
return text
def _folder_files(folder: Path) -> set[str]:
"""The unpacked folder's file set, source-relative POSIX paths."""
return {
p.relative_to(folder).as_posix()
for p in folder.rglob("*")
if p.is_file()
}
# ---------------------------------------------------------------------------
# 1. Upload → 202 + toast → ready-for-sync line, the row + folder on
# disk — and ZERO documents indexed (the scan is the Sync button's
# job, phase 90 A1)
# ---------------------------------------------------------------------------
def test_upload_does_not_scan(
page: Page, app_url: str, db_ready: None, tarball_v1: Path, upload_dir: Path
) -> None:
"""One real upload through the page: the button reads **Upload**
(phase 90 A3); the 202 fires the "Successfully uploaded — <source>"
toast; the settled result line points at the next step — "Uploaded
<source> — press Sync sources to import it." (A3) — with the
terminal no-count status payload (``{"message": "uploaded"}``,
null/0/0 progress — A2). The source row is present (Local badge +
the phase-89 "Ignore paths" control) and its folder exists on the
host under ``BOR_UPLOAD_DIR`` with all three files — but
**zero documents are indexed**: ``GET /api/docs`` is empty and the
RAG catalog settles on its empty state (phase 90 A1 — the scan is
the RAG page's Sync button's job)."""
page.set_default_timeout(30_000)
_admin_git_sources_page(page, app_url)
expect(page.locator("#git-sources-tbody tr")).to_have_count(0)
# The button reads exactly "Upload" (phase 90 A3 — the
# scan-suffixed label is gone).
btn = page.locator("#archive-upload-btn")
expect(btn).to_be_enabled()
expect(btn).to_have_text("Upload")
page.set_input_files("#archive-upload-file", str(tarball_v1))
btn.click()
# The 202 moment (phase-64 A2): a single .toast node, visible,
# role=status, naming the safe source name…
toast = page.locator(".toast")
expect(toast).to_have_count(1, timeout=UPLOAD_TIMEOUT_MS)
expect(toast).to_have_class(re.compile(r"\bis-visible\b"))
assert toast.get_attribute("role") == "status"
expect(toast).to_have_text(f"Successfully uploaded — {SOURCE_NAME}")
# …and the background run (unpack + register only) settles at the
# next 2 s poll tick: the ready-for-sync result line (A3), the
# never-stale restore (button + input)…
result = page.locator("#archive-upload-result")
expect(result).to_be_visible(timeout=UPLOAD_TIMEOUT_MS)
expect(result).to_have_text(
f"Uploaded {SOURCE_NAME} — press Sync sources to import it."
)
expect(btn).to_be_enabled()
expect(btn).to_have_text("Upload")
expect(page.locator("#archive-upload-file")).to_have_value("")
# …and the terminal status is the no-count payload with null/0/0
# progress (the phase-64 key set, phase 90 A2).
r = page.request.get(f"{app_url}/api/git-sources/upload/status")
assert r.status == 200, r.text
status = r.json()
assert status["state"] == "success", status
assert status["detail"] == {"message": "uploaded"}, status
assert status["current_file"] is None
assert status["files_done"] == 0 and status["files_total"] == 0
# The row landed — Local badge, the unpacked path in the mono cell,
# and the phase-89 "Ignore paths" control (the editor the
# deferral exists for — test 2 opens it).
expect(page.locator("#git-sources-tbody tr")).to_have_count(1, timeout=30_000)
row = page.locator("#git-sources-tbody tr", has_text=SOURCE_NAME)
expect(row).to_have_count(1)
expect(row.locator("span.git-source-kind")).to_have_text("Local")
expect(row.locator("td.git-source-url-cell code")).to_have_text(
str(upload_dir / SOURCE_NAME)
)
expect(row.locator("button.git-source-ignore")).to_have_count(1)
expect(row.locator("button.git-source-ignore")).to_have_text("Ignore paths")
# Phase 90 A1: the upload indexes NOTHING — the KB is empty…
assert _docs(page, app_url) == []
# …and the folder exists on the host with ALL THREE files unpacked
# (the notes/ file included — the ignore list, not the upload,
# decides what the sync walks).
folder = upload_dir / SOURCE_NAME
assert folder.is_dir(), f"the unpacked folder is missing: {folder}"
assert _folder_files(folder) == {"alpha.md", "beta.md", "notes/skipme.md"}, (
f"unexpected unpacked files: {_folder_files(folder)}"
)
# …and so is the RAG catalog: it settles on its empty state (the
# visible #sources-empty is the deterministic "the load finished
# with zero documents" signal — a count-0 check alone would race
# the boot loadDocs fetch).
page.goto(app_url + SOURCES_URL)
expect(page.locator("#sources-empty")).to_be_visible(timeout=30_000)
expect(page.locator("#docs-tbody tr")).to_have_count(0)
expect(page.locator("#stat-docs")).to_have_text("0")
# ---------------------------------------------------------------------------
# 2. The phase-89 ignore list is edited BEFORE the scan — and the RAG
# page's Sync sources button honors it (phase 90 A4)
# ---------------------------------------------------------------------------
def test_ignore_list_then_sync_scans(
page: Page, app_url: str, db_ready: None, tarball_v1: Path, upload_dir: Path
) -> None:
"""The whole deferred-scan loop: upload (nothing indexed) → open
the row's phase-89 "Ignore paths" editor → type ``notes`` → Save
(the row shows the "1 ignored" count tag; the stored row carries
``ignore_paths == ["notes"]`` — the A5 round-trip) → navigate to
the RAG page → click **Sync sources** → the sync settles
("Synced HH:MM" label, "2 added" result, the status endpoint's
counts: added 2 / pruned 0 / 2 of 2 files — the ignored file is
excluded from the walk AND the denominator) → the catalog lists
``alpha.md`` + ``beta.md`` for the source and **not**
``notes/skipme.md``: the edit made before the sync is honored
(phase 90 A4 — the sync's existing kind='local' + prune +
ignore_paths path, unchanged in this phase)."""
page.set_default_timeout(30_000)
_admin_git_sources_page(page, app_url)
# Upload: the ready-for-sync line, and still nothing indexed.
assert _upload_via_page(page, tarball_v1) == (
f"Uploaded {SOURCE_NAME} — press Sync sources to import it."
)
assert _docs(page, app_url) == []
expect(page.locator("#git-sources-tbody tr", has_text=SOURCE_NAME)).to_have_count(1)
# The phase-89 per-row editor: open it from the row's button…
row = page.locator("#git-sources-tbody tr", has_text=SOURCE_NAME)
row.locator("button.git-source-ignore").click()
dialog = page.locator("#ignore-editor-dialog")
expect(dialog).to_be_visible(timeout=15_000)
expect(page.locator("#ignore-editor-source")).to_have_text(
str(upload_dir / SOURCE_NAME)
)
# …type the ignore entry (one path per line; the prefix rule
# excludes everything under notes/), and save.
page.fill("#ignore-editor-textarea", IGNORE_ENTRY)
page.click("#ignore-editor-save")
expect(dialog).to_be_hidden(timeout=30_000)
# The A5 round-trip: the row re-loads and shows the "1 ignored"
# count tag…
expect(row.locator(".git-source-ignore-count")).to_have_text(
"1 ignored", timeout=30_000
)
# …and the stored row carries the normalized list the sync will
# read.
r = page.request.get(f"{app_url}/api/git-sources")
assert r.status == 200, r.text
body = r.json()
assert [(s["kind"], s["path"], s["ignore_paths"]) for s in body["sources"]] == [
("local", str(upload_dir / SOURCE_NAME), [IGNORE_ENTRY])
]
# The RAG page — the catalog is still empty before the scan…
page.goto(app_url + SOURCES_URL)
expect(page.locator("#sync-btn")).to_be_visible(timeout=30_000)
expect(page.locator("#sources-empty")).to_be_visible(timeout=30_000)
expect(page.locator("#sync-label")).to_have_text("Sync sources")
expect(page.locator("#sync-error-banner")).to_be_hidden()
# Click Sync sources (the button the deferral exists for). The
# short sync (2 files against the fast mock LLM) may settle between
# the 2 s poll ticks — the settled-state assertions below retry
# with generous timeouts instead of racing a live label.
page.click("#sync-btn")
expect(page.locator("#sync-error-banner")).to_be_hidden()
expect(page.locator("#sync-label")).to_have_text(
SYNCED_LABEL, timeout=SYNC_TIMEOUT_MS
)
expect(page.locator("#sync-result")).to_have_text("2 added", timeout=SYNC_TIMEOUT_MS)
expect(page.locator("#sync-btn")).to_be_enabled()
expect(page.locator("#sync-btn")).not_to_have_attribute("aria-busy")
# The status endpoint agrees: 2 added, nothing pruned, and the
# ignored file is absent from the walk's denominator (2 of 2 —
# the pre-walk uses the row's ignore list).
r = page.request.get(f"{app_url}/api/sync/status")
assert r.status == 200, r.text
status = r.json()
assert status["state"] == "success", status
assert status["detail"]["added"] == 2, status
assert status["detail"]["pruned"] == 0, status
assert status["current_file"] is None
assert status["files_done"] == 2 and status["files_total"] == 2
# The catalog: exactly the two non-ignored docs, for the source —
# and the ignored one is NOT there.
expect(page.locator("#docs-tbody tr")).to_have_count(2, timeout=30_000)
expect(page.locator("#docs-tbody tr", has_text=SOURCE_NAME)).to_have_count(2)
expect(page.locator("#docs-tbody tr", has_text="alpha.md")).to_have_count(1)
expect(page.locator("#docs-tbody tr", has_text="beta.md")).to_have_count(1)
expect(page.locator("#docs-tbody tr", has_text="notes/skipme.md")).to_have_count(0)
assert _docs(page, app_url) == [
(SOURCE_NAME, "alpha.md"),
(SOURCE_NAME, "beta.md"),
], f"the ignore list was not honored: {_docs(page, app_url)}"
# ---------------------------------------------------------------------------
# 3. Re-upload, same filename → in-place replace (no duplicate row,
# folder swap on disk) — and still nothing indexed
# ---------------------------------------------------------------------------
def test_reupload_replaces_without_scan(
page: Page,
app_url: str,
db_ready: None,
tarball_v1: Path,
tarball_v2: Path,
upload_dir: Path,
) -> None:
"""v1 then v2 under the SAME filename (``e2e-upload-no-scan.tar.gz``
— v2 modifies ``beta.md``, adds ``gamma.md``, drops ``alpha.md``):
both unpack + register only (phase 90). After the re-upload the
list has exactly ONE ``e2e-upload-no-scan`` row (the in-place
identity, phase-49 contract — no duplicate), the registry agrees
(one kind=local row at the unpacked path), the on-disk folder holds
ONLY v2's files (the atomic swap) — and the KB is empty
THROUGHOUT (zero documents indexed by either upload: the scan is
the Sync button's job, never the upload's)."""
page.set_default_timeout(30_000)
_admin_git_sources_page(page, app_url)
# Baseline: v1 through the page (202 → the ready-for-sync line,
# one row, no index).
assert _upload_via_page(page, tarball_v1) == (
f"Uploaded {SOURCE_NAME} — press Sync sources to import it."
)
expect(page.locator("#git-sources-tbody tr", has_text=SOURCE_NAME)).to_have_count(1)
folder = upload_dir / SOURCE_NAME
assert _folder_files(folder) == set(V1_FILES)
assert _docs(page, app_url) == []
# Re-upload v2 — SAME basename, different parent dir (the file
# input's selection is replaced wholesale).
assert _upload_via_page(page, tarball_v2) == (
f"Uploaded {SOURCE_NAME} — press Sync sources to import it."
)
# No duplicate: exactly ONE row for that source (and one row
# total)…
expect(page.locator("#git-sources-tbody tr", has_text=SOURCE_NAME)).to_have_count(1)
expect(page.locator("#git-sources-tbody tr")).to_have_count(1)
# …and the registry agrees: one kind=local row, the unpacked path.
r = page.request.get(f"{app_url}/api/git-sources")
assert r.status == 200, r.text
body = r.json()
assert [(s["kind"], s["path"]) for s in body["sources"]] == [
("local", str(upload_dir / SOURCE_NAME))
]
# The on-disk folder holds ONLY v2's files (the swap replaced the
# whole folder in place — no stale v1 file survived)…
assert _folder_files(folder) == set(V2_FILES), (
f"unexpected unpacked files: {_folder_files(folder)}"
)
# …and the KB is STILL empty (the upload never scans, phase 90 A1
# — the Sync button is what will index v2's files).
assert _docs(page, app_url) == []