All gates green. Final report: **Verified (phase 167 final pass — all 6 tasks already in `complete/`)** - Live re-measure of the headline criterion on the **1,000-doc live-replica corpus** (restored `data/bor_live_kb_replica.dump.sql` into a scratch DB `bor_audit`, alembic 0011→0027, dropped after; the shared dev DB was never touched): `GET /api/docs/tree` default depth=1 = **8,474 identity → 1,128 B with `Content-Encoding: gzip`** (< 40 KB bar; ~2 MB defect fixed), repeat load **304 / 0 bytes**; depth=8 265,147→23,515; node drill 20,844→1,065; `/api/docs` 246,366→45,104; `/api/documents/content` 892,124→60,776 (`LOOK`, accepted-with-why). Reproduces `AUDIT.md` to within 1 byte. - Transport contracts proven live: `POST /api/chat` **no** `Content-Encoding` + frames arriving live + budget line `… enc=identity duration_ms=7380 stream=1`; HTML still `no-cache` with the `?v=` rewrite (and gzipped); A20 headers on 200 **and** 304; ETag identical across identity/gzip; `depth=0|9|abc`→422; anonymous tree/node→403; unknown folder/source→404 `unknown node`. **Gates (all 100% green)** - `uv run pytest` → **3980 passed**; `uv run pytest --cov=app --cov-report=term-missing` → **TOTAL 99%** (`compression.py`, `http_cache.py`, `response_budget.py` 100%; `api/docs.py` 99%, one defensive dedupe line). - `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Phase + regression E2E **in isolation** (`--junitxml`, exit codes): `test_kb_tree_lazy_load` 11, `test_kb_tree` 8, `test_kb_tree_nav` 4, `test_navbar_refresh` 7, `test_document_dates` 6, `test_git_source_dates` 4, `test_sync_summary_visibility` 3, `test_table_ux_polish` 6, `test_sources_midstream_bug` 6, `test_ls_tree_drilldown` 3, `test_smoke` 3 — **0 failures/errors** (the earlier `-q -q` run had simply suppressed pytest's summary line). **Documentation** — README.md + AGENTS.md reviewed in full (1,436 / 80 lines): - Fixed (pre-existing, phase-167-exposed): README *Caching & deploys* claimed "One Starlette middleware (`app/core/caching.py`) applies the rule" while the section now describes three transport layers → rewritten to name caching/compression/http_cache, the real outermost→innermost order, and why `bytes=` is uncompressed. - AGENTS.md rule 5 (UI Structure Check) gained the phase-167 clause: a lazily-drilled level keeps the current level rendered, its in-flight `role="status"` / `role="alert"`+Retry row lives in the same bounded scroll panel, and stat cards read server `totals` (never a walk of a depth-limited tree). - Cross-checked and consistent: every `BOR_*` in `.env.example` is in the README table, every `Settings` field is documented, all 15 README-cited API routes exist in the app, no stale "one fetch / drill client-side" claim remains anywhere. **Notable decisions / deviations** - `compression.py` adds a documented `_RebufferSmallResponses` seam beyond the D1 sketch (the caching layer's `BaseHTTPMiddleware` re-framing otherwise defeats Starlette's `minimum_size` exemption and inflates tiny responses) — integration-pinned, math still Starlette's. - A node-route **404 = vanished folder → honest reset to top level** (per `00_phase.md` D4/D6), not task-06's "alert + Retry"; 500 shows alert+Retry with the level preserved — both pinned, nothing weakened. - Retrieval battery not measurable (shared E2E DB holds no corpus), but `git diff --name-only -- app/rag` is empty, so retrieval is unchanged by construction. `AUDIT.md` left byte-identical (it lives under `todo/`, off-limits to this pass); no git index/history commands run. **Next pending phase:** none — `167_api_payload_diet` is the only entry in `.agents/phases/todo/`; the harness moves it to `complete/` and commits.
1204 lines
52 KiB
Python
1204 lines
52 KiB
Python
"""Phase 167 task 06 E2E (Playwright, mock-free): the Sources view loads a
|
||
SMALL catalog, drills it level by level, and revalidates instead of
|
||
re-downloading.
|
||
|
||
The dedicated story suite for ``167_api_payload_diet`` (``TODO.md:5`` —
|
||
"GET /api/docs/tree loads 2 MB in production which causes long loading
|
||
times"): the owner-visible half of the payload diet, pinned in a real
|
||
browser against the real app. Every claim here is the browser's own
|
||
observable — the request log, the parsed response body, the wire headers,
|
||
the rendered rows — never a server-side helper.
|
||
|
||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||
|
||
uv run pytest tests/e2e/test_kb_tree_lazy_load.py -v --no-cov
|
||
|
||
Mock-only suite (the phase-97 convention): no chat, no sync, no vision
|
||
call happens on any path below — the mock LLM server is started ONLY
|
||
because ``create_app`` wants a reachable ``BOR_LLM_BASE_URL`` at boot.
|
||
``E2E_REAL_LLM`` is therefore irrelevant here.
|
||
|
||
KB fixture — seeded STRAIGHT INTO THE DB (the ``test_prose_word_wrap.py``
|
||
pattern: the tree is database-only, so no sync and no LLM are involved),
|
||
after a TRUNCATE (the ``test_kb_tree.py`` isolation pattern — this suite
|
||
asserts exact counts, and the E2E Postgres is shared). Eight documents
|
||
across THREE sources, twelve chunks:
|
||
|
||
* ``atlas`` (5) — the deep chain the drill walks: ``index.md`` at the
|
||
root, ``alpha/`` → ``beta/`` → ``gamma/`` with one document at every
|
||
level, plus the image document ``alpha/pic.png`` (real PNG bytes in a
|
||
scratch dir, so the thumbnail renders through the NODE route and not
|
||
just the full tree);
|
||
* ``beacon`` (2) — ``top.md`` + ``notes/short.md``, with NO stored
|
||
description anywhere: the pending-marker source;
|
||
* ``cobalt`` (1) — one flat file: the level with no folders at all.
|
||
|
||
Three stored ``folder_summaries`` rows (the descriptions are long on
|
||
purpose — the pre-phase 2 MB payload was mostly inlined description
|
||
text, and the gzip assertion below needs a body over the threshold): the
|
||
``atlas`` root, ``atlas/alpha/beta`` (a level reached ONLY through a
|
||
node fetch), and ``cobalt``.
|
||
|
||
Dates are fixed, and deliberately NOT monotone with depth: the newest
|
||
document in ``atlas`` is the ROOT-level ``index.md``, while the deepest
|
||
file (``alpha/beta/gamma/three.md``) is the newest one INSIDE ``alpha``
|
||
— so the truncated ``alpha`` row's Updated cell proves phase 106 D9 (the
|
||
recursive max-date map) survives the depth limit, and the atlas row's
|
||
proves it does not leak downward.
|
||
|
||
Test → observable mapping (Playwright Mapping Rule):
|
||
1. ``test_first_boot_is_one_small_tree_request`` — THE TODO REPRO: the
|
||
Sources boot issues EXACTLY ONE ``/api/docs/tree`` request (and zero
|
||
``/api/docs/tree/node``), its query is ``depth=1``, its body carries
|
||
``totals`` (documents 8 / chunks 12 / a last_indexed stamp), every
|
||
source node's ``documents`` count is RECURSIVE (atlas 5 with two
|
||
levels of files it does not list), no node deeper than level 1
|
||
exists, and every level-1 folder reports ``truncated: true``. The
|
||
stat cards read ``totals`` (not a client-side walk of the partial
|
||
tree).
|
||
2. ``test_tree_response_arrives_gzipped`` — the wire half of the fix:
|
||
the tree response arrives ``Content-Encoding: gzip`` with
|
||
``Accept-Encoding`` in ``Vary`` (Chromium advertises gzip natively,
|
||
so this is the real negotiation, not a stub), and the uncompressed
|
||
body is over the ``BOR_COMPRESS_MIN_BYTES`` threshold the app boots
|
||
with (the assertion could never pass vacuously).
|
||
3. ``test_drill_into_a_folder_fetches_that_level_once`` — clicking a
|
||
source costs NOTHING (the depth-1 payload holds that level);
|
||
clicking the truncated ``alpha`` folder costs EXACTLY ONE
|
||
``/api/docs/tree/node?source=atlas&path=alpha``; the level renders
|
||
what the pre-phase full tree rendered — the recursive count and the
|
||
recursive D9 date on the truncated ``beta`` row, the stored
|
||
description, the file rows with their Created dates, the image
|
||
thumbnail through the node route — and the phase-156 bounded scroll
|
||
panel + sticky header survive the lazy render.
|
||
4. ``test_two_levels_deep_then_back_uses_the_cache`` — ``alpha`` (fetch
|
||
1) → ``beta`` (fetch 2, the level block shows the description that
|
||
only a node fetch can deliver) → the ``alpha`` crumb: ZERO new
|
||
requests and the rows re-render from what the client holds → the
|
||
``Knowledge base`` crumb: still zero → re-entering ``alpha``: still
|
||
two fetches in total (``levelCache``, never a refetch).
|
||
5. ``test_pending_folder_marker_survives_the_lazy_level`` — the
|
||
description-less ``beacon`` source and its ``notes`` folder render
|
||
the phase-98 D4 "Summary pending" marker (class + text + tooltip —
|
||
text and color, never color alone) in a level reached by drilling.
|
||
6. ``test_reload_revalidates_to_304`` — a document reload sends
|
||
``If-None-Match`` and the WIRE answers ``304`` with a header-sized
|
||
transfer while the table still renders (the browser serves the
|
||
cached body). Chromium hands ``fetch`` the cached 200 — Playwright's
|
||
response event never sees the 304 — so the proof rides the CDP
|
||
Network events (the same two facts a DevTools Network panel shows).
|
||
7. ``test_view_switch_revalidates_too`` — the same cache hit through the
|
||
SPA: History → Sources re-issues the tree request with
|
||
``If-None-Match`` and gets a 304 (the mount-once view's refresh hook
|
||
still asks the server, and the server answers cheaply).
|
||
8. ``test_an_edit_busts_the_tag_so_fresh_content_lands`` — the never-stale
|
||
half (PLAN §7.4): a folder description saved through the admin API
|
||
changes the body, so the reload's conditional request answers 200
|
||
with the new payload and the new text renders on the row.
|
||
9. ``test_vanished_folder_resets_to_the_top_level`` — a node stub that
|
||
answers 404 (the folder was pruned under the open view) is the honest
|
||
reset: the breadcrumb is gone, the source rows render, no stale
|
||
``alpha`` rows and no loading row left behind.
|
||
10. ``test_failed_drill_keeps_the_level_and_retry_recovers`` — a 500
|
||
node stub: the ``role="alert"`` line names the folder and the real
|
||
Retry button sits beside it, the level the user was ON stays
|
||
rendered (rows + breadcrumb untouched); Retry (the stub lifted)
|
||
loads the level.
|
||
11. ``test_in_flight_drill_marks_the_link_and_ignores_a_second_click`` —
|
||
with the node request held by the harness: the clicked link carries
|
||
``aria-busy`` + ``aria-disabled``, the ONE ``role="status"``
|
||
"Loading alpha…" line sits in the panel, a second click issues NO
|
||
second request, and releasing the held request renders the level.
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import base64
|
||
import json
|
||
import os
|
||
import subprocess
|
||
import sys
|
||
import time
|
||
from collections.abc import Iterator
|
||
from datetime import UTC, datetime
|
||
from pathlib import Path
|
||
from typing import Any
|
||
from urllib.parse import parse_qs, urlparse
|
||
|
||
import httpx
|
||
import pytest
|
||
from playwright.sync_api import Locator, Page, Response, expect
|
||
from sqlalchemy import text
|
||
|
||
from app.config import Settings as _Settings
|
||
from app.db import SessionLocal
|
||
from app.models import Chunk, Document, FolderSummary
|
||
from e2e.auth_helpers import login
|
||
from e2e.conftest import (
|
||
ADMIN_PASSWORD,
|
||
SESSION_SECRET,
|
||
_wait_http,
|
||
)
|
||
|
||
REPO = Path(__file__).resolve().parents[2]
|
||
|
||
# The conftest session app owns 8123 in a combined run — this module app
|
||
# binds its own port (the test_kb_tree.py pattern). Env-overridable.
|
||
APP_PORT = int(os.environ.get("E2E_APP_PORT_KBLAZY", "8185"))
|
||
APP_URL = f"http://127.0.0.1:{APP_PORT}"
|
||
|
||
TREE_PATH = "/api/docs/tree"
|
||
NODE_PATH = "/api/docs/tree/node"
|
||
|
||
# --------------------------------------------------------------------------
|
||
# The seeded catalogue (fixed IDs-by-name, fixed dates, fixed counts)
|
||
# --------------------------------------------------------------------------
|
||
|
||
ATLAS = "atlas"
|
||
BEACON = "beacon"
|
||
COBALT = "cobalt"
|
||
|
||
ATLAS_INDEX = "index.md"
|
||
ALPHA_ONE = "alpha/one.md"
|
||
ALPHA_PIC = "alpha/pic.png"
|
||
BETA_TWO = "alpha/beta/two.md"
|
||
GAMMA_THREE = "alpha/beta/gamma/three.md"
|
||
BEACON_TOP = "top.md"
|
||
NOTES_SHORT = "notes/short.md"
|
||
COBALT_SOLO = "solo.md"
|
||
|
||
#: ``(source, path, title, created_at day-of, chunks)`` — the whole
|
||
#: catalogue in ONE table so the counts below cannot drift from the
|
||
#: seed. The dates are NOT monotone with depth (see the docstring).
|
||
SEED_DOCS: tuple[tuple[str, str, str, str, int], ...] = (
|
||
(ATLAS, ATLAS_INDEX, "Atlas Index", "2024-07-01", 2),
|
||
(ATLAS, ALPHA_ONE, "Atlas Alpha One", "2024-02-11", 1),
|
||
(ATLAS, ALPHA_PIC, "Atlas Alpha Pic", "2024-02-20", 1),
|
||
(ATLAS, BETA_TWO, "Atlas Beta Two", "2024-05-05", 3),
|
||
(ATLAS, GAMMA_THREE, "Atlas Gamma Three", "2024-06-06", 1),
|
||
(BEACON, BEACON_TOP, "Beacon Top", "2024-03-03", 2),
|
||
(BEACON, NOTES_SHORT, "Beacon Notes Short", "2024-04-04", 1),
|
||
(COBALT, COBALT_SOLO, "Cobalt Solo", "2024-01-01", 1),
|
||
)
|
||
|
||
TOTAL_DOCS = len(SEED_DOCS)
|
||
TOTAL_CHUNKS = sum(row[4] for row in SEED_DOCS)
|
||
|
||
#: ``source -> (recursive document count, the subtree MAX created_at)``.
|
||
#: ``alpha``'s max is the DEEPEST file's date — the recursive map, not
|
||
#: the level's own files; ``atlas``'s max is its root file.
|
||
SOURCE_COUNTS: dict[str, tuple[int, str]] = {
|
||
ATLAS: (5, "2024-07-01"),
|
||
BEACON: (2, "2024-04-04"),
|
||
COBALT: (1, "2024-01-01"),
|
||
}
|
||
|
||
#: The stored descriptions. The length is load-bearing: the pre-phase
|
||
#: 2 MB payload was mostly inlined folder text, and test 2 needs a
|
||
#: depth-1 body over the compression threshold.
|
||
ATLAS_ROOT_SUM = (
|
||
"Atlas is the deep fixture source: the index note sits at the root, "
|
||
"one folder chain runs three levels down to gamma, and a picture "
|
||
"documents the alpha level alongside its notes."
|
||
)
|
||
BETA_SUM = (
|
||
"Beta is the second level of the atlas chain. It holds the two.md "
|
||
"note and the gamma folder below it, and this description reaches the "
|
||
"view through the per-level node route alone."
|
||
)
|
||
COBALT_SUM = (
|
||
"Cobalt is the single-file source: one flat document, no folders at "
|
||
"all, kept to pin the level that renders no folder rows."
|
||
)
|
||
|
||
#: The image document's summary (the vision description the importer
|
||
#: stores) — the thumbnail's alt text.
|
||
PIC_DESCRIPTION = (
|
||
"A one-pixel PNG fixture that pins the image-document thumbnail "
|
||
"through the lazily loaded level."
|
||
)
|
||
|
||
#: The D4 pending tooltip (the phase-98 copy, verbatim).
|
||
PENDING_TITLE = "No stored description yet — the next sync will generate one."
|
||
|
||
#: The owner edit that busts the ETag (test 8) — distinctive enough to
|
||
#: never match a seeded row.
|
||
ALPHA_SUMMARY_EDIT = "Owner note: alpha holds the first level of the atlas chain. (RESE-167-01)"
|
||
|
||
PNG_1X1 = base64.b64decode(
|
||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=="
|
||
)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Fixtures
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def app_server(mock_llm: int, tmp_path_factory: pytest.TempPathFactory) -> Iterator[str]:
|
||
"""The real app under test, on its own port.
|
||
|
||
The conftest leak-guard pattern throughout: an operator's local
|
||
``.env`` cannot reach the app under test (every value the suite
|
||
asserts on is pinned from the code default or the E2E constant).
|
||
``compress_min_bytes`` is pinned to the CODE DEFAULT (test 2 asserts
|
||
the depth-1 body clears it, so gzip is the production setting, not
|
||
a harness gift).
|
||
"""
|
||
env = dict(os.environ)
|
||
env.pop("DEBUGPY", None)
|
||
env["BOR_ENVIRONMENT"] = "e2e"
|
||
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
||
env["BOR_LLM_BASE_URL"] = f"http://127.0.0.1:{mock_llm}/v1"
|
||
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
||
env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
|
||
env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
|
||
env["BOR_LLM_RETRY_DELAY"] = "0"
|
||
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
|
||
env.setdefault(
|
||
"BOR_DATABASE_URL", "postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese"
|
||
)
|
||
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
||
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
||
env["BOR_GIT_SOURCES"] = ""
|
||
env["BOR_DOCS_REPO"] = ""
|
||
env["BOR_SUGGESTIONS"] = json.dumps(_Settings.model_fields["suggestions"].default)
|
||
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
|
||
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
|
||
env["BOR_COMPRESS_MIN_BYTES"] = str(_Settings.model_fields["compress_min_bytes"].default)
|
||
# The image bytes live in this module's scratch dir (the seeded
|
||
# image_path must point at bytes the public route serves).
|
||
image_dir = tmp_path_factory.mktemp("bor_kb_lazy_images")
|
||
env["BOR_IMAGE_DIR"] = str(image_dir)
|
||
proc = subprocess.Popen(
|
||
[
|
||
sys.executable,
|
||
"-m",
|
||
"uvicorn",
|
||
"app.main:app",
|
||
"--host",
|
||
"127.0.0.1",
|
||
"--port",
|
||
str(APP_PORT),
|
||
"--log-level",
|
||
"warning",
|
||
],
|
||
cwd=REPO,
|
||
env=env,
|
||
)
|
||
try:
|
||
_wait_http(f"{APP_URL}/api/health")
|
||
yield APP_URL
|
||
finally:
|
||
proc.terminate()
|
||
try:
|
||
proc.wait(timeout=10)
|
||
except subprocess.TimeoutExpired:
|
||
proc.kill()
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def app_url(app_server: str) -> str:
|
||
return app_server
|
||
|
||
|
||
def _truncate_kb() -> None:
|
||
"""The E2E isolation pattern (``test_kb_tree.py``): the suites share
|
||
one Postgres, and this suite pins exact counts + totals, so the
|
||
catalogue starts empty (a leftover source would move the top-level
|
||
rows and the stat cards)."""
|
||
with SessionLocal() as db:
|
||
db.execute(
|
||
text(
|
||
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
||
"kb_overview, git_sources, folder_summaries"
|
||
)
|
||
)
|
||
db.commit() # without the commit the TRUNCATE rolls back
|
||
|
||
|
||
def _stamp(day: str) -> datetime:
|
||
"""The fixed UTC stamp for a seeded date (``indexed_at`` rides the
|
||
same day, one hour later, so the totals' last_indexed is stable)."""
|
||
moment = datetime.fromisoformat(day).replace(tzinfo=UTC)
|
||
return moment
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def seeded_kb(
|
||
app_server: str, tmp_path_factory: pytest.TempPathFactory
|
||
) -> Iterator[dict[str, Any]]:
|
||
"""The catalogue above, written straight through the ORM (the tree
|
||
is database-only: no sync, no chunks-of-truth, no LLM). The image
|
||
document's ``image_path`` points at a real PNG inside the app's own
|
||
``BOR_IMAGE_DIR``, and the fixture PROVES the public bytes route
|
||
serves it before any render assertion runs. Yields the identity map
|
||
(the document ids the assertions need), and truncates on teardown."""
|
||
try:
|
||
body = httpx.get(f"{APP_URL}/api/health", timeout=5).json()
|
||
except Exception: # noqa: BLE001 — unreachable is the skip case
|
||
body = {"db": "down"}
|
||
if body.get("db") != "up":
|
||
pytest.skip("Postgres not reachable — run `podman compose up -d db` first")
|
||
|
||
_truncate_kb()
|
||
scratch = tmp_path_factory.mktemp("bor_kb_lazy_png")
|
||
png = scratch / "pic.png"
|
||
png.write_bytes(PNG_1X1)
|
||
|
||
ids: dict[str, str] = {}
|
||
last_indexed = ""
|
||
with SessionLocal() as db:
|
||
for source, path, title, day, chunks in SEED_DOCS:
|
||
created = _stamp(day)
|
||
indexed = created.replace(hour=min(23, created.hour + 1))
|
||
is_image = path == ALPHA_PIC
|
||
doc = Document(
|
||
source=source,
|
||
path=path,
|
||
full_path=f"/fixture/{source}/{path}",
|
||
title=title,
|
||
content=PIC_DESCRIPTION if is_image else f"# {title}\n\nBody of {path}.\n",
|
||
summary=PIC_DESCRIPTION if is_image else None,
|
||
is_image=is_image,
|
||
image_path=str(png) if is_image else None,
|
||
content_hash=f"{abs(hash((source, path))):064x}"[:64],
|
||
created_at=created,
|
||
indexed_at=indexed,
|
||
)
|
||
db.add(doc)
|
||
db.flush()
|
||
ids[path] = str(doc.id)
|
||
for position in range(chunks):
|
||
db.add(
|
||
Chunk(
|
||
document_id=doc.id,
|
||
position=position,
|
||
content=f"{title} chunk {position}",
|
||
)
|
||
)
|
||
if indexed.isoformat() > last_indexed:
|
||
last_indexed = indexed.isoformat()
|
||
db.add_all(
|
||
[
|
||
FolderSummary(
|
||
source=ATLAS, folder_path="", summary=ATLAS_ROOT_SUM, manually_edited=False
|
||
),
|
||
FolderSummary(
|
||
source=ATLAS,
|
||
folder_path="alpha/beta",
|
||
summary=BETA_SUM,
|
||
manually_edited=False,
|
||
),
|
||
FolderSummary(
|
||
source=COBALT, folder_path="", summary=COBALT_SUM, manually_edited=False
|
||
),
|
||
]
|
||
)
|
||
db.commit()
|
||
assert len(ids) == TOTAL_DOCS
|
||
|
||
served = httpx.get(f"{APP_URL}/api/documents/{ids[ALPHA_PIC]}/image", timeout=10)
|
||
assert served.status_code == 200, "the seeded image bytes must serve through the route"
|
||
assert served.headers["content-type"] == "image/png"
|
||
|
||
yield {"ids": ids, "last_indexed": last_indexed, "png": str(png)}
|
||
|
||
with SessionLocal() as db: # the shared Postgres leaves clean for the next suite
|
||
db.execute(
|
||
text(
|
||
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
||
"kb_overview, git_sources, folder_summaries"
|
||
)
|
||
)
|
||
db.commit()
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Page + wire helpers
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def _folder_rows(page: Page) -> Locator:
|
||
"""The catalog's ONE folder table: the source rows at the top
|
||
level, the subfolder rows at every deeper level. The in-flight
|
||
loading/alert line shares that tbody and is NOT a catalog row, so
|
||
every row-count claim excludes it."""
|
||
return page.locator("#folders-tbody tr:not(.kb-level-loading)")
|
||
|
||
|
||
def _file_rows(page: Page) -> Locator:
|
||
return page.locator("#docs-tbody tr")
|
||
|
||
|
||
def _folder_link(page: Page, name: str) -> str:
|
||
return f'#folders-tbody a.folder-link:text-is("{name}")'
|
||
|
||
|
||
def _crumb_link(_page: Page, name: str) -> str:
|
||
"""The breadcrumb step back up to ``name`` ("Knowledge base" for the
|
||
top level) — a crumb click is a client-side navigation."""
|
||
return f'#kb-crumb a.kb-crumb-link:text-is("{name}")'
|
||
|
||
|
||
def _drill(page: Page, *names: str) -> None:
|
||
"""Walk one level per click (a source row, then the folder links).
|
||
Since phase 167 an UNCACHED level click triggers exactly one node
|
||
fetch, so the click is followed by the render it produced."""
|
||
for name in names:
|
||
page.click(_folder_link(page, name))
|
||
expect(page.locator(".kb-level-loading")).to_have_count(0)
|
||
|
||
|
||
def _wait_top_level(page: Page) -> None:
|
||
"""The boot's first paint: every seeded source is a row (the ONE
|
||
depth-1 fetch settled) and no breadcrumb shows."""
|
||
expect(_folder_rows(page)).to_have_count(len(SOURCE_COUNTS), timeout=30_000)
|
||
expect(page.locator("#kb-crumb")).to_be_hidden()
|
||
|
||
|
||
def _paths_of(requests: list[str], path: str) -> list[str]:
|
||
"""The logged request URLs whose PATH is exactly ``path`` — the
|
||
exact-path match is the point: ``/api/docs/tree/node`` must never
|
||
be counted as a tree request, and vice versa."""
|
||
return [url for url in requests if urlparse(url).path == path]
|
||
|
||
|
||
class Wire:
|
||
"""The wire truth behind the catalogue fetches, straight off the CDP
|
||
Network domain.
|
||
|
||
WHY CDP: Chromium's HTTP cache adds ``If-None-Match`` ITSELF and, on
|
||
a 304, hands ``fetch`` the cached 200 body — so Playwright's response
|
||
event reports 200 and its request headers never carry the conditional
|
||
header. The DevTools Network panel shows the 304; ``Network.enable``
|
||
is that same panel, and it is the only honest way to pin "the second
|
||
visit costs a 304 with no body" in a browser test.
|
||
|
||
CDP names the URL on ONE event only (``requestWillBeSent``) and the
|
||
cache layer's own events (the conditional headers, the wire response,
|
||
the transfer size) can land BEFORE it — so every event that arrives
|
||
without a URL stages itself by ``requestId`` and merges into the
|
||
entry the moment its URL is known. Order never matters, and an
|
||
unrelated request's stage is simply never adopted.
|
||
"""
|
||
|
||
_BLANK: dict[str, Any] = {
|
||
"request_id": "",
|
||
"url": "",
|
||
"path": "",
|
||
"query": "",
|
||
"if_none_match": None,
|
||
"status": None,
|
||
"etag": None,
|
||
"content_encoding": None,
|
||
"encoded_bytes": 0,
|
||
}
|
||
|
||
def __init__(self, page: Page) -> None:
|
||
self.entries: dict[str, dict[str, Any]] = {}
|
||
self.order: list[str] = []
|
||
self.stash: dict[str, dict[str, Any]] = {}
|
||
self.cdp = page.context.new_cdp_session(page)
|
||
self.cdp.on("Network.requestWillBeSent", self._on_request)
|
||
self.cdp.on("Network.requestWillBeSentExtraInfo", self._on_request_extra)
|
||
self.cdp.on("Network.responseReceivedExtraInfo", self._on_response_extra)
|
||
self.cdp.on("Network.loadingFinished", self._on_loading_finished)
|
||
self.cdp.send("Network.enable")
|
||
|
||
def _slot(self, request_id: str) -> dict[str, Any]:
|
||
"""The entry for a request, from the adopted table or the stage."""
|
||
return self.entries.get(request_id) or self.stash.setdefault(
|
||
request_id, dict(self._BLANK, request_id=request_id)
|
||
)
|
||
|
||
def _track(self, request_id: str, url: str) -> dict[str, Any]:
|
||
entry = self.entries.get(request_id)
|
||
if entry is None:
|
||
entry = dict(self._slot(request_id), url=url, path=urlparse(url).path,
|
||
query=urlparse(url).query)
|
||
self.stash.pop(request_id, None)
|
||
self.entries[request_id] = entry
|
||
self.order.append(request_id)
|
||
return entry
|
||
|
||
def _on_request(self, params: dict[str, Any]) -> None:
|
||
url = params["request"]["url"]
|
||
if urlparse(url).path in (TREE_PATH, NODE_PATH):
|
||
self._track(params["requestId"], url)
|
||
|
||
def _on_request_extra(self, params: dict[str, Any]) -> None:
|
||
headers = {k.lower(): v for k, v in params.get("headers", {}).items()}
|
||
if not headers.get("if-none-match"):
|
||
return
|
||
self._slot(params["requestId"])["if_none_match"] = headers["if-none-match"]
|
||
|
||
def _on_response_extra(self, params: dict[str, Any]) -> None:
|
||
headers = {k.lower(): v for k, v in params.get("headers", {}).items()}
|
||
entry = self._slot(params["requestId"])
|
||
entry["status"] = params["statusCode"]
|
||
entry["etag"] = headers.get("etag")
|
||
entry["content_encoding"] = headers.get("content-encoding")
|
||
|
||
def _on_loading_finished(self, params: dict[str, Any]) -> None:
|
||
"""The bytes the network MOVED for a tracked request — a 304
|
||
costs its headers only, the payload stays in the cache."""
|
||
entry = self.entries.get(params["requestId"])
|
||
if entry is not None:
|
||
entry["encoded_bytes"] += int(params.get("encodedDataLength") or 0)
|
||
|
||
def of(self, path: str) -> list[dict[str, Any]]:
|
||
return [self.entries[rid] for rid in self.order if self.entries[rid]["path"] == path]
|
||
|
||
def settled(self, path: str) -> list[dict[str, Any]]:
|
||
"""Only the entries whose wire response has ARRIVED — the
|
||
assertion never races the network."""
|
||
return [e for e in self.of(path) if e["status"] is not None]
|
||
|
||
|
||
def _admin_client() -> httpx.Client:
|
||
"""A cookie-authenticated admin client on the module app (the
|
||
``test_kb_tree.py`` pattern) — the API half of the never-stale
|
||
proof (an edit that changes the tree's bytes)."""
|
||
client = httpx.Client(base_url=APP_URL, timeout=30.0)
|
||
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||
assert r.status_code == 204, r.text
|
||
return client
|
||
|
||
|
||
def _body_of(responses: list[Response], path: str) -> dict[str, Any]:
|
||
"""The parsed JSON body of the ONE response whose path is ``path`` —
|
||
asserted on the PARSED body, never on bytes."""
|
||
matches = [r for r in responses if urlparse(r.url).path == path]
|
||
assert len(matches) == 1, [r.url for r in responses]
|
||
return matches[0].json() # type: ignore[Any]
|
||
|
||
|
||
def _walk(nodes: list[dict[str, Any]]) -> Iterator[dict[str, Any]]:
|
||
for node in nodes:
|
||
yield node
|
||
yield from _walk(node.get("children") or [])
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 1. The first load is small: one depth-1 request, totals, truncated names
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_first_boot_is_one_small_tree_request(
|
||
page: Page, seeded_kb: dict[str, Any], db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
responses: list[Response] = []
|
||
requests: list[str] = []
|
||
page.on("request", lambda r: requests.append(r.url))
|
||
page.on("response", lambda r: responses.append(r))
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
|
||
# THE TODO L5 REPRO, pinned: exactly ONE catalogue request, and it is
|
||
# the DEPTH-LIMITED one. No per-level node fetch happens unprompted.
|
||
tree_requests = _paths_of(requests, TREE_PATH)
|
||
assert len(tree_requests) == 1, tree_requests
|
||
assert _paths_of(requests, NODE_PATH) == []
|
||
assert parse_qs(urlparse(tree_requests[0]).query) == {"depth": ["1"]}, tree_requests[0]
|
||
|
||
body = _body_of(responses, TREE_PATH)
|
||
|
||
# The server-computed totals: the stat cards' only source of truth,
|
||
# because a depth-limited tree cannot be summed client-side.
|
||
totals = body["totals"]
|
||
assert totals["documents"] == TOTAL_DOCS, totals
|
||
assert totals["chunks"] == TOTAL_CHUNKS, totals
|
||
assert totals["last_indexed"], totals
|
||
|
||
# The top level is the sources, with RECURSIVE counts — atlas counts
|
||
# 5 documents while listing exactly two of its own children.
|
||
sources = {node["name"]: node for node in body["sources"]}
|
||
assert set(sources) == {ATLAS, BEACON, COBALT}
|
||
for name, node in sources.items():
|
||
assert node["documents"] == SOURCE_COUNTS[name][0], (name, node)
|
||
assert node["updated_at"].startswith(SOURCE_COUNTS[name][1]), (name, node)
|
||
|
||
# NOTHING deeper than level 1 is in the body: every node the payload
|
||
# carries is a source, a level-1 folder, or a level-1 file — and
|
||
# every level-1 folder says so instead of silently rendering short.
|
||
levels = [node["path"].count("/") + 1 for node in _walk(sources[ATLAS].get("children", []))]
|
||
assert levels, "atlas must list its level-1 children"
|
||
assert max(levels) == 1, levels
|
||
for node in _walk(list(sources.values())):
|
||
if node.get("kind") == "folder":
|
||
assert node.get("truncated") is True, node
|
||
assert node.get("children") == [], node
|
||
# D9 under truncation: the recursive max date survives the
|
||
# depth budget — `alpha`'s newest file is TWO levels down.
|
||
assert node["updated_at"], node
|
||
alpha = next(c for c in sources[ATLAS]["children"] if c["path"] == "alpha")
|
||
assert alpha["documents"] == 4, alpha
|
||
assert alpha["updated_at"].startswith("2024-06-06"), alpha
|
||
assert sources[ATLAS]["updated_at"].startswith("2024-07-01")
|
||
|
||
# The stat cards ride `totals` (the pre-phase walk of the partial
|
||
# tree would have read 2 documents for the whole catalogue).
|
||
expect(page.locator("#stat-docs")).to_have_text(str(TOTAL_DOCS))
|
||
expect(page.locator("#stat-chunks")).to_have_text(str(TOTAL_CHUNKS))
|
||
expect(page.locator("#stat-last")).not_to_have_text("–")
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 2. gzip on the wire (the browser's own negotiation)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_tree_response_arrives_gzipped(
|
||
page: Page, seeded_kb: dict[str, Any], db_ready: None
|
||
) -> None:
|
||
"""Chromium advertises ``Accept-Encoding: gzip, deflate, br`` on its
|
||
own, so this is the real negotiation — no stub, no spoofed header.
|
||
|
||
The threshold check is part of the assertion: the body must clear
|
||
the app's ``BOR_COMPRESS_MIN_BYTES`` (the code default, pinned in
|
||
the fixture env) so a payload that starved the compressor could
|
||
never turn this test green.
|
||
"""
|
||
page.set_default_timeout(30_000)
|
||
|
||
responses: list[Response] = []
|
||
page.on("response", lambda r: responses.append(r))
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
|
||
body = _body_of(responses, TREE_PATH)
|
||
threshold = _Settings.model_fields["compress_min_bytes"].default
|
||
uncompressed = len(json.dumps(body))
|
||
assert uncompressed > threshold, (uncompressed, threshold)
|
||
|
||
headers = next(r for r in responses if urlparse(r.url).path == TREE_PATH).headers
|
||
assert headers.get("content-encoding") == "gzip", headers
|
||
assert "accept-encoding" in (headers.get("vary") or "").lower(), headers
|
||
# The revalidation pair rides the same response (task 03), and the
|
||
# A20 security headers survive outside the compressor (A20).
|
||
assert (headers.get("etag") or "").startswith('W/"'), headers
|
||
assert headers.get("cache-control") == "private, must-revalidate", headers
|
||
assert headers.get("content-security-policy"), headers
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 3. The drill: one node request for an uncached level, rendered in full
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_drill_into_a_folder_fetches_that_level_once(
|
||
page: Page, seeded_kb: dict[str, Any], db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
ids = seeded_kb["ids"]
|
||
|
||
requests: list[str] = []
|
||
page.on("request", lambda r: requests.append(r.url))
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
|
||
# The top level renders the stored source-root descriptions and the
|
||
# recursive counts the payload carries.
|
||
rows = _folder_rows(page)
|
||
expect(rows.nth(0).locator("a.folder-link")).to_have_text(ATLAS)
|
||
expect(rows.nth(0).locator("td:nth-child(2)")).to_have_text("5")
|
||
expect(rows.nth(0).locator("td:nth-child(4) span.kb-desc-text")).to_have_text(ATLAS_ROOT_SUM)
|
||
|
||
# A SOURCE row costs nothing: the depth-1 payload already holds that
|
||
# level (the phase-97 zero-request drill, kept where it is free).
|
||
page.click(_folder_link(page, ATLAS))
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text(ATLAS)
|
||
assert _paths_of(requests, NODE_PATH) == []
|
||
expect(_folder_rows(page)).to_have_count(1) # alpha only
|
||
expect(_folder_rows(page).nth(0).locator("td:nth-child(2)")).to_have_text("4")
|
||
# The D9 recursive max on a node the payload truncated: the newest
|
||
# file inside alpha lives two levels below it.
|
||
updated = _folder_rows(page).nth(0).locator("td:nth-child(3)")
|
||
assert (updated.get_attribute("title") or "").startswith("2024-06-06")
|
||
|
||
# THE DRILL: opening a folder the payload only NAMES costs EXACTLY
|
||
# one node request for that level.
|
||
page.click(_folder_link(page, "alpha"))
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("alpha")
|
||
node_requests = _paths_of(requests, NODE_PATH)
|
||
assert len(node_requests) == 1, node_requests
|
||
query = parse_qs(urlparse(node_requests[0]).query)
|
||
assert query.get("source") == [ATLAS], node_requests[0]
|
||
assert query.get("path") == ["alpha"], node_requests[0]
|
||
|
||
# What the level renders is what the pre-phase full tree rendered.
|
||
# Folders first (beta, still truncated), then the level's own files
|
||
# in catalogue order.
|
||
folders = _folder_rows(page)
|
||
expect(folders).to_have_count(1)
|
||
expect(folders.nth(0).locator("a.folder-link")).to_have_text("beta")
|
||
expect(folders.nth(0).locator("td:nth-child(2)")).to_have_text("2") # recursive
|
||
expect(folders.nth(0).locator("td:nth-child(4) span.kb-desc-text")).to_have_text(BETA_SUM)
|
||
|
||
files = _file_rows(page)
|
||
expect(files).to_have_count(2)
|
||
expect(files.nth(0).locator("td:nth-child(1)")).to_have_text(ATLAS)
|
||
expect(files.nth(0).locator("a.doc-link")).to_have_text(ALPHA_ONE)
|
||
expect(files.nth(0).locator("td:nth-child(4)")).to_have_text("1")
|
||
assert (files.nth(0).locator("td:nth-child(5)").get_attribute("title") or "").startswith(
|
||
"2024-02-11"
|
||
)
|
||
expect(files.nth(1).locator("a.doc-link")).to_have_text(ALPHA_PIC)
|
||
assert (files.nth(1).locator("td:nth-child(5)").get_attribute("title") or "").startswith(
|
||
"2024-02-20"
|
||
)
|
||
|
||
# The image affordance survives the node route: the bounded image
|
||
# map rides the drill, so the thumbnail is the SAME route the full
|
||
# tree pointed at, and it carries the vision description as alt.
|
||
thumb = files.nth(1).locator("img.kb-doc-thumb-img")
|
||
expect(thumb).to_have_count(1)
|
||
assert thumb.get_attribute("src") == f"/api/documents/{ids[ALPHA_PIC]}/image"
|
||
assert thumb.get_attribute("alt") == PIC_DESCRIPTION
|
||
|
||
# Phase 156 (UI Structure Check): the bounded scroll panel and its
|
||
# sticky header survive a lazy render — the drill paints into the
|
||
# same card, it does not rebuild the page.
|
||
panel = page.locator("#folders-wrap")
|
||
wrap_style = panel.evaluate(
|
||
"el => { const s = getComputedStyle(el);"
|
||
"return { maxHeight: s.maxHeight, overflowY: s.overflowY, "
|
||
"borderRadius: s.borderTopLeftRadius, boxShadow: s.boxShadow }; }"
|
||
)
|
||
assert wrap_style["maxHeight"] not in ("none", ""), wrap_style
|
||
assert wrap_style["overflowY"] in ("auto", "scroll"), wrap_style
|
||
assert wrap_style["boxShadow"] != "none", wrap_style
|
||
head = page.locator("#folders-table thead th").first
|
||
head_style = head.evaluate(
|
||
"el => { const s = getComputedStyle(el); return { p: s.position, t: s.top }; }"
|
||
)
|
||
assert head_style["p"] == "sticky", head_style
|
||
assert head_style["t"] == "0px", head_style
|
||
|
||
# …and the lazily rendered level still rides the ONE 72rem centered
|
||
# reading column (AGENTS.md rule 5): a node-fetched level never
|
||
# grows the page or escapes the container.
|
||
geom = page.evaluate(
|
||
"""() => {
|
||
const shell = document.querySelector(".container.sources-shell");
|
||
const panel = document.querySelector("#folders-wrap");
|
||
const s = shell.getBoundingClientRect();
|
||
const p = panel.getBoundingClientRect();
|
||
return { shell: s.width, left: p.left - s.left, right: s.right - p.right };
|
||
}"""
|
||
)
|
||
assert geom["shell"] <= 1152 + 4, geom # 72rem at this 1280px viewport
|
||
assert geom["left"] >= -1 and geom["right"] >= -1, geom
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 4. Two levels deep, then back: the cache answers, the server does not
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_two_levels_deep_then_back_uses_the_cache(
|
||
page: Page, seeded_kb: dict[str, Any], db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
requests: list[str] = []
|
||
page.on("request", lambda r: requests.append(r.url))
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
_drill(page, ATLAS)
|
||
|
||
# Level 2 (`alpha`): fetch 1.
|
||
page.click(_folder_link(page, "alpha"))
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("alpha")
|
||
node_requests = _paths_of(requests, NODE_PATH)
|
||
assert len(node_requests) == 1, node_requests
|
||
assert parse_qs(urlparse(node_requests[0]).query).get("path") == ["alpha"], node_requests[0]
|
||
|
||
# Level 3 (`alpha/beta`): fetch 2 — and with it the description that
|
||
# NO depth-1 payload can carry (the level block is the unclamped
|
||
# surface, and this row lives below the budget).
|
||
page.click(_folder_link(page, "beta"))
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("beta")
|
||
assert len(_paths_of(requests, NODE_PATH)) == 2, _paths_of(requests, NODE_PATH)
|
||
assert parse_qs(urlparse(_paths_of(requests, NODE_PATH)[1]).query).get("path") == [
|
||
"alpha/beta"
|
||
], _paths_of(requests, NODE_PATH)
|
||
expect(_folder_rows(page)).to_have_count(1) # gamma
|
||
expect(_folder_rows(page).nth(0).locator("a.folder-link")).to_have_text("gamma")
|
||
expect(page.locator("#kb-level")).to_be_visible()
|
||
expect(page.locator("#kb-level-title")).to_have_text(f"{ATLAS}/alpha/beta")
|
||
expect(page.locator("#kb-level-summary")).to_have_text(BETA_SUM)
|
||
expect(_file_rows(page).nth(0).locator("a.doc-link")).to_have_text(BETA_TWO)
|
||
|
||
# The stat cards never move: they are catalogue-wide `totals`, not a
|
||
# walk of the level on screen.
|
||
expect(page.locator("#stat-docs")).to_have_text(str(TOTAL_DOCS))
|
||
expect(page.locator("#stat-chunks")).to_have_text(str(TOTAL_CHUNKS))
|
||
|
||
# BACK to `alpha` (a crumb): zero new requests, the rows come from
|
||
# what the client holds.
|
||
page.click(_crumb_link(page, "alpha"))
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("alpha")
|
||
assert len(_paths_of(requests, NODE_PATH)) == 2, _paths_of(requests, NODE_PATH)
|
||
expect(_folder_rows(page)).to_have_count(1)
|
||
expect(_folder_rows(page).nth(0).locator("a.folder-link")).to_have_text("beta")
|
||
expect(_file_rows(page)).to_have_count(2)
|
||
|
||
# BACK to the top (a crumb): still nothing on the wire.
|
||
page.click(_crumb_link(page, "Knowledge base"))
|
||
expect(page.locator("#kb-crumb")).to_be_hidden()
|
||
assert len(_paths_of(requests, NODE_PATH)) == 2, _paths_of(requests, NODE_PATH)
|
||
expect(_folder_rows(page)).to_have_count(len(SOURCE_COUNTS))
|
||
|
||
# RE-ENTERING levels the client already holds is free: the two
|
||
# fetches above are the whole session's tally, level by level.
|
||
_drill(page, ATLAS, "alpha", "beta")
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("beta")
|
||
assert len(_paths_of(requests, NODE_PATH)) == 2, _paths_of(requests, NODE_PATH)
|
||
# …while the level the cache has NEVER seen costs exactly its own
|
||
# one request (the deepest folder of the deepest chain).
|
||
page.click(_folder_link(page, "gamma"))
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("gamma")
|
||
assert len(_paths_of(requests, NODE_PATH)) == 3, _paths_of(requests, NODE_PATH)
|
||
assert parse_qs(urlparse(_paths_of(requests, NODE_PATH)[2]).query).get("path") == [
|
||
"alpha/beta/gamma"
|
||
], _paths_of(requests, NODE_PATH)
|
||
expect(_file_rows(page).nth(0).locator("a.doc-link")).to_have_text(GAMMA_THREE)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 5. The pending marker renders in a lazily loaded level
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_pending_folder_marker_survives_the_lazy_level(
|
||
page: Page, seeded_kb: dict[str, Any], db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
requests: list[str] = []
|
||
page.on("request", lambda r: requests.append(r.url))
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
|
||
# `beacon` has no stored description anywhere → the source row and
|
||
# its folder row both carry the phase-98 D4 marker (text + muted
|
||
# color + tooltip, never color alone).
|
||
beacon_row = _folder_rows(page).filter(has_text=BEACON)
|
||
marker = beacon_row.locator("span.kb-desc-text")
|
||
expect(marker).to_have_text("Summary pending")
|
||
assert "kb-summary-pending" in (marker.get_attribute("class") or "")
|
||
assert marker.get_attribute("title") == PENDING_TITLE
|
||
|
||
page.click(_folder_link(page, BEACON))
|
||
expect(_folder_rows(page)).to_have_count(1) # notes — no fetch (level held)
|
||
assert _paths_of(requests, NODE_PATH) == []
|
||
|
||
# The lazy level carries it too: the node route answers
|
||
# `summary_pending` exactly like the full tree did.
|
||
notes_marker = _folder_rows(page).nth(0).locator("span.kb-desc-text")
|
||
expect(notes_marker).to_have_text("Summary pending")
|
||
assert "kb-summary-pending" in (notes_marker.get_attribute("class") or "")
|
||
assert notes_marker.get_attribute("title") == PENDING_TITLE
|
||
page.click(_folder_link(page, "notes"))
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("notes")
|
||
assert len(_paths_of(requests, NODE_PATH)) == 1, _paths_of(requests, NODE_PATH)
|
||
# A PENDING level's block shows the D4 note instead of an empty box.
|
||
expect(page.locator("#kb-level")).to_be_visible()
|
||
expect(page.locator("#kb-level-summary")).to_have_text(
|
||
"No description stored yet — the next sync will generate one. "
|
||
"(You can write one yourself.)"
|
||
)
|
||
assert "kb-summary-pending" in (page.locator("#kb-level-summary").get_attribute("class") or "")
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 6. The second visit: If-None-Match on the way out, 304 on the wire
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def _conditional_tree_load(page: Page, wire: Wire, act: Any) -> dict[str, Any]:
|
||
"""Run ``act`` (a reload, a view switch), then return the tree
|
||
entry the wire settled, asserting the two halves of the claim:
|
||
the browser SENT the validator, the server answered 304."""
|
||
act()
|
||
_wait_top_level(page)
|
||
deadline = time.monotonic() + 10.0
|
||
settled = wire.settled(TREE_PATH)
|
||
while len(settled) < 2 and time.monotonic() < deadline:
|
||
page.wait_for_timeout(100)
|
||
settled = wire.settled(TREE_PATH)
|
||
assert len(settled) >= 2, settled
|
||
return settled[-1]
|
||
|
||
|
||
def test_reload_revalidates_to_304(page: Page, seeded_kb: dict[str, Any], db_ready: None) -> None:
|
||
page.set_default_timeout(30_000)
|
||
wire = Wire(page)
|
||
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
|
||
first = wire.settled(TREE_PATH)
|
||
assert len(first) == 1, first
|
||
assert first[0]["status"] == 200, first[0]
|
||
assert first[0]["if_none_match"] is None, first[0]
|
||
etag = first[0]["etag"]
|
||
assert etag and etag.startswith('W/"'), first[0]
|
||
assert first[0]["content_encoding"] == "gzip", first[0]
|
||
full_transfer = first[0]["encoded_bytes"]
|
||
assert full_transfer > 500, first[0]
|
||
|
||
second = _conditional_tree_load(page, wire, lambda: page.reload())
|
||
|
||
# The browser re-sent the validator, and the server answered 304 —
|
||
# the second visit costs headers, not the payload.
|
||
assert second["if_none_match"] == etag, second
|
||
assert second["status"] == 304, second
|
||
assert second["encoded_bytes"] < full_transfer / 2, (second, full_transfer)
|
||
|
||
# And the table still renders: the cached body is what the owner
|
||
# sees, row for row.
|
||
expect(_folder_rows(page)).to_have_count(len(SOURCE_COUNTS))
|
||
expect(_folder_rows(page).nth(0).locator("a.folder-link")).to_have_text(ATLAS)
|
||
expect(page.locator("#stat-docs")).to_have_text(str(TOTAL_DOCS))
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 7. The same cache hit through the SPA (History → Sources)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_view_switch_revalidates_too(
|
||
page: Page, seeded_kb: dict[str, Any], db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
wire = Wire(page)
|
||
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
assert len(wire.settled(TREE_PATH)) == 1
|
||
|
||
# The router's mount-once view re-fires its load on a re-show (phase
|
||
# 77) — phase 167 makes the SECOND of those cheap: same URL, same
|
||
# validator, a 304 back.
|
||
def switch() -> None:
|
||
page.click("#nav-history")
|
||
expect(page).to_have_url(APP_URL + "/history.html")
|
||
page.click("#nav-sources")
|
||
expect(page).to_have_url(APP_URL + "/sources.html")
|
||
|
||
second = _conditional_tree_load(page, wire, switch)
|
||
assert second["if_none_match"], second
|
||
assert second["status"] == 304, second
|
||
expect(_folder_rows(page)).to_have_count(len(SOURCE_COUNTS))
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 8. Fresh content still lands (the never-stale half of the contract)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_an_edit_busts_the_tag_so_fresh_content_lands(
|
||
page: Page, seeded_kb: dict[str, Any], db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
wire = Wire(page)
|
||
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
first = wire.settled(TREE_PATH)[0]
|
||
assert first["status"] == 200, first
|
||
|
||
# An admin saves a description on a level the payload truncated. The
|
||
# tag is a hash of the BODY (LOCKED A4), so this — not a generation
|
||
# bump — is what invalidates the client's copy.
|
||
client = _admin_client()
|
||
try:
|
||
r = client.patch(
|
||
"/api/folders/summary",
|
||
json={"source": ATLAS, "folder_path": "alpha", "summary": ALPHA_SUMMARY_EDIT},
|
||
)
|
||
assert r.status_code == 200, r.text
|
||
finally:
|
||
client.close()
|
||
|
||
def reload() -> None:
|
||
page.reload()
|
||
|
||
second = _conditional_tree_load(page, wire, reload)
|
||
assert second["if_none_match"], second
|
||
assert second["status"] == 200, second
|
||
assert second["etag"] != first["etag"], (first, second)
|
||
|
||
# The new text renders where the owner looks: on the row, at the
|
||
# level the description belongs to.
|
||
_drill(page, ATLAS)
|
||
row = _folder_rows(page).filter(has_text="alpha")
|
||
expect(row.locator("span.kb-desc-text")).to_have_text(ALPHA_SUMMARY_EDIT)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 9. A folder that vanished under the open view (§7.4)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_vanished_folder_resets_to_the_top_level(
|
||
page: Page, seeded_kb: dict[str, Any], db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
_drill(page, ATLAS)
|
||
|
||
# The node route's 404 IS the vanished-folder signal (task 03/04):
|
||
# the folder was pruned between the listing and the click.
|
||
page.route(
|
||
"**/api/docs/tree/node*",
|
||
lambda route: route.fulfill(
|
||
status=404, content_type="application/json", body=json.dumps({"detail": "unknown node"})
|
||
),
|
||
)
|
||
page.click(_folder_link(page, "alpha"))
|
||
|
||
# The honest reset: no stale level, no stale breadcrumb, no row
|
||
# left spinning — the top level renders, because that is all the
|
||
# client can prove.
|
||
expect(page.locator("#kb-crumb")).to_be_hidden()
|
||
expect(_folder_rows(page)).to_have_count(len(SOURCE_COUNTS))
|
||
expect(_folder_rows(page).nth(0).locator("a.folder-link")).to_have_text(ATLAS)
|
||
expect(page.locator(".kb-level-loading")).to_have_count(0)
|
||
expect(page.locator("#stat-docs")).to_have_text(str(TOTAL_DOCS))
|
||
page.unroute("**/api/docs/tree/node*")
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 10. A failed drill is honest, keeps the level, and recovers
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_failed_drill_keeps_the_level_and_retry_recovers(
|
||
page: Page, seeded_kb: dict[str, Any], db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
_drill(page, ATLAS)
|
||
|
||
# The level the user is ON before the click — it must survive.
|
||
expect(_folder_rows(page).nth(0).locator("a.folder-link")).to_have_text("alpha")
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text(ATLAS)
|
||
rows_before = _folder_rows(page).count()
|
||
|
||
page.route(
|
||
"**/api/docs/tree/node*",
|
||
lambda route: route.fulfill(
|
||
status=500, content_type="text/plain", body="embedding model is down"
|
||
),
|
||
)
|
||
page.click(_folder_link(page, "alpha"))
|
||
|
||
# The alert names the folder and the Retry is a REAL button with an
|
||
# accessible name (phase 49's .retry-btn pill).
|
||
alert = page.locator(".kb-level-loading [role='alert']")
|
||
expect(alert).to_have_count(1)
|
||
expect(alert).to_have_text("Couldn't load alpha — ")
|
||
retry = page.locator(".kb-level-loading button.retry-btn")
|
||
expect(retry).to_have_count(1)
|
||
expect(retry).to_have_text("Retry")
|
||
expect(retry).to_be_visible()
|
||
# The UI Structure Check (AGENTS.md rule 5): the recovery control is
|
||
# a >=44px target and it is keyboard-reachable — tabbing forward
|
||
# from the row's own Edit button lands ON the Retry, with the global
|
||
# :focus-visible ring applied (the same rule every control uses).
|
||
assert retry.evaluate("el => el.getBoundingClientRect().height") >= 44
|
||
page.locator("#folders-tbody .kb-summary-edit").first.evaluate("el => el.focus()")
|
||
page.keyboard.press("Tab")
|
||
focused = page.evaluate(
|
||
"() => { const el = document.activeElement;"
|
||
"return { cls: el.className, text: el.textContent, "
|
||
"visible: el.matches(':focus-visible'), outline: getComputedStyle(el).outlineWidth }; }"
|
||
)
|
||
assert "retry-btn" in focused["cls"], focused
|
||
assert focused["text"] == "Retry", focused
|
||
assert focused["visible"] is True, focused
|
||
assert focused["outline"] not in ("", "0px"), focused
|
||
|
||
# Nothing else moved: the level, the breadcrumb and the stat cards
|
||
# are exactly what they were (§7.4 — a failure never paints a hole).
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text(ATLAS)
|
||
expect(_folder_rows(page)).to_have_count(rows_before)
|
||
expect(_folder_rows(page).nth(0).locator("a.folder-link")).to_have_text("alpha")
|
||
|
||
# Retry, with the fault lifted, finishes the drill the click started.
|
||
page.unroute("**/api/docs/tree/node*")
|
||
retry.click()
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("alpha")
|
||
expect(page.locator(".kb-level-loading")).to_have_count(0)
|
||
expect(_folder_rows(page).nth(0).locator("a.folder-link")).to_have_text("beta")
|
||
expect(_file_rows(page)).to_have_count(2)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 11. The in-flight drill state, and the second click that costs nothing
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_in_flight_drill_marks_the_link_and_ignores_a_second_click(
|
||
page: Page, seeded_kb: dict[str, Any], db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
requests: list[str] = []
|
||
page.on("request", lambda r: requests.append(r.url))
|
||
login(page, APP_URL)
|
||
_wait_top_level(page)
|
||
_drill(page, ATLAS)
|
||
|
||
# Hold the node request in the harness (the house hold-the-request
|
||
# pattern) so the in-flight state is observed, never raced.
|
||
held: list[Any] = []
|
||
|
||
def hold(route: Any) -> None:
|
||
held.append(route)
|
||
|
||
page.route("**/api/docs/tree/node*", hold)
|
||
link = page.locator(_folder_link(page, "alpha"))
|
||
link.click()
|
||
|
||
# The clicked control says "busy" and refuses a parallel fetch…
|
||
expect(link).to_have_attribute("aria-busy", "true")
|
||
expect(link).to_have_attribute("aria-disabled", "true")
|
||
# …the ONE live line says what is loading…
|
||
status_line = page.locator(".kb-level-loading [role='status']")
|
||
expect(status_line).to_have_count(1)
|
||
expect(status_line).to_have_text("Loading alpha…")
|
||
# …and the level the user is LEAVING stays on screen (never stale).
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text(ATLAS)
|
||
expect(_folder_rows(page).nth(0).locator("a.folder-link")).to_have_text("alpha")
|
||
|
||
# A second click cannot start a second fetch.
|
||
page.click(_folder_link(page, "alpha"), force=True)
|
||
page.click(_folder_link(page, "alpha"), force=True)
|
||
page.wait_for_timeout(300)
|
||
assert len(_paths_of(requests, NODE_PATH)) == 1, _paths_of(requests, NODE_PATH)
|
||
assert len(held) == 1, held
|
||
|
||
# Release: the held request answers, the flags clear, the level
|
||
# paints. Every settle path owns its own cleanup (§7.4).
|
||
held[0].fallback()
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("alpha")
|
||
# The level re-rendered, so the clicked link is gone with it — the
|
||
# settle's cleanup is pinned as "no control anywhere still claims to
|
||
# be busy" (a flag surviving the paint would strand the view).
|
||
expect(page.locator("[aria-busy='true']")).to_have_count(0)
|
||
expect(page.locator("#folders-tbody [aria-disabled='true']")).to_have_count(0)
|
||
expect(page.locator(".kb-level-loading")).to_have_count(0)
|
||
expect(_folder_rows(page).nth(0).locator("a.folder-link")).to_have_text("beta")
|
||
page.unroute("**/api/docs/tree/node*")
|