Everything is verified green. Final report: **Phase 106 — Document dates (final verification pass; all 10 tasks already complete)** - Verified all phase artifacts: alembic `0020` (dev DB at `0020`), `doc_dates.py`, git `file_commit_dates`, importer `doc_dates_by_root`/`dates_updated`, both entry-point wirings, date APIs + tree `created_at`/`updated_at`, LLM surfaces (prompt block, `read` line 2, appended `ls` field), `apply_recency_boost` in `retrieve()`, UI columns/badge, admin editor, mock-LLM regex — all present and correct; no defects found, no fixes needed. - `uv run pytest --cov=app --cov-report=term-missing` → **2299 passed, TOTAL 99%** (>90% ✓) - `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` → **6/6 passed** in isolation (DB up) - 12 regression E2E suites (retrieval_quality, whole_document_context, agent_document_tools, ls_tree_drilldown, read_truncation_cap, kb_tree, kb_tree_nav, document_viewer, edit_summaries, import_documents, sync_button, hidden_folders_toggle, smoke) → **all green in isolation** - `uv run ruff check .` → clean; `uv run pyright` → **0 errors, 0 warnings** **Completion criteria:** 1) non-null `created_at` + 0020 upgrade/downgrade on dev DB ✓ (real-Alembic integration tests) 2) sync refresh/older/manual-persists/content-reset/no sources_meta bump ✓ 3) zip/tar mtime + future→today ✓ 4) LLM date surfaces + cross-check ✓ 5) UI Created/Updated/badge positions ✓ 6) admin editor set+revert round-trip ✓ 7) old-correct-beats-new-similar (defaults & boost-off) + near-tie + `BOR_RECENCY_BOOST=0` byte-identical ✓ 8) full gate ✓ 9) commit/phase-move — left to harness per instructions. - **Notable:** recency default tuned 0.001 → **0.0007** (task 07 step 5 explicitly permits; measured margins recorded in `test_recency_boost.py` docstring). - **Next pending phase:** none — `todo/` holds only this phase.
944 lines
39 KiB
Python
944 lines
39 KiB
Python
"""Phase 97 task 06 E2E (Playwright, mock-only): the RAG view's drill-down
|
||
catalog tree + the editable folder descriptions.
|
||
|
||
The dedicated story suite for ``97_kb_tree_catalog`` (owner request,
|
||
2026-09-11): the Knowledge base view lists the KB the way the agent's
|
||
``ls`` sees it (the phase-94 concept, one end to end) — sources at the
|
||
top, then per level the subfolders with their stored descriptions and
|
||
the level's files — and the owner edits (or clears) any directory's
|
||
description with the phase-57 inline affordance. Everything is pinned
|
||
against the real app + the deterministic mock.
|
||
|
||
Run in isolation (DB must be up: ``podman compose up -d db``):
|
||
|
||
uv run pytest tests/e2e/test_kb_tree.py -v --no-cov
|
||
|
||
KB fixture — a host temp-dir tree (``tmp_path_factory``; the app runs on
|
||
the same host) with TWO local sources registered through the
|
||
authenticated API, then the real in-process ``POST /api/sync`` pipeline
|
||
(the ``test_local_directory_sources.py`` / phase-94 registration +
|
||
real-Sync pattern; no git anywhere):
|
||
|
||
* ``alpha/`` — ``root-note.md`` at the source root, ``one/`` (2 docs),
|
||
``two/`` (2 docs);
|
||
* ``beta/`` — ``gamma/`` (2 docs).
|
||
|
||
Total: 7 documents; alpha counts 5 (root + 2 + 2), beta counts 2. Every
|
||
stored description is deterministic: the mock's EXISTING
|
||
``FOLDER_SUMMARY_MODE`` branch (phase 94 — no mock changes needed)
|
||
stores, per ≥ 2-doc folder, the canned one-liner naming the folder,
|
||
``Fixture folder summary for <source>[/<folder>].`` — the ``synced_kb``
|
||
module fixture pins those exact rows (all with
|
||
``manually_edited = false``) after the sync, and the tests assert on
|
||
that exact text.
|
||
|
||
MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the real
|
||
``lite`` does whatever it does with the folder prompts, while this
|
||
story's assertions key on the CANNED summaries (the phase-94
|
||
convention): the stored rows are a pure function of the request, so the
|
||
drill, the edit/clear, and the keep-through-sync observables are
|
||
byte-exact only against the mock.
|
||
|
||
The view is admin-only (phase 16): the admin flows drive the REAL form
|
||
login (``e2e.auth_helpers.login`` → ``/sources.html``); the anonymous
|
||
test uses a fresh context and asserts the sign-in gate + zero catalog
|
||
fetches.
|
||
|
||
Test → observable mapping (Playwright Mapping Rule):
|
||
1. ``test_top_level_lists_sources_with_descriptions`` — the source rows:
|
||
``alpha`` (5) + ``beta`` (2) in registry order with the canned
|
||
source-root descriptions (the top-level rows ARE the sources — the
|
||
``ls()`` equivalence); the top-level file table is HIDDEN (files are
|
||
per-source); the stat cards read 7 documents + the computed chunks
|
||
total (the tree walk = the former flat walk).
|
||
2. ``test_drill_into_source`` — click the ``alpha`` row → breadcrumb
|
||
``alpha`` (the ``aria-current`` segment); the level block shows
|
||
alpha's root description (the canned text, title = the source name);
|
||
the folder rows ``one`` / ``two`` each with count 2 + their canned
|
||
summaries; the file row ``root-note.md`` (Source column, title,
|
||
chunks column) in the level's file table.
|
||
3. ``test_drill_into_folder`` — drill to ``alpha`` → ``two`` →
|
||
breadcrumb ``alpha`` → ``two``; the level block shows ``two``'s
|
||
description (title = the full source-relative path); the file rows
|
||
``two-a`` / ``two-b`` (titles, the Source column ``alpha``);
|
||
``#folders-wrap`` hidden (no subfolders); the breadcrumb link on
|
||
``alpha`` goes back up to the source level.
|
||
4. ``test_edit_folder_description`` — on the ``alpha`` level, Edit on
|
||
the ``one/`` ROW (pinned: the row surface — the level-block surface
|
||
is test 6's) → the textarea prefilled with the canned text → set the
|
||
new text → Save → the new text renders in the row + status
|
||
"Description updated."; a SQL assert (the ``SessionLocal`` house
|
||
pattern) on ``folder_summaries``: the row's ``summary`` is the new
|
||
text AND ``manually_edited`` is true.
|
||
5. ``test_clear_folder_description`` — an empty save → the text is gone
|
||
(the row cell emptied — the always-present Edit button stays) +
|
||
status "Description cleared."; the SQL assert: no row for the folder
|
||
(the next KB-changing sync regenerates an AI description — the
|
||
reset path).
|
||
6. ``test_manual_description_survives_a_changed_sync`` — the
|
||
LEVEL-BLOCK edit on the ``beta`` source root (the static
|
||
``#kb-level-edit`` button, ``folder_path ""``); a new file is added
|
||
to the fixture dir; the real in-process sync (the mock regenerates
|
||
the OTHER folders' summaries) → the catalog re-fetch renders the
|
||
edited description UNCHANGED (the SQL row keeps the manual text +
|
||
the flag — the ``kept_manual`` path, E2E-pinned) while the untouched
|
||
folders show the canned regenerated text (flag back to false) and
|
||
the new document lands in the tree (the ``one`` folder counts 3).
|
||
7. ``test_reload_falls_back_to_top_level`` — drill to ``alpha/two``;
|
||
delete ``two/``'s documents + chunks directly (the house DB
|
||
pattern); re-show the RAG view (the nav link re-click — the
|
||
``bor:view-refresh`` trigger) → the breadcrumb is hidden and the top
|
||
level renders (the never-stale contract, PLAN §7.4).
|
||
8. ``test_anonymous_sees_the_gate`` — the RAG view for an anonymous
|
||
visitor: the sign-in gate visible, no stat cards, no folders/file
|
||
table, no Edit affordance, and NO ``/api/docs*`` request (the
|
||
phase-16 soft rule — asserted via the request log, the
|
||
``test_admin_auth.py`` pattern).
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import json
|
||
import os
|
||
import subprocess
|
||
import sys
|
||
import time
|
||
from collections.abc import Iterator
|
||
from pathlib import Path
|
||
from typing import Any
|
||
|
||
import httpx
|
||
import pytest
|
||
from playwright.sync_api import Page, expect
|
||
from sqlalchemy import select, text
|
||
|
||
from app.config import Settings as _Settings
|
||
from app.db import SessionLocal
|
||
from app.models import FolderSummary
|
||
from e2e.auth_helpers import login
|
||
from e2e.conftest import (
|
||
ADMIN_PASSWORD,
|
||
SESSION_SECRET,
|
||
USE_REAL_LLM,
|
||
_wait_http,
|
||
)
|
||
|
||
REPO = Path(__file__).resolve().parents[2]
|
||
|
||
# Phase 79 (task 04, full inventory): the conftest session app owns its
|
||
# port in a combined run — this module app binds its own port instead
|
||
# (a same-port second uvicorn dies on bind and would drive the wrong
|
||
# server). Env-overridable.
|
||
APP_PORT = int(os.environ.get("E2E_APP_PORT_KBTREE", "8139"))
|
||
APP_URL = f"http://127.0.0.1:{APP_PORT}"
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Fixture constants (deterministic, token-controlled)
|
||
# --------------------------------------------------------------------------
|
||
|
||
ALPHA = "alpha"
|
||
BETA = "beta"
|
||
|
||
ROOT_NOTE = "root-note.md"
|
||
ONE_A = "one/one-a.md"
|
||
ONE_B = "one/one-b.md"
|
||
ONE_C = "one/one-c.md" # test 6's KB-changing addition
|
||
TWO_A = "two/two-a.md"
|
||
TWO_B = "two/two-b.md"
|
||
GAMMA_A = "gamma/gamma-a.md"
|
||
GAMMA_B = "gamma/gamma-b.md"
|
||
|
||
ALPHA_COUNT = 5 # 1 root note + 2 one/ + 2 two/
|
||
BETA_COUNT = 2
|
||
TOTAL_DOCS = ALPHA_COUNT + BETA_COUNT
|
||
|
||
#: The sync-time folder descriptions the mock's canned
|
||
#: ``FOLDER_SUMMARY_MODE`` branch stores (the phase-94 byte-stable
|
||
#: template — the one-liner names the folder), in ``(source,
|
||
#: folder_path)`` order: one row per ≥ 2-doc folder (the
|
||
#: recursive-subtree rule) — the ``""`` rows are the source roots.
|
||
SUMMARY_FOR = "Fixture folder summary for {}."
|
||
ALPHA_ROOT_SUM = SUMMARY_FOR.format(ALPHA)
|
||
ONE_SUM = SUMMARY_FOR.format(f"{ALPHA}/one")
|
||
TWO_SUM = SUMMARY_FOR.format(f"{ALPHA}/two")
|
||
BETA_ROOT_SUM = SUMMARY_FOR.format(BETA)
|
||
GAMMA_SUM = SUMMARY_FOR.format(f"{BETA}/gamma")
|
||
|
||
EXPECTED_SUMMARIES: list[tuple[str, str, str]] = [
|
||
(ALPHA, "", ALPHA_ROOT_SUM),
|
||
(ALPHA, "one", ONE_SUM),
|
||
(ALPHA, "two", TWO_SUM),
|
||
(BETA, "", BETA_ROOT_SUM),
|
||
(BETA, "gamma", GAMMA_SUM),
|
||
]
|
||
assert [
|
||
(source, folder) for source, folder, _s in EXPECTED_SUMMARIES
|
||
] == sorted((source, folder) for source, folder, _s in EXPECTED_SUMMARIES)
|
||
|
||
#: The hand-edited descriptions (tests 4 and 6) — distinctive sentences
|
||
#: no part of the fixture or the canned template contains, so the
|
||
#: round-trip assertions can never pass against the old text.
|
||
NEW_ALPHA_ONE = (
|
||
"Owner override: alpha/one holds the one-a and one-b fixture notes — "
|
||
"the drill catalog's first hand-written description. (RESE-KBTREE-01)"
|
||
)
|
||
NEW_BETA_ROOT = (
|
||
"Owner override: beta is the second fixture source — gamma only. "
|
||
"(RESE-KBTREE-02)"
|
||
)
|
||
|
||
|
||
def _md(title: str, body: str) -> str:
|
||
return f"# {title}\n\n{body}\n"
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Fixtures
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def kb_tree_dirs(tmp_path_factory: pytest.TempPathFactory) -> tuple[Path, Path]:
|
||
"""The two-source temp tree (see the module docstring): the app
|
||
server runs on the same host, so the paths are visible to it. The
|
||
directory NAMES are the source names (``kind=local`` → the
|
||
directory's basename, phase 38)."""
|
||
root = tmp_path_factory.mktemp("bor_kb_tree")
|
||
alpha = root / ALPHA
|
||
beta = root / BETA
|
||
(alpha / "one").mkdir(parents=True)
|
||
(alpha / "two").mkdir(parents=True)
|
||
(beta / "gamma").mkdir(parents=True)
|
||
|
||
(alpha / ROOT_NOTE).write_text(
|
||
_md(
|
||
"Alpha Root Note",
|
||
"This file sits directly under the alpha source, not in any "
|
||
"folder.",
|
||
),
|
||
encoding="utf-8",
|
||
)
|
||
(alpha / ONE_A).write_text(
|
||
_md(
|
||
"Alpha One A",
|
||
"Alpha one fixture note A: covers topic A of the alpha "
|
||
"source tree.",
|
||
),
|
||
encoding="utf-8",
|
||
)
|
||
(alpha / ONE_B).write_text(
|
||
_md(
|
||
"Alpha One B",
|
||
"Alpha one fixture note B: covers topic B of the alpha "
|
||
"source tree.",
|
||
),
|
||
encoding="utf-8",
|
||
)
|
||
(alpha / TWO_A).write_text(
|
||
_md(
|
||
"Alpha Two A",
|
||
"Alpha two fixture note A: covers topic A of the alpha "
|
||
"source tree.",
|
||
),
|
||
encoding="utf-8",
|
||
)
|
||
(alpha / TWO_B).write_text(
|
||
_md(
|
||
"Alpha Two B",
|
||
"Alpha two fixture note B: covers topic B of the alpha "
|
||
"source tree.",
|
||
),
|
||
encoding="utf-8",
|
||
)
|
||
(beta / GAMMA_A).write_text(
|
||
_md(
|
||
"Beta Gamma A",
|
||
"Beta gamma fixture note A: covers topic A of the beta "
|
||
"source tree.",
|
||
),
|
||
encoding="utf-8",
|
||
)
|
||
(beta / GAMMA_B).write_text(
|
||
_md(
|
||
"Beta Gamma B",
|
||
"Beta gamma fixture note B: covers topic B of the beta "
|
||
"source tree.",
|
||
),
|
||
encoding="utf-8",
|
||
)
|
||
assert (alpha / TWO_A).is_file() and (beta / GAMMA_B).is_file()
|
||
return alpha, beta
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def app_server(mock_llm: int, kb_tree_dirs: tuple[Path, Path]) -> Iterator[str]:
|
||
"""The real app under test — per-module app (the conftest pattern,
|
||
cf. ``test_local_directory_sources.py`` / ``test_ls_tree_drilldown.
|
||
py``): NO ``BOR_GIT_SOURCES`` (the env fallback is git-only — the
|
||
sources here are DB-registered local directories), the mock LLM, the
|
||
mock-calibrated threshold, and the leak-guarded code defaults. The
|
||
session app is never started in this isolated run, so no port clash.
|
||
``kb_tree_dirs`` is a dependency only for the fixture ordering (the
|
||
temp tree exists before the app boots — the sync reads it)."""
|
||
env = dict(os.environ)
|
||
env.pop("DEBUGPY", None)
|
||
env["BOR_ENVIRONMENT"] = "e2e"
|
||
env["BOR_STATIC_DIR"] = str(REPO / "frontend")
|
||
env["BOR_LLM_BASE_URL"] = (
|
||
"https://aipi.reeseapps.com/v1"
|
||
if USE_REAL_LLM
|
||
else f"http://127.0.0.1:{mock_llm}/v1"
|
||
)
|
||
# Mock-calibrated threshold (conftest pattern): this suite never
|
||
# asks the chat model anything — the gate is never on a path.
|
||
env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
|
||
# Phase 67: instant retry waits + the code-default budget (the
|
||
# conftest leak-guard pattern).
|
||
env["BOR_LLM_RETRY_DELAY"] = "0"
|
||
env["BOR_LLM_RETRIES"] = str(_Settings.model_fields["llm_retries"].default)
|
||
env.setdefault(
|
||
"BOR_DATABASE_URL",
|
||
"postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
|
||
)
|
||
# Phase 16: admin auth must be set or create_app() refuses to boot.
|
||
env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
|
||
env["BOR_SESSION_SECRET"] = SESSION_SECRET
|
||
# The repo's .env file carries the owner's BOR_GIT_SOURCES (the app
|
||
# reads it from cwd) — override it with an EMPTY value (the env var
|
||
# beats the .env file): the registry must hold EXACTLY the two
|
||
# local directories this suite registers (a leftover env git list
|
||
# would pollute the top-level rows the whole story asserts on).
|
||
env["BOR_GIT_SOURCES"] = ""
|
||
# Leak guards (conftest pattern): an operator's local (gitignored)
|
||
# .env cannot leak corpus-specific settings into the app under test.
|
||
env["BOR_DOCS_REPO"] = ""
|
||
env["BOR_SUGGESTIONS"] = json.dumps(
|
||
_Settings.model_fields["suggestions"].default
|
||
)
|
||
env["BOR_INPUT_PLACEHOLDER"] = _Settings.model_fields["input_placeholder"].default
|
||
env["BOR_FOOTER_TEXT"] = _Settings.model_fields["footer_text"].default
|
||
proc = subprocess.Popen(
|
||
[sys.executable, "-m", "uvicorn", "app.main:app",
|
||
"--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning"],
|
||
cwd=REPO,
|
||
env=env,
|
||
)
|
||
try:
|
||
_wait_http(f"{APP_URL}/api/health")
|
||
yield APP_URL
|
||
finally:
|
||
proc.terminate()
|
||
try:
|
||
proc.wait(timeout=10)
|
||
except subprocess.TimeoutExpired:
|
||
proc.kill()
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def app_url(app_server: str) -> str:
|
||
return app_server
|
||
|
||
|
||
def _truncate_all() -> None:
|
||
"""Fresh registry + KB (the E2E isolation pattern): the E2E suites
|
||
share one Postgres, so a leftover git_sources row would pollute the
|
||
top-level rows and a leftover document would show up in the level
|
||
file tables and the stat cards the suite asserts on exactly."""
|
||
with SessionLocal() as db:
|
||
db.execute(
|
||
text(
|
||
"TRUNCATE chunks, documents, query_log, steering_notes, "
|
||
"kb_overview, git_sources, folder_summaries"
|
||
)
|
||
)
|
||
db.commit() # without the commit the TRUNCATE rolls back (the house pattern)
|
||
|
||
|
||
def _wait_sync_done_http(client: httpx.Client, timeout_s: float = 180.0) -> dict[str, Any]:
|
||
"""Poll the (cookie-authenticated) status endpoint until the run
|
||
reaches a terminal state (the test_local_directory_sources pattern,
|
||
over plain httpx)."""
|
||
deadline = time.monotonic() + timeout_s
|
||
body: dict[str, Any] = {}
|
||
while time.monotonic() < deadline:
|
||
r = client.get("/api/sync/status")
|
||
assert r.status_code == 200, r.text
|
||
body = r.json()
|
||
if body["state"] in ("success", "failed"):
|
||
return body
|
||
time.sleep(0.5)
|
||
raise AssertionError(f"sync did not reach a terminal state: {body}")
|
||
|
||
|
||
@pytest.fixture(scope="module")
|
||
def synced_kb(app_server: str, kb_tree_dirs: tuple[Path, Path]) -> None:
|
||
"""The story's precondition: the folder-structured KB synced under
|
||
the deterministic mock.
|
||
|
||
Registers the two temp directories through the authenticated API
|
||
(the ``test_local_directory_sources.py`` pattern — ``alpha`` FIRST,
|
||
committed separately, so the registry order — ``(added_at, id)`` —
|
||
lists alpha before beta, the top-level row order the suite
|
||
asserts), runs the REAL in-process sync (``POST /api/sync`` —
|
||
walk → chunk → embed → overview → folder summaries → version
|
||
bump), and pins the stored folder descriptions: the mock's canned
|
||
``FOLDER_SUMMARY_MODE`` branch (phase 94) makes the sync store one
|
||
deterministic row per ≥ 2-doc folder — the tests assert on that
|
||
exact text (and on the phase-97 ``manually_edited`` flag: every
|
||
stored row starts out AI-written).
|
||
"""
|
||
alpha, beta = kb_tree_dirs
|
||
_truncate_all()
|
||
with httpx.Client(base_url=app_server, timeout=30.0) as client:
|
||
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||
assert r.status_code == 204, r.text
|
||
r = client.post(
|
||
"/api/git-sources", json={"kind": "local", "path": str(alpha)}
|
||
)
|
||
assert r.status_code == 201, r.text
|
||
time.sleep(0.05) # distinct added_at: alpha before beta (registry order)
|
||
r = client.post(
|
||
"/api/git-sources", json={"kind": "local", "path": str(beta)}
|
||
)
|
||
assert r.status_code == 201, r.text
|
||
r = client.post("/api/sync")
|
||
assert r.status_code == 202, r.text
|
||
body = _wait_sync_done_http(client)
|
||
assert body["state"] == "success", body
|
||
detail = body["detail"]
|
||
assert detail["added"] == TOTAL_DOCS, detail
|
||
assert detail["pruned"] == 0, detail
|
||
assert detail["overview"] is True, detail
|
||
# The change-gated folder descriptions (phase 94) landed: one row
|
||
# per ≥ 2-doc folder, the mock's byte-stable text — and every row
|
||
# AI-written (the ``manually_edited`` flag starts false).
|
||
with SessionLocal() as db:
|
||
rows = db.execute(
|
||
select(
|
||
FolderSummary.source,
|
||
FolderSummary.folder_path,
|
||
FolderSummary.summary,
|
||
FolderSummary.manually_edited,
|
||
).order_by(FolderSummary.source, FolderSummary.folder_path)
|
||
).all()
|
||
assert [
|
||
(source, folder, summary, False)
|
||
for source, folder, summary in EXPECTED_SUMMARIES
|
||
] == [(s, f, t, m) for s, f, t, m in rows], rows
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# Page + DB helpers
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def _kb_totals() -> tuple[int, int]:
|
||
"""The KB-wide ``(documents, chunks)`` totals for the two sources,
|
||
straight from the DB — the values the stat cards' tree walk must
|
||
render (the former flat /api/docs walk, one level deeper)."""
|
||
with SessionLocal() as db:
|
||
docs = db.scalar(
|
||
text("SELECT count(*) FROM documents WHERE source IN (:a, :b)"),
|
||
{"a": ALPHA, "b": BETA},
|
||
)
|
||
chunks = db.scalar(
|
||
text(
|
||
"SELECT count(*) FROM chunks c JOIN documents d "
|
||
"ON c.document_id = d.id WHERE d.source IN (:a, :b)"
|
||
),
|
||
{"a": ALPHA, "b": BETA},
|
||
)
|
||
return int(docs), int(chunks)
|
||
|
||
|
||
def _doc_chunks(source: str, path: str) -> int:
|
||
"""One document's chunk count (the Chunks column value the file
|
||
row must render — the count ``GET /api/docs`` returns for it)."""
|
||
with SessionLocal() as db:
|
||
n = db.scalar(
|
||
text(
|
||
"SELECT count(*) FROM chunks c JOIN documents d "
|
||
"ON c.document_id = d.id WHERE d.source = :s AND d.path = :p"
|
||
),
|
||
{"s": source, "p": path},
|
||
)
|
||
return int(n)
|
||
|
||
|
||
def _folder_row(source: str, folder: str) -> tuple[str, bool] | None:
|
||
"""The stored ``(source, folder)`` description as
|
||
``(summary, manually_edited)`` (``None`` when no row exists) —
|
||
read back through a fresh session (the ``SessionLocal`` house
|
||
pattern)."""
|
||
with SessionLocal() as db:
|
||
row = db.scalar(
|
||
select(FolderSummary).where(
|
||
FolderSummary.source == source,
|
||
FolderSummary.folder_path == folder,
|
||
)
|
||
)
|
||
return (row.summary, row.manually_edited) if row is not None else None
|
||
|
||
|
||
def _run_sync(app_url: str) -> dict[str, Any]:
|
||
"""One admin sync through the API — the cookie-authenticated
|
||
``POST /api/sync`` + the status poll (the ``synced_kb`` pattern,
|
||
for test 6's mid-suite KB-changing sync)."""
|
||
with httpx.Client(base_url=app_url, timeout=30.0) as client:
|
||
r = client.post("/api/login", json={"password": ADMIN_PASSWORD})
|
||
assert r.status_code == 204, r.text
|
||
r = client.post("/api/sync")
|
||
assert r.status_code == 202, r.text
|
||
return _wait_sync_done_http(client)
|
||
|
||
|
||
def _wait_top_level(page: Page) -> None:
|
||
"""The admin boot has rendered the top level: the two source rows
|
||
are in the folders table (the tree's single fetch settled) and the
|
||
top-level file table is empty (files are per-source — always hidden
|
||
at the top)."""
|
||
expect(page.locator("#folders-tbody tr")).to_have_count(2, timeout=30_000)
|
||
expect(page.locator("#docs-tbody tr")).to_have_count(0)
|
||
|
||
|
||
def _drill(page: Page, *names: str) -> None:
|
||
"""Drill one level at a time (client-side — no fetch, no URL
|
||
change): each name is the EXACT text of the source/folder link at
|
||
the current level (the row builders' link text: the source name,
|
||
or the folder's last path segment)."""
|
||
for name in names:
|
||
page.click(f'#folders-tbody a.folder-link:text-is("{name}")')
|
||
|
||
|
||
def _expect_level(page: Page, title: str, summary: str) -> None:
|
||
"""The level block shows the current directory's stored
|
||
description: the full source-relative path as the title, the
|
||
description as the text."""
|
||
expect(page.locator("#kb-level")).to_be_visible()
|
||
expect(page.locator("#kb-level-title")).to_have_text(title)
|
||
expect(page.locator("#kb-level-summary")).to_have_text(summary)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 1. The top level: the source rows (the ls() equivalence) + stat cards
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_top_level_lists_sources_with_descriptions(
|
||
page: Page, app_url: str, synced_kb: None, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
docs, chunks = _kb_totals()
|
||
assert docs == TOTAL_DOCS # the fixture's precondition (7 documents)
|
||
|
||
login(page, app_url) # → /sources.html (the RAG view)
|
||
_wait_top_level(page)
|
||
|
||
# The source rows: registry order (alpha registered first), the
|
||
# recursive count, the stored (source, "") description — the
|
||
# top-level rows ARE the sources (the ls() equivalence: name,
|
||
# count, description).
|
||
rows = page.locator("#folders-tbody tr")
|
||
expect(rows.nth(0).locator("a.folder-link")).to_have_text(ALPHA)
|
||
expect(rows.nth(0).locator("td:nth-child(2)")).to_have_text(str(ALPHA_COUNT))
|
||
expect(rows.nth(0).locator("td:nth-child(4) span")).to_have_text(ALPHA_ROOT_SUM)
|
||
expect(rows.nth(1).locator("a.folder-link")).to_have_text(BETA)
|
||
expect(rows.nth(1).locator("td:nth-child(2)")).to_have_text(str(BETA_COUNT))
|
||
expect(rows.nth(1).locator("td:nth-child(4) span")).to_have_text(BETA_ROOT_SUM)
|
||
# The edit affordance is ALWAYS present (a description can be
|
||
# CREATED where none is stored) — the row surface, one per row.
|
||
expect(page.locator("#folders-tbody .kb-summary-edit")).to_have_count(2)
|
||
|
||
# The top level carries no breadcrumb and no level block; the
|
||
# empty state is off (a source IS listed).
|
||
expect(page.locator("#kb-crumb")).to_be_hidden()
|
||
expect(page.locator("#kb-level")).to_be_hidden()
|
||
expect(page.locator("#sources-empty")).to_be_hidden()
|
||
|
||
# The file table is HIDDEN at the top level (files are seen per
|
||
# source, as with ls(source) — the ls() equivalence).
|
||
expect(page.locator("#docs-table")).to_be_hidden()
|
||
|
||
# The stat cards: the KB-wide walk of the WHOLE tree — identical
|
||
# values to the former flat /api/docs walk.
|
||
expect(page.locator("#stat-docs")).to_have_text(str(TOTAL_DOCS))
|
||
expect(page.locator("#stat-chunks")).to_have_text(str(chunks))
|
||
expect(page.locator("#stat-last")).not_to_have_text("–")
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 2. Drill into a source: breadcrumb, level block, folder rows, files
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_drill_into_source(
|
||
page: Page, app_url: str, synced_kb: None, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
root_chunks = _doc_chunks(ALPHA, ROOT_NOTE)
|
||
|
||
login(page, app_url)
|
||
_wait_top_level(page)
|
||
_drill(page, ALPHA)
|
||
|
||
# Breadcrumb: the top-level link + the current source segment.
|
||
expect(page.locator("#kb-crumb")).to_be_visible()
|
||
links = page.locator("#kb-crumb a.kb-crumb-link")
|
||
expect(links).to_have_count(1)
|
||
expect(links.nth(0)).to_have_text("Knowledge base")
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text(ALPHA)
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_attribute(
|
||
"aria-current", "page"
|
||
)
|
||
|
||
# The level block: the source's stored root description (title =
|
||
# the source name — the source root is folder "") + the level's
|
||
# own Edit button (task 05).
|
||
_expect_level(page, ALPHA, ALPHA_ROOT_SUM)
|
||
expect(page.locator("#kb-level-edit")).to_be_visible()
|
||
|
||
# The subfolder rows: path order, the recursive count, the stored
|
||
# (AI) description.
|
||
rows = page.locator("#folders-tbody tr")
|
||
expect(rows).to_have_count(2)
|
||
expect(rows.nth(0).locator("a.folder-link")).to_have_text("one")
|
||
expect(rows.nth(0).locator("td:nth-child(2)")).to_have_text("2")
|
||
expect(rows.nth(0).locator("td:nth-child(4) span")).to_have_text(ONE_SUM)
|
||
expect(rows.nth(1).locator("a.folder-link")).to_have_text("two")
|
||
expect(rows.nth(1).locator("td:nth-child(2)")).to_have_text("2")
|
||
expect(rows.nth(1).locator("td:nth-child(4) span")).to_have_text(TWO_SUM)
|
||
|
||
# The level's direct files: root-note.md — the 6-column contract
|
||
# (makeRow): Source | Path | Title | Chunks | Created | Indexed
|
||
# (phase 106, task 08, D8: Created BEFORE Indexed — the cell
|
||
# carries the locale date, the full ISO on its title).
|
||
expect(page.locator("#docs-table")).to_be_visible()
|
||
frows = page.locator("#docs-tbody tr")
|
||
expect(frows).to_have_count(1)
|
||
expect(frows.nth(0).locator("td:nth-child(1)")).to_have_text(ALPHA)
|
||
expect(frows.nth(0).locator("a.doc-link")).to_have_text(ROOT_NOTE)
|
||
expect(frows.nth(0).locator("td:nth-child(3)")).to_have_text("Alpha Root Note")
|
||
expect(frows.nth(0).locator("td:nth-child(4)")).to_have_text(str(root_chunks))
|
||
expect(frows.nth(0).locator("td:nth-child(6)")).not_to_have_text("")
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 3. Drill into a folder: the file level + the breadcrumb going back up
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_drill_into_folder(
|
||
page: Page, app_url: str, synced_kb: None, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
login(page, app_url)
|
||
_wait_top_level(page)
|
||
_drill(page, ALPHA, "two")
|
||
|
||
# Breadcrumb: the top-level link + the source link + the current
|
||
# folder segment.
|
||
links = page.locator("#kb-crumb a.kb-crumb-link")
|
||
expect(links).to_have_count(2)
|
||
expect(links.nth(0)).to_have_text("Knowledge base")
|
||
expect(links.nth(1)).to_have_text(ALPHA)
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("two")
|
||
|
||
# The level block: two's stored description, the FULL
|
||
# source-relative path as the title.
|
||
_expect_level(page, f"{ALPHA}/two", TWO_SUM)
|
||
|
||
# The file rows: two-a / two-b (path order), the Source column
|
||
# carries the source (the file nodes are source-scoped in the tree
|
||
# — the row object restores the flat shape makeRow reads).
|
||
frows = page.locator("#docs-tbody tr")
|
||
expect(frows).to_have_count(2)
|
||
expect(frows.nth(0).locator("td:nth-child(1)")).to_have_text(ALPHA)
|
||
expect(frows.nth(0).locator("a.doc-link")).to_have_text(TWO_A)
|
||
expect(frows.nth(0).locator("td:nth-child(3)")).to_have_text("Alpha Two A")
|
||
expect(frows.nth(1).locator("td:nth-child(1)")).to_have_text(ALPHA)
|
||
expect(frows.nth(1).locator("a.doc-link")).to_have_text(TWO_B)
|
||
expect(frows.nth(1).locator("td:nth-child(3)")).to_have_text("Alpha Two B")
|
||
|
||
# No subfolders under two: the folders table is hidden.
|
||
expect(page.locator("#folders-wrap")).to_be_hidden()
|
||
|
||
# The breadcrumb link on the source goes back UP to the source
|
||
# level (client-side — no fetch, no URL change).
|
||
page.click(f'#kb-crumb a.kb-crumb-link:text-is("{ALPHA}")')
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text(ALPHA)
|
||
expect(page.locator("#kb-crumb a.kb-crumb-link")).to_have_count(1)
|
||
rows = page.locator("#folders-tbody tr")
|
||
expect(rows).to_have_count(2)
|
||
expect(rows.nth(0).locator("a.folder-link")).to_have_text("one")
|
||
expect(rows.nth(1).locator("a.folder-link")).to_have_text("two")
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 4. Edit a folder description (the row surface) → manually_edited
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_edit_folder_description(
|
||
page: Page, app_url: str, synced_kb: None, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
login(page, app_url)
|
||
_wait_top_level(page)
|
||
_drill(page, ALPHA)
|
||
|
||
rows = page.locator("#folders-tbody tr")
|
||
expect(rows).to_have_count(2)
|
||
row = rows.nth(0)
|
||
expect(row.locator("a.folder-link")).to_have_text("one")
|
||
|
||
# Edit → the inline editor: textarea PREFILLED with the stored
|
||
# (canned) text, Save / Cancel, and the role=status live region.
|
||
row.locator(".kb-summary-edit").click()
|
||
editor = page.locator(".kb-summary-editor")
|
||
expect(editor).to_have_count(1)
|
||
expect(editor).to_be_visible()
|
||
expect(editor).to_have_value(ONE_SUM)
|
||
expect(page.locator(".kb-summary-save")).to_be_visible()
|
||
expect(page.locator(".kb-summary-cancel")).to_be_visible()
|
||
status = row.locator(".kb-summary-status")
|
||
expect(status).to_have_attribute("role", "status")
|
||
expect(status).to_have_attribute("aria-live", "polite")
|
||
|
||
# Replace the text with the distinctive hand-edit and Save.
|
||
page.fill(".kb-summary-editor", NEW_ALPHA_ONE)
|
||
page.click(".kb-summary-save")
|
||
|
||
# The live-region confirmation; the new text renders in the row
|
||
# (the textContent re-render — the canned text is gone); the
|
||
# always-present Edit button is back.
|
||
expect(status).to_have_text("Description updated.")
|
||
expect(row.locator("td:nth-child(4) span")).to_have_text(NEW_ALPHA_ONE)
|
||
expect(row.locator(".kb-summary-edit")).to_be_visible()
|
||
|
||
# The server row: the new text AND the manually_edited flag (the
|
||
# task-01 keep/keep-out rules apply from this save on — the
|
||
# sync-time generator must never rewrite or prune the row).
|
||
stored = _folder_row(ALPHA, "one")
|
||
assert stored == (NEW_ALPHA_ONE, True), stored
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 5. An empty save CLEARS — the row is deleted (the reset path)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_clear_folder_description(
|
||
page: Page, app_url: str, synced_kb: None, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
login(page, app_url)
|
||
_wait_top_level(page)
|
||
_drill(page, ALPHA)
|
||
|
||
rows = page.locator("#folders-tbody tr")
|
||
expect(rows).to_have_count(2)
|
||
row = rows.nth(0)
|
||
expect(row.locator("a.folder-link")).to_have_text("one")
|
||
# The row holds a description (the module's canned row — or test
|
||
# 4's manual edit when the tests run in order); the editor opens
|
||
# either way (the button is always present).
|
||
row.locator(".kb-summary-edit").click()
|
||
expect(page.locator(".kb-summary-editor")).to_be_visible()
|
||
|
||
# Select-all + delete — clear the prefilled editor — then Save.
|
||
page.fill(".kb-summary-editor", "")
|
||
page.click(".kb-summary-save")
|
||
|
||
status = row.locator(".kb-summary-status")
|
||
expect(status).to_have_text("Description cleared.")
|
||
# The text is gone: the cell empties (the always-present Edit
|
||
# button stays — a description can be re-created from the empty
|
||
# cell).
|
||
expect(row.locator("td:nth-child(4) span")).to_have_text("")
|
||
expect(row.locator(".kb-summary-edit")).to_be_visible()
|
||
|
||
# The SQL assert: no row for the folder (the next KB-changing
|
||
# sync regenerates an AI description — the reset path).
|
||
assert _folder_row(ALPHA, "one") is None
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 6. The manual description survives a KB-changing sync (kept_manual)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_manual_description_survives_a_changed_sync(
|
||
page: Page,
|
||
app_url: str,
|
||
synced_kb: None,
|
||
kb_tree_dirs: tuple[Path, Path],
|
||
db_ready: None,
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
alpha, _beta = kb_tree_dirs
|
||
|
||
login(page, app_url)
|
||
_wait_top_level(page)
|
||
_drill(page, BETA)
|
||
|
||
# The beta level: the stored root description + the gamma row
|
||
# (both canned — AI-written at the module sync).
|
||
_expect_level(page, BETA, BETA_ROOT_SUM)
|
||
rows = page.locator("#folders-tbody tr")
|
||
expect(rows).to_have_count(1)
|
||
expect(rows.nth(0).locator("a.folder-link")).to_have_text("gamma")
|
||
expect(rows.nth(0).locator("td:nth-child(4) span")).to_have_text(GAMMA_SUM)
|
||
|
||
# The LEVEL-BLOCK Edit (the level surface — test 4 pins the row
|
||
# surface): the static #kb-level-edit button, the source root
|
||
# (folder_path ""). The textarea is prefilled with the stored text.
|
||
page.click("#kb-level-edit")
|
||
editor = page.locator(".kb-summary-editor")
|
||
expect(editor).to_have_count(1)
|
||
expect(editor).to_have_value(BETA_ROOT_SUM)
|
||
page.fill(".kb-summary-editor", NEW_BETA_ROOT)
|
||
page.click(".kb-summary-save")
|
||
expect(page.locator("#kb-level .kb-summary-status")).to_have_text(
|
||
"Description updated."
|
||
)
|
||
expect(page.locator("#kb-level-summary")).to_have_text(NEW_BETA_ROOT)
|
||
|
||
# The KB-changing sync: a new file lands in alpha/one (the fixture
|
||
# dir — the app runs on the same host), then the real in-process
|
||
# sync (the mock's FOLDER_SUMMARY_MODE branch regenerates the
|
||
# non-manual folders' summaries).
|
||
(alpha / ONE_C).write_text(
|
||
_md(
|
||
"Alpha One C",
|
||
"Alpha one fixture note C: added for the changed-sync test.",
|
||
),
|
||
encoding="utf-8",
|
||
)
|
||
body = _run_sync(app_url)
|
||
assert body["state"] == "success", body
|
||
assert body["detail"]["added"] == 1, body["detail"]
|
||
|
||
# The catalog re-fetch (the nav re-click — the bor:view-refresh
|
||
# trigger, no document load): the current position (the beta root)
|
||
# still exists, so the view stays put and re-renders from the NEW
|
||
# tree — the EDITED description is UNCHANGED (kept_manual: the
|
||
# generator skipped the owner's row, no lite burn on it)…
|
||
page.click("#nav-sources")
|
||
expect(page.locator("#kb-level-summary")).to_have_text(NEW_BETA_ROOT)
|
||
# …while the untouched folder shows the canned REGENERATED text.
|
||
expect(
|
||
page.locator("#folders-tbody tr")
|
||
.nth(0)
|
||
.locator("td:nth-child(4) span")
|
||
).to_have_text(GAMMA_SUM)
|
||
|
||
# The new document landed in the tree: back to the top (the
|
||
# breadcrumb's top-level link), then into alpha — the one folder
|
||
# now counts 3 with its regenerated canned description.
|
||
page.locator("#kb-crumb a.kb-crumb-link").nth(0).click()
|
||
_drill(page, ALPHA)
|
||
one_row = page.locator("#folders-tbody tr").nth(0)
|
||
expect(one_row.locator("a.folder-link")).to_have_text("one")
|
||
expect(one_row.locator("td:nth-child(2)")).to_have_text("3")
|
||
expect(one_row.locator("td:nth-child(4) span")).to_have_text(ONE_SUM)
|
||
|
||
# The SQL asserts: the manual row survived (text + flag), the
|
||
# untouched rows were regenerated (the canned text, the flag back
|
||
# to false — the rows are AI-written again).
|
||
assert _folder_row(BETA, "") == (NEW_BETA_ROOT, True)
|
||
assert _folder_row(BETA, "gamma") == (GAMMA_SUM, False)
|
||
assert _folder_row(ALPHA, "one") == (ONE_SUM, False)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 7. A vanished location resets the view to the top level (never stale)
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_reload_falls_back_to_top_level(
|
||
page: Page, app_url: str, synced_kb: None, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
login(page, app_url)
|
||
_wait_top_level(page)
|
||
_drill(page, ALPHA, "two")
|
||
expect(page.locator("#kb-crumb .kb-crumb-current")).to_have_text("two")
|
||
|
||
# The folder vanishes: its documents + chunks deleted directly (the
|
||
# house DB pattern — no sync, no API).
|
||
with SessionLocal() as db:
|
||
db.execute(
|
||
text(
|
||
"DELETE FROM chunks WHERE document_id IN "
|
||
"(SELECT id FROM documents WHERE source = :s AND path LIKE 'two/%')"
|
||
),
|
||
{"s": ALPHA},
|
||
)
|
||
db.execute(
|
||
text("DELETE FROM documents WHERE source = :s AND path LIKE 'two/%'"),
|
||
{"s": ALPHA},
|
||
)
|
||
db.commit()
|
||
|
||
# Re-show the RAG view: the nav-link RE-CLICK is the
|
||
# bor:view-refresh trigger (the router's re-fetch on the mounted
|
||
# view — no document load).
|
||
page.click("#nav-sources")
|
||
|
||
# The vanished location resets the view to the top level BEFORE
|
||
# rendering (PLAN §7.4 — no stale breadcrumb, no stale block): the
|
||
# breadcrumb and the level block are hidden, the source rows
|
||
# re-render.
|
||
expect(page.locator("#kb-crumb")).to_be_hidden()
|
||
expect(page.locator("#kb-level")).to_be_hidden()
|
||
rows = page.locator("#folders-tbody tr")
|
||
expect(rows).to_have_count(2)
|
||
expect(rows.nth(0).locator("a.folder-link")).to_have_text(ALPHA)
|
||
expect(rows.nth(1).locator("a.folder-link")).to_have_text(BETA)
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 8. Anonymous: the sign-in gate — and zero catalog fetches
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
def test_anonymous_sees_the_gate(
|
||
page: Page, app_url: str, synced_kb: None, db_ready: None
|
||
) -> None:
|
||
page.set_default_timeout(30_000)
|
||
|
||
docs_calls: list[str] = []
|
||
page.on(
|
||
"request",
|
||
lambda r: docs_calls.append(r.url) if "/api/docs" in r.url else None,
|
||
)
|
||
|
||
# Fresh context (the function-scoped page fixture — no login).
|
||
page.goto(app_url + "/sources.html")
|
||
gate = page.locator("#sources-gate")
|
||
expect(gate).to_be_visible(timeout=30_000)
|
||
expect(gate).to_contain_text("Sign in to view the full catalog")
|
||
|
||
# The catalog surfaces stay hidden (the tree ships hidden and never
|
||
# fills for anonymous)…
|
||
expect(page.locator("#stat-cards")).to_be_hidden()
|
||
expect(page.locator("#folders-wrap")).to_be_hidden()
|
||
expect(page.locator("#docs-table")).to_be_hidden()
|
||
expect(page.locator("#kb-crumb")).to_be_hidden()
|
||
expect(page.locator("#kb-level")).to_be_hidden()
|
||
expect(page.locator("#sources-empty")).to_be_hidden()
|
||
expect(page.locator("#folders-tbody tr")).to_have_count(0)
|
||
expect(page.locator("#docs-tbody tr")).to_have_count(0)
|
||
# …and there is no Edit affordance anywhere (the row buttons are
|
||
# never built; the static level button sits in the hidden block).
|
||
expect(page.locator(".kb-summary-edit:visible")).to_have_count(0)
|
||
expect(page.locator("#kb-level-edit")).to_be_hidden()
|
||
|
||
# The soft rule's wire-level proof (the phase-16 pattern, the
|
||
# tree edition): not one catalog request — the anonymous gate
|
||
# branch never fetches /api/docs/tree.
|
||
assert docs_calls == [], f"anonymous RAG view fetched the catalog: {docs_calls}"
|