brain-of-reese/tests/e2e/test_fast_path_deferred_content.py

"""Phase 185 E2E (Playwright, mock-only): the deferred column still shows up
where the owner reads it.

Source: owner incident 2026-10-15 (the same run phases 183/184 fixed). The
unchanged fast path looked each file up by a WHOLE-ENTITY ``select(Document)``
— ``documents.content`` included — and ``app/db.py`` builds sessions with
``expire_on_commit=False``, so the 2026-10-15 sync inspected ~27k files and
held ~252.6 MB of markdown (17,407 rows × avg 15,217 chars) resident to
discover that 217 files had changed.

The fix (``app/rag/importer.py``): the four fast-path row loads share the ONE
statement ``_doc_row_stmt``, which defers ``Document.content`` — exactly ONE
column (LOCKED A1: no ``load_only`` allowlist, so a future column can never
silently become lazy) and deliberately NO ``raiseload``/``noload`` (LOCKED
A2: the phase-118 summary-NULL backfill genuinely reads the stored markdown
and must pay one per-row, one-time SELECT instead of crashing). The
statement-level proof — the captured fast-path ``SELECT`` names no
``documents.content``, one ``SELECT`` per unchanged file with a summary, one
lazy follow-up for the backfill — lives in
``tests/integration/test_import_fast_path_columns.py``. What a browser can
see is what THIS module pins: the deferral is invisible to the owner.

Run in isolation (DB must be up: ``podman compose up -d db``):

    uv run pytest tests/e2e/test_fast_path_deferred_content.py -v --no-cov

MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is skipped — the backfilled summary is
compared BYTE-IDENTICAL against the summary the first import stored, which
is a property of the deterministic mock summarizer; against the real endpoint
the comparison would be a measurement, not a pin.

One module app on a dedicated port (the conftest session app owns 8123 in a
combined run), booting against the mock LLM with ``BOR_GIT_SOURCES`` forced
empty (an operator's local ``.env`` must never leak a real source into a
run) and its own ``BOR_SOURCES_DIR`` / ``BOR_PDF_PAGE_DIR`` under
``tmp_path_factory``. The fixture KB is ONE plain (non-git) directory named
:data:`SOURCE_NAME` (``kind='local'`` rows index under ``Path(path).name``,
so the source label is stable whatever temp root pytest picked) holding TWO
``.md`` files; both carry a sentinel phrase on their FIRST content line and
another on their LAST line, so "the viewer shows the FULL stored text" is
provable at both ends of a file.

Isolation: the shared E2E Postgres is truncated BEFORE **and** AFTER every
test (``chunks, documents, query_log, kb_overview, git_sources, sources_meta,
folder_summaries``), after waiting for the app to hold no running sync (the
run state lives in the app's memory — a leaked run would keep importing into
the next test's truncated KB). The fixture TREE is re-seeded to its original
bytes at the top of every test too (test 3 rewrites one file — a lone re-run
of this module must still see ``2 added``, not ``0 added · 2 unchanged``),
and each test registers its own source row.

Test → observable mapping (Playwright Mapping Rule):
1. ``test_summary_null_backfill_survives_the_deferral`` — with one row's
   ``documents.summary`` nulled AND its stale ``is_summary`` chunk deleted
   directly (the phase-118 staging, extended so the chunk's REAPPEARANCE
   proves the backfill ran), the Sources-page re-sync still settles
   ``0 added · 2 unchanged`` (a backfill is not an ``updated``/``added``),
   the document page shows a non-empty summary identical to the one run 1
   stored — produced from the STORED content through the deferred-column
   lazy read — and exactly ONE ``is_summary`` chunk exists for the doc
   (no duplicate).
2. ``test_unchanged_document_viewer_shows_full_stored_text`` — a second
   sync of untouched files settles ``0 added · 2 unchanged`` and the
   document view then renders the FULL stored text: the head AND the tail
   sentinel both present, every stored line present, no truncation, no
   empty body. The row lookup the deferral thinned is the same lookup the
   fast path answers with; the viewer reads its row separately and still
   gets every byte.
3. ``test_changed_file_reindexes_under_the_deferral`` — one file is
   rewritten on disk (new head/tail sentinels, more sections, different
   length); the re-sync settles ``0 added · 1 updated · 1 unchanged``, the
   viewer shows the NEW text with NO stale fragment, the stored content
   equals the new file bytes, the content-chunk count moved, the
   untouched neighbour row did not, and a THIRD sync reports all
   ``unchanged`` — the fast path still fires after a deferral-mediated
   write.
"""
from __future__ import annotations

import os
import re
import subprocess
import sys
import time
from collections.abc import Iterator
from datetime import datetime
from pathlib import Path
from typing import Any

import httpx
import pytest
from playwright.sync_api import Page, expect
from sqlalchemy import delete, func, select, text

from app.db import SessionLocal
from app.models import Chunk, Document
from e2e.auth_helpers import login
from e2e.conftest import (
    ADMIN_PASSWORD,
    MOCK_PORT,
    SESSION_SECRET,
    USE_REAL_LLM,
    _wait_http,
)

REPO = Path(__file__).resolve().parents[2]

#: Dedicated module port (the house env-overridable pattern): the conftest
#: session app owns 8123 in a combined run.
APP_PORT = int(os.environ.get("E2E_APP_PORT_DEFERCONTENT", "8202"))
APP_URL = f"http://127.0.0.1:{APP_PORT}"

SOURCES_URL = "/sources.html"

#: The fixture directory's basename — the ``source`` the importer records for
#: a ``kind='local'`` row (``Path(path).name``), stable whatever temp root
#: this run got.
SOURCE_NAME = "defer-fast-path"

ALPHA_PATH = "notes/alpha.md"
BETA_PATH = "notes/beta.md"
N_FILES = 2

#: Head/tail sentinels: the FIRST content line and the LAST line of each
#: fixture file, so "the viewer shows the FULL stored text" is proven at
#: both ends (test 2), "no stale fragment" is proven at both ends (test 3).
ALPHA_HEAD = "RESE-DEFER-ALPHA-HEAD-3c81"
ALPHA_TAIL = "RESE-DEFER-ALPHA-TAIL-7e52"
ALPHA_MIDDLE = "RESE-DEFER-ALPHA-MIDDLE-0f64"
BETA_HEAD = "RESE-DEFER-BETA-HEAD-5b19"
BETA_TAIL = "RESE-DEFER-BETA-TAIL-8a64"
#: What test 3 rewrites alpha.md into (and what must have replaced every
#: ALPHA_* marker in the KB and in the viewer).
REVISED_HEAD = "RESE-DEFER-ALPHA2-HEAD-e208"
REVISED_TAIL = "RESE-DEFER-ALPHA2-TAIL-41cb"

#: Test 1 stages THIS row (phase-118's summary-NULL backfill case).
BACKFILL_PATH = BETA_PATH

#: "Synced HH:MM" — the local-time last-result label (sources.js's
#: fmtSyncTime), any hour/minute (the test_sync_button.py pattern).
SYNCED_LABEL = re.compile(r"Synced \d{1,2}:\d{2}$")

#: A two-file mock sync (walk + embeds + summary + KB overview) lands in
#: seconds; this is an order of magnitude of head-room with no client-side
#: hard timeout (the test_sync_session_reset.py budget).
SETTLE_TIMEOUT_MS = 90_000


# ---------------------------------------------------------------------------
# The fixture source
# ---------------------------------------------------------------------------


def _alpha_text() -> str:
    """The ORIGINAL bytes of ``notes/alpha.md`` (test 2's full-text subject,
    test 3's rewrite subject). Two H2 sections — a section is the md chunker's
    unit, so the rewrite's extra sections move the chunk count."""
    return (
        "# Deferred content alpha\n"
        "\n"
        f"Head marker {ALPHA_HEAD} opens this file.\n"
        "\n"
        f"Middle marker {ALPHA_MIDDLE} sits between the sections.\n"
        "\n"
        "## Alpha body\n"
        "\n"
        "Alpha prose that the unchanged fast path must never need to read: "
        "the walk wants the row metadata, not these paragraphs. It keeps "
        "growing so the file is unmistakably a document and not a stub.\n"
        "\n"
        "## Alpha notes\n"
        "\n"
        "More alpha prose so the content chunk count is more than one. Each "
        "section is its own chunk, which is what test 3 counts when it "
        "rewrites this file under the deferral.\n"
        "\n"
        f"Tail marker {ALPHA_TAIL} closes this file.\n"
    )


def _revised_alpha_text() -> str:
    """The rewrite: NEW head/tail sentinels, two extra sections, clearly
    different length — every old marker is gone from the bytes."""
    return (
        "# Deferred content alpha revised\n"
        "\n"
        f"Head marker {REVISED_HEAD} opens the revised file.\n"
        "\n"
        "## Alpha body\n"
        "\n"
        "The revised alpha prose. The deferral changed which columns the "
        "unchanged fast path reads; a CHANGED file must still re-index its "
        "full text, so this paragraph and its siblings must land in the KB "
        "verbatim, and every marker of the original file must be gone.\n"
        "\n"
        "## Alpha notes\n"
        "\n"
        "Revised notes prose, longer than before on purpose so the stored "
        "length and the viewer both prove the write path carried the new "
        "text through the same session that read the row with content "
        "deferred.\n"
        "\n"
        "## Alpha appendix one\n"
        "\n"
        "A brand new section that did not exist before the rewrite, added "
        "so the content chunk count of this document must change when the "
        "re-indexed file is re-chunked.\n"
        "\n"
        "## Alpha appendix two\n"
        "\n"
        "A second brand new section, because the point of the phase is "
        "that NOTHING about what a run does changed — only which columns "
        "travel on an unchanged row lookup.\n"
        "\n"
        f"Tail marker {REVISED_TAIL} closes the revised file.\n"
    )


def _beta_text() -> str:
    """``notes/beta.md`` — test 1's backfill subject. Ordinary prose with
    its own head/tail sentinels."""
    return (
        "# Deferred content beta\n"
        "\n"
        f"Head marker {BETA_HEAD} opens this file.\n"
        "\n"
        "## Beta body\n"
        "\n"
        "Beta is the document whose stored summary test 1 nulls directly, "
        "so the Sources-page re-sync must backfill it from the stored "
        "content — the ONE fast-path read that pulls the deferred column "
        "back, one SELECT, one time, by design.\n"
        "\n"
        f"Tail marker {BETA_TAIL} closes this file.\n"
    )


def _seed_source(root: Path) -> Path:
    """(Re-)write BOTH fixture files under ``root / SOURCE_NAME``.

    Idempotent on purpose: every test calls it first, so the tree is at its
    ORIGINAL bytes whatever an earlier test (test 3!) left behind — a lone
    re-run of any single test still sees a fresh-KB ``2 added``.
    """
    base = root / SOURCE_NAME
    files = {ALPHA_PATH: _alpha_text(), BETA_PATH: _beta_text()}
    for rel, body in files.items():
        target = base / rel
        target.parent.mkdir(parents=True, exist_ok=True)
        target.write_text(body, encoding="utf-8")
    assert not (base / ".git").exists()  # a plain directory — no git anywhere
    return base


@pytest.fixture(scope="module")
def source_dir(tmp_path_factory: pytest.TempPathFactory) -> Path:
    """The fixture tree the module app walks (the app server runs on the same
    host, so the path is visible to it)."""
    return _seed_source(tmp_path_factory.mktemp("bor_fast_path_deferred"))


# ---------------------------------------------------------------------------
# The module app
# ---------------------------------------------------------------------------


@pytest.fixture(scope="module")
def app_server(
    mock_llm: int, source_dir: Path, tmp_path_factory: pytest.TempPathFactory
) -> Iterator[str]:
    """The real app under test — per-module env: the mock LLM, the
    mock-calibrated gate (the conftest convention), NO git fallback, and a
    dedicated ``BOR_SOURCES_DIR`` so no run writes into the operator's.

    The phase's subject needs no knob: the ``defer(Document.content)`` fast
    path is the code default (LOCKED A1/A2) — this app is the app every
    other suite runs with, walked over a fixture small enough to read.
    """
    if USE_REAL_LLM:
        pytest.skip(
            "mock-only suite — the byte-identical backfilled summary is a "
            "property of the deterministic mock (E2E_REAL_LLM unsupported)"
        )
    assert mock_llm and source_dir
    scratch = tmp_path_factory.mktemp("bor_fast_path_deferred_app")
    env = dict(os.environ)
    env.pop("DEBUGPY", None)
    env["BOR_ENVIRONMENT"] = "e2e"
    env["BOR_STATIC_DIR"] = str(REPO / "frontend")
    env["BOR_LLM_BASE_URL"] = f"http://127.0.0.1:{MOCK_PORT}/v1"
    # Mock-calibrated gate (the conftest pattern): both floors must stay <=
    # the threshold or the app refuses to boot (app/config.py).
    env["BOR_RELEVANCE_THRESHOLD"] = "0.30"
    env["BOR_LEXICAL_SUPPORT_FLOOR"] = "0.15"
    env["BOR_SOURCE_USEFULNESS_FLOOR"] = "0.15"
    env.setdefault(
        "BOR_DATABASE_URL",
        "postgresql+psycopg://reese:reese@localhost:5432/brain_of_reese",
    )
    # Phase 16: admin auth must be set or create_app() refuses to boot.
    env["BOR_ADMIN_PASSWORD"] = ADMIN_PASSWORD
    env["BOR_SESSION_SECRET"] = SESSION_SECRET
    # No env-fallback rows: the registry each test writes through the
    # authenticated API is the only source list.
    env["BOR_GIT_SOURCES"] = ""
    env["BOR_SOURCES_DIR"] = str(scratch / "checkouts")
    env["BOR_PDF_PAGE_DIR"] = str(scratch / "pdf-pages")
    proc = subprocess.Popen(
        [
            sys.executable, "-m", "uvicorn", "app.main:app",
            "--host", "127.0.0.1", "--port", str(APP_PORT), "--log-level", "warning",
        ],
        cwd=REPO,
        env=env,
    )
    try:
        _wait_http(f"{APP_URL}/api/health")
        yield APP_URL
    finally:
        proc.terminate()
        try:
            proc.wait(timeout=10)
        except subprocess.TimeoutExpired:
            proc.kill()


@pytest.fixture(scope="module")
def app_url(app_server: str) -> str:
    return app_server


# ---------------------------------------------------------------------------
# Isolation
# ---------------------------------------------------------------------------


def _truncate_all() -> None:
    """Fresh registry + KB (the E2E isolation pattern, the phase-184 list:
    ``sources_meta`` and ``folder_summaries`` reset with the KB)."""
    with SessionLocal() as db:
        db.execute(
            text(
                "TRUNCATE chunks, documents, query_log, kb_overview, "
                "git_sources, sources_meta, folder_summaries"
            )
        )
        db.commit()  # without the commit the TRUNCATE rolls back


def _wait_sync_settled(base_url: str, timeout_s: float = 180.0) -> None:
    """No run may leak across a truncate: wait for a non-running status with
    an own admin session (the endpoint is admin-only). Usually a no-op — the
    tests settle before they end."""
    deadline = time.monotonic() + timeout_s
    while time.monotonic() < deadline:
        with httpx.Client(base_url=base_url, timeout=5.0) as client:
            client.post("/api/login", json={"password": ADMIN_PASSWORD})
            r = client.get("/api/sync/status")
            assert r.status_code == 200, r.text
            if r.json()["state"] != "running":
                return
        time.sleep(0.2)
    raise AssertionError(f"a sync was still running at a test boundary: {base_url}")


@pytest.fixture(autouse=True)
def _clean(app_url: str, db_ready: None) -> Iterator[None]:
    """Per-test KB isolation: the shared E2E Postgres is truncated before AND
    after every test, the app holding no running sync at both boundaries."""
    _wait_sync_settled(app_url)
    _truncate_all()
    yield
    _wait_sync_settled(app_url)
    _truncate_all()


# ---------------------------------------------------------------------------
# Helpers: the UI side
# ---------------------------------------------------------------------------


def _add_local_source(page: Page, base_url: str, path: Path) -> None:
    """Register the fixture directory through the authenticated API (the
    phase-49 shape — the admin cookie rides ``page.request``)."""
    r = page.request.post(
        f"{base_url}/api/git-sources",
        data={"kind": "local", "path": str(path)},
    )
    assert r.status == 201, f"expected 201 for {path}: {r.status} {r.text}"


def _started_at(body: dict[str, Any]) -> float:
    """The status body's ``started_at`` as a wall-clock epoch (the server
    stamps it in UTC and this process runs on the same host)."""
    started = body.get("started_at")
    if not started:
        return 0.0
    try:
        return datetime.fromisoformat(str(started)).timestamp()
    except ValueError:
        return 0.0


def _sync_from_sources_page(page: Page, base_url: str) -> dict[str, Any]:
    """Click the Sources page **Sync sources** button and wait for THIS run's
    terminal state (the module app is shared, so a terminal counts only after
    a ``running`` tick or when its ``started_at`` post-dates the click).

    Settles the UI on success: the button re-enabled, ``Synced HH:MM``, no
    error banner (the test_sync_session_reset.py helper)."""
    btn = page.locator("#sync-btn")
    expect(btn).to_be_visible(timeout=30_000)
    expect(btn).to_be_enabled()
    clicked_at = time.time()
    btn.click()
    expect(btn).to_be_disabled()
    expect(page.locator("#sync-label")).to_have_text("Syncing…")

    deadline = time.monotonic() + SETTLE_TIMEOUT_MS / 1000
    saw_running = False
    body: dict[str, Any] = {}
    while time.monotonic() < deadline:
        r = page.request.get(f"{base_url}/api/sync/status")
        assert r.status == 200, r.text
        body = r.json()
        state = body["state"]
        if state == "running":
            saw_running = True
        elif state in ("success", "failed", "cancelled") and (
            saw_running or _started_at(body) >= clicked_at - 1.0
        ):
            break
        time.sleep(0.2)
    else:
        raise AssertionError(f"sync never reached a terminal state: {body}")

    assert body["state"] == "success", f"sync failed: {body}"
    assert body["error"] is None, body
    expect(btn).to_be_enabled(timeout=15_000)
    expect(page.locator("#sync-label")).to_have_text(SYNCED_LABEL, timeout=15_000)
    expect(page.locator("#sync-error-banner")).to_be_hidden()
    return body


def _expect_counts_line(page: Page, expected: str) -> None:
    """The counts line — the owner-visible sync report on the single sync
    text surface (phase 150; sources.js fmtSyncResult; the zero "updated"
    part is omitted)."""
    expect(page.locator("#sync-progress-text")).to_have_text(expected, timeout=15_000)


def _goto_sources(page: Page, base_url: str) -> None:
    """Back to the Sources page (the sync button's home) after a viewer
    detour — the session cookie rides the navigation."""
    page.goto(base_url + SOURCES_URL)
    expect(page.locator("#sync-btn")).to_be_visible(timeout=30_000)
    expect(page.locator("#sync-btn")).to_be_enabled()


def _open_document_page(page: Page, base_url: str, path: str) -> None:
    """Open the standalone viewer for one fixture document (the phase-10
    surface; the session cookie from ``login`` rides the navigation)."""
    page.goto(
        f"{base_url}/document.html?source={SOURCE_NAME}"
        f"&path={path.replace('/', '%2F')}"
    )
    expect(page.locator("#doc-content .doc-md")).to_have_count(1, timeout=30_000)


# ---------------------------------------------------------------------------
# Helpers: the DB side
# ---------------------------------------------------------------------------


def _summary_map() -> dict[str, str | None]:
    """``path`` → ``documents.summary`` for both fixture rows."""
    with SessionLocal() as db:
        rows = db.execute(
            select(Document.path, Document.summary).where(
                Document.source == SOURCE_NAME
            )
        ).all()
    return {str(path): (None if summary is None else str(summary)) for path, summary in rows}


def _summary_chunk_contents(path: str) -> list[str]:
    """The ``is_summary`` chunk contents of one document — the duplicate
    check (phase 30: at most one per doc at a time)."""
    with SessionLocal() as db:
        return [
            str(content)
            for (content,) in db.execute(
                select(Chunk.content)
                .join(Document, Document.id == Chunk.document_id)
                .where(
                    Document.source == SOURCE_NAME,
                    Document.path == path,
                    Chunk.is_summary.is_(True),
                )
            ).all()
        ]


def _content_chunk_counts() -> dict[str, int]:
    """``path`` → content-chunk count per fixture doc (the ``position >= 0``
    rows — the ``is_summary`` position −1 chunk stays out)."""
    with SessionLocal() as db:
        rows = db.execute(
            select(Document.path, func.count(Chunk.id))
            .join(Chunk, Chunk.document_id == Document.id)
            .where(
                Document.source == SOURCE_NAME,
                Chunk.position >= 0,
            )
            .group_by(Document.path)
        ).all()
    return {str(path): int(count) for path, count in rows}


def _stored_content(path: str) -> str:
    with SessionLocal() as db:
        content = db.scalar(
            select(Document.content).where(
                Document.source == SOURCE_NAME, Document.path == path
            )
        )
    assert content is not None, f"{path} is not in the KB"
    return str(content)


def _row_stamp(path: str) -> tuple[Any, Any]:
    """``(content_hash, file_hash)`` of one fixture row — the neighbour's
    byte-still proof (a re-index would re-stamp both)."""
    with SessionLocal() as db:
        row = db.execute(
            select(Document.content_hash, Document.file_hash).where(
                Document.source == SOURCE_NAME, Document.path == path
            )
        ).one()
    return row[0], row[1]


def _stage_null_summary(path: str) -> None:
    """The phase-118 backfill case, staged directly (the way phase 118's
    tests null the column), extended for this phase: the stale
    ``is_summary`` CHUNK goes too, so the chunk's REAPPEARANCE after the
    re-sync proves the backfill ran rather than merely survived."""
    with SessionLocal() as db:
        doc_id = db.scalar(
            select(Document.id).where(
                Document.source == SOURCE_NAME, Document.path == path
            )
        )
        assert doc_id is not None, f"{path} is not in the KB"
        db.execute(delete(Chunk).where(Chunk.document_id == doc_id, Chunk.is_summary.is_(True)))
        db.execute(
            text(
                "UPDATE documents SET summary = NULL "
                "WHERE source = :s AND path = :p"
            ),
            {"s": SOURCE_NAME, "p": path},
        )
        db.commit()
    # The stage landed — otherwise every later assertion would pass vacuously.
    assert _summary_map()[path] is None
    assert _summary_chunk_contents(path) == []


# ---------------------------------------------------------------------------
# 1. The summary-NULL backfill survives the deferral
# ---------------------------------------------------------------------------


def test_summary_null_backfill_survives_the_deferral(
    page: Page, app_url: str, source_dir: Path, db_ready: None
) -> None:
    """Run 1 indexes both docs and summarizes them (phase 118 A2: markdown
    docs are summarized too). One row is then staged by hand —
    ``documents.summary = NULL`` AND its ``is_summary`` chunk deleted — and
    the Sources-page re-sync runs WITH the deferral in place:

    * the run reports the doc as ``unchanged``, not ``added``/``updated``
      (the settle line and the terminal detail both say so — the backfill
      must not masquerade as a re-index, and ``dates_updated``/``summaries``
      stay 0);
    * the document page shows a NON-EMPTY summary again, byte-identical to
      the one run 1 stored — produced from the STORED content through the
      one lazy read the deferral leaves on this branch (LOCKED A2);
    * exactly ONE ``is_summary`` chunk exists for the doc (it was recreated,
      not duplicated)."""
    page.set_default_timeout(30_000)
    _seed_source(source_dir.parent)  # idempotent — the tree starts at its original bytes
    login(page, app_url, next=SOURCES_URL)
    _add_local_source(page, app_url, source_dir)

    # --- run 1: everything is new, both docs get a summary ----------------
    body1 = _sync_from_sources_page(page, app_url)
    _expect_counts_line(page, f"{N_FILES} added")
    assert body1["detail"]["added"] == N_FILES, body1["detail"]
    originals = _summary_map()
    assert set(originals) == {ALPHA_PATH, BACKFILL_PATH}, originals
    stored_summary = originals[BACKFILL_PATH]
    assert stored_summary, "run 1 must have summarized the markdown doc (phase 118 A2)"

    # --- stage the phase-118 case directly --------------------------------
    _stage_null_summary(BACKFILL_PATH)

    # --- run 2: the unchanged re-sync backfills from the STORED content ----
    body2 = _sync_from_sources_page(page, app_url)
    _expect_counts_line(page, f"0 added · {N_FILES} unchanged")
    detail = body2["detail"]
    assert detail["added"] == 0 and detail["updated"] == 0, detail
    assert detail["unchanged"] == N_FILES, detail
    assert detail["errors"] == 0, detail
    # The counts stay honest: a backfill is neither a re-summary of changed
    # content (``summaries``) nor a date refresh (``dates_updated``).
    assert detail["summaries"] == 0, detail
    assert detail["dates_updated"] == 0, detail
    assert detail["pruned"] == 0, detail

    # --- the DB side: the summary is back, byte-identical, one chunk ------
    after = _summary_map()
    assert after[BACKFILL_PATH] == stored_summary, (
        "the backfilled summary must equal what run 1 stored from the same "
        "content (the deterministic mock makes byte-equality a pin)"
    )
    chunks = _summary_chunk_contents(BACKFILL_PATH)
    assert len(chunks) == 1, f"exactly one is_summary chunk, got {len(chunks)}"
    assert chunks[0] == stored_summary

    # --- the owner's surface: the document page shows the summary ---------
    _open_document_page(page, app_url, BACKFILL_PATH)
    expect(page.locator("#doc-title")).to_have_text("Deferred content beta")
    summary_text = page.locator("#doc-content .doc-summary-text")
    expect(summary_text).to_have_count(1)
    expect(summary_text).not_to_be_empty()
    expect(summary_text).to_have_text(stored_summary)
    # The deferral did not thin the doc for the viewer either.
    expect(page.locator("#doc-content")).to_contain_text(BETA_HEAD)
    expect(page.locator("#doc-content")).to_contain_text(BETA_TAIL)


# ---------------------------------------------------------------------------
# 2. An unchanged document's viewer still shows the FULL stored text
# ---------------------------------------------------------------------------


def test_unchanged_document_viewer_shows_full_stored_text(
    page: Page, app_url: str, source_dir: Path, db_ready: None
) -> None:
    """Run 1 indexes, run 2 changes NOTHING (untouched bytes, so every file
    walks the deferred-row fast path) — and the document view then renders
    the FULL stored text: the sentinel at the START and the sentinel at the
    END are both present, every stored line is present, no truncation, no
    empty body. The deferral must not have made the viewer's source row
    thinner."""
    page.set_default_timeout(30_000)
    _seed_source(source_dir.parent)
    login(page, app_url, next=SOURCES_URL)
    _add_local_source(page, app_url, source_dir)

    _sync_from_sources_page(page, app_url)
    _expect_counts_line(page, f"{N_FILES} added")
    _sync_from_sources_page(page, app_url)
    _expect_counts_line(page, f"0 added · {N_FILES} unchanged")

    stored = _stored_content(ALPHA_PATH)
    assert stored == _alpha_text(), "the stored content is the fixture's original bytes"

    _open_document_page(page, app_url, ALPHA_PATH)
    expect(page.locator("#doc-title")).to_have_text("Deferred content alpha")
    body = page.locator("#doc-content .doc-md")
    expect(body).not_to_be_empty()

    # Both ENDS of the file, in the rendered view (the task's pin): a viewer
    # fed by a truncated row would be missing one of them.
    expect(body).to_contain_text(ALPHA_HEAD)
    expect(body).to_contain_text(ALPHA_TAIL)
    expect(body).to_contain_text(ALPHA_MIDDLE)

    # The strong form: EVERY stored line renders (headings render without
    # their leading '#', the fixture carries no list/inline markup, so the
    # comparison is line-for-line on whitespace-collapsed text — a viewer
    # fed by a thinner row, or truncated anywhere in the middle, drops one
    # of these).
    rendered = " ".join(body.inner_text().split())
    for line in stored.splitlines():
        if not line.strip():
            continue
        expected = " ".join(line.lstrip("#").split())
        assert expected in rendered, f"stored line not rendered: {line!r}"


# ---------------------------------------------------------------------------
# 3. A changed file re-indexes under the deferral
# ---------------------------------------------------------------------------


def test_changed_file_reindexes_under_the_deferral(
    page: Page, app_url: str, source_dir: Path, db_ready: None
) -> None:
    """One file is rewritten on disk (new head/tail sentinels, two extra
    sections, clearly different length). The re-sync runs with the row read
    deferred and must behave exactly as before the phase: ``1 updated`` with
    the neighbour ``unchanged``, the viewer shows the NEW text with NO stale
    fragment, the stored content equals the new bytes, the content-chunk
    count moved, and the untouched neighbour's hashes did not. A THIRD sync
    then reports everything ``unchanged`` — the fast path still fires after
    a deferral-mediated write."""
    page.set_default_timeout(30_000)
    _seed_source(source_dir.parent)
    login(page, app_url, next=SOURCES_URL)
    _add_local_source(page, app_url, source_dir)

    _sync_from_sources_page(page, app_url)
    _expect_counts_line(page, f"{N_FILES} added")
    chunks_before = _content_chunk_counts()
    stamps_before = _row_stamp(BETA_PATH)
    assert set(chunks_before) == {ALPHA_PATH, BETA_PATH}, chunks_before

    # --- rewrite alpha on disk (new sentinels, longer, more sections) -----
    (source_dir / ALPHA_PATH).write_text(_revised_alpha_text(), encoding="utf-8")

    body = _sync_from_sources_page(page, app_url)
    _expect_counts_line(page, f"0 added · 1 updated · {N_FILES - 1} unchanged")
    detail = body["detail"]
    assert detail["added"] == 0 and detail["updated"] == 1, detail
    assert detail["unchanged"] == N_FILES - 1, detail
    assert detail["errors"] == 0, detail

    # The stored content is the NEW file bytes — the write path carried the
    # full text through the same session whose row read deferred it.
    assert _stored_content(ALPHA_PATH) == _revised_alpha_text()

    # The content-chunk count moved (the rewrite added two whole sections).
    chunks_after = _content_chunk_counts()
    assert chunks_after[ALPHA_PATH] != chunks_before[ALPHA_PATH], (
        f"chunk count must move on a re-index: {chunks_before} -> {chunks_after}"
    )
    assert chunks_after[ALPHA_PATH] > 0 and chunks_before[ALPHA_PATH] > 0
    # The untouched neighbour row did not move (both hashes byte-still).
    assert _row_stamp(BETA_PATH) == stamps_before

    # The viewer shows the NEW text and NO stale fragment.
    _open_document_page(page, app_url, ALPHA_PATH)
    expect(page.locator("#doc-title")).to_have_text("Deferred content alpha revised")
    rendered = page.locator("#doc-content .doc-md")
    expect(rendered).to_contain_text(REVISED_HEAD)
    expect(rendered).to_contain_text(REVISED_TAIL)
    expect(rendered).not_to_contain_text(ALPHA_HEAD)
    expect(rendered).not_to_contain_text(ALPHA_TAIL)
    expect(rendered).not_to_contain_text(ALPHA_MIDDLE)

    # --- run 3: the fast path still fires after the deferral-mediated write
    _goto_sources(page, app_url)
    body3 = _sync_from_sources_page(page, app_url)
    _expect_counts_line(page, f"0 added · {N_FILES} unchanged")
    assert body3["detail"]["updated"] == 0, body3["detail"]
    assert body3["detail"]["unchanged"] == N_FILES, body3["detail"]