brain-of-reese/tests/integration/test_import_fast_path_columns.py

"""Integration: phase 185 — what the UNCHANGED fast path SELECTs (the probe).

**VERDICT: CONFIRMED (task 01) → FIXED (phase 185 task 02,
``defer(Document.content)`` in ``app/rag/importer.py::_doc_row_stmt``).**
The fast path used to SELECT ``content`` for every
file it inspects. Each inspected file is looked up by
``app/rag/importer.py::_index_file`` (``select(Document).where(
Document.source == source, Document.path == rel)``), which is a WHOLE-ENTITY
load: its select list names ``documents.content`` — the full stored markdown,
**15,217** chars on average — next to the metadata the fast path actually
reads. The three sibling loads that share the phase-147 unchanged path (the
PDF branch, the PDF-embedded-image branch, the standalone-image branch)
compile to the same statement, so the whole KB walk pays this for EVERY file
it looks at, not only for the ones that changed. Nothing is released
afterwards: ``app/db.py:30`` builds every session with
``expire_on_commit=False``, so a row the run read — and then dirtied with the
``file_hash`` stamp (phase 184) — stays resident with its markdown until the
run ends.

**What the fix changed (measured by this file, post-fix).** The four fast-path
row loads now share ``_doc_row_stmt``, whose projection is every ``Document``
column EXCEPT ``content``, and the unchanged run issues **7** documents
SELECTs for the same four files: the **4** metadata-only row loads, **1**
primary-key follow-up (``SELECT documents.content AS documents_content …
WHERE documents.id = …``) — paid by the ONE summary-NULL row whose phase-118
backfill genuinely reads the stored markdown — and the same **2** per-source
post-passes. A file that already has a summary still costs exactly **1**
documents SELECT and zero follow-ups (``test_unchanged_file_is_one_select``).

**The incident (owner, 2026-10-15 — the same run phases 183/184 fixed).**
``documents`` held **17,407** rows at ``avg(length(content)) = 15,217``
characters = **252.6 MB of markdown**. The sync inspected ~27k files and
re-indexed **217**; the remaining ~26.8k took the fast path and still hydrated
their markdown to discover that nothing had changed. Phase 183 took ``content``
out of the prune's row (six light columns), phase 184 bounded what the walk
RETAINS (``session.expunge_all()`` every ``BOR_SYNC_SESSION_RESET_FILES``
files); this phase takes the markdown out of what the walk READS per unchanged
file, so a full re-sync's memory scales with row metadata instead of with the
corpus.

**Measured BY THIS FILE (2026-10-15, the dev Postgres; fixture: 4 markdown
files of ~8 KB seeded through the real ``import_sources``, then a SECOND
``import_sources`` over the same tree — the unchanged run — captured with
``before_cursor_execute``, the phase-183 task-01 technique).** PRE-FIX the
unchanged run issued **6** statements over ``documents``:

* **4** per-file ``(source, path)`` row loads — exactly one per inspected
  file, the fast path's ONE lookup, every one of them naming
  ``documents.content`` (the load-bearing pin, flipped by task 02);
* **0** primary-key follow-up SELECTs over ``documents`` — nothing was
  deferred yet, so nothing could be re-read; every file that already has a
  summary costs exactly **1** documents SELECT in total;
* **2** per-source post-pass walks (``_enrich_pending_images`` +
  ``_backfill_image_contexts``, both ``WHERE documents.source = … AND
  documents.is_image IS true``) — twice per RUN, independent of the file
  count, and outside this phase's scope (D1 names exactly the four fast-path
  row loads);
* the one staged row (``documents.summary = NULL``, its stale ``is_summary``
  chunk stamped so replacement is observable) cost **1** documents SELECT
  total — the row load, whose already-hydrated ``content`` the phase-118
  backfill consumed directly. Post-task-02 that row is the ONLY one that pays
  a second, single-column SELECT.

The captured fast-path statement, verbatim for one file **POST-FIX** — every
column of the PRE-FIX list below EXCEPT ``documents.content`` (which sat
between ``title`` and ``content_hash``), so no markdown travels with an
inspected row; the fast path reads ``file_hash``, then ``summary`` /
``created_at`` / ``page_count`` / ``path`` / ``full_path`` — never the text
this projection drops:

```
SELECT documents.id, documents.source, documents.path, documents.full_path,
documents.title, documents.content_hash, documents.file_hash,
documents.indexed_at, documents.created_at, documents.created_at_manual,
documents.summary, documents.is_image, documents.image_path,
documents.image_context, documents.page_count, documents.page_strip_mode
FROM documents
WHERE documents.source = %(source_1)s::VARCHAR AND documents.path =
%(path_1)s::VARCHAR
```

**The PRE-FIX shape** (task 01's evidence — the statement above WITH
``documents.content`` between ``title`` and ``content_hash``, which is what
put ~250 MB of markdown in front of a run that re-indexed 217 of ~27k files)
and **the follow-up a deferred row pays when it genuinely needs the
markdown** — the phase-118 summary-NULL backfill, ``_unchanged_doc`` step 3,
and nothing else on the fast path (measured 2026-10-15 by putting the same
row behind an ad-hoc ``defer(Document.content)`` load and then reading
``doc.content``)::

    SELECT documents.content AS documents_content
    FROM documents
    WHERE documents.id = %(pk_1)s::UUID

That is the whole trade (phase 185 D2 / LOCKED A2): a doc WITH a stored
summary pays ZERO extra queries, a summary-NULL backfill pays ONE per-row
one-time SELECT, and deliberately NO ``raiseload``/``noload`` — those would
turn the backfill into a crash instead of a query.

**What this file does NOT touch:** the two per-source post-pass walks above,
``_prune``'s light row (phase 184), the identity-map cadence (phase 184),
``app/db.py:30``, ``iter_importable_files`` (phase 185 LOCKED A3) and the
``doc.chunks = [...]`` re-embed path (phase 183 LOCKED A2).

**Why phase 194 left these pins UNEDITED (the D5 sweep).** The phase-194
pre-scan is gated on the caller passing ``progress`` — and every
``import_sources`` call in this file passes NO ``progress`` (the statement
capture is the point, so the run needs no hook). The pre-scan therefore
never runs here: the captured fast-path ROW statement above stays the only
statement over ``documents`` per inspected file, the "exactly one
``SELECT`` per unchanged file" pin and the "one PK follow-up for the
summary-NULL backfill" pin are byte-identical, and the pre-scan's own
scalar statement (``SELECT documents.file_hash``, ``_pending_hash_stmt``)
appears nowhere in the capture — it is pinned in
``tests/unit/test_sync_pending_count.py`` and
``tests/integration/test_sync_pending_count.py`` instead, where a
``progress`` caller does exist.
"""
from __future__ import annotations

import asyncio
import logging
import re
from collections.abc import Iterator
from pathlib import Path
from typing import Any, NamedTuple

import pytest
from sqlalchemy import event, func, select, text
from sqlalchemy.orm import Session

from app.config import Settings
from app.db import SessionLocal, engine
from app.models import Chunk, Document
from app.rag.importer import ImportSummary, import_sources
from tests.fakes import FakeEmbedder

# ---------------------------------------------------------------------------
# The fixture: 4 markdown files of ~8 KB, one seed run + one unchanged run
# ---------------------------------------------------------------------------

#: The probe's own source name — nothing else in the KB uses it.
PROBE_SOURCE = "fastpath-probe"

#: Four files: enough that "one row load per file" is arithmetic rather than
#: a single sample, few enough that the seed run is instant.
FILE_COUNT = 4

#: The row whose stored ``summary`` is nulled before the second run — the
#: phase-118 backfill case, the ONE fast-path branch that genuinely needs the
#: stored markdown.
BACKFILL_PATH = "probe-02.md"

#: What the pre-existing ``is_summary`` chunk of that row is stamped with, so
#: "the backfill replaced the chunk" is observable instead of assumed (the
#: replacement is the only thing that can make the chunk read ``expected``).
STALE_SUMMARY = "STALE SUMMARY CHUNK — the backfill must replace this"

#: One repeated sentence, so every file has the same shape (and therefore the
#: same chunk count) as every other.
_SENTENCE = (
    "The homelab runs podman quadlets on the hnb host with a static DNS "
    "zone, nightly restic backups and a systemd timer that probes uptime. "
)

#: ~8 KB per file — production ``content`` averages 15.2 KB, so a whole-entity
#: row load is tens of kilobytes per inspected file and ``content`` is
#: unmistakably the fat column of the row (and stays under
#: ``BOR_SUMMARY_MAX_CHARS``, so the backfill's summary call is uncapped).
CONTENT_CHARS_PER_FILE = 8_000

#: The documents SELECTs of the two per-source post-passes, measured
#: 2026-10-15 — asserted so a future per-file extra SELECT cannot hide among
#: them (and so a post-pass that stopped querying is noticed too).
POST_PASS_DOC_SELECTS = 2

#: **0 pre-phase-185** — the whole-entity row load already carries
#: ``content``, so the phase-118 backfill needs no follow-up query. **1
#: **0 pre-phase-185** — the whole-entity row load already carried
#: ``content``, so the phase-118 backfill needed no follow-up query. **1
#: post-phase-185 (the value now)** — the row load defers ``content``, so the
#: ONE backfill pays exactly ONE lazy ``SELECT documents.content … WHERE
#: documents.id = …``. The arithmetic: the backfilled file costs ``1 row load
#: + EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL`` documents SELECTs, every other
#: unchanged file costs exactly 1 (``test_unchanged_file_is_one_select``).
#: Everything else about the backfill is identical across the flip: the stored
#: summary, its ``is_summary`` chunk, the run's counts and its log words
#: (``test_unchanged_run_keeps_every_row_intact``).
EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL = 1


class Captured(NamedTuple):
    """One executed statement plus its bind parameters.

    The parameters make the probe attributable: the fast-path row load carries
    ``path_1`` (the file it looked up) while the deferred column's lazy
    follow-up carries only the primary key — so "one SELECT per unchanged
    file" and "one follow-up for the backfill" are two different, separately
    asserted claims.
    """

    text: str
    params: dict[str, Any]


class UnchangedRun(NamedTuple):
    """The unchanged run, snapshotted at the moment it ENDED.

    ``captured`` is a copy: the listener stays installed for the rest of the
    test, so the assertions must not race the probe's own read queries.
    """

    captured: list[Captured]
    summary: ImportSummary
    llm: FakeEmbedder

    @property
    def lite_prompts(self) -> list[str]:
        """The user-message text of every ``lite`` call the run made."""
        return [
            str(message.get("content"))
            for call in self.llm.chat_calls
            for message in call
            if message.get("role") == "user"
        ]


def _file_body(n: int) -> str:
    """One ~8 KB markdown document, carrying a per-file sentinel per section."""
    parts = [f"# Probe document {n}\n"]
    section = 0
    while sum(len(p) for p in parts) < CONTENT_CHARS_PER_FILE:
        parts.append(f"\n## Section {section} of probe {n} (fastpath-sentinel-{n})\n\n")
        parts.extend(_SENTENCE for _ in range(4))
        parts.append("\n")
        section += 1
    return "".join(parts)


def _seed_files(tmp: Path) -> Path:
    """``FILE_COUNT`` markdown files under a fresh source root; returns it."""
    root = tmp / PROBE_SOURCE
    root.mkdir(parents=True, exist_ok=True)
    for n in range(FILE_COUNT):
        body = _file_body(n)
        assert 8_000 <= len(body) <= 9_000, len(body)
        (root / f"probe-{n:02d}.md").write_text(body)
    return root


def _probe_llm() -> FakeEmbedder:
    """The deterministic fake embedder/summarizer on the SHIPPED defaults
    (``sync_session_reset_files=200`` — no identity-map release lands inside a
    four-file walk, so nothing but the row loads is in play here)."""
    llm = FakeEmbedder()
    llm.settings = Settings(_env_file=None)  # pyright: ignore[reportCallIssue]
    return llm


def _run_import(root: Path, llm: FakeEmbedder) -> ImportSummary:
    """One real ``import_sources`` pass over *root*, in its own session."""
    session = SessionLocal()
    try:
        return asyncio.run(
            import_sources([root], llm, prune=False, session=session)
        )
    finally:
        session.close()


def _doc_id(db: Session, path: str) -> Any:
    """The id of one probe row (lazy follow-ups are keyed by primary key)."""
    doc_id = db.scalar(
        select(Document.id).where(
            Document.source == PROBE_SOURCE, Document.path == path
        )
    )
    assert doc_id is not None
    return doc_id


def _stage_the_backfill(db: Session, path: str) -> None:
    """The phase-118 state: a row whose stored ``summary`` is NULL while its
    content and ``file_hash`` are current (the direct UPDATE of
    ``tests/integration/test_importer_dates.py`` — exactly what a pre-phase-118
    import left behind).

    The stale ``is_summary`` chunk is stamped rather than deleted: after the
    run a chunk carrying the backfill text can only have got there through
    :func:`app.rag.importer._store_summary`'s replacement rule, which is what
    makes "the backfill ran" a fact rather than an assumption.
    """
    doc_id = _doc_id(db, path)
    db.execute(
        text(
            "UPDATE documents SET summary = NULL WHERE source = :s AND path = :p"
        ),
        {"s": PROBE_SOURCE, "p": path},
    )
    db.execute(
        text("UPDATE chunks SET content = :stale WHERE document_id = :d AND is_summary"),
        {"stale": STALE_SUMMARY, "d": doc_id},
    )
    db.commit()
    db.expunge_all()


def _two_runs(db: Session, tmp_path: Path, captured: list[Captured]) -> UnchangedRun:
    """Seed the KB, then run the UNCHANGED pass with the capture active.

    The seed run's SQL is cleared away at the boundary: this file pins what
    the unchanged walk costs, not what first indexing costs.
    """
    root = _seed_files(tmp_path)
    first = _run_import(root, _probe_llm())
    assert (first.files, first.added, first.unchanged, first.errors) == (
        FILE_COUNT,
        FILE_COUNT,
        0,
        0,
    )
    _stage_the_backfill(db, BACKFILL_PATH)

    captured.clear()
    llm = _probe_llm()
    second = _run_import(root, llm)
    run = UnchangedRun(list(captured), second, llm)

    assert (second.files, second.added, second.unchanged, second.errors) == (
        FILE_COUNT,
        0,
        FILE_COUNT,
        0,
    )
    assert second.chunks == 0  # the fast path re-embeds no content
    assert second.summary_backfilled == 1  # the staged row was backfilled
    return run


# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------


@pytest.fixture(autouse=True)
def clean_kb(db: Session) -> Iterator[None]:
    """``documents`` / ``chunks`` are global state: reset around every test
    (the ``test_sync_api.py`` truncate pattern) and drop whatever the session
    cached, so the next read is the database's truth."""
    db.execute(text("TRUNCATE chunks, documents"))
    db.commit()
    db.expunge_all()
    yield
    db.execute(text("TRUNCATE chunks, documents"))
    db.commit()
    db.expunge_all()


@pytest.fixture()
def statement_capture() -> Iterator[list[Captured]]:
    """Every SQL the engine executes for the duration of ONE test.

    The ``before_cursor_execute`` technique of phase 183 task 01
    (``tests/integration/test_prune_cascade_statements.py``): the column list
    a SELECT uses is invisible in the ORM's Python API and plain to see in the
    statement it compiles, so a capture is the only deterministic way to pin
    what the fast path's row actually contains. Installed on the engine and
    REMOVED in teardown, so no other test of the session ever sees the
    listener (verified by the full ``uv run pytest`` run being green with this
    file in it).
    """
    captured: list[Captured] = []

    def _listener(
        conn: object,  # noqa: ARG001
        cursor: object,  # noqa: ARG001
        statement: str,
        parameters: object,
        context: object,  # noqa: ARG001
        executemany: bool,  # noqa: ARG001
    ) -> None:
        params = dict(parameters) if isinstance(parameters, dict) else {}
        captured.append(Captured(statement, params))

    event.listen(engine, "before_cursor_execute", _listener)
    try:
        yield captured
    finally:
        event.remove(engine, "before_cursor_execute", _listener)


# ---------------------------------------------------------------------------
# Helpers — reading a captured statement
# ---------------------------------------------------------------------------

#: ``… FROM documents`` — case-insensitive, word-bounded.
_FROM_DOCUMENTS_RE = re.compile(r"\bfrom\s+documents\b", re.IGNORECASE)

#: Where the projection ends: SQLAlchemy compiles ``FROM`` onto its own line,
#: and everything after it is a predicate, not a column the row hydrates.
_FROM_RE = re.compile(r"\bFROM\b")

#: The ``content`` COLUMN TOKEN, aliased or plain. The trailing ``\\b`` is what
#: keeps the neighbours out: ``_`` is a WORD character, so the ``documents.content``
#: inside ``documents.content_hash`` has no boundary after the ``t`` and never
#: matches. That is why this is a token regex and not a substring test —
#: ``content_hash`` is in EVERY one of these select lists, and a substring
#: match would report a content load for every row of every run, pre- and
#: post-fix alike.
_CONTENT_TOKEN_RE = re.compile(r"\bdocuments\.content\b")


def doc_selects(captured: list[Captured]) -> list[Captured]:
    """The captured statements that SELECT the ``documents`` table."""
    return [
        c
        for c in captured
        if c.text.lstrip().lower().startswith("select")
        and _FROM_DOCUMENTS_RE.search(c.text)
    ]


def loads_content(statement: Captured | str) -> bool:
    """``True`` when the statement's SELECT LIST names the ``content`` column.

    Only the projection is inspected (see :data:`_FROM_RE`), and only the
    column token matches (see :data:`_CONTENT_TOKEN_RE`).
    """
    text = statement.text if isinstance(statement, Captured) else statement
    from_at = _FROM_RE.search(text)
    projection = text[: from_at.start()] if from_at else text
    return _CONTENT_TOKEN_RE.search(projection) is not None


def test_the_content_matcher_names_the_column_and_nothing_else() -> None:
    """The matcher's own pin — every other assertion here rests on it.

    ``SELECT … documents.content, documents.content_hash …`` is a content load
    because of the FIRST token; ``documents.content_hash`` alone is not. If
    this ever stops holding, the pre-/post-fix assertion above is meaningless
    in BOTH directions, so it is asserted directly rather than trusted.
    """
    assert loads_content(
        "SELECT documents.id, documents.content, documents.content_hash\n"
        "FROM documents\nWHERE documents.id = %(pk_1)s"
    )
    assert loads_content(
        "SELECT documents.content AS documents_content\nFROM documents\n"
        "WHERE documents.id = %(pk_1)s"
    )
    assert not loads_content(
        "SELECT documents.id, documents.content_hash, documents.file_hash\n"
        "FROM documents\nWHERE documents.source = %(source_1)s"
    )
    assert not loads_content(
        "SELECT documents.summary, documents.is_image, documents.page_count\n"
        "FROM documents\nWHERE documents.id = %(pk_1)s"
    )
    # A predicate mention is not a hydration.
    assert not loads_content(
        "SELECT documents.id\nFROM documents\nWHERE documents.content = %(c_1)s"
    )


def _predicate(statement: str) -> str:
    """Everything after the first ``FROM`` — the statement's predicate."""
    from_at = _FROM_RE.search(statement)
    return statement[from_at.end():] if from_at else ""


def row_loads(captured: list[Captured]) -> list[Captured]:
    """The ``(source, path)`` row loads — the fast path's ONE lookup per file.

    Matched on the PREDICATE, not the projection: the four sites of phase 185
    D1 (``_index_file`` plus the PDF / PDF-embedded-image / image branch loads)
    all compile to ``WHERE documents.source = … AND documents.path = …``. The
    two per-source post-pass walks filter on ``documents.is_image`` instead
    and are therefore excluded here — they are not the fast path's row load.
    """
    return [
        c
        for c in doc_selects(captured)
        if "documents.source" in _predicate(c.text)
        and "documents.path" in _predicate(c.text)
    ]


def pk_followups(captured: list[Captured]) -> list[Captured]:
    """Documents SELECTs keyed by PRIMARY KEY — the deferred-column shape.

    A ``defer``-red column loads as ``SELECT documents.content … WHERE
    documents.id = …``: no ``source``/``path`` predicate at all. Pre-fix this
    list is EMPTY (the row was loaded whole, so nothing is deferred); post-fix
    it holds the ONE query the phase-118 backfill pays.
    """
    return [
        c
        for c in doc_selects(captured)
        if "documents.source" not in _predicate(c.text)
        and "documents.path" not in _predicate(c.text)
    ]


def _summary_chunks(db: Session, path: str) -> list[Chunk]:
    """The ``is_summary`` chunks of one document, read by ``document_id`` so
    the parent row's ``content`` is never what this probe reads."""
    return list(
        db.scalars(
            select(Chunk)
            .where(
                Chunk.document_id == _doc_id(db, path),
                Chunk.is_summary.is_(True),
            )
            .order_by(Chunk.id)
        )
    )


# ---------------------------------------------------------------------------
# The probe
# ---------------------------------------------------------------------------


def test_unchanged_run_defers_content_post_fix(
    db: Session, tmp_path: Path, statement_capture: list[Captured]
) -> None:
    """THE PROBE (post-fix): no file the unchanged walk inspects SELECTs its
    markdown.

    Four unchanged files → four ``(source, path)`` row loads, and NONE of them
    names ``documents.content`` in its projection — the fast path reads row
    metadata only (phase 185, ``_doc_row_stmt``). The only follow-up the run
    may pay is the backfill row's, attributed per primary key in
    :func:`test_summary_null_backfill_reads_stored_content`, while
    :func:`test_unchanged_file_is_one_select` pins that every
    summary-bearing file pays none. The rest of the run's documents traffic
    is the two per-source image post-passes, pinned apart so a future
    per-file extra SELECT cannot hide inside the total.

    (Task 01's PRE-fix assertion was the inverse — ``all(loads_content(c))``
    with zero follow-ups — and flipped here exactly as planned. A deferred
    column that is genuinely READ still shows up, as a PK follow-up, so
    neither a re-appeared ``content`` nor a new lazy reader hides here.)
    """
    run = _two_runs(db, tmp_path, statement_capture)

    selects = doc_selects(run.captured)
    loads = row_loads(run.captured)
    assert len(loads) == FILE_COUNT, (
        f"a {FILE_COUNT}-file unchanged walk issued {len(loads)} (source, "
        "path) row loads — the fast path looks each file up exactly once"
    )
    assert len(selects) >= FILE_COUNT

    # Post-fix (phase 185 task 02): no content in the fast-path SELECT.
    assert not any(loads_content(c) for c in loads), (
        "a fast-path row load SELECTs documents.content again — the deferral "
        "in _doc_row_stmt is gone or bypassed, which re-hydrates the corpus "
        "(252.6 MB on the incident box) for a walk that reads metadata only"
    )
    followups = pk_followups(run.captured)
    assert len(followups) == EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL, (
        f"an unchanged run over {FILE_COUNT} files (one with summary IS NULL) "
        f"paid {len(followups)} primary-key follow-up SELECTs over documents "
        f"while EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL is "
        f"{EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL} — the backfill's single "
        "deferred-column read is the only one the fast path may need"
    )
    post_passes = [c for c in selects if c not in loads and c not in followups]
    assert len(post_passes) == POST_PASS_DOC_SELECTS, (
        f"the unchanged run did {len(post_passes)} documents SELECTs beyond "
        f"the {FILE_COUNT} row loads — expected the {POST_PASS_DOC_SELECTS} "
        "per-source image post-passes and nothing else"
    )


def test_unchanged_file_is_one_select(
    db: Session, tmp_path: Path, statement_capture: list[Captured]
) -> None:
    """One documents SELECT per unchanged file — MUST STAY GREEN POST-FIX.

    A file WITH a stored summary reads nothing beyond what its row load
    already carries: the ``file_hash`` comparison, then ``_unchanged_doc``'s
    ``summary`` / ``created_at`` / ``page_count`` bookkeeping. Pre-fix that is
    one whole-entity load; post-fix it must still be exactly ONE — this is the
    guard that ``defer(Document.content)`` does not decay into a lazy-load
    storm (``summary``, then ``created_at``, then ``page_count`` … one SELECT
    apiece), which would trade 15 KB of residency for five round trips.
    """
    run = _two_runs(db, tmp_path, statement_capture)

    for n in range(FILE_COUNT):
        path = f"probe-{n:02d}.md"
        if path == BACKFILL_PATH:
            continue  # asserted separately, with its documented extra
        one_file = [c for c in row_loads(run.captured) if c.params.get("path_1") == path]
        assert one_file, (
            f"{path} was walked with no (source, path) row load carrying "
            "path_1 — the row-load statement shape changed, so this probe is "
            "attributing nothing"
        )
        assert len(one_file) == 1, (
            f"{path} cost {len(one_file)} documents row loads — the fast path "
            "looks a file up ONCE"
        )
        doc_id = str(_doc_id(db, path))
        followups = [
            c for c in pk_followups(run.captured) if str(c.params.get("pk_1")) == doc_id
        ]
        assert followups == [], (
            f"{path} paid {len(followups)} deferred-column follow-up SELECT(s) "
            "despite having a stored summary — nothing on its fast path may "
            "read content, so a follow-up means a new reader crept in"
        )


def test_summary_null_backfill_reads_stored_content(
    db: Session, tmp_path: Path, statement_capture: list[Captured]
) -> None:
    """The ONE branch that genuinely needs the markdown: ``summary IS NULL``.

    ``_unchanged_doc`` step 3 (phase 118, A2) backfills the summary FROM THE
    STORED ``doc.content`` — never by re-reading the file, re-extracting or
    re-describing. So the probe pins the behaviour AND its cost:

    * the run backfills exactly one summary (``summary_backfilled=1`` while
      ``summaries`` stays ``0`` — a backfill is not a KB-change signal), the
      stored summary comes back and its single ``is_summary`` chunk is the
      REPLACED one (the pre-existing chunk still carried :data:`STALE_SUMMARY`
      when the run started, and replacement is the only way it changes);
    * the text the ``lite`` call was given IS the stored markdown (~8 KB, with
      the file's own sentinel in every section) — the bytes travelled from
      ``documents.content``, not from a re-read of the file;
    * the documents SELECT count for that file is ``1 +
      EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL``: the row load plus, once task
      02 defers ``content``, exactly one lazy follow-up keyed by primary key
      (D2 — a per-row, one-time, already-rare cost, accepted precisely instead
      of a ``raiseload`` that would crash this branch where it should query).
    """
    run = _two_runs(db, tmp_path, statement_capture)
    n = int(BACKFILL_PATH.removeprefix("probe-").removesuffix(".md"))
    expected = f"Summary of #\nSource: {PROBE_SOURCE}/{BACKFILL_PATH}"

    # The backfill happened, from the stored content.
    assert run.summary.summary_backfilled == 1
    assert run.summary.summaries == 0
    stored = db.scalar(
        select(Document.content).where(
            Document.source == PROBE_SOURCE, Document.path == BACKFILL_PATH
        )
    )
    assert stored is not None and len(stored) >= 8_000
    assert f"fastpath-sentinel-{n}" in stored

    doc = db.scalar(
        select(Document).where(
            Document.source == PROBE_SOURCE, Document.path == BACKFILL_PATH
        )
    )
    assert doc is not None
    assert doc.summary == expected
    assert doc.file_hash is not None

    chunks = _summary_chunks(db, BACKFILL_PATH)
    assert [c.content for c in chunks] == [expected], (
        "the summary chunk was not replaced by the backfill — a stale chunk "
        "survived, or more than one is_summary chunk exists"
    )
    assert chunks[0].position == -1
    assert chunks[0].embedding is not None

    # The lite call got the STORED markdown, sentinel and all — uncapped.
    assert run.lite_prompts, "the backfill made no lite-model call"
    assert any(stored == prompt for prompt in run.lite_prompts), (
        "the summary model was not fed documents.content verbatim — the "
        "phase-118 backfill reads something other than the stored markdown"
    )

    # And the cost of that branch, as a documented constant.
    doc_id = str(_doc_id(db, BACKFILL_PATH))
    loads = [c for c in row_loads(run.captured) if c.params.get("path_1") == BACKFILL_PATH]
    followups = [c for c in pk_followups(run.captured) if str(c.params.get("pk_1")) == doc_id]
    assert len(loads) == 1
    assert len(followups) == EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL, (
        f"the summary-NULL backfill paid {len(followups)} lazy content "
        f"SELECT(s) while EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL is "
        f"{EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL} — 0 while the row load "
        "carries content (pre task 02), 1 once content is deferred"
    )
    assert all(loads_content(c) for c in followups)


def test_unchanged_run_keeps_every_row_intact(
    db: Session, tmp_path: Path, caplog: pytest.LogCaptureFixture
) -> None:
    """The observable contract the deferral must not move (phase 185 D4).

    The reference for task 02's byte-identity claim: the unchanged run's whole
    ``ImportSummary`` shape and the whole KB after it — four unchanged files,
    one backfill, no re-embed, no date refresh, every file still holding its
    full ~8 KB of content, its summary and all of its chunks. ``changed_paths``
    stays empty (the backfill is not a document change, so the folder-summary
    regeneration never sees it) and no source is pruned.
    """
    root = _seed_files(tmp_path)
    first = _run_import(root, _probe_llm())
    assert (first.files, first.added, first.summary_backfilled) == (
        FILE_COUNT,
        FILE_COUNT,
        0,
    )
    content_chunks_after_seed = db.scalar(
        select(func.count()).select_from(Chunk).where(Chunk.is_summary.is_(False))
    )
    assert content_chunks_after_seed is not None and content_chunks_after_seed > FILE_COUNT
    _stage_the_backfill(db, BACKFILL_PATH)

    caplog.clear()
    with caplog.at_level(logging.INFO, logger="app.importer"):
        second = _run_import(root, _probe_llm())
    messages = [r.getMessage() for r in caplog.records if r.name == "app.importer"]

    # The phase-147/118 log words, verbatim (D4: task 02 may change WHICH
    # COLUMNS travel, never what the run says it did). ``import: summary
    # source=`` is the FRESH-summary line — it must not appear on an unchanged
    # run, where only the backfill's own wording is allowed.
    assert sum(m.startswith("import: unchanged ") for m in messages) == FILE_COUNT
    assert sum(m.startswith("import: summary-backfill ") for m in messages) == 1
    assert not [m for m in messages if m.startswith("import: summary source=")]
    assert not [m for m in messages if m.startswith("import: date-refreshed ")]
    assert not [m for m in messages if m.startswith("import: added ")]
    assert not [m for m in messages if m.startswith("import: updated ")]

    assert (second.files, second.added, second.updated) == (FILE_COUNT, 0, 0)
    assert (second.unchanged, second.pruned, second.errors) == (FILE_COUNT, 0, 0)
    assert second.summary_backfilled == 1
    assert second.summaries == 0
    assert second.summary_errors == 0
    assert second.dates_updated == 0
    assert second.chunks == 0
    assert second.changed_paths == []
    assert second.formats == {"md": FILE_COUNT}
    assert second.format_counts() == "md:4"

    # Nothing was re-embedded, nothing was lost.
    assert db.scalar(
        select(func.count()).select_from(Chunk).where(Chunk.is_summary.is_(False))
    ) == content_chunks_after_seed
    assert db.scalar(
        select(func.count())
        .select_from(Document)
        .where(Document.source == PROBE_SOURCE, Document.summary.is_not(None))
    ) == FILE_COUNT
    assert all(
        len(c) >= 8_000
        for c in db.scalars(
            select(Document.content).where(Document.source == PROBE_SOURCE)
        )
    )
    # Exactly one summary chunk per document (the backfill replaced, not added).
    assert db.scalar(
        select(func.count()).select_from(Chunk).where(Chunk.is_summary.is_(True))
    ) == FILE_COUNT