phase: 118_summary_seed_context

**Phase 118 final verification pass — complete.** All criteria verified; 4 pre-existing defects found and fixed. - **Verified:** summary-seed wiring (`select_suggested` top-5 no-floor → summary blocks, no full text in HIGH prompt), all-doc markdown summaries + NULL backfill (`summary_backfilled`, no `sources_meta` bump), `read` adds full text with `read_docs`-only dedupe, `done.sources` = suggested+read / durable record = suggested+related+read + `suggested=N` log line (seen live in E2E), byte-locked PERSONA/LOW/TOOLS_SECTION, battery gate PASS recorded in `TOOL_CALLING_TESTING.md` §10 (turbo 2026-09-16: 1/2/4 GREEN, cond-3 reported 9/10 per A7, contract 21/21, caps 0). - **Defects fixed (all pre-existing, none phase-118):** ① `ChatMessage` schema missing the phase-113 `related` key → `extra="forbid"` 422'd every done-time auto-save of grounded turns with a related tier, leaving `message_count=1` (root cause of `test_share_chat` 3F; browser-level instrumentation proved the PUT 422) — added the field + unit/integration pins; ② `test_theme_semantic_completion` pins stale vs phase-117 debox (border/chip removed) — re-targeted to assert border/chip *absence*; ③ `test_header_consistency` `<26`px pin red on 26.125px native date-input line — bound relaxed to `<34` (wrap-detection intent kept); ④ `test_navbar_refresh` bor.chat.v1 key set updated for `related`. - **Test/lint/coverage:** `uv run pytest --cov=app --cov-report=term-missing` → **2506 passed, app/ 99%** (>90%); `uv run ruff check . && uv run pyright` → clean, 0 errors. - **E2E:** new story suite in isolation → **2 passed**; full 103-suite matrix sweep (each isolated) → **all 103 green** after the fixes; `test_share_chat` 4 passed, `test_theme_semantic_completion` 8 passed, `test_header_consistency` 3 passed, `test_navbar_refresh` 7 passed. - **Deviations:** none from LOCKED decisions. Note: orphaned diagnostic uvicorn processes briefly made E2E sessions exercise stale code — killed and re-verified; a sweep-regenerated tracked screenshot was restored. No commits made (harness commits). - **Completion criteria:** all 7 ✅ (commit/phase-move is the harness's step). - **Next pending phase:** none — `todo/` holds only this phase's overview pending the harness move.
2026-09-16 06:57:49 -04:00
parent 21aad84a6d
commit 9820c361b0
80 changed files with 4690 additions and 1302 deletions
@@ -4,8 +4,10 @@ The walk tests are pure filesystem (``tmp_path``); the delta and summary
 tests run against the local compose Postgres (preferred — a real vector
 table), skipping with clear instructions when the stack is not up.

-Summaries (phase 30): non-markdown files get a ``lite``-model summary via
-the fake's deterministic ``chat`` (``"Summary of <first token>"``); the
+Summaries (phase 30; phase 118, A2: every file, markdown included):
+every file gets a ``lite``-model summary via the fake's deterministic
+``chat`` (``"Summary of <first token>"``); an unchanged doc whose summary
+is NULL is backfilled on the next run (``summary_backfilled``); the
 sentinel word ``SUMMARY-BLOWUP`` makes ``chat`` raise :class:`LLMError`
 for the fail-soft path.
 """
@@ -13,10 +15,12 @@ from __future__ import annotations

 import asyncio
 import logging
+from datetime import UTC, datetime
 from pathlib import Path

 import pytest
 from sqlalchemy import func, select
+from sqlalchemy.orm import Session

 import app.rag.importer as importer
 from app.config import Settings
@@ -29,7 +33,7 @@ from app.rag.importer import (
    iter_importable_files,
    match_extension,
 )
-from app.rag.llm import EmbeddingError
+from app.rag.llm import EmbeddingError, LLMError
 from tests.fakes import FakeEmbedder

 #: The original seven A9 formats as dotted suffixes (pre-phase-47 default
@@ -47,6 +51,15 @@ class _PoisonEmbedder(FakeEmbedder):
        return await super().embed(texts)


+class _FailingChatEmbedder(FakeEmbedder):
+    """A ``lite`` model that always fails (drives the summary fail-soft
+    path — including the phase-118 backfill)."""
+
+    async def chat(self, messages: list[dict[str, str]], model: str | None = None) -> str:
+        self.chat_calls.append(list(messages))
+        raise LLMError("simulated lite-model failure (test sentinel)")
+
+
 class _CapEmbedder(FakeEmbedder):
    """Simulates the endpoint's ~1024-token input cap at ~1.1 chars/token:
    any single text over 1000 chars is rejected (URL-dense worst case)."""
@@ -266,16 +279,19 @@ def test_added_then_unchanged_then_updated_then_pruned(db, tmp_path: Path) -> No
    try:
        s1 = asyncio.run(import_sources([root], llm, session=db))
        assert (s1.files, s1.added, s1.unchanged, s1.updated, s1.pruned) == (2, 2, 0, 0, 0)
-        # a.md has two sections (2 chunks), b.md one (1 chunk).
+        # a.md has two sections (2 chunks), b.md one (1 chunk) — content
+        # chunks only; the ``is_summary`` chunks live in ``summaries``.
        assert s1.chunks == 3
-        # Embeddings are stored with the configured dimension.
+        assert s1.summaries == 2  # phase 118 (A2): markdown is summarized too
+        # Embeddings are stored with the configured dimension: 3 content
+        # chunks + 2 ``is_summary`` chunks (one per doc, phase 118 A2).
        n = db.scalar(
            select(func.count())
            .select_from(Chunk)
            .join(Document, Document.id == Chunk.document_id)
            .where(Document.source == root.name)
        )
-        assert n == 3
+        assert n == 5
        for c in db.scalars(
            select(Chunk)
            .join(Document, Document.id == Chunk.document_id)
@@ -300,14 +316,15 @@ def test_added_then_unchanged_then_updated_then_pruned(db, tmp_path: Path) -> No
        assert db.scalar(
            select(Document).where(Document.source == root.name, Document.path == "a.md")
        ) is None
-        # Chunks of the pruned document are gone (FK cascade).
+        # Chunks of the pruned document are gone (FK cascade); b.md's
+        # content chunk + its ``is_summary`` chunk survive (phase 118 A2).
        n_after = db.scalar(
            select(func.count())
            .select_from(Chunk)
            .join(Document, Document.id == Chunk.document_id)
            .where(Document.source == root.name)
        )
-        assert n_after == 1
+        assert n_after == 2
    finally:
        _cleanup_source(db, root.name)

@@ -438,8 +455,11 @@ def test_chunk_positions_and_titles(db, tmp_path: Path) -> None:
            select(Document).where(Document.source == root.name, Document.path == "multi.md")
        )
        assert doc is not None
-        positions = sorted(c.position for c in doc.chunks)
-        assert positions == list(range(len(doc.chunks))) and len(doc.chunks) >= 2
+        # 0-based CONTENT positions (the ``is_summary`` chunk sits at −1,
+        # phase 30/118).
+        content = [c for c in doc.chunks if not c.is_summary]
+        positions = sorted(c.position for c in content)
+        assert positions == list(range(len(content))) and len(content) >= 2
    finally:
        _cleanup_source(db, root.name)

@@ -535,7 +555,7 @@ def test_quadlet_and_j2_files_get_stem_titles_and_per_format_counts(
        _cleanup_source(db, root.name)


-# ---------- phase 30: lite-model summaries for non-markdown files ----------
+# ---------- phase 30: lite-model summaries (phase 118: every file) ----------


 def test_non_markdown_file_gets_stored_and_indexed_summary(db, tmp_path: Path) -> None:
@@ -571,9 +591,11 @@ def test_non_markdown_file_gets_stored_and_indexed_summary(db, tmp_path: Path) -
        _cleanup_source(db, root.name)


-def test_markdown_file_never_gets_summary(db, tmp_path: Path) -> None:
-    """Markdown is already natural language: no summary, no ``is_summary``
-    chunk, and the ``lite`` model is never called."""
+def test_markdown_file_gets_stored_summary(db, tmp_path: Path) -> None:
+    """Phase 118 (A2): markdown is summarized too (the phase-30 exclusion
+    is retired) — ``documents.summary`` is set and one ``is_summary``
+    chunk (position −1, embedded) is indexed alongside the content
+    chunks."""
    root = tmp_path / "mdsrc"
    root.mkdir()
    (root / "note.md").write_text("# Note\n\nmarkdown body\n")
@@ -581,14 +603,240 @@ def test_markdown_file_never_gets_summary(db, tmp_path: Path) -> None:
    try:
        summary = asyncio.run(import_sources([root], llm, session=db))
        assert summary.added == 1
-        assert summary.summaries == 0 and summary.summary_errors == 0
-        assert llm.chat_calls == []  # the model was never asked
+        assert summary.summaries == 1 and summary.summary_errors == 0
+        assert llm.chat_calls  # the lite model WAS asked (phase 118)
        doc = db.scalar(
            select(Document).where(Document.source == root.name, Document.path == "note.md")
        )
        assert doc is not None
-        assert doc.summary is None
-        assert doc.chunks and all(not c.is_summary for c in doc.chunks)
+        assert doc.summary is not None
+        # Deterministic fake reply + the code-appended pointer line.
+        assert doc.summary.startswith("Summary of")
+        assert doc.summary.endswith(f"Source: {root.name}/note.md")
+        schunks = [c for c in doc.chunks if c.is_summary]
+        assert len(schunks) == 1
+        assert schunks[0].position == -1
+        assert schunks[0].content == doc.summary
+        assert schunks[0].embedding is not None and len(schunks[0].embedding) == 768
+        # Content chunks stay 0-based and are never flagged as summaries.
+        content = [c for c in doc.chunks if not c.is_summary]
+        assert sorted(c.position for c in content) == list(range(len(content)))
+    finally:
+        _cleanup_source(db, root.name)
+
+
+# ---------- phase 118 (A2): NULL-summary backfill on the unchanged path ----------
+
+
+def _clear_stored_summary(db: Session, doc: Document) -> None:
+    """Simulate a NULL-summary row (a pre-phase-30 row, or a cleared
+    summary): the content stays, only the summary + its chunk go away."""
+    doc.summary = None
+    for c in [c for c in doc.chunks if c.is_summary]:
+        doc.chunks.remove(c)
+    db.commit()
+
+
+def test_unchanged_doc_with_null_summary_is_backfilled(db, tmp_path: Path) -> None:
+    """Phase 118 (A2): an unchanged doc whose summary is NULL gets a
+    summary-only backfill on the next sync: summary stored + one embedded
+    ``is_summary`` chunk, counted ``summary_backfilled`` — never
+    ``summaries``, never added/updated/pruned, no content re-embed."""
+    root = tmp_path / "bfill"
+    root.mkdir()
+    (root / "note.md").write_text("# Note\n\nmarkdown body\n")
+    llm = FakeEmbedder()
+    try:
+        first = asyncio.run(import_sources([root], llm, session=db))
+        assert (first.added, first.summaries) == (1, 1)
+        doc = db.scalar(
+            select(Document).where(Document.source == root.name, Document.path == "note.md")
+        )
+        assert doc is not None and doc.summary is not None
+        _clear_stored_summary(db, doc)  # the NULL-summary row the backfill targets
+
+        embed_before = len(llm.calls)
+        second = asyncio.run(import_sources([root], llm, session=db))
+        assert (second.added, second.updated, second.pruned) == (0, 0, 0)
+        assert second.unchanged == 1
+        assert second.summary_backfilled == 1
+        assert second.summaries == 0 and second.summary_errors == 0
+        # No content re-embed: exactly one new embed batch, the summary
+        # text only.
+        assert len(llm.calls) == embed_before + 1
+
+        db.expire_all()
+        doc = db.scalar(
+            select(Document).where(Document.source == root.name, Document.path == "note.md")
+        )
+        assert doc is not None
+        assert doc.summary is not None
+        assert llm.calls[-1] == [doc.summary]  # only the backfilled summary
+        schunks = [c for c in doc.chunks if c.is_summary]
+        assert len(schunks) == 1
+        assert schunks[0].position == -1
+        assert schunks[0].content == doc.summary
+        assert schunks[0].embedding is not None and len(schunks[0].embedding) == 768
+        # The content chunk is untouched.
+        content = [c for c in doc.chunks if not c.is_summary]
+        assert len(content) == 1 and content[0].embedding is not None
+    finally:
+        _cleanup_source(db, root.name)
+
+
+def test_unchanged_doc_with_stored_summary_never_resummarizes(db, tmp_path: Path) -> None:
+    """Phase 118 (A2): an unchanged doc that ALREADY has a summary (the
+    third sync of the lifecycle) makes no summary LLM call at all and
+    gains no chunks — owner-edited (non-NULL) summaries are never
+    touched."""
+    root = tmp_path / "noref"
+    root.mkdir()
+    (root / "note.md").write_text("# Note\n\nmarkdown body\n")
+    llm = FakeEmbedder()
+    try:
+        asyncio.run(import_sources([root], llm, session=db))
+        chat_before = len(llm.chat_calls)
+        embed_before = len(llm.calls)
+        chunk_before = db.scalar(
+            select(func.count())
+            .select_from(Chunk)
+            .join(Document, Document.id == Chunk.document_id)
+            .where(Document.source == root.name)
+        )
+        second = asyncio.run(import_sources([root], llm, session=db))
+        assert second.unchanged == 1
+        assert second.summary_backfilled == 0 and second.summaries == 0
+        assert second.summary_errors == 0
+        assert len(llm.chat_calls) == chat_before  # the model was never asked
+        assert len(llm.calls) == embed_before  # no embedding of any kind
+        chunk_after = db.scalar(
+            select(func.count())
+            .select_from(Chunk)
+            .join(Document, Document.id == Chunk.document_id)
+            .where(Document.source == root.name)
+        )
+        assert chunk_after == chunk_before  # no new chunk of any kind
+    finally:
+        _cleanup_source(db, root.name)
+
+
+def test_unchanged_doc_with_empty_string_summary_is_never_backfilled(
+    db, tmp_path: Path
+) -> None:
+    """Phase 118 (A2, strict ``is None``): an empty-string summary is
+    owner-set (phase 57) — the backfill skips it, the ``lite`` model is
+    never called, and the value stays byte-identical."""
+    root = tmp_path / "emptysum"
+    root.mkdir()
+    (root / "note.md").write_text("# Note\n\nmarkdown body\n")
+    llm = FakeEmbedder()
+    try:
+        asyncio.run(import_sources([root], llm, session=db))
+        doc = db.scalar(
+            select(Document).where(Document.source == root.name, Document.path == "note.md")
+        )
+        assert doc is not None
+        doc.summary = ""  # the owner-set empty string (never NULL)
+        db.commit()
+
+        chat_before = len(llm.chat_calls)
+        second = asyncio.run(import_sources([root], llm, session=db))
+        assert second.unchanged == 1
+        assert second.summary_backfilled == 0 and second.summaries == 0
+        assert second.summary_errors == 0
+        assert len(llm.chat_calls) == chat_before  # the model was never asked
+
+        db.expire_all()
+        doc = db.scalar(
+            select(Document).where(Document.source == root.name, Document.path == "note.md")
+        )
+        assert doc is not None
+        assert doc.summary == ""  # byte-identical — never overwritten
+    finally:
+        _cleanup_source(db, root.name)
+
+
+def test_backfill_runs_on_manually_dated_doc_without_touching_the_date(
+    db, tmp_path: Path
+) -> None:
+    """Phase 118 (A2, assumption 7): ``created_at_manual`` protects the
+    DATE only (phase 106, D1) — a manually-dated, NULL-summary doc still
+    gets its backfilled summary, and the stored date stays byte-untouched
+    even though a refresh was due."""
+    root = tmp_path / "manualdate"
+    root.mkdir()
+    (root / "note.md").write_text("# Note\n\nmarkdown body\n")
+    llm = FakeEmbedder()
+    try:
+        asyncio.run(import_sources([root], llm, session=db))
+        doc = db.scalar(
+            select(Document).where(Document.source == root.name, Document.path == "note.md")
+        )
+        assert doc is not None
+        manual = datetime(2020, 5, 4, 12, 0, 0, tzinfo=UTC)
+        doc.created_at = manual
+        doc.created_at_manual = True
+        _clear_stored_summary(db, doc)  # the NULL-summary row the backfill targets
+
+        second = asyncio.run(import_sources([root], llm, session=db))
+        assert second.unchanged == 1
+        assert second.summary_backfilled == 1 and second.summary_errors == 0
+        # A date refresh WAS due (the mtime differs from the 2020
+        # correction) but the manual flag withheld it — the backfill
+        # never touches the date either.
+        assert second.dates_updated == 0
+
+        db.expire_all()
+        doc = db.scalar(
+            select(Document).where(Document.source == root.name, Document.path == "note.md")
+        )
+        assert doc is not None
+        assert doc.summary is not None  # the backfill landed
+        assert doc.created_at == manual  # byte-untouched
+        assert doc.created_at_manual is True
+    finally:
+        _cleanup_source(db, root.name)
+
+
+def test_backfill_failure_is_fail_soft_and_date_still_refreshes(
+    db, tmp_path: Path
+) -> None:
+    """Phase 118 (A2): a backfill whose ``lite`` call fails rolls back
+    its own session work only — ``summary_errors=1``, the doc row
+    untouched — while the UNCHANGED path's date refresh still runs
+    afterwards."""
+    root = tmp_path / "bfillfail"
+    root.mkdir()
+    (root / "note.md").write_text("# Note\n\nmarkdown body\n")
+    llm = FakeEmbedder()
+    try:
+        asyncio.run(import_sources([root], llm, session=db))
+        doc = db.scalar(
+            select(Document).where(Document.source == root.name, Document.path == "note.md")
+        )
+        assert doc is not None
+        _clear_stored_summary(db, doc)  # the NULL-summary row the backfill targets
+        # Force a date drift so the refresh is DUE on this run.
+        doc.created_at = datetime(2020, 1, 1, tzinfo=UTC)
+        db.commit()
+
+        second = asyncio.run(import_sources([root], _FailingChatEmbedder(), session=db))
+        assert second.unchanged == 1
+        assert (second.added, second.updated, second.pruned) == (0, 0, 0)
+        assert second.summary_errors == 1
+        assert second.summary_backfilled == 0 and second.summaries == 0
+
+        db.expire_all()
+        doc = db.scalar(
+            select(Document).where(Document.source == root.name, Document.path == "note.md")
+        )
+        assert doc is not None
+        assert doc.summary is None  # the failed backfill left the row untouched
+        assert not any(c.is_summary for c in doc.chunks)
+        # …but the date refresh ran (the failure only rolled back the
+        # summary's own session work).
+        assert second.dates_updated == 1
+        assert doc.created_at != datetime(2020, 1, 1, tzinfo=UTC)
    finally:
        _cleanup_source(db, root.name)

@@ -720,11 +968,13 @@ def test_import_summary_log_line_includes_summary_counters(
    caplog: pytest.LogCaptureFixture,
 ) -> None:
    """PLAN §9 summary line: the phase-30 counters sit between
-    ``embed_batches`` and ``formats``; the phase-106 date-refresh
-    counter sits between ``summary_errors`` and ``formats``."""
+    ``embed_batches`` and ``formats``; the phase-118 backfill counter
+    sits between ``summary_errors`` and ``dates_updated``; the
+    phase-106 date-refresh counter sits before ``formats``."""
    s = ImportSummary()
    s.files, s.added, s.chunks, s.embed_batches = 3, 3, 5, 4
    s.summaries, s.summary_errors = 2, 1
+    s.summary_backfilled = 1
    s.dates_updated = 0
    s.formats = {"md": 1, "yaml": 2}
    with caplog.at_level(logging.INFO, logger="app.importer"):
@@ -732,8 +982,8 @@ def test_import_summary_log_line_includes_summary_counters(
    line = caplog.records[-1].getMessage()
    assert line == (
        "import: summary files=3 added=3 updated=0 unchanged=0 pruned=0 errors=0 "
-        "chunks=5 embed_batches=4 summaries=2 summary_errors=1 dates_updated=0 "
-        "formats=yaml:2,md:1"
+        "chunks=5 embed_batches=4 summaries=2 summary_errors=1 summary_backfilled=1 "
+        "dates_updated=0 formats=yaml:2,md:1"
    )