# Story: Document Summaries (lite-model summaries for non-markdown docs)

**Phase:** `30_document_summaries.md` · **E2E:** `tests/e2e/test_document_summaries.py`

## Narrative

As **any user of Reese**, I ask questions about notes that live in
machine-formatted files (yaml, json, py, txt). Raw flags and keys embed
badly, so retrieval misses those documents. I want a small model to
**analyze every non-markdown document at import time** and store a
natural-language summary with a pointer back to the source. When
retrieval hits a summary, Reese follows the pointer and answers from the
**full source document** — the summary is a retrieval target, never the
answer.

- **Given** a knowledge base containing non-markdown A9 documents (yaml,
  yml, json, py, txt)
- **When** I run `import_docs` and then ask a question those documents
  answer
- **Then** each non-markdown document carries a `lite`-model summary
  (`documents.summary` plus one embedded `is_summary` chunk at position
  −1), markdown documents carry neither, and a question whose best match
  is a summary chunk gets an answer grounded in the **full source
  document**, with the hit counted in the per-turn log line
  (`summary_hits=N`).

## Acceptance criteria
1. `BOR_LLM_SUMMARY_MODEL` (default `lite`) + non-streaming
   `LLMClient.chat()` — A5 extended (same OpenAI-compatible endpoint),
   empty/missing content raises `LLMError` (no silent summaries).
2. Migration 0004: `documents.summary TEXT NULL` +
   `chunks.is_summary BOOLEAN NOT NULL DEFAULT FALSE` (reversible,
   integration-tested up/down).
3. `app/rag/summarizer.py`: the `SUMMARY_MODE` prompt (document content
   capped at `BOR_SUMMARY_MAX_CHARS`, overflow marked with the shared
   truncation marker), the lite call, output validation, and the
   **code-deterministic** `Source: <source>/<path>` pointer line (never
   model-generated).
4. Importer: every non-markdown file gets its summary stored + indexed
   (best-effort — a lite failure logs, counts in `summary_errors`, and
   still leaves the document fully indexed); markdown files get neither;
   re-import replaces the old summary chunk (exactly one at a time).
5. `is_summary` flows through the retriever's vector and lexical
   candidate lists and `fuse`; `TurnPlan.summary_hits` counts hit chunks
   with `is_summary` whose parent document made the selected top-N
   context; the per-turn log line records `summary_hits=N` after
   `fts_hits=N` (PLAN §9 extension).
6. A summary hit resolves to its parent through the unchanged
   chunk→document mapping — the LLM receives the full source document
   (A7 revised: never truncated).
7. `.env.example` + README document `BOR_LLM_SUMMARY_MODEL` /
   `BOR_SUMMARY_MAX_CHARS` and the fail-soft behavior; the deterministic
   mock LLM answers `SUMMARY_MODE` with a byte-stable 24-token digest.
8. Unit + integration green, `app/` coverage >90%, story E2E green in
   isolation, `ruff` + `pyright` clean, one `--no-gpg-sign` commit.

## Playwright Mapping Rule
`tests/e2e/test_document_summaries.py` — one story, one file, run in
isolation. It imports the story-dedicated fixture KB
(`tests/fixtures/summary_kb/`: one yaml document with a tail sentinel +
one markdown control document) through the real importer against the
deterministic mock LLM, asserts the yaml document's import state (one
embedded `is_summary` chunk at position −1 whose text is the byte-stable
mock digest + the deterministic `Source:` pointer), asserts the summary
chunk is the yaml document's best fused chunk, then asks the question
whose best yaml match is that summary chunk and asserts the rendered
answer quotes the document's tail sentinel `RESE-SUMMARY-SENTINEL-7f3a`
(only possible if the full source document — not the summary digest —
reached the LLM), the source chip cites the yaml path, the turn is not
deflected, and the markdown control document carries no summary chunk.