71 lines
3.8 KiB
Markdown
71 lines
3.8 KiB
Markdown
# Story: Document Summaries (lite-model summaries for non-markdown docs)
|
||
|
||
**Phase:** `30_document_summaries.md` · **E2E:** `tests/e2e/test_document_summaries.py`
|
||
|
||
## Narrative
|
||
|
||
As **any user of Reese**, I ask questions about notes that live in
|
||
machine-formatted files (yaml, json, py, txt). Raw flags and keys embed
|
||
badly, so retrieval misses those documents. I want a small model to
|
||
**analyze every non-markdown document at import time** and store a
|
||
natural-language summary with a pointer back to the source. When
|
||
retrieval hits a summary, Reese follows the pointer and answers from the
|
||
**full source document** — the summary is a retrieval target, never the
|
||
answer.
|
||
|
||
- **Given** a knowledge base containing non-markdown A9 documents (yaml,
|
||
yml, json, py, txt)
|
||
- **When** I run `import_docs` and then ask a question those documents
|
||
answer
|
||
- **Then** each non-markdown document carries a `lite`-model summary
|
||
(`documents.summary` plus one embedded `is_summary` chunk at position
|
||
−1), markdown documents carry neither, and a question whose best match
|
||
is a summary chunk gets an answer grounded in the **full source
|
||
document**, with the hit counted in the per-turn log line
|
||
(`summary_hits=N`).
|
||
|
||
## Acceptance criteria
|
||
1. `BOR_LLM_SUMMARY_MODEL` (default `lite`) + non-streaming
|
||
`LLMClient.chat()` — A5 extended (same OpenAI-compatible endpoint),
|
||
empty/missing content raises `LLMError` (no silent summaries).
|
||
2. Migration 0004: `documents.summary TEXT NULL` +
|
||
`chunks.is_summary BOOLEAN NOT NULL DEFAULT FALSE` (reversible,
|
||
integration-tested up/down).
|
||
3. `app/rag/summarizer.py`: the `SUMMARY_MODE` prompt (document content
|
||
capped at `BOR_SUMMARY_MAX_CHARS`, overflow marked with the shared
|
||
truncation marker), the lite call, output validation, and the
|
||
**code-deterministic** `Source: <source>/<path>` pointer line (never
|
||
model-generated).
|
||
4. Importer: every non-markdown file gets its summary stored + indexed
|
||
(best-effort — a lite failure logs, counts in `summary_errors`, and
|
||
still leaves the document fully indexed); markdown files get neither;
|
||
re-import replaces the old summary chunk (exactly one at a time).
|
||
5. `is_summary` flows through the retriever's vector and lexical
|
||
candidate lists and `fuse`; `TurnPlan.summary_hits` counts hit chunks
|
||
with `is_summary` whose parent document made the selected top-N
|
||
context; the per-turn log line records `summary_hits=N` after
|
||
`fts_hits=N` (PLAN §9 extension).
|
||
6. A summary hit resolves to its parent through the unchanged
|
||
chunk→document mapping — the LLM receives the full source document
|
||
(A7 revised: never truncated).
|
||
7. `.env.example` + README document `BOR_LLM_SUMMARY_MODEL` /
|
||
`BOR_SUMMARY_MAX_CHARS` and the fail-soft behavior; the deterministic
|
||
mock LLM answers `SUMMARY_MODE` with a byte-stable 24-token digest.
|
||
8. Unit + integration green, `app/` coverage >90%, story E2E green in
|
||
isolation, `ruff` + `pyright` clean, one `--no-gpg-sign` commit.
|
||
|
||
## Playwright Mapping Rule
|
||
`tests/e2e/test_document_summaries.py` — one story, one file, run in
|
||
isolation. It imports the story-dedicated fixture KB
|
||
(`tests/fixtures/summary_kb/`: one yaml document with a tail sentinel +
|
||
one markdown control document) through the real importer against the
|
||
deterministic mock LLM, asserts the yaml document's import state (one
|
||
embedded `is_summary` chunk at position −1 whose text is the byte-stable
|
||
mock digest + the deterministic `Source:` pointer), asserts the summary
|
||
chunk is the yaml document's best fused chunk, then asks the question
|
||
whose best yaml match is that summary chunk and asserts the rendered
|
||
answer quotes the document's tail sentinel `RESE-SUMMARY-SENTINEL-7f3a`
|
||
(only possible if the full source document — not the summary digest —
|
||
reached the LLM), the source chip cites the yaml path, the turn is not
|
||
deflected, and the markdown control document carries no summary chunk.
|