phase: 113_source_chip_quality

All gates green — no defects found; this pass was verification only.

**Phase 113 final verification pass — report**

- Verified (no code changes needed): `select_documents_tiered` cited/related tiering + `select_documents` wrapper, `TurnPlan.related_docs`, `ChatDoneEvent.related` (additive, old payloads parse), `appendRelated` UI row (`.related-doc`, never `.source-chip`), done-frame + restore-path wiring, two settings with validators, `.env.example` entries
- `uv run pytest --cov=app --cov-report=term-missing` → 2422 passed, app/ coverage **99%** (>90% gate)
- `uv run pytest tests/e2e/test_source_chip_quality.py -v --no-cov` (isolated) → 2 passed
- Regression E2E `test_retrieval_quality.py` + `test_honest_deflection.py` + `test_chat_rag.py` + `test_sources_midstream_bug.py` → 17 passed
- `uv run ruff check . && uv run pyright` → clean (0 errors); `bash .agents/validate.sh` → "validation OK"

Completion criteria:
1. Single-doc question → exactly one `.source-chip` (E2E): ✅ passed
2. Weak 2nd doc only in de-emphasized related row, never `.source-chip` (unit + E2E): ✅ passed
3. Deflected turn → zero citation chips, weak hits in related row: ✅ passed
4. Full suite green, coverage >90%, isolated E2E green, lint/types clean: ✅ passed
5. `--no-gpg-sign` commit + phase dir move: left to harness per pass rules (task files already in `complete/`)

No deviations. Next pending phase: `114_embed_question_length`.

app/api/chat.py

@@ -164,7 +164,7 @@ import json
 import logging
 import time
 from collections.abc import AsyncIterator, Sequence
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from typing import Any
 
 from fastapi import APIRouter, Depends
@@ -194,7 +194,7 @@ from app.rag.llm import (
 )
 from app.rag.overview import load_kb_overview
 from app.rag.prompts import build_deflect_prompt, build_high_prompt, history_to_messages
-from app.rag.retriever import RetrievedChunk, retrieve, select_documents, weak_hit_titles
+from app.rag.retriever import RetrievedChunk, retrieve, select_documents_tiered, weak_hit_titles
 from app.rag.scaffolding import ScaffoldingFilter  # phase 71: the streaming filter
 from app.rag.suggestions import derive_suggestions
 from app.schemas import (
@@ -257,8 +257,9 @@ class TurnPlan:
     fts_hits: int  # lexical (OR-tsquery) candidates matched
     deflected: bool
     system_prompt: str
-    docs: list[Document]  # cited sources (weak hits when deflected)
-    suggestions: list[str]  # "Maybe try" chips (deflected turns only)
+    docs: list[Document]  # cited sources (phase 113: the bar-clearing tier)
+    related_docs: list[Document] = field(default_factory=list)  # phase 113
+    suggestions: list[str] = field(default_factory=list)  # "Maybe try" chips (deflected turns only)
     tuning_count: int = 0  # steering notes injected into the system prompt
     #: Hit chunks with ``is_summary`` whose parent document made it into
     #: *docs* (phase 30; per-turn log line ``summary_hits=N``).
@@ -306,13 +307,29 @@ def plan_turn(
     ``summary_hits`` (phase 30) counts the hit chunks with
     ``is_summary`` whose parent document is among the selected
     top-N documents — both the HIGH and the LOW branch record it.
+
+    Phase 113 (the usefulness bar, LOCKED A2): retrieval documents are
+    tiered before either branch — ``docs`` (cited) are the distinct
+    parent documents whose best hit-chunk cosine clears
+    ``settings.source_usefulness_floor`` (at most ``top_n_docs`` — the
+    ceiling, never a quota); ``related_docs`` are the next ranked
+    documents (at most ``related_max_docs``) that did not earn a cited
+    slot. On a deflected turn the weak hits fall to ``related_docs``
+    (the cited tier is usually empty — nothing below the bar earned a
+    citation slot); the LOW prompt itself is unchanged (weak-hit titles
+    only).
     """
     steering = list(notes or [])
     kb_text = (kb_overview or "").strip()
     kb_chars = len(kb_text)
     best_cosine = max((c.cosine for c in chunks), default=0.0)
     fts_hits = sum(1 for c in chunks if c.fts_hit)
-    docs = select_documents(chunks, n=settings.top_n_docs)
+    docs, related_docs = select_documents_tiered(
+        chunks,
+        n=settings.top_n_docs,
+        floor=settings.source_usefulness_floor,
+        related_cap=settings.related_max_docs,
+    )
     selected_ids = {d.id for d in docs}
     summary_hits = sum(1 for c in chunks if c.is_summary and c.document.id in selected_ids)
     lexical_supported = fts_hits > 0 and best_cosine >= settings.lexical_support_floor
@@ -323,6 +340,7 @@ def plan_turn(
             False,
             build_high_prompt(docs, notes=steering, kb_overview=kb_text),
             docs,
+            related_docs,
             [],
             len(steering),
             summary_hits,
@@ -335,6 +353,7 @@ def plan_turn(
         True,
         build_deflect_prompt(titles, notes=steering, kb_overview=kb_text),
         docs,
+        related_docs,
         derive_suggestions(titles, settings.suggestions),
         len(steering),
         summary_hits,
@@ -747,24 +766,36 @@ async def chat(
 
                 # 4. Durable record + required per-turn log line (PLAN §9).
                 #    Phase 37: the agent's read documents join the
-                #    retrieval's — deduped by (source, path), order preserved.
-                #    The combined list feeds query_log.sources and the log
-                #    line (retrieval docs even on deflected turns —
-                #    observability, the phase-113 A3 precedent). Phase 112:
+                #    retrieval's — deduped by (source, path), order
+                #    preserved (the read doc stays last). Phase 113:
+                #    the retrieval now arrives in two tiers — the cited
+                #    docs (``plan.docs``) and the related docs
+                #    (``plan.related_docs``, the scored-but-below-the-bar
+                #    documents). The DURABLE record keeps the full
+                #    retrieval (LOCKED A3: query_log records retrieval,
+                #    not citations — even on deflected turns, where the
+                #    weak hits live in the related tier). Phase 112:
                 #    done.sources is the CITATION surface — it carries the
-                #    combined list on grounded turns and [] on deflected
-                #    ones (a deflected answer cites nothing; the weak hits
-                #    stay in the durable record). A cancelled turn (the
-                #    generator closed by the consumer) never reaches this
-                #    step — no query_log row.
+                #    cited docs + the agent-read docs on grounded turns
+                #    and [] on deflected ones (a deflected answer cites
+                #    nothing; the weak hits stay in the durable record).
+                #    A cancelled turn (the generator closed by the
+                #    consumer) never reaches this step — no query_log row.
                 cited_docs: list[Document] = []
-                seen: set[tuple[str, str]] = set()
+                cited_seen: set[tuple[str, str]] = set()
                 for doc in [*plan.docs, *holder.read_docs]:
+                    key = (doc.source, doc.path)
+                    if key not in cited_seen:
+                        cited_seen.add(key)
+                        cited_docs.append(doc)
+                record_docs: list[Document] = []
+                seen: set[tuple[str, str]] = set()
+                for doc in [*plan.docs, *plan.related_docs, *holder.read_docs]:
                     key = (doc.source, doc.path)
                     if key not in seen:
                         seen.add(key)
-                        cited_docs.append(doc)
-                source_paths = [f"{d.source}/{d.path}" for d in cited_docs]
+                        record_docs.append(doc)
+                source_paths = [f"{d.source}/{d.path}" for d in record_docs]
                 total_ms = int((time.monotonic() - started) * 1000)
                 try:
                     with SessionLocal() as log_db:
@@ -810,18 +841,31 @@ async def chat(
                 # as a citation; the weak hits are scored docs, not
                 # citations). The retrieval stays durably recorded above
                 # (query_log.sources + the log line — observability
-                # unchanged); the phase-113 related-doc tier is the home
-                # for the weak hits' visibility.
+                # unchanged).
+                # Phase 113 (A2/A4): done.related carries the related
+                # tier — the scored documents that did not clear the
+                # usefulness bar (deduped against the cited list, the
+                # same (source, path) pattern as cited_docs: an agent-
+                # read related doc is a citation, never a "nearby doc"
+                # — and capped by related_max_docs in the tiering). The
+                # UI renders it as the de-emphasized related-docs row,
+                # never a citation chip; old clients ignore the field.
                 cited_refs: list[SourceRef] = []
                 if not plan.deflected:
                     cited_refs = [
                         SourceRef(source=d.source, path=d.path, title=d.title)
                         for d in cited_docs
                     ]
+                related_refs = [
+                    SourceRef(source=d.source, path=d.path, title=d.title)
+                    for d in plan.related_docs
+                    if (d.source, d.path) not in cited_seen
+                ]
                 yield sse_event(
                     ChatDoneEvent(
                         deflected=plan.deflected,
                         sources=cited_refs,
+                        related=related_refs,
                         suggestions=plan.suggestions,
                     ).model_dump()
                 )