phase: 191_source_name_labels_tags
All gates re-verified against a stable tree. Report: **Verified (all 6 tasks in `complete/`; this was the final pass)** - `SourceLabel`+alembic `0030` (dev DB `alembic current` = `0030 (head)`, table present, 0 rows); admin-only `GET/PUT/DELETE /api/source-labels` (`require_admin` router-wide, `resolve_source_name`, partial-update caps/404s); Sources-page "Names, labels and tags" card (read-only derived-name `<code>`, per-row `saveInFlight`, 404 note that survives the settle, chips reuse `.git-source-hidden-count`); `question_name_hits` D2 rule; ONE `all_labels` read per turn in `chat.py` feeding both the `source_name_hit` flag and the prompt surfaces; `source_name_hit_bonus=0.005` in `_selection_order`; ` name=`/` label=` in `build_high_prompt` + `build_overview_prompt(names=…)`. - **Completion criteria:** E2E `tests/e2e/test_source_name_labels.py` ✅ 7 passed (arms a–e: editor all kinds, control/named question, rename-never-breaks-identity, 404 note, reset) · full suite ✅ · coverage ✅ · README/AGENTS ✅ · phase dir in `complete/` ✅ · commit left to the harness. - **Gates:** `uv run pytest --cov=app --cov-report=term-missing` → **exit 0, TOTAL 7160 stmts / 16 missed = 99%** (>90%) · `uv run pytest tests/e2e/test_source_name_labels.py -v --no-cov` → **7 passed, exit 0** · regression E2E isolated, exit 0: `test_document_viewer`, `test_admin_auth`, `test_api_tokens`, `test_git_sources_admin`, `test_kb_tree`, `test_local_directory_sources` · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors. - **Battery (task 04 criterion):** `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` at default vs `BOR_SOURCE_NAME_HIT_BONUS=0` → **`diff` empty (line-for-line identical)**, matching the recorded arms (dev KB is empty, so every question deflects; the observable boost is unit/E2E-pinned). - **Documentation:** README — fixed the `/git-sources.html` rows (nav label is **Sources**) and cross-linked the phase-191 card from that section; full audit otherwise clean: all 86 `BOR_*` settings documented with defaults matching `app/config.py`, 0 broken file/dir references and 0 broken anchors in README+AGENTS, AGENTS rule 17 re-checked line-by-line against the code. - **Notable / deviations:** (1) a **concurrent pipeline process** (`phased-execution/progress.mjs` for `191_…_00_phase`) was mutating this same working tree mid-pass — it rewrote `tests/e2e/test_source_name_labels.py` (11→7 tests), `frontend/assets/git-sources.js`, `tests/unit/test_source_labels_editor_ui.py` and `AGENTS.md` at 05:27; every gate above was (re-)run after those edits with the tree no source file newer than the green full run. (2) The task-06 report/phase text claims regression suites `test_chat_token_gate.py`, `test_git_sources_page.py`, `test_rag_browser.py` are green — **those files have never existed in git history** (those runs exit 4); I ran the real equivalents listed above instead (phase files are not mine to edit). (3) **No live changes**: the VM DB already has `source_labels` (0 rows) but the deployed app predates this uncommitted code, so deploying `/api/source-labels` to the live service is the owner's post-commit step, not part of this phase's criteria. - **Next pending phase:** none — `191_source_name_labels_tags` is the only entry in `.agents/phases/todo/`.
This commit is contained in:
@@ -13,6 +13,7 @@ import json
|
||||
import uuid
|
||||
from collections.abc import Iterator, Sequence
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import pytest
|
||||
@@ -27,6 +28,7 @@ from app.rag.agent import AGENT_TOOLS, SHOW_TEXT_TOOL
|
||||
from app.rag.llm import StreamPiece, ToolCallPiece
|
||||
from app.rag.prompts import SUGGEST_INTRO, build_deflect_prompt
|
||||
from app.rag.retriever import RetrievedChunk, weak_hit_titles
|
||||
from app.rag.source_labels import SourceLabelView
|
||||
from app.rag.suggestions import MAX_SUGGESTIONS, derive_suggestions
|
||||
from tests.conftest import ADMIN_PASSWORD
|
||||
|
||||
@@ -1817,3 +1819,114 @@ def test_deflected_turn_resolves_nothing_and_logs_zero(
|
||||
done = [f for f in frames if f.get("type") == "done"]
|
||||
assert done and done[0]["deflected"] is True
|
||||
assert "answer_text" not in done[0]
|
||||
|
||||
|
||||
# ---------- phase 191 (D4, task 05): the turn's prompt learns the name ----------
|
||||
|
||||
|
||||
def _label_view(
|
||||
source: str = "Homelab",
|
||||
display_name: str | None = None,
|
||||
label: str | None = None,
|
||||
) -> SourceLabelView:
|
||||
return SourceLabelView(source=source, display_name=display_name, label=label, tags=())
|
||||
|
||||
|
||||
def test_plan_turn_high_prompt_carries_the_source_name() -> None:
|
||||
"""The HIGH branch hands the label views to ``build_high_prompt``: the
|
||||
suggested block gains `` name="…"`` / `` label="…"`` right after
|
||||
``source="…"`` — the model sees the name→docs mapping."""
|
||||
doc = _doc("Gitea Upgrade", "GITEA_DOC_CONTENT", summary="Upgrading Gitea.")
|
||||
names = {"Homelab": _label_view(display_name="the git server", label="self-hosted forge")}
|
||||
plan = chat_api.plan_turn(
|
||||
[_chunk(doc, 0.90, cosine=0.90)],
|
||||
_settings(threshold=0.30),
|
||||
names=names,
|
||||
)
|
||||
assert plan.deflected is False
|
||||
assert 'source="Homelab" name="the git server" label="self-hosted forge" path=' in (
|
||||
plan.system_prompt
|
||||
)
|
||||
assert "Upgrading Gitea." in plan.system_prompt # the body is untouched
|
||||
|
||||
|
||||
def test_plan_turn_without_names_is_byte_identical() -> None:
|
||||
"""THE kill pin on the call-through: ``names`` omitted, ``None`` and
|
||||
``{}`` each build the exact pre-phase prompt (and every other TurnPlan
|
||||
field is identical)."""
|
||||
doc = _doc("Gitea Upgrade", "GITEA_DOC_CONTENT", summary="Upgrading Gitea.")
|
||||
chunks = [_chunk(doc, 0.90, cosine=0.90)]
|
||||
settings = _settings(threshold=0.30)
|
||||
pre = chat_api.plan_turn(chunks, settings)
|
||||
for names in (None, {}):
|
||||
plan = chat_api.plan_turn(chunks, settings, names=names)
|
||||
assert plan.system_prompt == pre.system_prompt
|
||||
assert plan.deflected == pre.deflected
|
||||
assert [d.id for d in plan.suggested_docs] == [d.id for d in pre.suggested_docs]
|
||||
assert ' name="' not in pre.system_prompt
|
||||
|
||||
|
||||
def test_plan_turn_low_prompt_ignores_names() -> None:
|
||||
"""A deflected turn has no document blocks: the LOW prompt is
|
||||
byte-identical whether or not the source is named (D4 touches the
|
||||
suggested tier only)."""
|
||||
doc = _doc("Mongolia Trivia", "MONGOLIA_CONTENT")
|
||||
settings = _settings(threshold=0.62)
|
||||
chunks = [_chunk(doc, 0.05, cosine=0.10, fts_hit=True)]
|
||||
pre = chat_api.plan_turn(chunks, settings)
|
||||
plan = chat_api.plan_turn(chunks, settings, names={"Homelab": _label_view("Homelab", "x")})
|
||||
assert plan.deflected is True
|
||||
assert plan.system_prompt == pre.system_prompt
|
||||
# The pre-phase deflected build, unchanged, is what the LOW branch owes.
|
||||
assert plan.system_prompt == build_deflect_prompt(
|
||||
weak_hit_titles(chunks), notes=[], kb_overview=""
|
||||
)
|
||||
|
||||
|
||||
def test_chat_module_issues_exactly_one_label_read_per_turn() -> None:
|
||||
"""The marker pin for the "one query per turn" contract: the endpoint
|
||||
reads the label rows ONCE (``all_labels``) and derives both the D2 hit
|
||||
set and the prompt map from that read — no second label query, and task
|
||||
04's convenience wrapper (which would re-query) is gone from the
|
||||
module."""
|
||||
src = Path(chat_api.__file__).read_text(encoding="utf-8")
|
||||
assert src.count("all_labels(") == 1
|
||||
assert "question_name_hit_sources(" not in src
|
||||
assert src.count("names=labels") == 1, "the single read is what the prompt receives"
|
||||
assert "question_name_hits(" in src, "the pure D2 rule stays applied per view"
|
||||
|
||||
|
||||
def test_endpoint_single_label_read_feeds_flags_and_prompt(
|
||||
client: TestClient,
|
||||
gate_env: tuple[_FakeSession, _CannedLLM],
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The one-read contract on the wire (task 04 + task 05): the ONE
|
||||
``all_labels`` result of a turn sets the D2 boost flags AND the
|
||||
``<document>`` name/label attributes — one query, two surfaces."""
|
||||
_session, llm = gate_env
|
||||
doc = _doc("Gitea Upgrade", "GITEA_DOC_CONTENT", summary="Upgrading Gitea.")
|
||||
chunk = _chunk(doc, 0.90, cosine=0.90)
|
||||
assert chunk.source_name_hit is False # the pre-flag state
|
||||
monkeypatch.setattr(chat_api, "retrieve", _fake_retriever([chunk]))
|
||||
views = {
|
||||
"Homelab": SourceLabelView(
|
||||
source="Homelab",
|
||||
display_name="the git server",
|
||||
label="self-hosted forge",
|
||||
tags=("gitea",),
|
||||
)
|
||||
}
|
||||
reads: list[int] = []
|
||||
|
||||
def _counting_all_labels(_db: Any) -> dict[str, SourceLabelView]:
|
||||
reads.append(1)
|
||||
return views
|
||||
|
||||
monkeypatch.setattr(chat_api, "all_labels", _counting_all_labels)
|
||||
frames = _ask(client, "how do I restart the git server?")
|
||||
assert [f for f in frames if f.get("type") == "done"], "the turn completed"
|
||||
assert len(reads) == 1, "ONE label read for the whole turn"
|
||||
assert chunk.source_name_hit is True, "the same read sets the task-04 boost flag"
|
||||
system = llm.seen[0][0]["content"]
|
||||
assert 'source="Homelab" name="the git server" label="self-hosted forge"' in system
|
||||
|
||||
Reference in New Issue
Block a user