phase: 142_kb_overview_freshness
Build and Push Containers / build-and-push-app (push) Successful in 2m33s
Build and Push Containers / build-and-push-db (push) Successful in 12s

All verification complete. Final report:

**Phase 142 final verification pass — all green**

- **Verified implemented** (working tree matches design; app diff is exactly 2 lines of logic): `app/rag/overview.py` — `regenerate_overview` now orders the lite-model document list `created_at` desc, `(source, path)` tie-break (newest docs survive the cap); `app/rag/prompts.py` — `_KB_INTRO` rewritten to the honest intro (recency bias, "map not inventory", verify-with-`ls`/`grep`; "(generated at import time)" gone).
- **Completion criteria:**
  1. Forced-cap outline leads with newest docs — ✅ E2E `test_kb_overview_freshness.py` test 1 (cap 300, digest from `zz_new/z.md`, no "old" tokens) + integration repro (cap 150 → `mydocs zz new 03 md zz new 03`)
  2. New honest intro served in `<knowledge_base>` — ✅ unit pin (verbatim) + E2E test 2 (`build_kb_section` surface, tag/cap byte-stable)
  3. Gates — ✅ `uv run pytest --cov=app --cov-report=term-missing`: **3359 passed**, `app/` TOTAL **99%** (>90%); `uv run pytest tests/e2e/test_kb_overview_freshness.py -v --no-cov`: **3 passed** in isolation; `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov`: **4 passed** in isolation; `uv run ruff check . && uv run pyright`: **clean, 0 errors**
  4. No behavior change in completed phases — ✅ cap env knob, change-gated sync trigger, `build_kb_section` cap/marker, deflect path all byte-identical (pinned pins pass in the 3359)
- **Documentation:** README.md reviewed in full (every `.env.example` var documented with matching defaults, all endpoints/routes verified against `app/api/`, Quick Start/deploy/troubleshooting claims verified) and AGENTS.md re-reviewed (rules + quick reference accurate) — **verified consistent, no changes needed**.
- **Decisions:** none required — no defects found; no live-infra changes (only the already-running local `db` container was used by tests).
- **Next pending phase:** `.agents/phases/todo/143_mobile_keyboard_after_answer`.
This commit is contained in:
2026-10-01 04:25:03 -04:00
parent a8c0402ad2
commit aa65d2d691
30 changed files with 1738 additions and 38 deletions
@@ -0,0 +1,13 @@
All verification complete. Final report:
**Phase 142 final verification pass — all green**
- **Verified implemented** (working tree matches design; app diff is exactly 2 lines of logic): `app/rag/overview.py` — `regenerate_overview` now orders the lite-model document list `created_at` desc, `(source, path)` tie-break (newest docs survive the cap); `app/rag/prompts.py` — `_KB_INTRO` rewritten to the honest intro (recency bias, "map not inventory", verify-with-`ls`/`grep`; "(generated at import time)" gone).
- **Completion criteria:**
1. Forced-cap outline leads with newest docs — ✅ E2E `test_kb_overview_freshness.py` test 1 (cap 300, digest from `zz_new/z.md`, no "old" tokens) + integration repro (cap 150 → `mydocs zz new 03 md zz new 03`)
2. New honest intro served in `<knowledge_base>` — ✅ unit pin (verbatim) + E2E test 2 (`build_kb_section` surface, tag/cap byte-stable)
3. Gates — ✅ `uv run pytest --cov=app --cov-report=term-missing`: **3359 passed**, `app/` TOTAL **99%** (>90%); `uv run pytest tests/e2e/test_kb_overview_freshness.py -v --no-cov`: **3 passed** in isolation; `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov`: **4 passed** in isolation; `uv run ruff check . && uv run pyright`: **clean, 0 errors**
4. No behavior change in completed phases — ✅ cap env knob, change-gated sync trigger, `build_kb_section` cap/marker, deflect path all byte-identical (pinned pins pass in the 3359)
- **Documentation:** README.md reviewed in full (every `.env.example` var documented with matching defaults, all endpoints/routes verified against `app/api/`, Quick Start/deploy/troubleshooting claims verified) and AGENTS.md re-reviewed (rules + quick reference accurate) — **verified consistent, no changes needed**.
- **Decisions:** none required — no defects found; no live-infra changes (only the already-running local `db` container was used by tests).
- **Next pending phase:** `.agents/phases/todo/143_mobile_keyboard_after_answer`.
@@ -0,0 +1,118 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
............................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 179 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 422 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 56 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 154 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 424 0 100%
--------------------------------------------------
TOTAL 5498 22 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,19 @@
All done. Final verification summary — everything is in the designed state.
## Report — task `01_cap_ordering_repro` (phase 142)
**Implemented (tests-only, no app code changed):**
- `tests/unit/test_overview.py`: `_phase_dir_rows()` (20 owner-shape rows, `01_old_a.md`…`20_new_z.md`, 100 chars/line); `test_cap_keeps_the_given_prefix_and_cuts_the_tail` — GREEN pin of the current cap behavior (lexicographic prefix survives, tail cut, marker on its own line, cut exactly at cap); `test_document_list_reaching_the_model_leads_with_newest_docs` — RED repro: `regenerate_overview` leads the model's list with the OLDEST doc (`01_old_a`), asserts the post-fix intent (newest first, oldest last).
- `tests/integration/test_import_docs_overview.py`: `OverviewDigestEmbedder` (reproduces the mock's `KB_OVERVIEW_MODE` 8-token digest, imports `TOKEN_RE` from `tests/e2e/mock_llm.py` so it's byte-identical); `owner_kb` fixture (one source, `old/` batch mtimed 2026-01 vs `zz_new/` batch 2026-09 via `os.utime` — recency opposes lexicographic order); `test_overview_outline_names_the_newest_docs_when_the_cap_bites` — RED repro with `BOR_OVERVIEW_INPUT_MAX_CHARS=150` forced: pre-fix the stored outline is `mydocs old 01 md old 01 summary of` (the exact "summaries only up to phase 113" shape); asserts the post-fix digest `mydocs zz new 03 md zz new 03` + no `old` token.
**Test / lint / coverage (exact outcomes):**
- `uv run pytest` → `2 failed, 3356 passed` — the 2 failures are exactly the new repros, failing for the documented root cause (failure output names it); everything else 100% green.
- `uv run ruff check .` → clean; `uv run pyright` → `0 errors, 0 warnings`.
- `uv run pytest --cov=app` → `app/` coverage **99%** (gate >90%).
- `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov` → 4 passed (no regression).
**README/AGENTS.md:** no doc impact (tests-only task).
**Notable:** red-is-by-design per the task's completion criteria (task 02 flips both green); `git diff` touches only the two test files.
**Next pending task:** `.agents/phases/todo/142_kb_overview_freshness/02_recency_ordering.md`.
@@ -0,0 +1,240 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
...............................................................F........ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 90%]
........................................................................ [ 92%]
.......F................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.............................................. [100%]
=================================== FAILURES ===================================
_________ test_document_list_reaching_the_model_leads_with_newest_docs _________
db = <sqlalchemy.orm.session.Session object at 0x7f625b1edf40>
clean_overview = None
def test_document_list_reaching_the_model_leads_with_newest_docs(
db: Session, clean_overview
) -> None:
"""Phase 142, task 01 — RED pre-fix (the intent the fix must meet).
The document list reaching the lite model must LEAD with the NEWEST
rows (``created_at`` desc — the document's date, phase 106) so that,
when the cap bites, the newest documents — not the lexicographically
first — are the ones the outline covers. Pre-fix the query is plain
``order_by(source, path)``: with the phase-dir rows below (recency
strictly OPPOSITE to the path order) that puts ``01_old_a.md`` — the
OLDEST — first and ``20_new_z.md`` — the NEWEST — last, the traced
"summaries only up to phase 113" failure at unit scale. The
assertion is the EXPECTED POST-FIX order (task 02 flips it green).
"""
_truncate_documents(db)
try:
base = datetime(2026, 1, 1, tzinfo=UTC)
for i, (source, path, title, summary) in enumerate(_phase_dir_rows()):
_add_doc(db, source, path, title, summary)
# Row i (0-based) is i days after the base: index 0
# (``01_old_a.md``) is the OLDEST, index 19 (``20_new_z.md``)
# the NEWEST — recency strictly opposite to the path order.
db.execute(
text("UPDATE documents SET created_at = :d WHERE path = :p"),
{"d": base + timedelta(days=i), "p": path},
)
db.commit()
llm = _FakeLLM()
assert asyncio.run(regenerate_overview(llm, db)) is True
finally:
_truncate_documents(db)
user = next(m["content"] for m in llm.messages if m["role"] == "user")
lines = user.splitlines()
assert len(lines) == 20 # no cap at this size — the ORDER is what is asserted
> assert lines[0].startswith("s — 20_new_z.md"), (
"the list must lead with the NEWEST doc (20_new_z) — pre-fix it "
"leads with the lexicographic prefix (01_old_a, the oldest)"
)
E AssertionError: the list must lead with the NEWEST doc (20_new_z) — pre-fix it leads with the lexicographic prefix (01_old_a, the oldest)
E assert False
E + where False = <built-in method startswith of str object at 0x7f625b12fdd0>('s — 20_new_z.md')
E + where <built-in method startswith of str object at 0x7f625b12fdd0> = 's — 01_old_a.md — Phase 01 — xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'.startswith
tests/unit/test_overview.py:379: AssertionError
----------------------------- Captured stdout call -----------------------------
2026-10-01 01:46:55 INFO app.rag.overview :: overview: regenerated docs=20 chars=61
------------------------------ Captured log call -------------------------------
INFO app.rag.overview:overview.py:199 overview: regenerated docs=20 chars=61
________ test_overview_outline_names_the_newest_docs_when_the_cap_bites ________
db = <sqlalchemy.orm.session.Session object at 0x7f5febb62300>
owner_kb = PosixPath('/tmp/pytest-of-ducoterra/pytest-49/test_overview_outline_names_th0/MyDocs')
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f622c6472a0>
capsys = <_pytest.capture.CaptureFixture object at 0x7f5fb305e000>
def test_overview_outline_names_the_newest_docs_when_the_cap_bites(
db: Session,
owner_kb: Path,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
) -> None:
"""Phase 142, task 01 — RED pre-fix (the KB-shape repro of the
"summaries only up to phase 113" failure).
The owner's KB shape: one source whose OLDER documents sort
lexicographically FIRST (``old/*``) and whose NEWER documents sort
LAST (``zz_new/*``). With ``BOR_OVERVIEW_INPUT_MAX_CHARS`` forced to
150, the 6-doc list (275 chars) exceeds the cap and the cut keeps a
PREFIX. Pre-fix that prefix is the lexicographic one — the three
OLDER ``old/*`` docs — so the stored outline's 8-token digest names
the old docs (``mydocs old 01 md old 01 summary of``) and the newer
``zz_new/*`` docs never reach the model. The assertion below is the
EXPECTED POST-FIX outcome: the ``created_at``-desc ordering (task 02)
puts the newest docs first, so they survive the cap and are the ones
the outline names (``mydocs zz new 03 md zz new 03``).
"""
llm = OverviewDigestEmbedder()
get_settings.cache_clear()
monkeypatch.setenv("BOR_OVERVIEW_INPUT_MAX_CHARS", "150")
try:
rc, out = _run_main(monkeypatch, llm, ["--source", str(owner_kb)], capsys)
finally:
get_settings.cache_clear()
assert rc == 0
assert "added=6" in out
# 6 doc summaries (phase 118) + the overview + 3 folder summaries
# (source root, old/, zz_new/) — the overview gate fired (added > 0).
assert "overview=updated" in out
assert len(llm.chat_calls) == 10
overview_call = next(
m for m in llm.chat_calls
if "KB_OVERVIEW_MODE" in m[0]["content"]
)
user = next(m["content"] for m in overview_call if m["role"] == "user")
assert "[…truncated…]" in user # the 150-char cap bit (the repro's precondition)
row = _row(db)
assert row is not None
# POST-fix digest: the capped list leads with the NEWEST docs —
# "MyDocs — zz_new/03.md — Zz New 03 — Summary of #" → first 8
# tokens "mydocs zz new 03 md zz new 03". Pre-fix the stored row is
# "mydocs old 01 md old 01 summary of" (the oldest batch) — that
# mismatch is the red the fix must flip.
> assert row.content == "Knowledge base outline:\n- mydocs zz new 03 md zz new 03"
E AssertionError: assert 'Knowledge ba...01 summary of' == 'Knowledge ba... md zz new 03'
E
E Knowledge base outline:
E - - mydocs zz new 03 md zz new 03
E + - mydocs old 01 md old 01 summary of
tests/integration/test_import_docs_overview.py:457: AssertionError
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 179 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 422 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 56 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 154 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 424 0 100%
--------------------------------------------------
TOTAL 5498 22 99%
=========================== short test summary info ============================
FAILED tests/unit/test_overview.py::test_document_list_reaching_the_model_leads_with_newest_docs
FAILED tests/integration/test_import_docs_overview.py::test_overview_outline_names_the_newest_docs_when_the_cap_bites
tests FAILED: unit + integration suite did not pass
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation FAILED (see output above)
@@ -0,0 +1,16 @@
## Report — task `01_cap_ordering_repro` (phase 142), retry
**Fixed:** the two repro tests now carry `@pytest.mark.xfail(strict=True)` with the root cause in the reason string — the test bodies still fail on pre-fix code (verified: pytest reports XFAIL, not XPASS), so the red repro is preserved, but the suite is green. `strict=True` turns a forgotten marker in task 02 into an XPASS failure, forcing the flip-to-green. The pre-fix cap-behavior pin (`test_cap_keeps_the_given_prefix_and_cuts_the_tail`) stays a plain green test; docstrings updated to match.
**Repro content (unchanged from last attempt):**
- Unit: 20 owner-phase-dir rows (`01_old_a.md`…`20_new_z.md`, 100 chars/line); xfail body asserts the list reaching the model leads with `20_new_z` (newest) — pre-fix it leads with `01_old_a` (oldest).
- Integration: `owner_kb` fixture (`old/` mtimed 2026-01 vs `zz_new/` 2026-09 via `os.utime`), `OverviewDigestEmbedder` (mock-identical 8-token digest via the mock's `TOKEN_RE`), forced `BOR_OVERVIEW_INPUT_MAX_CHARS=150`; xfail body asserts the stored outline is `mydocs zz new 03 md zz new 03` — pre-fix it is `mydocs old 01 md old 01 summary of` (the "summaries only up to phase 113" shape).
**Results:**
- `uv run pytest --cov=app --cov-report=term` → **3356 passed, 2 xfailed, 0 failed**; `app/` coverage **99%** (>90% gate).
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings.
- `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov` → 4 passed (checked previously, app code untouched since).
**Docs:** no doc impact (tests-only). **No application code changed** (git diff: two test files only).
**Next pending task:** `.agents/phases/todo/142_kb_overview_freshness/02_recency_ordering.md` (landing the fix will XPASS both xfail tests → remove the markers there).
@@ -0,0 +1,118 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
...............................................................x........ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 90%]
........................................................................ [ 92%]
.......x................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.............................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 179 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 422 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 56 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 154 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 424 0 100%
--------------------------------------------------
TOTAL 5498 22 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,18 @@
All changes verified and all gates green. Final report:
**Implemented (phase 142, task 02 — recency ordering)**
- `app/rag/overview.py`: `regenerate_overview` now SELECTs `order_by(Document.created_at.desc(), Document.source, Document.path)` — newest docs lead the lite model's list so the cap drops the OLDEST; module + prompt docstrings updated (cap, cut+marker, fail-soft, upsert all unchanged)
- `tests/unit/test_overview.py`: strict-xfail repro flipped green (marker removed); new 6-doc/2-batch ordering pin (newest batch first, `(source,path)` tie-break); `build_overview_prompt` byte-identical pins untouched
- `tests/integration/test_import_docs_overview.py`: strict-xfail KB-shape repro flipped green (digest now names `zz_new/03`, the newest); fixture mtimes made run-relative (de-timebombs `normalize_doc_date`'s future-skew rule)
- `tests/e2e/test_kb_overview.py`: re-pinned test 4 — dates pinned explicitly (checkout mtimes aren't git-stable), stored digest re-pinned to the newest doc's line (`docs homelab tables md service port table this`), first-line recency assertion added
- Defect fixed from task 01: repro asserted path `20_new_z.md` but the row generator produces suffixes a–t (`20_new_t.md`) — assertions/docstrings corrected
**Gates (all post-edit)**
- `uv run pytest --no-cov -v`: 3358 passed, 0 failed
- `uv run pytest --cov=app --cov-report=term-missing`: 3358 passed, TOTAL 99% (overview.py 100%)
- `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov`: 4 passed (isolated); `test_sync_button.py` spot-check: 3 passed
- `uv run ruff check . && uv run pyright`: clean
**Docs:** no doc impact (no env/CLI/API changes; README's overview trigger + knob docs stay accurate)
**Deviations:** none
**Next pending task:** `.agents/phases/todo/142_kb_overview_freshness/03_honest_kb_intro.md`
@@ -0,0 +1,118 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
.............................................. [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 179 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 422 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 56 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 154 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 424 0 100%
--------------------------------------------------
TOTAL 5498 22 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,10 @@
All green. The diff is clean and complete.
**Report — phase 142, task 03 (honest `<knowledge_base>` intro)**
- Rewrote `_KB_INTRO` in `app/rag/prompts.py`: drops "(generated at import time)"; now states the outline is a compact, possibly partial picture favoring recent documents (a map, not a complete inventory) and to verify "latest / complete set" questions with `ls` (drill-down) / `grep` before answering from it. `build_kb_section` code untouched (tag/cap/marker/pathological behavior byte-stable); constant doc comment updated with the phase-142 rationale.
- `tests/unit/test_prompts.py`: new `test_kb_intro_phase142_honest_copy` pins the intro verbatim (incl. absence of the old phrase + the three contracts, in both HIGH and LOW builds); re-pinned the two budget-edge caps 200→600 to the new intro length (frame 422 chars; behavior unchanged, as the task permits); module docstring noted. No-overview byte-identity, section-order, and deflect-prompt pins pass unchanged.
- Test/lint results: `uv run pytest --cov=app --cov-report=term-missing` → **3359 passed, `app/` coverage 99%** (>90%); `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors, 0 warnings; `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov` → **4 passed** in isolation (mock echo unaffected — intro is not a `-` bullet).
- Docs: no README/AGENTS.md impact (no commands/config/env/API changes; intro text not documented there).
- Notable: pre-existing stale comment "frame alone is 121 chars" (actual 135 pre-change) corrected to the accurate 422.
- Next pending task: `todo/142_kb_overview_freshness/04_e2e_outline_freshness.md`.
@@ -0,0 +1,118 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
............................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 179 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 422 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 56 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 154 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 424 0 100%
--------------------------------------------------
TOTAL 5498 22 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
@@ -0,0 +1,10 @@
**Task 04 complete — all gates green.**
- Added `tests/e2e/test_kb_overview_freshness.py` (dedicated Playwright file, mock-only, run in isolation):
- `test_regenerated_outline_leads_with_newest_doc` — owner-shaped fixture (`old/a..c.md` past mtimes, sort first; `zz_new/y..z.md` fresh mtimes, sort last), `BOR_OVERVIEW_INPUT_MAX_CHARS=300` forced via env + `get_settings` cache clear so the cap bites; real `regenerate_overview` against the mock's `KB_OVERVIEW_MODE` → stored row's 8 digest tokens computed from the fixture (`reese zz new z md zz new z`) and proven to come from the NEWEST doc's uncut line; "old" absent.
- `test_kb_section_carries_honest_intro` — `build_kb_section` surface: lead line is the phase-142 honest intro pinned verbatim, "(generated at import time)" gone, recency/map/verify-with-`ls`/`grep` contracts present, tag + exact-fit/tight-budget cap behavior intact.
- `test_kb_section_reaches_a_rendered_turn` — seeded row's first-bullet `(kb: …)` echo reaches a rendered turn (phase-31 injection path, standalone).
- Tests/lint: `uv run pytest tests/e2e/test_kb_overview_freshness.py -v --no-cov` → 3 passed in isolation (ran twice); `uv run pytest tests/e2e/test_kb_overview.py -v --no-cov` → 4 passed in isolation; `uv run pytest --cov=app` → 100% green, `app/` coverage 99% (>90%); `uv run ruff check .` clean; `uv run pyright` 0 errors.
- Docs: no doc impact (E2E-only task; README/AGENTS.md overview claims still accurate — cap knob, change-gated trigger, env table all unchanged).
- Decisions: digest tokens derived in-test from the fixture's source/path/title (exactly 8 tokens, summary-independent); test 3 doesn't pin the deflection flag (echo is the injection contract in both modes). No app code touched; no phase files/commits.
- Next pending task: `143_mobile_keyboard_after_answer/01_coarse_pointer_focus_gate.md` (phase 142 is now fully complete — 01–04 done).
@@ -0,0 +1,118 @@
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 6%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 12%]
........................................................................ [ 15%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 27%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 42%]
........................................................................ [ 45%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 51%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 57%]
........................................................................ [ 60%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 66%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 72%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 87%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 96%]
........................................................................ [ 98%]
............................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 268 1 99%
app/api/chat_images.py 50 0 100%
app/api/chats.py 110 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 179 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 183 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 55 0 100%
app/config.py 264 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 39 0 100%
app/core/errors.py 5 0 100%
app/core/logging.py 13 0 100%
app/core/rate_limit.py 44 0 100%
app/core/security_headers.py 20 0 100%
app/core/theming.py 38 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 68 0 100%
app/models.py 133 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 422 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 56 0 100%
app/rag/importer.py 705 9 99%
app/rag/llm.py 246 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 71 0 100%
app/rag/pdf.py 124 0 100%
app/rag/prompts.py 154 0 100%
app/rag/retriever.py 248 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/schemas.py 424 0 100%
--------------------------------------------------
TOTAL 5498 22 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK