phase: 141_read_range_dedupe_fix
Build and Push Containers / build-and-push-app (push) Successful in 2m37s
Build and Push Containers / build-and-push-db (push) Successful in 12s

All gates green and verified. Final report:

**Phase 141 final verification pass — complete**

- **Verified (tasks 01–04 already in `complete/`):** `AgentHolder.read_coverage` span recording (all 4 read outcomes, refusals record nothing); coverage dedupe matrix (range read of uncovered lines proceeds, fully-covered re-read → `ALREADY_IN_CONTEXT` with a now-true claim, partial plain re-read → `REREAD_PARTIAL` naming merged spans, `REREAD_TRUNCATED` retired, fast path retired per task 02); `TOOLS_SECTION` re-read sentence re-cut to coverage-true form (prompt lock re-pinned); owner-scenario integration repro (pure + truncated variants) and mock `read two ranges` flow + dedicated E2E — all match the task specs exactly; no code defects found.
- **Tests/lint/coverage:** `uv run pytest --cov=app --cov-report=term-missing` → exit 0, TOTAL 99% (5498 stmts, `app/rag/agent.py` 99%) — run twice, both green; `uv run pytest tests/e2e/test_read_range_dedupe.py -v --no-cov` → 2 passed in isolation; `uv run ruff check . && uv run pyright` → All checks passed / 0 errors.
- **Completion criteria:** (1) two different ranges both reach the model, re-covered re-read refused truthfully — ✅ (integration + E2E); (2) partial plain re-read gets the `offset`/`limit` teaching, false claim gone both directions — ✅; (3) plain-read + phase-131 result shapes byte-identical, existing pins pass unchanged — ✅; (4) all gate commands green, >90% — ✅; (5) no behavior change in completed phases (grep/ls, truncation notice, `read_docs`/`read_truncations` signals) — ✅ (whole project suite green).
- **Documentation:** README.md — fixed pre-existing phase-140 gap: `BOR_PDF_PICTORIAL` / `BOR_PDF_PICTORIAL_MIN_CHARS` were in `.env.example`/Settings but undocumented (README claims every `.env.example` var is covered); added the pictorial-PDF-pass bullet to the PDF section + two config-reference rows. AGENTS.md — verified consistent (rules, quick-ref commands, phase refs all check out) — no changes needed.
- **Notable:** overview's "fast path survives for holder-state rows" note is superseded by task 02's authoritative decision (every dedupe row resolves — "full" needs the true line count); implementation follows task 02. No phase files touched; changes left in the working tree for the harness commit.
- **Next pending phase:** `142_kb_overview_freshness`.
This commit is contained in:
2026-10-01 01:27:46 -04:00
parent de49a03c4d
commit a8c0402ad2
33 changed files with 3520 additions and 203 deletions
+22 -1
View File
@@ -623,7 +623,26 @@ uploaded archives):
descriptions (a failed page gets a `page N — description unavailable`
placeholder — fail-soft, the doc still lands). Clean text layers
never burn a fallback vision call. The run counts the affected PDFs
on the import summary line (`pdfs_screenshotted=N`).
on the import summary line (`pdfs_screenshotted=N`). A garbage-text
PDF takes this fallback instead of the pictorial pass below — the
precedence is locked, never double vision.
- **Pictorial pages get vision-described (phase 140, on by default).**
For a PDF whose text layer is usable but thin — the IKEA
assembly-instruction class (20 of 24 pages: 4–73 chars of markers
around vector-drawn pictograms) — every page whose STRIPPED text is
under `BOR_PDF_PICTORIAL_MIN_CHARS`
characters (default `100`; a deterministic per-page threshold, no LLM
in the gate) is rendered to a PNG and described by the chat model,
and the description appends to that page's content as
`[Page N illustration: …]` — so the embeddings carry what the thin
text layer couldn't, and a question about the illustrations hits.
NO page cap: every candidate page is described (A21 — the LLM calls
are free; a slower sync is acceptable). A PDF with zero candidate
pages burns no render or vision call — its indexed content is
byte-identical to pre-phase-140. The run counts the affected PDFs and
pages on the import summary line (`pdfs_pictorial=N`
`pdf_pictorial_pages=N`). `BOR_PDF_PICTORIAL=0` turns the whole pass
off (byte-identical pre-phase-140 content).
- **Served as text.** The document viewer renders the extracted text —
never the binary PDF.
@@ -749,6 +768,8 @@ to `.env` and adjust.
| Env | Default | Meaning |
|-----|---------|---------|
| `BOR_IMPORT_EXTENSIONS` | A9 family | Importable text formats — the A9 default is `md,markdown,txt,yaml,yml,json,py,container,network,volume,image,pod,kube,swap,os,endpoint,j2`; any well-formed extension is allowed. Add the `pdf` token to make PDFs first-class, retrievable documents (phase 126, opt-in). Images are **not** added here — see `BOR_IMAGES` |
| `BOR_PDF_PICTORIAL` | `1` | Master switch for the pictorial PDF pass (phase 140, on by default): a usable-text PDF's sparse pages (stripped text under `BOR_PDF_PICTORIAL_MIN_CHARS`) are rendered + vision-described into the indexed content (`[Page N illustration: …]`); `0` ⇒ content byte-identical to pre-phase-140 (no render, no vision call) |
| `BOR_PDF_PICTORIAL_MIN_CHARS` | `100` | Sparseness bar for the pictorial pass (phase 140): a page whose STRIPPED text is STRICTLY shorter than this many chars is a candidate (a page with exactly this many is not); `0` = every page (A21); a negative value fails startup loudly |
| `BOR_GIT_SOURCES` | — (empty) | CSV of git repo URLs — **fallback while the admin Git sources page's list is empty** |
| `BOR_SOURCES_DIR` | `~/bor-sources` | Where git repos are cloned/pulled |
| `BOR_UPLOAD_DIR` | `~/bor-sources/uploads` | Where uploaded source archives are unpacked |