phase: 141_read_range_dedupe_fix
All gates green and verified. Final report: **Phase 141 final verification pass — complete** - **Verified (tasks 01–04 already in `complete/`):** `AgentHolder.read_coverage` span recording (all 4 read outcomes, refusals record nothing); coverage dedupe matrix (range read of uncovered lines proceeds, fully-covered re-read → `ALREADY_IN_CONTEXT` with a now-true claim, partial plain re-read → `REREAD_PARTIAL` naming merged spans, `REREAD_TRUNCATED` retired, fast path retired per task 02); `TOOLS_SECTION` re-read sentence re-cut to coverage-true form (prompt lock re-pinned); owner-scenario integration repro (pure + truncated variants) and mock `read two ranges` flow + dedicated E2E — all match the task specs exactly; no code defects found. - **Tests/lint/coverage:** `uv run pytest --cov=app --cov-report=term-missing` → exit 0, TOTAL 99% (5498 stmts, `app/rag/agent.py` 99%) — run twice, both green; `uv run pytest tests/e2e/test_read_range_dedupe.py -v --no-cov` → 2 passed in isolation; `uv run ruff check . && uv run pyright` → All checks passed / 0 errors. - **Completion criteria:** (1) two different ranges both reach the model, re-covered re-read refused truthfully — ✅ (integration + E2E); (2) partial plain re-read gets the `offset`/`limit` teaching, false claim gone both directions — ✅; (3) plain-read + phase-131 result shapes byte-identical, existing pins pass unchanged — ✅; (4) all gate commands green, >90% — ✅; (5) no behavior change in completed phases (grep/ls, truncation notice, `read_docs`/`read_truncations` signals) — ✅ (whole project suite green). - **Documentation:** README.md — fixed pre-existing phase-140 gap: `BOR_PDF_PICTORIAL` / `BOR_PDF_PICTORIAL_MIN_CHARS` were in `.env.example`/Settings but undocumented (README claims every `.env.example` var is covered); added the pictorial-PDF-pass bullet to the PDF section + two config-reference rows. AGENTS.md — verified consistent (rules, quick-ref commands, phase refs all check out) — no changes needed. - **Notable:** overview's "fast path survives for holder-state rows" note is superseded by task 02's authoritative decision (every dedupe row resolves — "full" needs the true line count); implementation follows task 02. No phase files touched; changes left in the working tree for the harness commit. - **Next pending phase:** `142_kb_overview_freshness`.
This commit is contained in:
@@ -623,7 +623,26 @@ uploaded archives):
|
||||
descriptions (a failed page gets a `page N — description unavailable`
|
||||
placeholder — fail-soft, the doc still lands). Clean text layers
|
||||
never burn a fallback vision call. The run counts the affected PDFs
|
||||
on the import summary line (`pdfs_screenshotted=N`).
|
||||
on the import summary line (`pdfs_screenshotted=N`). A garbage-text
|
||||
PDF takes this fallback instead of the pictorial pass below — the
|
||||
precedence is locked, never double vision.
|
||||
- **Pictorial pages get vision-described (phase 140, on by default).**
|
||||
For a PDF whose text layer is usable but thin — the IKEA
|
||||
assembly-instruction class (20 of 24 pages: 4–73 chars of markers
|
||||
around vector-drawn pictograms) — every page whose STRIPPED text is
|
||||
under `BOR_PDF_PICTORIAL_MIN_CHARS`
|
||||
characters (default `100`; a deterministic per-page threshold, no LLM
|
||||
in the gate) is rendered to a PNG and described by the chat model,
|
||||
and the description appends to that page's content as
|
||||
`[Page N illustration: …]` — so the embeddings carry what the thin
|
||||
text layer couldn't, and a question about the illustrations hits.
|
||||
NO page cap: every candidate page is described (A21 — the LLM calls
|
||||
are free; a slower sync is acceptable). A PDF with zero candidate
|
||||
pages burns no render or vision call — its indexed content is
|
||||
byte-identical to pre-phase-140. The run counts the affected PDFs and
|
||||
pages on the import summary line (`pdfs_pictorial=N`
|
||||
`pdf_pictorial_pages=N`). `BOR_PDF_PICTORIAL=0` turns the whole pass
|
||||
off (byte-identical pre-phase-140 content).
|
||||
- **Served as text.** The document viewer renders the extracted text —
|
||||
never the binary PDF.
|
||||
|
||||
@@ -749,6 +768,8 @@ to `.env` and adjust.
|
||||
| Env | Default | Meaning |
|
||||
|-----|---------|---------|
|
||||
| `BOR_IMPORT_EXTENSIONS` | A9 family | Importable text formats — the A9 default is `md,markdown,txt,yaml,yml,json,py,container,network,volume,image,pod,kube,swap,os,endpoint,j2`; any well-formed extension is allowed. Add the `pdf` token to make PDFs first-class, retrievable documents (phase 126, opt-in). Images are **not** added here — see `BOR_IMAGES` |
|
||||
| `BOR_PDF_PICTORIAL` | `1` | Master switch for the pictorial PDF pass (phase 140, on by default): a usable-text PDF's sparse pages (stripped text under `BOR_PDF_PICTORIAL_MIN_CHARS`) are rendered + vision-described into the indexed content (`[Page N illustration: …]`); `0` ⇒ content byte-identical to pre-phase-140 (no render, no vision call) |
|
||||
| `BOR_PDF_PICTORIAL_MIN_CHARS` | `100` | Sparseness bar for the pictorial pass (phase 140): a page whose STRIPPED text is STRICTLY shorter than this many chars is a candidate (a page with exactly this many is not); `0` = every page (A21); a negative value fails startup loudly |
|
||||
| `BOR_GIT_SOURCES` | — (empty) | CSV of git repo URLs — **fallback while the admin Git sources page's list is empty** |
|
||||
| `BOR_SOURCES_DIR` | `~/bor-sources` | Where git repos are cloned/pulled |
|
||||
| `BOR_UPLOAD_DIR` | `~/bor-sources/uploads` | Where uploaded source archives are unpacked |
|
||||
|
||||
Reference in New Issue
Block a user