phase: 126_pdf_support
All gates verified green — no code changes were needed on this pass. Final report: ## Phase 126 — PDF support: final verification pass **Verified (all tasks 01–05 already complete in working tree):** - `pymupdf` dep in `pyproject.toml`/`uv.lock`; `.env.example` documents the `pdf` token - `app/rag/pdf.py` (text + image extraction, MIME mapping), page-aware `_chunk_pdf`, importer `_index_pdf_file` + embedded-image pass, archives covered via extension walk **Test / lint / coverage results:** - `uv run pytest` → 2997 passed - `uv run pytest tests/e2e/test_pdf_support.py -v --no-cov` (isolation) → 6 passed - `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%** (>90% gate), 2997 passed - `uv run ruff check . && uv run pyright` → clean (0 errors) **Completion criteria:** 1. PDF → doc with text/chunks/embedding/summary — ✅ (integration + E2E Sources-page tests) 2. `BOR_IMAGES=true`: embedded images → standalone image docs — ✅ (integration + E2E chat-chip test) 3. `BOR_IMAGES=false`: images skipped, text indexed — ✅ (both suites) 4. PDF renders as extracted text in viewer — ✅ (E2E) 5. Test/coverage/lint gates — ✅ (above) 6. Commit + phase move — harness handles (per protocol, no `git` on this pass) **Decisions:** none new; inline `[Embedded image: …]` placeholders were a documented task-04 deviation (simplified spec wins) — not a defect. No defects found; repo left functional with changes in the working tree. **Next pending phase:** none — `todo/` is empty once 126 is moved.
This commit is contained in:
@@ -0,0 +1,25 @@
|
||||
All gates verified green — no code changes were needed on this pass. Final report:
|
||||
|
||||
## Phase 126 — PDF support: final verification pass
|
||||
|
||||
**Verified (all tasks 01–05 already complete in working tree):**
|
||||
- `pymupdf` dep in `pyproject.toml`/`uv.lock`; `.env.example` documents the `pdf` token
|
||||
- `app/rag/pdf.py` (text + image extraction, MIME mapping), page-aware `_chunk_pdf`, importer `_index_pdf_file` + embedded-image pass, archives covered via extension walk
|
||||
|
||||
**Test / lint / coverage results:**
|
||||
- `uv run pytest` → 2997 passed
|
||||
- `uv run pytest tests/e2e/test_pdf_support.py -v --no-cov` (isolation) → 6 passed
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%** (>90% gate), 2997 passed
|
||||
- `uv run ruff check . && uv run pyright` → clean (0 errors)
|
||||
|
||||
**Completion criteria:**
|
||||
1. PDF → doc with text/chunks/embedding/summary — ✅ (integration + E2E Sources-page tests)
|
||||
2. `BOR_IMAGES=true`: embedded images → standalone image docs — ✅ (integration + E2E chat-chip test)
|
||||
3. `BOR_IMAGES=false`: images skipped, text indexed — ✅ (both suites)
|
||||
4. PDF renders as extracted text in viewer — ✅ (E2E)
|
||||
5. Test/coverage/lint gates — ✅ (above)
|
||||
6. Commit + phase move — harness handles (per protocol, no `git` on this pass)
|
||||
|
||||
**Decisions:** none new; inline `[Embedded image: …]` placeholders were a documented task-04 deviation (simplified spec wins) — not a defect. No defects found; repo left functional with changes in the working tree.
|
||||
|
||||
**Next pending phase:** none — `todo/` is empty once 126 is moved.
|
||||
@@ -0,0 +1,113 @@
|
||||
........................................................................ [ 2%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 98%]
|
||||
............................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 262 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 179 1 99%
|
||||
app/api/git_sources.py 241 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 139 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 256 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 132 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 374 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 123 0 100%
|
||||
app/rag/git_sources.py 38 0 100%
|
||||
app/rag/importer.py 563 9 98%
|
||||
app/rag/llm.py 244 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 81 0 100%
|
||||
app/rag/prompts.py 144 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 49 0 100%
|
||||
app/schemas.py 400 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5114 22 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -0,0 +1,19 @@
|
||||
Task complete — all criteria green. Report:
|
||||
|
||||
**Implemented**
|
||||
- Added `pymupdf>=1.24,<2.0` to `pyproject.toml` dependencies (after `python-multipart`, before dev group) with a Phase 126 provenance comment, per locked decision A1.
|
||||
- `uv sync` installed `pymupdf==1.28.2` (satisfies the pin; `uv.lock` updated accordingly).
|
||||
|
||||
**Verification**
|
||||
- `uv sync` — completed, `+ pymupdf==1.28.2`
|
||||
- `uv run python -c "import fitz; print(fitz.VersionBind)"` — OK, `VersionBind: 1.28.2`
|
||||
- `uv run pytest --cov=app --cov-report=term` — 2926 passed, TOTAL coverage **99%** (>90% gate)
|
||||
- `uv run ruff check .` — all checks passed; `uv run pyright` — 0 errors, 0 warnings
|
||||
|
||||
**Notable**
|
||||
- `import fitz` emits a deprecation warning in pymupdf 1.28 ("use `import pymupdf` instead"). The phase file locks `fitz` as the import name (A1), so I kept it; task 02 can equally use `import pymupdf as fitz` to silence the warning without deviating from the API.
|
||||
|
||||
**Next pending task**
|
||||
- `.agents/phases/todo/126_pdf_support/02_pdf_parser.md` — `app/rag/pdf.py`: text extraction, image extraction, MIME mapping.
|
||||
|
||||
No commit made (harness commits at phase completion); working tree left with `pyproject.toml` + `uv.lock` modified.
|
||||
@@ -0,0 +1,111 @@
|
||||
........................................................................ [ 2%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 22%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 34%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 56%]
|
||||
........................................................................ [ 59%]
|
||||
........................................................................ [ 61%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 83%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 98%]
|
||||
.............................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 262 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 179 1 99%
|
||||
app/api/git_sources.py 241 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 139 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 256 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 132 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 374 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 206 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 123 0 100%
|
||||
app/rag/git_sources.py 38 0 100%
|
||||
app/rag/importer.py 432 10 98%
|
||||
app/rag/llm.py 244 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/prompts.py 144 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 49 0 100%
|
||||
app/schemas.py 400 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 4888 23 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -0,0 +1,16 @@
|
||||
Task 02 complete — all criteria green.
|
||||
|
||||
**Implemented**
|
||||
- `app/rag/pdf.py`: `extract_pdf_text` (`---PAGE-N---` 1-indexed separators, A2), `extract_pdf_images` (original encoded bytes via `doc.extract_image`, per-occurrence page/index), `PdfImage` dataclass, `_mime_from_image_info` (ext first → type code → `application/octet-stream`)
|
||||
- `tests/unit/test_pdf.py`: 39 tests — multi/single-page text, image-only → `""`, blank middle page, embedded/duplicate images, missing/garbage files (no raise), broken-page/image skip branches, MIME mapping (22 cases), old-pymupdf `xrefs` fallback
|
||||
|
||||
**Validation**
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` → **2965 passed**, TOTAL **99%** (gate >90%); `app/rag/pdf.py` **100%**
|
||||
- `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors, 0 warnings
|
||||
|
||||
**Notable decisions**
|
||||
- `import pymupdf as fitz` (per task-01 report) — avoids the 1.28 deprecation warning, same API
|
||||
- Images use `extract_image` original bytes (no lossy Pixmap re-encode); `get_image_info()`'s `ext` is `None` and `type` key absent in PyMuPDF 1.28, so `extract_image`'s reported `ext` drives the MIME
|
||||
- Pages without text get no separator (satisfies "image-only → empty string" spec); task-file type codes (8=tiff, 9=webp) kept despite `00_phase.md` having 8/9 swapped
|
||||
|
||||
**Next pending task:** `.agents/phases/todo/126_pdf_support/03_pdf_chunking.md`
|
||||
@@ -0,0 +1,113 @@
|
||||
........................................................................ [ 2%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 80%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
............. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 262 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 179 1 99%
|
||||
app/api/git_sources.py 241 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 139 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 256 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 132 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 374 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 206 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 123 0 100%
|
||||
app/rag/git_sources.py 38 0 100%
|
||||
app/rag/importer.py 432 10 98%
|
||||
app/rag/llm.py 244 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 81 0 100%
|
||||
app/rag/prompts.py 144 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 49 0 100%
|
||||
app/schemas.py 400 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 4969 23 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -0,0 +1 @@
|
||||
Now let me look at the current chunker, existing tests, and completed task files for this phase.
|
||||
@@ -0,0 +1,113 @@
|
||||
........................................................................ [ 2%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 80%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
............. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 262 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 179 1 99%
|
||||
app/api/git_sources.py 241 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 139 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 256 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 132 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 374 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 206 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 123 0 100%
|
||||
app/rag/git_sources.py 38 0 100%
|
||||
app/rag/importer.py 432 10 98%
|
||||
app/rag/llm.py 244 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 81 0 100%
|
||||
app/rag/prompts.py 144 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 49 0 100%
|
||||
app/schemas.py 400 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 4969 23 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -0,0 +1 @@
|
||||
Now let me look at the current chunker, existing tests, and completed task files for this phase.
|
||||
@@ -0,0 +1,113 @@
|
||||
........................................................................ [ 2%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 29%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 53%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 77%]
|
||||
........................................................................ [ 80%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 99%]
|
||||
............. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 262 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 179 1 99%
|
||||
app/api/git_sources.py 241 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 139 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 256 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 132 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 374 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 206 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 123 0 100%
|
||||
app/rag/git_sources.py 38 0 100%
|
||||
app/rag/importer.py 432 10 98%
|
||||
app/rag/llm.py 244 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 81 0 100%
|
||||
app/rag/prompts.py 144 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 49 0 100%
|
||||
app/schemas.py 400 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 4969 23 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -0,0 +1,12 @@
|
||||
Everything is verified. Final state:
|
||||
|
||||
- **Implemented**
|
||||
- `app/rag/chunker.py` — `_chunk_pdf` page-aware chunker (`---PAGE-N---` markers as primary boundary, `Page N:` anchor on every chunk, anchor-budgeted to the hard cap) + `.pdf` dispatch. **This backfills task 03, whose task file was marked complete but whose code was never written** (truncated executor report; the chunker had no PDF branch).
|
||||
- `app/rag/importer.py` — PDF branch in `_index_file` (after image, before text); `_index_pdf_file` (extract → text-digest upsert with the full unchanged/D1/D4 dance → shared chunk+embed → lite summary → image pass); `_extract_pdf_title` (metadata title, 500-char cap, stem fallback); `_index_pdf_embedded_images`/`_index_one_embedded_image` (A3: `BOR_IMAGES`-gated, the single `_describe_or_skip` seam, `image_dir` copy written only after a successful description, stale-copy delete on change, no-orphan cleanup on embed failure, synthetic `#page=N#img=I` paths, added/updated counts).
|
||||
- Tests: +17 unit in `test_pdf.py` (title ×6, index paths, images-off, unchanged+backfill+manual-date, corrupt/empty PDF, unsupported mime, describe- and embed-failure, changed-image copy replacement), +7 chunker unit, +3 integration (real seam).
|
||||
|
||||
- **Results**: `uv run pytest --cov=app` → **2993 passed**, TOTAL **99%** (gate >90%; touched modules: chunker 98%, importer 98%, new code fully covered); `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors.
|
||||
|
||||
- **Notable decisions**: task-04 file's simplified spec wins over the 00_phase sketch — PDF `content` is the raw extracted text (no inline `[Embedded image: …]` placeholders); unchanged PDF (same text digest) skips the image pass entirely, so a transient embedded-image describe failure is retried on the next PDF text change (documented in docstrings); per-image embed failure is caught (rollback + copy delete + `errors`), not propagated. A chunk-order flake was fixed with `order_by(Chunk.position)` in the new tests.
|
||||
|
||||
- **Next pending task**: `.agents/phases/todo/126_pdf_support/05_pdf_archive_and_tests.md`
|
||||
@@ -0,0 +1,113 @@
|
||||
........................................................................ [ 2%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 98%]
|
||||
......................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 262 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 179 1 99%
|
||||
app/api/git_sources.py 241 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 139 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 256 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 132 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 374 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 123 0 100%
|
||||
app/rag/git_sources.py 38 0 100%
|
||||
app/rag/importer.py 555 9 98%
|
||||
app/rag/llm.py 244 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 81 0 100%
|
||||
app/rag/prompts.py 144 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 49 0 100%
|
||||
app/schemas.py 400 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5106 22 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -0,0 +1,19 @@
|
||||
All green — the previous rejection was a harness issue (missing final report), not a code failure. All changes from the prior attempt remain intact in the working tree and have been fully re-verified.
|
||||
|
||||
**Report**
|
||||
|
||||
**Implemented**
|
||||
- `app/rag/chunker.py` — `_chunk_pdf` page-aware chunker (`---PAGE-N---` as primary boundary, `Page N:` anchor on every chunk, anchor-budgeted to the hard cap) + `.pdf` dispatch (backfills task 03, whose code was never written despite the file being marked complete)
|
||||
- `app/rag/importer.py` — PDF branch in `_index_file` (after image, before text); `_index_pdf_file` (extract → text-digest upsert with the unchanged/D1/D4 dance → shared chunk+embed → lite summary → image pass); `_extract_pdf_title` (metadata title, 500-char cap, stem fallback); `_index_pdf_embedded_images`/`_index_one_embedded_image` (A3: `BOR_IMAGES`-gated, single `_describe_or_skip` seam, `image_dir` copy only after successful description, stale-copy delete, no-orphan cleanup on embed failure, `#page=N#img=I` paths)
|
||||
- Tests: +17 unit in `test_pdf.py`, +7 chunker unit, +3 integration (real describe seam)
|
||||
|
||||
**Results**
|
||||
- `uv run pytest --cov=app` → **2993 passed**, TOTAL **99%** (gate >90%; touched modules: chunker 98%, importer 98%, new code fully covered)
|
||||
- `uv run ruff check .` → all checks passed; `uv run pyright` → 0 errors, 0 warnings
|
||||
|
||||
**Notable decisions**
|
||||
- Task-04's simplified spec wins over the 00_phase sketch: PDF content is raw extracted text (no inline image placeholders); unchanged PDF (same text digest) skips the image pass — transient describe failures retry on the next text change (documented); per-image embed failure is caught (rollback + copy delete + `errors`)
|
||||
- Chunk-order flake fixed with `order_by(Chunk.position)` in new tests
|
||||
|
||||
**Next pending task**
|
||||
- `.agents/phases/todo/126_pdf_support/05_pdf_archive_and_tests.md`
|
||||
@@ -0,0 +1,113 @@
|
||||
........................................................................ [ 2%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 98%]
|
||||
......................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 262 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 179 1 99%
|
||||
app/api/git_sources.py 241 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 139 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 256 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 132 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 374 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 123 0 100%
|
||||
app/rag/git_sources.py 38 0 100%
|
||||
app/rag/importer.py 555 9 98%
|
||||
app/rag/llm.py 244 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 81 0 100%
|
||||
app/rag/prompts.py 144 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 49 0 100%
|
||||
app/schemas.py 400 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5106 22 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
@@ -0,0 +1,21 @@
|
||||
All green in the final state. Summary report:
|
||||
|
||||
**Phase 126 task 05 — PDFs in archives + full test suite: complete**
|
||||
|
||||
Implemented:
|
||||
- Verified archive path needs no code: `unpack_archive` extracts all files; the `pdf` token in `BOR_IMPORT_EXTENSIONS` admits PDFs (proven end-to-end via the E2E upload→sync flow)
|
||||
- `.env.example`: documented the `pdf` token under `BOR_IMPORT_EXTENSIONS`
|
||||
- **Defect fix (task 04 work):** `_prune` destroyed PDF-embedded image docs in the same sync that indexed them (synthetic `#page=N#img=I` paths never enter the walk's `seen`); added the parent-PDF rule (skip while the parent rel is walked; prune with the parent when it leaves)
|
||||
- **Parity fix:** `_index_one_embedded_image` now runs the phase-125 task-03 markdown context scan (embedded images referenced by a same-source markdown are enriched in both walk orderings, `image_context` stored, context → vision prompt + content/summary)
|
||||
- `tests/integration/test_importer.py` +4 tests: no-text PDF (empty doc, no content chunks), prune parent rule, markdown-first (record stored + enrichment), PDF-first (ordering-independent enrichment)
|
||||
- `tests/e2e/test_pdf_support.py` (new, 6 tests): module apps images ON/OFF, fitz-built PDF (2 text pages + embedded PNG) via the real admin upload → Sources rows (text row + thumbnail) → viewer renders extracted text (A4) + image doc → chat grounds on the PDF (scripted read echo) with the embedded image's auto-cited chip + figure (ON) / no image chip (OFF)
|
||||
|
||||
Results:
|
||||
- `uv run pytest` → 2997 passed; `uv run pytest --cov=app --cov-report=term-missing` → TOTAL 99% (gate >90%)
|
||||
- `uv run pytest tests/e2e/test_pdf_support.py -v --no-cov` → 6 passed (isolation)
|
||||
- `uv run ruff check . && uv run pyright` → clean (0 errors)
|
||||
- Regression E2E: `test_image_documents.py` + `test_image_anchor_context.py` → 8 passed; `test_archive_upload_sources.py` → 5 passed
|
||||
|
||||
Notable: the two importer fixes above were required for the phase-126 completion criteria ("images ON → embedded images become image documents") to hold under a real `prune=True` sync.
|
||||
|
||||
Next pending task: none — task 05 was this phase's final task (phase dir move + atomic commit are harness-gated).
|
||||
+113
@@ -0,0 +1,113 @@
|
||||
........................................................................ [ 2%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 16%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 21%]
|
||||
........................................................................ [ 24%]
|
||||
........................................................................ [ 26%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 40%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 45%]
|
||||
........................................................................ [ 48%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 64%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 69%]
|
||||
........................................................................ [ 72%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 88%]
|
||||
........................................................................ [ 91%]
|
||||
........................................................................ [ 93%]
|
||||
........................................................................ [ 96%]
|
||||
........................................................................ [ 98%]
|
||||
............................................. [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 262 1 99%
|
||||
app/api/chat_images.py 50 0 100%
|
||||
app/api/chats.py 110 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 179 1 99%
|
||||
app/api/git_sources.py 241 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 139 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 256 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 39 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/theming.py 38 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 68 0 100%
|
||||
app/models.py 132 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 374 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 123 0 100%
|
||||
app/rag/git_sources.py 38 0 100%
|
||||
app/rag/importer.py 563 9 98%
|
||||
app/rag/llm.py 244 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 71 0 100%
|
||||
app/rag/pdf.py 81 0 100%
|
||||
app/rag/prompts.py 144 0 100%
|
||||
app/rag/retriever.py 248 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 49 0 100%
|
||||
app/schemas.py 400 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 5114 22 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
Reference in New Issue
Block a user