phase: 125_image_anchor_context
All completion criteria verified. The repository is left functional with all changes in the working tree (no commits made, per the harness protocol). ## Final report — phase 125 `image_anchor_context` (verification pass) All 6 tasks were already complete; this pass verified every criterion and ran the full gates: - **Migration 0023** `documents.image_context` (TEXT nullable, reversible) — applied (`alembic current` = 0023 head); round-trip test (4 passed) exercises real upgrade → downgrade 0022 → upgrade. - **Criterion 1** (referenced image enriched + context line in retrieval): integration `test_markdown_referenced_image_gets_context_and_enriched_chunks` + E2E `test_sync_enriches_the_image_doc_with_markdown_context`, `test_chat_answer_quotes_the_context_line` (wire prompt pinned: `<document>` block leads with the one-line `Referenced in …: "alt"` line) — verified present and passing. - **Criterion 2** (standalone image byte-identical): E2E `test_standalone_*` (NULL `image_context`, block body exactly the bare description, no `Referenced in` anywhere in the prompt) + unit `test_describe_image_without_context_is_byte_identical` — passing. - **Criterion 3** (markdown-first: record stored on markdown doc, image picks up context later): integration `test_markdown_indexed_before_image_stores_record_and_image_picks_it_up` + post-index enrichment pass test (markdown file gone from disk) — passing. - **Gates:** `uv run pytest` → 2938 passed, exit 0 · `uv run pytest --cov=app --cov-report=term-missing` → exit 0, TOTAL **99%** (>90%) · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors, 0 warnings. - **Phase E2E in isolation:** `uv run pytest tests/e2e/test_image_anchor_context.py -v --no-cov` → 4 passed. - **Mock-LLM change regression check:** neighboring image E2E suites green in isolation — `test_smoke` (3), `test_image_documents` (16), `test_chat_image_questions` (17), `test_answer_images` (17). - No defects found; no code changes needed. No live-infrastructure changes (local dev DB only). No commit made (harness commits). - Next pending phase: check `.agents/phases/todo/` after the harness moves this phase.
This commit is contained in:
@@ -0,0 +1,54 @@
|
||||
# Phase 125 — Image-anchor context: markdown image references enrich image documents with surrounding text
|
||||
|
||||
**Source:** `TODO.md` — "improve image handling: use context of surrounding text to describe images, use alt text from markdown image syntax `` to give context to the summary."
|
||||
**Story:** n/a (improvement on phase-122/124 image capability).
|
||||
**Context:** `app/config.py` — `Settings` (`images` L379, `image_extensions` L390, `image_dir` L413). `app/rag/importer.py` — `iter_importable_files` (L235, extension walk), `import_sources` (L279), `_index_file` (L435 — text branch for markdown, binary branch delegates to `_index_image_file`), `_index_image_file` (L801 — bytes digest, persistent copy, `describe_image` seam, chunk + embed pipeline), `_store_summary` (L582), `_describe_or_skip` (L762 — calls `describe_image`). `app/rag/summarizer.py` — `describe_image` (L208 — multimodal chat call, DESCRIBE_PROMPT constant), `DESCRIBE_INSTRUCTION` (L89 — the locked prompt text), `generate_summary` (L182), `build_summary_prompt` (L162). `app/rag/chunker.py` — markdown section splitter on `##` headings, paragraph packing. `app/models.py` — `Document` (L102: `content`, `summary`, `is_image`, `image_path`), `Chunk` (L171). `app/rag/retriever.py` — `_selection_order` (L666, shared selection walk), `select_suggested` (L864, seeds prompt summaries), `source_ref_with_image` (L945). `app/rag/prompts.py` — `<document>` block body (L438, uses `doc.summary`), `build_high_prompt` (L461 — the LOCKED `<tools>` section, phase-124 `show_image` clause). `app/api/chat.py` — done-frame build (L1119-1133, `cited_docs` merge from task 05 of phase 124). `alembic/versions/0022_documents_image.py` — the phase-122 migration (the pattern for additive, reversible migrations).
|
||||
|
||||
## Objective
|
||||
When a markdown file references an image (e.g. ``), the image document's description is enriched with the alt text and surrounding paragraphs — so the image becomes retrievable not just by what it shows, but by why it exists in the documentation. This closes the gap between standalone image descriptions (phase 122) and the rich contextual text that lives in the parent markdown document.
|
||||
|
||||
## Dependencies
|
||||
- `122_image_documents` (complete) — the image-doc pipeline: `Document.is_image`/`image_path`, `describe_image`, the binary index path, `BOR_IMAGES` toggle, the vision description as content+summary.
|
||||
- `124_image_chips_and_answers` (complete) — auto-cite for image docs, `BOR_ANSWER_IMAGES` toggle, the `show_image` tool.
|
||||
- Code dependencies (all complete): phase 30 summary pipeline, phase 118 summary seeding, phase 119 read-docs citation tier.
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
- **The `image_context` field (task 01):** `Document.image_context: str | None = None` (TEXT, nullable). Stores the surrounding text extracted from the markdown file that references the image — alt text + N paragraphs before and after the reference. NULL for image docs that were not found to have a markdown reference (standalone images, images referenced in non-markdown formats, pre-phase rows). Migration: one additive, reversible migration (A13 precedent: `0023_documents_image_context.py`).
|
||||
- **Markdown image extraction (task 02):** a new module-level function `extract_image_refs(content: str) -> list[ImageRef]` in `app/rag/importer.py` (or `app/rag/md_images.py` if preferred) that parses markdown content and extracts: (a) the `` pattern via regex; (b) for each reference, the N paragraphs before and after the line containing the reference (configurable, default 2); (c) the line number of the reference for context window calculation. The regex: `r'!\[([^\]]*)\]\(([^)]+)\)'` — captures alt text and the image path. The context window: split content into paragraphs (blank-line separated, respecting code fences), find the paragraph containing the reference, take N before and N after. Result: `ImageRef(path, alt_text, context_text, line_number)`.
|
||||
- **Enrichment flow (task 03):** after `_index_file` processes a markdown file, check if it contains image references. For each reference whose image path resolves to an existing image document in the same source, load the image doc and enrich its content: `new_content = f"{alt_text}\n\n{context}\n\n{original_description}"` — the alt text first (short, purposeful), then the surrounding context (why the image exists), then the vision description (what the image shows). Then re-chunk and re-embed the enriched content. If the image doc doesn't exist yet (not indexed), store the reference on the markdown doc's `image_context` field so the enrichment can happen when the image is later indexed.
|
||||
- **Enrichment at image-index time (task 04):** in `_index_image_file`, BEFORE calling `describe_image`, check if any markdown file in the same source references this image. If so, load the markdown content, extract the reference + context, and pass it to `describe_image` as additional context in the prompt (the vision model can acknowledge the context: "This diagram, referenced in `<markdown_path>` as `<alt_text>`, shows..."). This is the primary enrichment path — the image gets context during its own indexing. The `image_context` field on the image doc stores the extracted context text for retrieval display.
|
||||
- **Retrieval display (task 05):** when an image doc is in the suggested tier, the `<document>` block (which currently shows `doc.summary`) is extended to also show the `image_context` (if present) — a short "Referenced in `<source>/<markdown_path>`: `<alt_text>`" line before the description. This is a small addition to `build_document_block` in `app/rag/prompts.py` — only for `is_image` docs with non-NULL `image_context`.
|
||||
- **NOT touched:** the `BOR_IMAGES` toggle (the feature is gated by it, unchanged), the `BOR_ANSWER_IMAGES` toggle (unchanged), the `show_image` tool (unchanged), the image serve route (unchanged), the document viewer (unchanged), and the prune logic (unchanged). Existing image docs without context are unaffected — `image_context` is NULL, the prompt block shows only the description (byte-identical to pre-phase).
|
||||
- **Locked assumptions:** **A1** — `image_context` stores the surrounding text from the referencing markdown file. **A2** — enrichment happens at image-index time (the image doc picks up context from its markdown parent). **A3** — the vision model receives context in the prompt but the description remains the primary content. **A4** — retrieval display adds a short context line for enriched image docs; pre-phase image docs are byte-identical.
|
||||
|
||||
## Tasks
|
||||
1. `01_migration_image_context.md` — `Document.image_context` column + migration.
|
||||
2. `02_extract_image_refs.md` — `extract_image_refs()` function for markdown image reference extraction.
|
||||
3. `03_enrich_image_description.md` — pass context to `describe_image`; enrich image doc content + re-embed.
|
||||
4. `04_store_image_context.md` — store context on image docs; handle the case where markdown is indexed before/after the image.
|
||||
5. `05_retrieval_display.md` — show context line for enriched image docs in the `<document>` block.
|
||||
6. `06_image_anchor_tests.md` — unit + integration + E2E tests.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_image_anchor_context.py` (new, task 06) — `extract_image_refs` (basic reference, multiple references, references in code fences, references with no alt text, references in headings, edge cases); the enrichment pipeline (context extraction, content enrichment, re-chunking, re-embedding); the `describe_image` prompt extension (context included in the text part when provided); the retrieval display (context line present for enriched docs, absent for non-enriched).
|
||||
- Integration: `tests/integration/test_importer.py` (extend) — a markdown file referencing an image in the same source → the image doc has `image_context` set; the image description includes the context; the chunks are re-embedded with the enriched content. A markdown file referencing a non-existent image → the markdown doc's `image_context` is set (the image will be enriched when indexed). A standalone image (no markdown reference) → `image_context` is NULL (byte-identical to pre-phase).
|
||||
- E2E: `tests/e2e/test_image_anchor_context.py` (new, task 06) — isolated run: upload a fixture markdown file that references an image in the same archive → sync → the image doc has enriched context → ask a question that requires the context to answer correctly (the mock LLM should be able to use the context to give a better answer) → the `<document>` block shows the context line.
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] A markdown file referencing an image in the same source → the image document's description is enriched with alt text + surrounding context; the retrieval block shows the context line.
|
||||
- [ ] A standalone image (no markdown reference) → `image_context` is NULL; behavior byte-identical to pre-phase.
|
||||
- [ ] A markdown file referencing a non-existent image → the context is stored on the markdown doc; when the image is later indexed, it picks up the context.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agents/phases/complete/` by the pipeline gate.
|
||||
|
||||
## Locked decisions
|
||||
- **A1 — `image_context` stores the surrounding text (alt text + N paragraphs before/after) from the referencing markdown file (owner-confirmed 2026-09-26, roadmap confirmation).**
|
||||
- **A2 — enrichment happens at image-index time: the image doc picks up context from its markdown parent (owner-confirmed 2026-09-26, roadmap confirmation).**
|
||||
- **A3 — the vision model receives context in the prompt but the description remains the primary content (owner-confirmed 2026-09-26, roadmap confirmation).**
|
||||
- **A4 — retrieval display adds a short context line for enriched image docs; pre-phase image docs are byte-identical (owner-confirmed 2026-09-26, roadmap confirmation).**
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add app/ alembic/ tests/ .agents/phases/ && git commit --no-gpg-sign -m "feat(rag): enrich image descriptions with markdown context (alt text + surrounding paragraphs)"
|
||||
```
|
||||
@@ -0,0 +1,34 @@
|
||||
# Task 01 — Migration: `Document.image_context` column
|
||||
|
||||
**Phase:** `125_image_anchor_context` · **Source:** `00_phase.md` — the `image_context` field
|
||||
|
||||
## Objective
|
||||
Add `Document.image_context: str | None` (TEXT, nullable) to the database schema — the column that stores the surrounding text extracted from the markdown file referencing the image. One additive, reversible migration (A13 precedent).
|
||||
|
||||
## Work
|
||||
1. `app/models.py` — add to `Document` (after `image_path`, L168):
|
||||
```python
|
||||
#: Surrounding text from the markdown file that references this image
|
||||
#: (phase 125, LOCKED A1): alt text + N paragraphs before/after the
|
||||
#: reference. NULL for standalone images and pre-phase rows. The
|
||||
#: embedding model embeds this alongside the vision description when
|
||||
#: the image is enriched (task 03).
|
||||
image_context: Mapped[str | None] = mapped_column(Text, default=None)
|
||||
```
|
||||
2. `alembic/versions/0023_documents_image_context.py` — new migration:
|
||||
- `revision = "0023"`, `down_revision = "0022"`
|
||||
- `upgrade()`: `op.add_column("documents", sa.Column("image_context", sa.Text(), nullable=True))`
|
||||
- `downgrade()`: `op.drop_column("documents", "image_context")`
|
||||
- Docstring following the `0022_documents_image.py` pattern: explain the column, its purpose, and that it's fully reversible.
|
||||
3. ASSUMPTION: none. The column is nullable (NULL for all pre-phase rows) and additive (no existing code breaks).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_models.py` (extend) — `Document(image_context="some context")` parses; `Document(image_context=None)` parses; the column is nullable (no NOT NULL constraint).
|
||||
- Integration: `tests/integration/test_migrations.py` (extend) — run `alembic upgrade head` (the new migration applies cleanly); run `alembic downgrade 0022` (the downgrade restores the 0022 schema); run `alembic upgrade head` again (idempotent). Assert that pre-phase image docs (NULL `image_context`) round-trip correctly.
|
||||
- Coverage: **>90%** on the touched modules.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `alembic upgrade head` applies the new migration cleanly.
|
||||
- [ ] `alembic downgrade 0022` + `upgrade head` round-trips the schema.
|
||||
- [ ] `Document` model has the `image_context` field; NULL is valid.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Task 02 — Extract image references from markdown content
|
||||
|
||||
**Phase:** `125_image_anchor_context` · **Source:** `00_phase.md` — the `extract_image_refs()` function
|
||||
|
||||
## Objective
|
||||
A pure function that parses markdown content and extracts all `` references, each with its alt text, the surrounding paragraph context (N before and N after), and the line number. Returns a list of `ImageRef` dataclasses.
|
||||
|
||||
## Work
|
||||
1. `app/rag/md_images.py` — new module (or add to `app/rag/importer.py` if preferred — the executor's call, but a separate module keeps the importer lean):
|
||||
- `@dataclass class ImageRef: path: str; alt_text: str; context: str; line_number: int` — the extracted reference with its context window.
|
||||
- `CONTEXT_PARAGRAPHS: int = 2` — configurable number of paragraphs before/after (the default gives ~2-4 sentences of surrounding text).
|
||||
- `def extract_image_refs(content: str, context_paragraphs: int = CONTEXT_PARAGRAPHS) -> list[ImageRef]`:
|
||||
- Split content into lines.
|
||||
- Track code fence state (a fenced code block is NOT context — image references inside code fences are just text, not actual references).
|
||||
- For each line, check for the image reference regex: `r'!\[([^\]]*)\]\(([^)]+)\)'`.
|
||||
- When a reference is found:
|
||||
- Record the line number (1-indexed).
|
||||
- Find the paragraph containing this line (paragraphs are separated by blank lines, but code fences are atomic).
|
||||
- Collect N paragraphs before and N after the reference paragraph.
|
||||
- The `context` is the joined paragraphs (excluding the reference line itself, which is already captured as `alt_text` + `path`).
|
||||
- The `alt_text` is the captured group (may be empty).
|
||||
- The `path` is the captured group (may be relative or absolute).
|
||||
- Return the list of `ImageRef` objects.
|
||||
- Edge cases to handle:
|
||||
- References inside code fences → skip (the reference is literal text, not an actual image).
|
||||
- Multiple references on the same line → extract each separately.
|
||||
- References with no alt text (``) → `alt_text = ""`.
|
||||
- References with no surrounding text (start/end of file) → context is whatever is available.
|
||||
- Image paths with special characters (spaces, parentheses) → the regex should handle common cases; edge cases with nested parentheses in paths are acceptable to miss (they're rare in practice).
|
||||
2. ASSUMPTION: none. This is a pure function — no I/O, no LLM calls, no database access.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_md_images.py` (new, task 06) — test `extract_image_refs` against various inputs:
|
||||
- Single reference: `` → one `ImageRef` with alt="alt", context from surrounding paragraphs.
|
||||
- Multiple references: two `![...]` on different lines → two `ImageRef`s, each with its own context.
|
||||
- Reference in code fence: ````  ```` → skipped (not extracted).
|
||||
- Reference with no alt text: `` → `alt_text = ""`.
|
||||
- Reference at start of file → context is only paragraphs after.
|
||||
- Reference at end of file → context is only paragraphs before.
|
||||
- Reference in a heading: `## ` → extracted, but context is the surrounding text (the heading line itself is not included in context).
|
||||
- Empty content → empty list.
|
||||
- Content with no references → empty list.
|
||||
- Verify that `line_number` is 1-indexed and correct.
|
||||
- Coverage: **>90%** on the new module.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `extract_image_refs("")` returns one `ImageRef` with correct alt, path, context, and line number.
|
||||
- [ ] References inside code fences are skipped.
|
||||
- [ ] Multiple references are extracted separately with correct context windows.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,49 @@
|
||||
# Task 03 — Enrich image descriptions with context
|
||||
|
||||
**Phase:** `125_image_anchor_context` · **Source:** `00_phase.md` — pass context to `describe_image`; enrich image doc content + re-embed
|
||||
|
||||
## Objective
|
||||
When indexing an image file, check if any markdown file in the same source references it. If so, extract the alt text and surrounding context, and pass it to `describe_image` so the vision model can produce a context-aware description. Store the extracted context on the image doc's `image_context` field.
|
||||
|
||||
## Work
|
||||
1. `app/rag/summarizer.py` — extend `describe_image`:
|
||||
- Add an optional `context: str | None = None` parameter.
|
||||
- When `context` is provided, prepend it to the DESCRIBE_PROMPT text part:
|
||||
```python
|
||||
context_prefix = f"\n\nThis image is referenced in documentation with the following context:\n{context}" if context else ""
|
||||
messages = [{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": DESCRIBE_PROMPT + context_prefix},
|
||||
{"type": "image_url", "image_url": {"url": data_url}},
|
||||
],
|
||||
}]
|
||||
```
|
||||
- The context prefix teaches the vision model what the image is for, enabling a more useful description. The existing DESCRIBE_PROMPT (the locked instruction) is unchanged — the context is appended after it.
|
||||
- The context is NOT stored on the doc (that's task 04's job); this task only passes it to the LLM.
|
||||
2. `app/rag/importer.py` — in `_index_image_file`, BEFORE calling `describe_image`:
|
||||
- Check if any markdown file in the same source references this image. The simplest approach: scan all `.md`/`.markdown` files in the same source directory for references to this image's filename. Use `extract_image_refs` from task 02.
|
||||
- For each markdown file that references this image, collect the `ImageRef` objects where the image path matches (handle both relative paths like `image.png` and paths like `folder/image.png`).
|
||||
- If references are found, combine them into a single context string:
|
||||
```
|
||||
Referenced in <source>/<markdown_path>:
|
||||
- Alt text: "<alt_text>"
|
||||
Context: <context_paragraphs>
|
||||
```
|
||||
- Pass this context to `describe_image` as the `context` parameter.
|
||||
- Store the combined context string on the image doc's `image_context` field (this is the primary storage — the retrieval display in task 05 reads from here).
|
||||
3. `app/rag/importer.py` — in `_index_image_file`, after `describe_image` returns:
|
||||
- If `image_context` is set (from step 2), the image doc's `content` should be enriched: `content = f"{image_context}\n\n{description}"` — the context first (for retrieval), then the description. Then the normal chunk pipeline embeds this enriched content.
|
||||
- If `image_context` is NULL (no markdown reference found), the content is just the description (byte-identical to pre-phase).
|
||||
- The `summary` field: for image docs, `summary = content` (the phase-122 convention). So the summary is also enriched.
|
||||
4. ASSUMPTION: none. The context extraction uses the `extract_image_refs` function from task 02. The `describe_image` extension is backward-compatible (the `context` parameter defaults to `None`).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_image_anchor_context.py` (task 06) — `describe_image` with `context=None` → same behavior as pre-phase (the DESCRIBE_PROMPT is unchanged); `describe_image` with `context="some context"` → the prompt includes the context prefix; the enriched content is `context + "\n\n" + description`.
|
||||
- Integration: `tests/integration/test_importer.py` (extend) — seed a source with both a markdown file referencing an image and the image file itself → sync with `images=True` → the image doc has `image_context` set, its `content` includes the context, its chunks are embedded from the enriched content. A standalone image (no markdown reference) → `image_context` is NULL, `content` is just the description.
|
||||
- Coverage: **>90%** on the touched modules.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] An image referenced in a markdown file → the description is context-aware (the LLM receives the context), and the image doc's `content` includes the context.
|
||||
- [ ] A standalone image → `image_context` is NULL, `content` is just the description (byte-identical to pre-phase).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,48 @@
|
||||
# Task 04 — Store context on markdown docs for future image enrichment
|
||||
|
||||
**Phase:** `125_image_anchor_context` · **Source:** `00_phase.md` — handle the case where markdown is indexed before/after the image
|
||||
|
||||
## Objective
|
||||
When a markdown file is indexed and references an image that doesn't yet exist as a document (e.g., the image hasn't been synced yet, or it's in a different source), store the image references on the markdown doc's `image_context` field. This enables a post-indexing enrichment pass that can pick up the context when the image is later indexed.
|
||||
|
||||
## Work
|
||||
1. `app/rag/importer.py` — in the markdown/text branch of `_index_file` (after the Document upsert, before the chunk pipeline):
|
||||
- After extracting the title (L505-509), check if the file is markdown (`full_path.suffix.lower() in (".md", ".markdown")`).
|
||||
- Call `extract_image_refs(content)` to get all image references.
|
||||
- For each reference, check if an image document already exists in the same source with a matching path. Use a database query: `select(Document).where(Document.source == source, Document.path == image_ref.path, Document.is_image == True)`.
|
||||
- For references where the image document DOES NOT exist, store the reference info on the markdown doc's `image_context` field:
|
||||
```python
|
||||
# Build a structured context string for references to non-existent images
|
||||
missing_refs = [r for r in image_refs if not image_exists_in_source(session, source, r.path)]
|
||||
if missing_refs:
|
||||
doc.image_context = _format_missing_image_refs(missing_refs, source)
|
||||
```
|
||||
- `_format_missing_image_refs(refs, source) -> str`: formats the missing references as a structured string:
|
||||
```
|
||||
IMAGE_CONTEXT:
|
||||
- [image.png] (line 42): "Alt text here" — 2 paragraphs of context
|
||||
- [diagram.jpg] (line 100): "" — 1 paragraph of context
|
||||
```
|
||||
This format is parseable by the enrichment pass (task 04b).
|
||||
- For references where the image document DOES exist, the enrichment happens in task 03 (at image-index time). The markdown doc's `image_context` remains NULL.
|
||||
2. `app/rag/importer.py` — a new helper `_format_missing_image_refs(refs, source) -> str` (module-level, pure function):
|
||||
- Takes a list of `ImageRef` objects and the source name.
|
||||
- Returns a formatted string as described above.
|
||||
- Used by both `_index_file` (markdown branch) and the enrichment pass.
|
||||
3. `app/rag/importer.py` — after `_index_file` returns for all files in a source (in `import_sources`), run a **post-indexing enrichment pass**:
|
||||
- Query all image documents in the source that have `image_context IS NULL` (not yet enriched).
|
||||
- For each, check if any markdown document in the same source has `image_context` set (contains references to this image).
|
||||
- If a match is found, extract the reference info from the markdown doc's `image_context`, enrich the image doc (same logic as task 03: re-call `describe_image` with context, update `content`, re-embed chunks).
|
||||
- This pass handles the case where the markdown was indexed first (storing references) and the image is indexed later.
|
||||
4. ASSUMPTION: none. The enrichment pass is idempotent — if an image doc already has `image_context` set, it's skipped. The markdown doc's `image_context` is not cleared after enrichment (it's useful for retrieval display too).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_image_anchor_context.py` (task 06) — `_format_missing_image_refs` (single ref, multiple refs, empty list → empty string); the enrichment pass (markdown with missing refs → image doc enriched when found; image doc already enriched → skipped).
|
||||
- Integration: `tests/integration/test_importer.py` (extend) — seed a source with a markdown file referencing an image that doesn't exist yet → sync → the markdown doc has `image_context` set; then add the image file → sync again → the image doc is enriched with the context from the markdown doc. A markdown file referencing an image that exists → the image doc is enriched at index time (task 03 path).
|
||||
- Coverage: **>90%** on the touched modules.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] A markdown file referencing a non-existent image → the markdown doc's `image_context` stores the reference info.
|
||||
- [ ] When the image is later indexed (or re-synced), it picks up the context from the markdown doc.
|
||||
- [ ] A markdown file referencing an existing image → the image is enriched at index time (task 03 path), the markdown doc's `image_context` is NULL.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Task 05 — Retrieval display: show context line for enriched image docs
|
||||
|
||||
**Phase:** `125_image_anchor_context` · **Source:** `00_phase.md` — show context line for enriched image docs in the `<document>` block
|
||||
|
||||
## Objective
|
||||
When an enriched image doc (one with non-NULL `image_context`) appears in the suggested tier, the `<document>` block shows a short context line before the description — so the LLM knows why the image exists in the documentation.
|
||||
|
||||
## Work
|
||||
1. `app/rag/prompts.py` — extend the document block builder (`_document_block_body`, L438):
|
||||
- Currently it returns `doc.summary` (stripped). For image docs with `image_context`, prepend a context line:
|
||||
```python
|
||||
summary = (doc.summary or "").strip()
|
||||
if doc.is_image and doc.image_context:
|
||||
# Extract the first reference's alt text for a short context line
|
||||
# The image_context is formatted as "Referenced in <source>/<path>: ..."
|
||||
# Show: "Referenced in <path>: <alt_text>"
|
||||
context_line = _extract_context_line(doc.image_context)
|
||||
if context_line:
|
||||
summary = f"{context_line}\n\n{summary}"
|
||||
return summary
|
||||
```
|
||||
- `_extract_context_line(image_context: str) -> str | None`: parses the `image_context` string (set by task 04) and extracts a short context line:
|
||||
```python
|
||||
# image_context format: "Referenced in <source>/<markdown_path>:\n- [image.png] (line 42): \"Alt text here\""
|
||||
# Return: "Referenced in <markdown_path>: \"Alt text here\""
|
||||
```
|
||||
If parsing fails or the format is unexpected, return `None` (graceful degradation — the summary shows without the context line).
|
||||
2. ASSUMPTION: none. The context line is short (one line) so it doesn't bloat the prompt. Pre-phase image docs (NULL `image_context`) are byte-identical — the `if doc.is_image and doc.image_context` guard ensures this.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_prompts.py` (extend) — `_document_block_body` for a non-image doc → unchanged (byte-identical to pre-phase); for an image doc with `image_context = None` → unchanged (byte-identical to pre-phase); for an image doc with `image_context` set → the summary includes the context line before the description; `_extract_context_line` (valid format → extracts the line; invalid format → returns None).
|
||||
- Integration: `tests/integration/test_chat_api.py` (extend) — a grounded turn whose suggested tier contains an enriched image doc → the `<documents>` section in the system prompt includes the context line; a non-enriched image doc → no context line.
|
||||
- Coverage: **>90%** on the touched modules.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] An enriched image doc in the suggested tier → the `<document>` block shows a context line before the description.
|
||||
- [ ] A non-enriched image doc → byte-identical to pre-phase (no context line).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,57 @@
|
||||
# Task 06 — Tests: unit, integration, and E2E for image-anchor context
|
||||
|
||||
**Phase:** `125_image_anchor_context` · **Source:** `00_phase.md` — the full test suite
|
||||
|
||||
## Objective
|
||||
Comprehensive tests across all new logic: the `extract_image_refs` function, the enrichment pipeline, the `describe_image` context extension, the retrieval display, and the end-to-end flow.
|
||||
|
||||
## Work
|
||||
1. `tests/unit/test_md_images.py` (new) — tests for `extract_image_refs`:
|
||||
- Single reference with alt text and surrounding paragraphs.
|
||||
- Multiple references on different lines, each with correct context windows.
|
||||
- References inside code fences (skipped).
|
||||
- References with no alt text (``).
|
||||
- References at start/end of file (context is partial).
|
||||
- References in headings.
|
||||
- Empty content, content with no references.
|
||||
- Verify `line_number` is 1-indexed and correct.
|
||||
- Edge cases: nested brackets in alt text, special characters in paths.
|
||||
2. `tests/unit/test_image_anchor_context.py` (new) — tests for the enrichment pipeline:
|
||||
- `describe_image` with `context=None` → same behavior as pre-phase.
|
||||
- `describe_image` with `context="..."` → the prompt includes the context prefix.
|
||||
- `_extract_context_line` (valid format → extracts the line; invalid format → returns None).
|
||||
- `_document_block_body` for enriched vs non-enriched image docs.
|
||||
- `_format_missing_image_refs` (single ref, multiple refs, empty list).
|
||||
- The enrichment pass (markdown with missing refs → image enriched; image already enriched → skipped).
|
||||
3. `tests/integration/test_importer.py` (extend) — integration tests:
|
||||
- Seed a source with a markdown file referencing an image → the image doc has `image_context` set.
|
||||
- A standalone image (no markdown reference) → `image_context` is NULL.
|
||||
- A markdown file referencing a non-existent image → the markdown doc stores the reference info.
|
||||
- Re-sync after adding the missing image → the image doc is enriched.
|
||||
- Verify that the enriched image doc's chunks are embedded from the enriched content.
|
||||
4. `tests/integration/test_chat_api.py` (extend) — integration tests:
|
||||
- A grounded turn whose suggested tier contains an enriched image doc → the done frame's system prompt includes the context line.
|
||||
- A non-enriched image doc → no context line (byte-identical to pre-phase).
|
||||
5. `tests/e2e/test_image_anchor_context.py` (new) — isolated E2E suite:
|
||||
- Module apps: `BOR_IMAGES=true` for this suite's app instance.
|
||||
- KB fixture: upload an archive containing a markdown file that references an image within the same archive (e.g., `docs/guide.md` references `diagram.png`).
|
||||
- Sync → the image doc is enriched with the markdown context.
|
||||
- Ask a question that requires the context to answer correctly (the mock LLM should use the context line to give a better answer).
|
||||
- Verify the `<document>` block in the prompt includes the context line (via the saved chat record).
|
||||
- A standalone image (no markdown reference) → the answer is byte-identical to pre-phase.
|
||||
- Run in isolation: `uv run pytest tests/e2e/test_image_anchor_context.py -v --no-cov`.
|
||||
6. Run the full test suite: `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Testing & Quality
|
||||
- All new logic is covered by unit tests (the `extract_image_refs` function, the enrichment pipeline, the context extraction/display).
|
||||
- Integration tests cover the full import + chat flow with seeded data.
|
||||
- E2E tests cover the real user path: upload → sync → ask → verify enriched answer.
|
||||
- The mock LLM should be configured to acknowledge the context line in its answer (extend the mock's prompt parser to recognize the context line format).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_md_images.py -v --no-cov` passes.
|
||||
- [ ] `uv run pytest tests/unit/test_image_anchor_context.py -v --no-cov` passes.
|
||||
- [ ] `uv run pytest tests/integration/test_importer.py -v --no-cov` passes (extended).
|
||||
- [ ] `uv run pytest tests/e2e/test_image_anchor_context.py -v --no-cov` passes in isolation.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%; `ruff` + `pyright` clean.
|
||||
- [ ] No regression in existing E2E suites (run `test_image_documents.py` and `test_answer_images.py` to verify).
|
||||
@@ -0,0 +1,79 @@
|
||||
# Phase 126 — PDF support: PDFs become first-class, retrievable documents
|
||||
|
||||
**Source:** `TODO.md` — "add PDF support" (implied by the image handling assessment: "PDFs: Not supported. No PDF parsing library is in pyproject.toml.")
|
||||
**Story:** n/a (new file format support).
|
||||
**Context:** `app/config.py` — `Settings` (`import_extensions` L343, `import_extension_set` L733 — the extension filter; `images` L379, `image_extensions` L390, `image_dir` L413 — the image pipeline); `app/rag/importer.py` — `iter_importable_files` (L235, extension walk via `match_extension`), `import_sources` (L279, `prune` L283), `_index_file` (L435 — text branch: `read_text`, sha256 over text, Document upsert, chunk + embed pipeline), `_index_image_file` (L801 — binary branch: bytes digest, persistent copy, `describe_image`), `_store_summary` (L582), `_prune` (L649). `app/rag/chunker.py` — format-aware chunker (markdown, yaml, json, py, txt dispatch). `app/rag/summarizer.py` — `describe_image` (L208, vision model), `generate_summary` (L182, lite model). `app/rag/retriever.py` — `_selection_order` (L666), `select_suggested` (L864), `source_ref_with_image` (L945). `app/models.py` — `Document` (L102), `Chunk` (L171). `app/rag/llm.py` — the LLM client. `app/api/docs.py` — document content endpoint. `app/rag/archive_upload.py` — archive unpack (PDFs in archives should also be indexed).
|
||||
|
||||
## Objective
|
||||
PDF files become first-class, retrievable documents: text is extracted page by page, embedded images are extracted and described (joining the standalone image pipeline), and the full PDF content is chunked, embedded, and summarized — all behind a new `BOR_IMPORT_EXTENSIONS` token (`pdf`).
|
||||
|
||||
## Dependencies
|
||||
- `122_image_documents` (complete) — the image-doc pipeline: `describe_image`, `Document.is_image`/`image_path`, the vision model description path. Embedded PDF images reuse this pipeline.
|
||||
- `125_image_anchor_context` (complete) — the `image_context` field and enrichment pipeline (PDF-embedded images can be enriched if referenced by a markdown file in the same source).
|
||||
- Code dependencies (all complete): phase 30 summary pipeline, phase 118 summary seeding, the chunker, the LLM client.
|
||||
|
||||
## Design (shared by all tasks — the executor reads this, not the chat)
|
||||
- **Dependency (task 01):** add `pymupdf` (PyMuPDF, the `fitz` package) to `pyproject.toml` dependencies. PyMuPDF is chosen over `pdfplumber` because it natively supports both text extraction AND embedded image extraction in a single API, and it's a well-maintained package with good performance. The `fitz` module is the import name.
|
||||
- **PDF parsing module (task 02):** `app/rag/pdf.py` — new module with:
|
||||
- `def extract_pdf_text(pdf_path: Path) -> str`: extracts text from all pages, joined with `---PAGE-N---` separators (where N is 1-indexed). The separators enable the chunker to respect page boundaries.
|
||||
- `def extract_pdf_images(pdf_path: Path) -> list[PdfImage]`: extracts embedded images from all pages. Returns a list of `PdfImage` dataclasses: `page: int`, `index: int` (image index on the page), `width: int`, `height: int`, `data: bytes`, `mime: str` (derived from the image type code via PyMuPDF's `page.get_image_info()`).
|
||||
- `_mime_from_image_info(info: dict) -> str`: maps PyMuPDF's image type codes to MIME types (e.g., `2` → `image/jpeg`, `3` → `image/png`, `4` → `image/gif`, `8` → `image/webp`, `9` → `image/tiff`). Falls back to `application/octet-stream` for unknown types.
|
||||
- All functions are pure (except I/O for reading the PDF file) — fully unit-testable with a temp PDF fixture.
|
||||
- **PDF chunking (task 03):** extend `app/rag/chunker.py`'s `chunk_document` dispatch to handle `.pdf` files. The PDF chunker:
|
||||
- Splits on `---PAGE-N---` separators first (one chunk per page, or sub-split if a page is too large).
|
||||
- Each chunk keeps its page anchor (e.g., "Page 3:" prefix) so hits are readable in context.
|
||||
- Code blocks within PDFs are handled by the existing paragraph packing (PDFs don't have code fences, so paragraph boundaries = blank lines).
|
||||
- The `HARD_MAX_CHARS` cap (1200) still applies.
|
||||
- For very large PDFs (>50 pages), the chunk target is reduced to keep individual chunks manageable.
|
||||
- **PDF indexing (task 04):** in `app/rag/importer.py`'s `_index_file`:
|
||||
- Add a new branch for `.pdf` files (checked AFTER the image branch, BEFORE the text branch):
|
||||
```python
|
||||
if full_path.suffix.lower() == ".pdf":
|
||||
return await _index_pdf_file(session, source, rel, full_path, llm, summary, raw_date)
|
||||
```
|
||||
- `_index_pdf_file`:
|
||||
1. Extract text via `extract_pdf_text` (task 02).
|
||||
2. Extract embedded images via `extract_pdf_images` (task 02).
|
||||
3. Create the PDF document row with the extracted text as `content`, the file's sha256 (over the text) as `content_hash`, and the title from the PDF's metadata (if available) or the file stem.
|
||||
4. Chunk and embed the text (task 03's chunker handles `.pdf`).
|
||||
5. Generate a lite summary of the PDF content.
|
||||
6. For each embedded image:
|
||||
- If the image extension matches `BOR_IMAGE_EXTENSIONS` and `BOR_IMAGES` is on: create a standalone image document (reuse `_index_image_file`'s logic, but the image bytes come from the PDF, not a file on disk). The image's `path` is `<pdf_source>/<pdf_path>#page=<N>#img=<index>` (a synthetic path that uniquely identifies the embedded image).
|
||||
- If the image extension doesn't match or `BOR_IMAGES` is off: log a warning and skip the image.
|
||||
- The embedded image's description becomes the image doc's content+summary (the phase-122 pipeline).
|
||||
7. The PDF document's content includes inline references to embedded images: after each page's text, append `[Embedded image: <image_type> on page <N>]` (a placeholder that the LLM can use to know images exist, even if they're not described).
|
||||
- **PDF in archives (task 05):** PDFs inside uploaded archives should be indexed too. The archive unpack (`app/rag/archive_upload.py`) already extracts all files matching the import extensions. Add `pdf` to the archive's accepted file types (archives already unpack everything; the extension filter in `iter_importable_files` handles the rest).
|
||||
- **NOT touched:** the `BOR_IMAGES` toggle (PDF embedded images reuse the existing image pipeline, gated by it), the `show_image` tool (unchanged), the image serve route (unchanged), the document viewer (PDFs are served as text content, not as binary PDFs — the viewer renders the extracted text), and the prune logic (unchanged).
|
||||
- **Locked assumptions:** **A1** — `pymupdf` is the PDF parser (text + image extraction). **A2** — one document per PDF (not per page); pages are separated by `---PAGE-N---` markers. **A3** — embedded images become standalone image documents (reusing the phase-122 pipeline). **A4** — PDFs are served as extracted text in the document viewer (not as binary PDFs).
|
||||
|
||||
## Tasks
|
||||
1. `01_pdf_dependency.md` — add `pymupdf` to `pyproject.toml`.
|
||||
2. `02_pdf_parser.md` — `app/rag/pdf.py`: text extraction, image extraction, MIME mapping.
|
||||
3. `03_pdf_chunking.md` — extend the chunker to handle `.pdf` files.
|
||||
4. `04_pdf_indexing.md` — `_index_pdf_file` in the importer: text + embedded images + summary.
|
||||
5. `05_pdf_archive_and_tests.md` — PDFs in archives, unit + integration + E2E tests.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_pdf.py` (new, task 05) — `extract_pdf_text` (multi-page PDF → text with page separators; single-page PDF → text without extra separators; empty PDF → empty string); `extract_pdf_images` (PDF with embedded images → list of `PdfImage` with correct page/index/size/data/mime; PDF with no images → empty list); `_mime_from_image_info` (known types → correct MIME; unknown type → fallback).
|
||||
- Integration: `tests/integration/test_importer.py` (extend) — seed a source with a PDF file → sync → the PDF document has extracted text, chunks, and embedding; embedded images (if `BOR_IMAGES=true`) become image documents; a PDF without embedded images → no image docs.
|
||||
- E2E: `tests/e2e/test_pdf_support.py` (new, task 05) — isolated run: upload a PDF via the admin upload flow → sync → the PDF appears on the Sources page → open the document viewer → the extracted text renders; ask a question the PDF answers → the chat grounds on the PDF content.
|
||||
- Coverage: **>90%** on `app/` (validate.sh gate).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] A PDF file in a git/local source or uploaded archive becomes a document with extracted text, chunks, embedding, and summary.
|
||||
- [ ] With `BOR_IMAGES=true`: embedded images in the PDF become standalone image documents (described by the vision model).
|
||||
- [ ] With `BOR_IMAGES=false`: embedded images are skipped (logged), the PDF text is indexed normally.
|
||||
- [ ] The PDF renders as extracted text in the document viewer.
|
||||
- [ ] `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
- [ ] One `--no-gpg-sign` commit; phase dir moved to `.agents/phases/complete/` by the pipeline gate.
|
||||
|
||||
## Locked decisions
|
||||
- **A1 — `pymupdf` (PyMuPDF) is the PDF parser; it handles both text and embedded image extraction (owner-confirmed 2026-09-26, roadmap confirmation).**
|
||||
- **A2 — one document per PDF; pages are separated by `---PAGE-N---` markers in the content (owner-confirmed 2026-09-26, roadmap confirmation).**
|
||||
- **A3 — embedded images become standalone image documents, reusing the phase-122 pipeline (gated by `BOR_IMAGES`) (owner-confirmed 2026-09-26, roadmap confirmation).**
|
||||
- **A4 — PDFs are served as extracted text in the document viewer (not as binary PDFs) (owner-confirmed 2026-09-26, roadmap confirmation).**
|
||||
|
||||
## Commit
|
||||
```bash
|
||||
git add app/ pyproject.toml tests/ .agents/phases/ && git commit --no-gpg-sign -m "feat(import): add PDF support — text extraction, embedded images, chunking"
|
||||
```
|
||||
@@ -0,0 +1,19 @@
|
||||
# Task 01 — Add `pymupdf` dependency
|
||||
|
||||
**Phase:** `126_pdf_support` · **Source:** `00_phase.md` — the `pymupdf` dependency
|
||||
|
||||
## Objective
|
||||
Add `pymupdf` to `pyproject.toml` dependencies. PyMuPDF provides both PDF text extraction and embedded image extraction via the `fitz` module.
|
||||
|
||||
## Work
|
||||
1. `pyproject.toml` — add `pymupdf>=1.24,<2.0` to the `dependencies` list (after the existing dependencies, before the dev group). Version pin: `1.24` is a stable release with good text and image extraction support; `<2.0` follows the project's convention of pinning major versions.
|
||||
2. ASSUMPTION: none. `pymupdf` is a pure-Python package with optional C extensions (the binary wheels are available for common platforms).
|
||||
|
||||
## Testing & Quality
|
||||
- No code tests needed — this is a dependency addition.
|
||||
- Verify: `uv sync` completes without errors; `import fitz` works in a Python shell.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv sync` completes successfully.
|
||||
- [ ] `import fitz` works; `fitz.VersionBind` shows the installed version.
|
||||
- [ ] `uv run pytest` still green (no regression from the dependency addition).
|
||||
@@ -0,0 +1,173 @@
|
||||
# Task 02 — PDF parser module: `app/rag/pdf.py`
|
||||
|
||||
**Phase:** `126_pdf_support` · **Source:** `00_phase.md` — text extraction, image extraction, MIME mapping
|
||||
|
||||
## Objective
|
||||
A new `app/rag/pdf.py` module with pure functions for PDF text extraction, embedded image extraction, and MIME type mapping. All functions are I/O-bound only (read a PDF file, return parsed data) — fully unit-testable with temp PDF fixtures.
|
||||
|
||||
## Work
|
||||
1. `app/rag/pdf.py` — new module:
|
||||
```python
|
||||
"""PDF parsing utilities (phase 126).
|
||||
|
||||
Pure functions for extracting text and embedded images from PDF files
|
||||
using PyMuPDF (the ``fitz`` module).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
import fitz # pymupdf
|
||||
|
||||
logger = logging.getLogger("app.pdf")
|
||||
|
||||
#: Page separator used in extracted text (chunker splits on this).
|
||||
PAGE_SEPARATOR = "---PAGE-{page}---"
|
||||
|
||||
@dataclass
|
||||
class PdfImage:
|
||||
"""One embedded image from a PDF page."""
|
||||
page: int #: 1-indexed page number
|
||||
index: int #: image index on the page (0-based, from fitz)
|
||||
width: int
|
||||
height: int
|
||||
data: bytes #: raw image bytes
|
||||
mime: str #: MIME type (e.g., "image/png")
|
||||
|
||||
#: Mapping from PyMuPDF image type codes to MIME types.
|
||||
#: Type codes: 2=JPEG, 3=PNG, 4=GIF, 8=TIFF, 9=WebP (per fitz docs).
|
||||
_TYPE_TO_MIME: dict[int, str] = {
|
||||
2: "image/jpeg",
|
||||
3: "image/png",
|
||||
4: "image/gif",
|
||||
8: "image/tiff",
|
||||
9: "image/webp",
|
||||
}
|
||||
|
||||
def _mime_from_image_info(info: dict) -> str:
|
||||
"""Map a PyMuPDF image info dict to a MIME type.
|
||||
|
||||
Falls back to ``application/octet-stream`` for unknown types.
|
||||
"""
|
||||
# Try the 'ext' key first (file extension), then the 'type' code.
|
||||
ext = info.get("ext", "").lower()
|
||||
if ext == "png":
|
||||
return "image/png"
|
||||
if ext in ("jpg", "jpeg"):
|
||||
return "image/jpeg"
|
||||
if ext == "gif":
|
||||
return "image/gif"
|
||||
if ext == "webp":
|
||||
return "image/webp"
|
||||
if ext == "tiff" or ext == "tif":
|
||||
return "image/tiff"
|
||||
# Fallback to type code.
|
||||
type_code = info.get("type")
|
||||
if type_code is not None:
|
||||
return _TYPE_TO_MIME.get(type_code, "application/octet-stream")
|
||||
return "application/octet-stream"
|
||||
|
||||
def extract_pdf_text(pdf_path: Path) -> str:
|
||||
"""Extract text from all pages of a PDF, joined with page separators.
|
||||
|
||||
Returns the full text with ``---PAGE-N---`` separators between pages.
|
||||
Returns an empty string if the PDF has no text or cannot be opened.
|
||||
"""
|
||||
try:
|
||||
doc = fitz.open(str(pdf_path))
|
||||
except Exception as e:
|
||||
logger.warning("pdf: could not open %s — %s", pdf_path, e)
|
||||
return ""
|
||||
|
||||
parts: list[str] = []
|
||||
for page_num in range(len(doc)):
|
||||
page = doc[page_num]
|
||||
text = page.get_text("text").strip()
|
||||
if text:
|
||||
parts.append(f"{PAGE_SEPARATOR.format(page=page_num + 1)}\n{text}")
|
||||
else:
|
||||
# Even empty pages get a separator (for chunker consistency).
|
||||
parts.append(f"{PAGE_SEPARATOR.format(page=page_num + 1)}")
|
||||
doc.close()
|
||||
return "\n\n".join(parts)
|
||||
|
||||
def extract_pdf_images(pdf_path: Path) -> list[PdfImage]:
|
||||
"""Extract all embedded images from a PDF.
|
||||
|
||||
Returns a list of ``PdfImage`` dataclasses, one per embedded image.
|
||||
Images are extracted in page order, then by their index on the page.
|
||||
Returns an empty list if the PDF has no embedded images or cannot
|
||||
be opened.
|
||||
"""
|
||||
try:
|
||||
doc = fitz.open(str(pdf_path))
|
||||
except Exception as e:
|
||||
logger.warning("pdf: could not open %s for image extraction — %s", pdf_path, e)
|
||||
return []
|
||||
|
||||
images: list[PdfImage] = []
|
||||
for page_num in range(len(doc)):
|
||||
page = doc[page_num]
|
||||
image_list = page.get_images(full=True)
|
||||
for img_index, img_info in enumerate(image_list):
|
||||
try:
|
||||
xref = img_info[0]
|
||||
# Get the raw image bytes.
|
||||
pix = fitz.Pixmap(doc, xref)
|
||||
# Convert to RGB if needed (CMYK → RGB).
|
||||
if pix.n - pix.alpha > 3:
|
||||
pix = fitz.Pixmap(fitz.csRGB, pix)
|
||||
data = pix.tobytes("png") if pix.n == 4 else pix.tobytes("jpeg")
|
||||
# Determine the best bytes representation.
|
||||
# If the image is grayscale or has alpha, use PNG.
|
||||
# Otherwise, use JPEG for smaller size.
|
||||
if pix.alpha > 0 or pix.n <= 2:
|
||||
data = pix.tobytes("png")
|
||||
mime = "image/png"
|
||||
else:
|
||||
data = pix.tobytes("jpeg")
|
||||
mime = "image/jpeg"
|
||||
# Fallback: if tobytes fails, use the raw data.
|
||||
if not data:
|
||||
data = pix.tobytes()
|
||||
mime = _mime_from_image_info(img_info)
|
||||
|
||||
images.append(PdfImage(
|
||||
page=page_num + 1,
|
||||
index=img_index,
|
||||
width=pix.width,
|
||||
height=pix.height,
|
||||
data=data,
|
||||
mime=mime,
|
||||
))
|
||||
pix = None # free the pixmap
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"pdf: could not extract image page=%d index=%d — %s",
|
||||
page_num + 1, img_index, e,
|
||||
)
|
||||
doc.close()
|
||||
return images
|
||||
```
|
||||
2. ASSUMPTION: none. The module uses only `fitz` (from `pymupdf`) and stdlib.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_pdf.py` (task 05) — create temp PDF fixtures using `fitz` itself:
|
||||
- A single-page PDF with text → `extract_pdf_text` returns the text with one page separator.
|
||||
- A multi-page PDF → text with multiple page separators.
|
||||
- A PDF with no text (image-only) → empty string.
|
||||
- A PDF with embedded images → `extract_pdf_images` returns the correct list.
|
||||
- A PDF with no embedded images → empty list.
|
||||
- `_mime_from_image_info` (known types → correct MIME; unknown → fallback).
|
||||
- Error handling: non-existent PDF → empty string / empty list (no exception).
|
||||
- Coverage: **>90%** on the new module.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `extract_pdf_text` returns correct text with page separators for a multi-page PDF.
|
||||
- [ ] `extract_pdf_images` returns correct image data for a PDF with embedded images.
|
||||
- [ ] `_mime_from_image_info` maps known type codes correctly; unknown types get the fallback.
|
||||
- [ ] Non-existent or unreadable PDFs return empty results (no exceptions).
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,56 @@
|
||||
# Task 03 — PDF chunking: extend the chunker for `.pdf` files
|
||||
|
||||
**Phase:** `126_pdf_support` · **Source:** `00_phase.md` — extend the chunker to handle `.pdf` files
|
||||
|
||||
## Objective
|
||||
Extend `app/rag/chunker.py`'s `chunk_document` dispatch to handle `.pdf` files. PDF chunks are split on page separators (`---PAGE-N---`), with each page as a logical unit.
|
||||
|
||||
## Work
|
||||
1. `app/rag/chunker.py` — extend the `chunk_document` dispatch:
|
||||
- Add `.pdf` to the suffix check that dispatches to the paragraph packer (PDFs don't have a format-specific splitter like markdown's heading-based splitting).
|
||||
- The PDF chunker uses the same `chunk_text` function as `txt` (paragraph packing), but with page-aware splitting:
|
||||
```python
|
||||
def _chunk_pdf(content: str, rel: str, target: int, overlap: int) -> list[str]:
|
||||
"""Chunk PDF content, respecting page separators.
|
||||
|
||||
Pages are the primary split boundary. Each page's text is packed
|
||||
into chunks of ``target`` chars with ``overlap`` chars carry-over.
|
||||
"""
|
||||
# Split on page separators.
|
||||
pages = re.split(r'---PAGE-\d+---', content)
|
||||
# pages[0] is empty (content starts with the separator).
|
||||
result: list[str] = []
|
||||
for i, page_text in enumerate(pages[1:], start=1):
|
||||
page_text = page_text.strip()
|
||||
if not page_text:
|
||||
continue
|
||||
# Prepend the page anchor for readability.
|
||||
anchored = f"Page {i}:\n{page_text}"
|
||||
# Chunk the anchored text using paragraph packing.
|
||||
chunks = chunk_text(anchored, target=target, overlap=overlap)
|
||||
result.extend(chunks)
|
||||
return result
|
||||
```
|
||||
- Register `.pdf` in the `chunk_document` dispatch table:
|
||||
```python
|
||||
CHUNKERS: dict[str, Callable] = {
|
||||
...
|
||||
".pdf": _chunk_pdf,
|
||||
}
|
||||
```
|
||||
2. ASSUMPTION: none. The PDF chunker reuses the existing `chunk_text` paragraph packer — it only adds page-aware splitting.
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_chunker.py` (extend) — add PDF-specific tests:
|
||||
- A single-page PDF → one or more chunks with "Page 1:" prefix.
|
||||
- A multi-page PDF → chunks with correct "Page N:" prefixes.
|
||||
- An empty PDF → empty list.
|
||||
- A very large page (>HARD_MAX_CHARS) → split by line (the `chunk_text` behavior).
|
||||
- Verify that page separators in the input are removed from the output chunks.
|
||||
- Coverage: **>90%** on the touched code.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `chunk_document` dispatches `.pdf` files to the page-aware chunker.
|
||||
- [ ] Chunks have "Page N:" prefixes and no page separator markers.
|
||||
- [ ] Multi-page PDFs produce chunks with correct page anchors.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,262 @@
|
||||
# Task 04 — PDF indexing: `_index_pdf_file` in the importer
|
||||
|
||||
**Phase:** `126_pdf_support` · **Source:** `00_phase.md` — text + embedded images + summary
|
||||
|
||||
## Objective
|
||||
Add PDF indexing to the importer: `_index_pdf_file` extracts text, creates the document, chunks + embeds, generates a summary, and handles embedded images.
|
||||
|
||||
## Work
|
||||
1. `app/rag/importer.py` — add the import:
|
||||
```python
|
||||
from app.rag.pdf import extract_pdf_text, extract_pdf_images, PdfImage
|
||||
```
|
||||
2. `app/rag/importer.py` — in `_index_file`, add the PDF branch (AFTER the image branch, BEFORE the text branch):
|
||||
```python
|
||||
# Phase 126: PDF files take the PDF index path.
|
||||
if full_path.suffix.lower() == ".pdf":
|
||||
return await _index_pdf_file(
|
||||
session, source=source, rel=rel, full_path=full_path,
|
||||
llm=llm, summary=summary, raw_date=raw_date,
|
||||
)
|
||||
```
|
||||
3. `app/rag/importer.py` — new `_index_pdf_file` function (follow the `_index_file` text branch pattern):
|
||||
```python
|
||||
async def _index_pdf_file(
|
||||
session: Session,
|
||||
*,
|
||||
source: str,
|
||||
rel: str,
|
||||
full_path: Path,
|
||||
llm: Embedder,
|
||||
summary: ImportSummary,
|
||||
raw_date: datetime | None = None,
|
||||
) -> None:
|
||||
"""Index one PDF file: extract text, create doc, chunk + embed,
|
||||
summarize, and extract embedded images.
|
||||
"""
|
||||
settings = llm.settings
|
||||
# Extract text.
|
||||
content = extract_pdf_text(full_path)
|
||||
if not content:
|
||||
# No text in the PDF — still create a doc (it might have
|
||||
# embedded images), but with empty content.
|
||||
logger.warning("import: PDF has no extractable text source=%s path=%s", source, rel)
|
||||
|
||||
# Digest over the extracted text (content identity).
|
||||
digest = hashlib.sha256(content.encode("utf-8")).hexdigest() if content else hashlib.sha256(b"").hexdigest()
|
||||
|
||||
# Get PDF metadata for the title.
|
||||
title = _extract_pdf_title(full_path, fallback=full_path.stem)
|
||||
|
||||
# Date: use the file's mtime (D2 fallback) — PDFs don't have
|
||||
# standard creation dates accessible via the filesystem.
|
||||
if raw_date is None:
|
||||
raw_date = file_mtime_datetime(full_path)
|
||||
|
||||
# Upsert the document row.
|
||||
doc = session.scalar(select(Document).where(Document.source == source, Document.path == rel))
|
||||
if doc is None:
|
||||
doc = Document(
|
||||
source=source,
|
||||
path=rel,
|
||||
full_path=str(full_path),
|
||||
title=title,
|
||||
content=content,
|
||||
content_hash=digest,
|
||||
indexed_at=datetime.now(UTC),
|
||||
created_at=normalize_doc_date(raw_date),
|
||||
)
|
||||
session.add(doc)
|
||||
else:
|
||||
# Check for content change.
|
||||
if doc.content_hash == digest and doc.content == content:
|
||||
summary.unchanged += 1
|
||||
logger.info("import: unchanged source=%s path=%s", source, rel)
|
||||
# Summary backfill (phase 118, A2).
|
||||
if doc.summary is None:
|
||||
await _store_summary(
|
||||
session, doc=doc, source=source, rel=rel,
|
||||
content=content, llm=llm, summary=summary, backfill=True,
|
||||
)
|
||||
if doc.created_at_manual:
|
||||
return
|
||||
target = normalize_doc_date(raw_date)
|
||||
if target != doc.created_at:
|
||||
doc.created_at = target
|
||||
session.commit()
|
||||
summary.dates_updated += 1
|
||||
return
|
||||
|
||||
doc.full_path = str(full_path)
|
||||
doc.title = title
|
||||
doc.content = content
|
||||
doc.content_hash = digest
|
||||
doc.indexed_at = datetime.now(UTC)
|
||||
doc.created_at = normalize_doc_date(raw_date)
|
||||
doc.created_at_manual = False
|
||||
|
||||
session.flush()
|
||||
|
||||
# Chunk and embed.
|
||||
if content:
|
||||
target = max(400, settings.chunk_target_chars)
|
||||
while True:
|
||||
chunks_text = chunk_document(content, rel, target, settings.chunk_overlap_chars)
|
||||
doc.chunks = [
|
||||
Chunk(document_id=doc.id, position=i, content=c)
|
||||
for i, c in enumerate(chunks_text)
|
||||
]
|
||||
session.flush()
|
||||
if not doc.chunks:
|
||||
break
|
||||
try:
|
||||
vectors = await llm.embed([c.content for c in doc.chunks])
|
||||
for row, vec in zip(doc.chunks, vectors, strict=True):
|
||||
row.embedding = vec
|
||||
break
|
||||
except EmbeddingError as e:
|
||||
if "token cap" not in str(e) or target <= 400:
|
||||
raise
|
||||
logger.info(
|
||||
"import: re-chunking PDF at %d chars after endpoint token cap: %s",
|
||||
target // 2, rel,
|
||||
)
|
||||
target //= 2
|
||||
|
||||
session.commit()
|
||||
|
||||
# Verb count.
|
||||
if doc.content_hash == digest and doc.content == content and doc.id in [d.id for d in []]:
|
||||
# This is a bit awkward — we need to track whether this was
|
||||
# new or updated. Use the session's identity map.
|
||||
pass # handled by the unchanged path above
|
||||
else:
|
||||
summary.added += 1 if doc.id not in session.identity_map else 0
|
||||
# Actually, let's just use the verb tracking from _index_file.
|
||||
# The simplest approach: track added/updated before the upsert.
|
||||
|
||||
# Generate summary (phase 30, A2).
|
||||
await _store_summary(
|
||||
session, doc=doc, source=source, rel=rel,
|
||||
content=content, llm=llm, summary=summary, backfill=False,
|
||||
)
|
||||
|
||||
# Extract and index embedded images.
|
||||
await _index_pdf_embedded_images(
|
||||
session, source=source, rel=rel, full_path=full_path,
|
||||
llm=llm, summary=summary, raw_date=raw_date,
|
||||
)
|
||||
|
||||
logger.info("import: %s source=%s path=%s chunks=%d",
|
||||
"added" if newly_created else "updated", source, rel, len(doc.chunks))
|
||||
```
|
||||
|
||||
Wait — the above is getting complex. Let me simplify. The key insight is that `_index_pdf_file` should follow the same pattern as `_index_file` but with PDF-specific extraction. Let me refactor.
|
||||
|
||||
Actually, let me simplify this significantly. The `_index_pdf_file` function should:
|
||||
1. Extract text
|
||||
2. Upsert the document
|
||||
3. Chunk + embed
|
||||
4. Generate summary
|
||||
5. Extract embedded images
|
||||
6. Log the result
|
||||
|
||||
The unchanged/hash path is the same as `_index_file` — check the hash, skip if unchanged.
|
||||
|
||||
4. `app/rag/importer.py` — `_extract_pdf_title(full_path, fallback) -> str`:
|
||||
- Open the PDF with `fitz`, try to get the metadata title (`doc.metadata.get("title", "")`).
|
||||
- If no title in metadata, return the file stem (the fallback).
|
||||
- Strip whitespace and limit to 500 chars (the `Document.title` column width).
|
||||
|
||||
5. `app/rag/importer.py` — `_index_pdf_embedded_images(session, source, rel, full_path, llm, summary, raw_date)`:
|
||||
- Extract embedded images via `extract_pdf_images(full_path)`.
|
||||
- For each image:
|
||||
- If `settings.images` is False → skip (log a debug message).
|
||||
- If the image extension doesn't match `settings.image_extension_set` → skip (log a warning).
|
||||
- Create a synthetic path for the embedded image: `<rel>#page=<N>#img=<index>.<ext>` (e.g., `report.pdf#page=3#img=0.png`).
|
||||
- Call `_index_image_file` with the synthetic path and the image bytes (the `_index_image_file` function expects a `full_path` on disk — we need to either write a temp file or refactor `_index_image_file` to accept bytes directly).
|
||||
- **Alternative approach:** instead of calling `_index_image_file`, handle the embedded image inline: copy the bytes to `settings.image_dir`, describe them via `describe_image`, create the image doc row, chunk + embed the description. This avoids the temp file and the `_index_image_file` refactoring.
|
||||
|
||||
Let me go with the inline approach — it's simpler and avoids refactoring `_index_image_file`:
|
||||
```python
|
||||
async def _index_pdf_embedded_images(...):
|
||||
images = extract_pdf_images(full_path)
|
||||
for img in images:
|
||||
ext = _ext_from_mime(img.mime) # "image/png" → ".png"
|
||||
if ext not in settings.image_extension_set:
|
||||
logger.warning("import: PDF embedded image has unsupported extension: %s", img.mime)
|
||||
continue
|
||||
|
||||
# Write to image_dir.
|
||||
img_id = uuid.uuid4()
|
||||
img_filename = f"{img_id}{ext}"
|
||||
img_dir_path = Path(settings.image_dir)
|
||||
img_dir_path.mkdir(parents=True, exist_ok=True)
|
||||
img_path = str(img_dir_path / img_filename)
|
||||
Path(img_path).write_bytes(img.data)
|
||||
|
||||
# Describe the image.
|
||||
description = await _describe_or_skip(
|
||||
llm, data=img.data, source=source, rel=f"{rel}#page={img.page}#img={img.index}",
|
||||
mime=img.mime, full_path=Path(img_path),
|
||||
)
|
||||
if description is None:
|
||||
summary.images_failed += 1
|
||||
Path(img_path).unlink(missing_ok=True)
|
||||
continue
|
||||
|
||||
# Create the image doc row.
|
||||
img_doc = Document(
|
||||
source=source,
|
||||
path=f"{rel}#page={img.page}#img={img.index}{ext}",
|
||||
full_path=img_path,
|
||||
title=f"Embedded image from {full_path.stem} page {img.page}",
|
||||
content=description,
|
||||
content_hash=hashlib.sha256(img.data).hexdigest(),
|
||||
indexed_at=datetime.now(UTC),
|
||||
created_at=normalize_doc_date(raw_date),
|
||||
is_image=True,
|
||||
image_path=img_path,
|
||||
)
|
||||
session.add(img_doc)
|
||||
session.flush()
|
||||
|
||||
# Chunk + embed the description.
|
||||
chunks_text = chunk_document(description, img_doc.path,
|
||||
settings.chunk_target_chars, settings.chunk_overlap_chars)
|
||||
img_doc.chunks = [
|
||||
Chunk(document_id=img_doc.id, position=i, content=c)
|
||||
for i, c in enumerate(chunks_text)
|
||||
]
|
||||
session.flush()
|
||||
try:
|
||||
vectors = await llm.embed([c.content for c in img_doc.chunks])
|
||||
for row, vec in zip(img_doc.chunks, vectors, strict=True):
|
||||
row.embedding = vec
|
||||
except EmbeddingError:
|
||||
pass # fail soft, like the text path
|
||||
|
||||
# Summary = description (phase-122 convention for image docs).
|
||||
img_doc.summary = description
|
||||
is_summary_chunk = Chunk(
|
||||
document_id=img_doc.id, position=-1, content=description,
|
||||
is_summary=True,
|
||||
)
|
||||
img_doc.chunks.append(is_summary_chunk)
|
||||
session.flush()
|
||||
|
||||
session.commit()
|
||||
```
|
||||
|
||||
6. ASSUMPTION: none. The embedded image indexing reuses the same patterns as standalone image indexing (phase 122).
|
||||
|
||||
## Testing & Quality
|
||||
- Unit: `tests/unit/test_pdf.py` (extend) — `_extract_pdf_title` (PDF with metadata title → returns it; PDF without → returns fallback).
|
||||
- Integration: `tests/integration/test_importer.py` (extend) — seed a source with a PDF file → sync → the PDF document has extracted text, chunks, and embedding; embedded images (if `BOR_IMAGES=true`) become image documents.
|
||||
- Coverage: **>90%** on the touched modules.
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] A PDF file in a source → the PDF document has extracted text, chunks, embedding, and summary.
|
||||
- [ ] With `BOR_IMAGES=true`: embedded images become image documents.
|
||||
- [ ] With `BOR_IMAGES=false`: embedded images are skipped.
|
||||
- [ ] `uv run pytest` green; `uv run ruff check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,60 @@
|
||||
# Task 05 — PDFs in archives and full test suite
|
||||
|
||||
**Phase:** `126_pdf_support` · **Source:** `00_phase.md` — PDFs in archives, unit + integration + E2E tests
|
||||
|
||||
## Objective
|
||||
Ensure PDFs inside uploaded archives are indexed, and write the full test suite (unit, integration, E2E).
|
||||
|
||||
## Work
|
||||
1. `app/rag/archive_upload.py` — no code changes needed. Archives are unpacked into `upload_dir`, and the extension walk (`iter_importable_files`) already filters by `import_extension_set`. Adding `pdf` to the set means PDFs in archives are automatically picked up. Verify that the archive unpack doesn't skip `.pdf` files (it shouldn't — archives unpack all files).
|
||||
|
||||
2. `app/config.py` — no code changes needed. `pdf` is added to `BOR_IMPORT_EXTENSIONS` by the user (the config already accepts any well-formed extension via `import_extensions`). Document it in `.env.example` alongside the existing extension list:
|
||||
```
|
||||
# BOR_IMPORT_EXTENSIONS=md,markdown,txt,yaml,yml,json,py,pdf
|
||||
```
|
||||
|
||||
3. `tests/unit/test_pdf.py` (new) — comprehensive unit tests:
|
||||
- Create temp PDF fixtures using `fitz`:
|
||||
- Single-page PDF with text → `extract_pdf_text` returns text with one page separator.
|
||||
- Multi-page PDF (3 pages) → text with 3 page separators.
|
||||
- PDF with no text (blank pages) → empty string.
|
||||
- PDF with embedded images (draw shapes, add text) → `extract_pdf_images` returns the images.
|
||||
- PDF with no embedded images → empty list.
|
||||
- `_mime_from_image_info` (all known type codes → correct MIME; unknown → fallback).
|
||||
- `_extract_pdf_title` (PDF with metadata → returns title; PDF without → returns fallback).
|
||||
- Error handling: non-existent PDF → empty results.
|
||||
- `tests/unit/test_chunker.py` (extend) — PDF-specific chunking tests:
|
||||
- Single-page PDF → correct chunks with "Page 1:" prefix.
|
||||
- Multi-page PDF → correct chunks with "Page N:" prefixes.
|
||||
- Empty PDF → empty list.
|
||||
- Large page → split by line.
|
||||
|
||||
4. `tests/integration/test_importer.py` (extend) — integration tests:
|
||||
- Seed a source with a PDF file (create a temp PDF fixture) → sync → the PDF document has extracted text, chunks, and embedding.
|
||||
- A PDF with embedded images + `BOR_IMAGES=true` → embedded images become image documents.
|
||||
- A PDF with embedded images + `BOR_IMAGES=false` → embedded images are skipped.
|
||||
- A PDF with no text → document is created with empty content (no chunks, no embedding).
|
||||
- A PDF referenced in a markdown file → the markdown doc stores the reference info (phase 125 enrichment path).
|
||||
|
||||
5. `tests/e2e/test_pdf_support.py` (new) — isolated E2E suite:
|
||||
- Module apps: one app with `BOR_IMAGES=true`, one with `BOR_IMAGES=false`.
|
||||
- KB fixture: create a test PDF with text and an embedded image (using `fitz` in the test setup). Upload via the admin upload flow → sync.
|
||||
- Verify: the PDF appears on the Sources page → open the document viewer → the extracted text renders → ask a question the PDF answers → the chat grounds on the PDF content.
|
||||
- With `BOR_IMAGES=true`: the embedded image appears as a source chip (if retrieved).
|
||||
- With `BOR_IMAGES=false`: no image chip, the PDF text is indexed normally.
|
||||
- Run in isolation: `uv run pytest tests/e2e/test_pdf_support.py -v --no-cov`.
|
||||
|
||||
6. Run the full test suite: `uv run pytest` green; `uv run pytest --cov=app --cov-report=term-missing` TOTAL >90%; `uv run ruff check . && uv run pyright` clean.
|
||||
|
||||
## Testing & Quality
|
||||
- All new logic covered by unit tests (PDF parsing, chunking, title extraction).
|
||||
- Integration tests cover the full import flow with seeded PDF data.
|
||||
- E2E tests cover the real user path: upload → sync → view → ask.
|
||||
- No regression in existing tests (run `test_image_documents.py` and `test_image_anchor_context.py` to verify).
|
||||
|
||||
## Completion Criteria
|
||||
- [ ] `uv run pytest tests/unit/test_pdf.py -v --no-cov` passes.
|
||||
- [ ] `uv run pytest tests/integration/test_importer.py -v --no-cov` passes (extended).
|
||||
- [ ] `uv run pytest tests/e2e/test_pdf_support.py -v --no-cov` passes in isolation.
|
||||
- [ ] `uv run pytest` green; coverage TOTAL >90%; `ruff` + `pyright` clean.
|
||||
- [ ] No regression in existing E2E suites.
|
||||
Reference in New Issue
Block a user