phase: 181_svg_document_preview
Build and Push Containers / build-and-push-app (push) Successful in 1m57s
Build and Push Containers / build-and-push-db (push) Successful in 15s

**Phase 181 — final verification pass (all tasks already complete; verified + docs pass)**

- Verified: `app/rag/svg_text.py` (100 % cov), vector branch in `_index_image_file`, `_row_kind_mismatch` text→image migration, `IMAGE_MIMES[".svg"]`, sandboxed CSP on `GET /api/documents/{id}/image`, `.doc-image-img--vector` in `document.js`/`styles.css`, figure/thumbnail deliberate non-changes.
- Tests/lint (exact commands): `uv run pytest --cov=app --cov-report=term-missing` → **4265 passed, TOTAL 99 %** (exit 0) · `uv run pytest tests/e2e/test_svg_document_preview.py -v --no-cov` → **5 passed in isolation** · `uv run ruff check .` → clean · `uv run pyright` → **0 errors**.
- Regression (each run in isolation, `--no-cov`, all rc=0): `test_image_documents`, `test_chat_image_questions`, `test_answer_images`, `test_chat_answer_clusters`, `test_image_anchor_context`, `test_image_upload_feedback`, `test_markdown_inline_links_images`, `test_prose_word_wrap`, `test_pdf_support`, `test_pictorial_pdf`, `test_garbage_pdf`, `test_pdf_image_extraction`, `test_pdf_image_context`, `test_pdf_pages_in_sources`, `test_pdf_strip_additive_selfheal`, `test_sync_image_progress`.
- Completion criteria: ①SVG→`is_image`+`image_path`, no vision call, own `<title>/<desc>/<text>` — **met**; ②`image/svg+xml` + `default-src 'none'; sandbox` + `private, max-age=3600`, raster headers byte-unchanged — **met**; ③chat figure → modal `<img>`, no XML — **met**; ④`/document.html` full-width vector in `.doc-image` + text below — **met**; ⑤`![alt](/api/documents/<id>/image)` paints — **met**; ⑥raster/PDF unchanged — **met**; ⑦all gates green — **met**; ⑧commit + phase-dir move — harness-owned.
- Documentation: found and fixed README inconsistencies — stale forward reference "left to phase 181 / is phase 181's job" → completed-tense sizing note; "Updating the documents" standalone-image list missing `svg`; new **Raster only** bullet for question uploads (`.svg` → 422) documenting the `chat_images` guard; missing `svg_indexed=N` / `import: svg indexed` log contract; env-table heading `122/123/124` → `122/123/124/181`; new troubleshooting entry for an `.svg` showing raw XML. Full audit: every `Settings` field and every `.env.example` var is documented; pages/commands/scripts all exist. AGENTS.md verified accurate (phase-181 §5 clause present) — no changes needed beyond what the phase wrote.
- Notable deviation (kept from task 02): `app/api/chat_images.py` subtracts `_VECTOR_IMAGE_EXTENSIONS` from the question-image accept set — a required consequence of LOCKED A7 (`svg` in the default image set) since `_CHAT_IMAGE_FILENAME_RE` refuses to *serve* `.svg`; the regex itself is untouched, as the phase specified.
- Note for the commit: changes include `README.md`, `AGENTS.md` and `.env.example` — the phase file's `git add app/ frontend/ tests/ .env.example .agents/phases/` line omits the two markdown files; stage them too.
- Next pending phase: `182_verbatim_show_text`.
This commit is contained in:
2026-10-14 22:54:43 -04:00
parent 6880bd0866
commit 814a400e6d
70 changed files with 4460 additions and 52 deletions
+39 -7
View File
@@ -68,6 +68,7 @@ from pydantic import ValidationError
from app.api import chat as chat_api
from app.api import chat_images
from app.api.chat_images import _VECTOR_IMAGE_EXTENSIONS
from app.config import Settings
from app.main import app as fastapi_app
from app.models import Document
@@ -354,6 +355,32 @@ def test_upload_rejects_a_non_image_extension_with_the_fixed_detail(
assert not image_dir.exists() # the dir is created on the accepted path only
def test_upload_rejects_a_vector_svg_even_though_it_indexes(
client: TestClient, image_dir: Path
) -> None:
"""Phase 181: ``svg`` joins the DEFAULT image set for the DOCUMENT
walk, and question attachments stay RASTER-only — the upload 422s
with the six-extension detail and writes nothing.
Without the subtraction this would be a silent trap: the stored
``<uuid>.svg`` is refused by the serve route's filename regex (a
broken figure in the user's own bubble) and an OpenAI-compatible
vision endpoint rejects ``image/svg+xml``, so the turn could never
be answered. The vector family belongs to INDEXING, not to
attachments."""
assert ".svg" in Settings(_env_file=None).image_extension_set # pyright: ignore[reportCallIssue]
assert frozenset({".svg"}) == _VECTOR_IMAGE_EXTENSIONS
r = _upload(
client, "diagram.svg", b'<svg xmlns="http://www.w3.org/2000/svg"></svg>',
mime="image/svg+xml",
)
assert r.status_code == 422
assert r.json()["detail"] == (
"only .bmp, .gif, .jpeg, .jpg, .png, .webp images are accepted"
)
assert not image_dir.exists()
def test_upload_rejects_a_name_without_an_extension(
client: TestClient, image_dir: Path
) -> None:
@@ -916,7 +943,10 @@ def test_the_attach_reveal_is_gated_on_the_config_flag() -> None:
def test_the_client_precheck_matches_the_servers_six_extensions() -> None:
"""ONE list of the six, on both sides: the client pre-check (the
file NAME's extension — the accept attribute is advisory) and the
server's phase-122 frozenset (the upload endpoint's authority).
server's phase-122 frozenset MINUS the vector members (the upload
endpoint's authority — phase 181: question images stay RASTER-only,
a vision endpoint rejects ``image/svg+xml`` and the serve route's
filename regex would never serve one).
The client list must stay the code default's six — a drift here
would either block a valid pick or waste a round-trip on a 422."""
js = _app_js()
@@ -931,12 +961,14 @@ def test_the_client_precheck_matches_the_servers_six_extensions() -> None:
"png",
"webp",
}
# and it equals the server's default set (dots stripped — the
# client list is dotless, the server's is dotted).
server_set = {
e.lstrip(".")
for e in Settings(_env_file=None).image_extension_set # pyright: ignore[reportCallIssue]
}
# and it equals the server's default raster set (dots stripped — the
# client list is dotless, the server's is dotted). ``svg`` is in the
# DEFAULT image set for the document walk (phase 181) and
# deliberately NOT attachable here (see
# ``chat_images._VECTOR_IMAGE_EXTENSIONS``).
raw_default = Settings(_env_file=None).image_extension_set # pyright: ignore[reportCallIssue]
assert ".svg" in raw_default # the vector member (phase 181)
server_set = {e.lstrip(".") for e in raw_default - _VECTOR_IMAGE_EXTENSIONS}
assert client_set == server_set
# the file input's accept attribute names the same six (mime form).
m = re.search(r'<input type="file" id="attach-file"\s+accept="([^"]+)"', _index_html())
+191
View File
@@ -94,6 +94,20 @@ format table as node behavioral runs over the real module, and the
``.doc-prose`` CSS (the 72rem reading column + ``pre-wrap`` +
``overflow-wrap: anywhere``, tokens only) beside the ``.doc-raw``
contract that stays locked (phase 116 A1 / phase 165 A3).
Phase 181 (task 04) — the SVG vector preview: ``docImageBlock`` adds
the ``doc-image-img--vector`` modifier ONLY when
``doc.format === "svg"`` (the base class is always kept) — a
``viewBox``-only SVG has NO intrinsic size, so the default 300×150
replaced-element box would sit small inside the full-width
``.doc-image`` card — pinned as a node behavioral run of the real
``docImageBlock`` (svg → the compound class, ``src`` from
``image_url`` never inlined, ``alt`` = summary with the title
fallback; png → the class string byte-identical to the base class)
plus the ``.doc-image-img--vector`` CSS pins (``width: 100%`` +
``height: auto`` + ``object-fit: contain`` under the base rule's
70vh clamp, zero colour literals, the base ``.doc-image-img`` rule
untouched).
"""
from __future__ import annotations
@@ -2102,3 +2116,180 @@ def test_doc_prose_css_wraps_in_the_reading_column() -> None:
raw_rule = css[raw_i : css.find("}", raw_i)]
assert "white-space: pre" in raw_rule
assert "overflow-x: auto" in raw_rule
# ---------------------------------------------------------------------------
# Phase 181 (task 04) — the SVG vector preview: docImageBlock's format
# gate (base class + the doc-image-img--vector modifier, LOCKED A3:
# bytes only ever through <img>) and the .doc-image-img--vector CSS
# (a viewBox-only SVG has no intrinsic size — width: 100% spans the
# .doc-image card, object-fit: contain survives the 70vh clamp,
# palette-neutral). The chat figure (app.js) and the Sources
# thumbnail (sources.js) are VERIFIED non-changes (design D7): both
# key off image_url alone and their boxes pin the img
# (.source-image-img 96px cap, .kb-doc-thumb-img fixed 48px box).
# ---------------------------------------------------------------------------
def test_doc_image_block_gates_the_vector_modifier_on_format() -> None:
"""Source pins (node-free): docImageBlock selects the compound
class on ``doc.format === "svg"`` and the bare base class
otherwise (every current .doc-image-img rule keeps applying);
the bytes stay an <img> src property assignment — never inlined
(LOCKED A3), never innerHTML. The comment records the rationale
(no-intrinsic-size / 300×150) and the legacy-TEXT honesty
(XML-in-.doc-raw is the truth; no content sniffing)."""
js = _read(DOCUMENT_JS)
fn = _fn(js, "docImageBlock")
assert 'doc.format === "svg"' in fn
assert '"doc-image-img doc-image-img--vector"' in fn
assert ': "doc-image-img";' in fn, "the raster path keeps the bare base class"
assert "img.src = doc.image_url;" in fn, "the bytes URL stays a property assignment"
assert "innerHTML" not in fn
# The rationale + honesty comments (house style: why, not what).
for citation in ("Phase 181", "viewBox", "300\u00d7150", "LOCKED A3", ".doc-raw"):
assert citation in fn, f"the comment must cite {citation!r}"
@pytest.mark.skipif(not HAVE_NODE, reason="node not available")
def test_doc_image_block_svg_yields_the_vector_class_png_unchanged() -> None:
"""Behavioral (node, the house fake DOM): the REAL docImageBlock —
an SVG doc renders class="doc-image-img doc-image-img--vector"
with src = image_url and alt = the summary (title fallback when
the summary is null/blank); a PNG doc's class string is the base
class ONLY (byte-identical to pre-phase — the phase-122/133
raster surfaces cannot drift); the wrapper stays .doc-image."""
js = _read(DOCUMENT_JS)
# The docs ride in via json.dumps (the summary carries quotes and
# an em-dash — the alt contract must survive verbatim).
cases = {
"svg": {
"format": "svg",
"image_url": "/api/documents/7/image",
"summary": 'Vector diagram "topo" \u2014 3 shapes',
"title": "topo",
},
"svgNoSummary": {
"format": "svg",
"image_url": "/api/documents/9/image",
"summary": None,
"title": "rack",
},
"svgBlankSummary": {
"format": "svg",
"image_url": "/api/documents/10/image",
"summary": " ",
"title": "blank",
},
"png": {
"format": "png",
"image_url": "/api/documents/8/image",
"summary": "A photo",
"title": "t",
},
}
script = (
_CONTENT_BRANCH_FAKE_DOM
+ _fn(js, "docImageBlock")
+ "\nconst cases = "
+ json.dumps(cases)
+ """
;
function imageFor(doc) {
const wrap = docImageBlock(doc);
const img = wrap.children[0];
return {
wrapClass: wrap.className, imgClass: img.className, src: img.src, alt: img.alt,
};
}
const out = {};
for (const [k, doc] of Object.entries(cases)) out[k] = imageFor(doc);
console.log(JSON.stringify(out));
"""
)
out = json.loads(_run_node(script))
assert out["svg"] == {
"wrapClass": "doc-image",
"imgClass": "doc-image-img doc-image-img--vector",
"src": "/api/documents/7/image",
"alt": 'Vector diagram "topo" \u2014 3 shapes',
}
# alt falls back to the title (the WCAG alt contract, unchanged).
assert out["svgNoSummary"]["alt"] == "rack"
assert out["svgBlankSummary"]["alt"] == "blank"
assert out["svgBlankSummary"]["imgClass"] == "doc-image-img doc-image-img--vector"
# The raster class string is byte-identical to pre-phase (base only).
assert out["png"]["imgClass"] == "doc-image-img"
assert out["png"]["src"] == "/api/documents/8/image"
def test_doc_image_vector_css_spans_the_card_without_squashing() -> None:
"""styles.css: ``.doc-image-img--vector`` (immediately after
``.doc-image-img``) is ``width: 100%`` + ``height: auto`` +
``object-fit: contain`` — a viewBox-only SVG (no intrinsic size:
the default replaced-element box is 300\u00d7150) spans the
full-width .doc-image card, and contain letterboxes instead of
squashing when the base rule's max-height: 70vh clamps a tall
drawing. Zero colour literals (palette-neutral — the monochrome
invariant); the base .doc-image-img rule is UNTOUCHED (the
phase-122 raster contract)."""
css = _read(STYLES_CSS)
base_i = css.find(".doc-image-img {")
vec_i = css.find(".doc-image-img--vector {")
assert -1 < base_i < vec_i, ".doc-image-img--vector must sit next to the base rule"
# Provenance comment directly above the rule (house style).
comment_start = css.rfind("/*", 0, vec_i)
provenance = css[comment_start:vec_i]
assert -1 < comment_start < vec_i
for citation in ("phase 181", "viewBox", "300\u00d7150", "object-fit", "doc.format"):
assert citation in provenance, f"the comment must cite {citation!r}"
assert "no animation" in provenance
rule = css[vec_i : css.find("}", vec_i)]
for decl in ("width: 100%", "height: auto", "object-fit: contain"):
assert decl in rule, f".doc-image-img--vector must carry {decl!r}"
# Palette-neutral — zero colour literals (no new hue).
assert not re.search(r"#[0-9a-fA-F]{3,8}\b", rule)
assert "rgb(" not in rule and "hsl(" not in rule and "color-mix(" not in rule
# The untouched base rule (the raster + onerror contract).
base_rule = css[base_i : css.find("}", base_i)]
for decl in ("max-width: 100%", "max-height: 70vh", "height: auto"):
assert decl in base_rule, f"the base rule must keep {decl!r}"
def test_figure_and_thumbnail_key_off_image_url_alone() -> None:
"""Design D7's verified non-changes: the chat figure
(appendSourceImageFigure) and the Sources KB-tree thumbnail
(docThumb) resolve their bytes from ``image_url`` alone — no
format field on the wire, none added — and their CSS boxes pin
the img (.source-image-img: 96px cap + object-fit: contain; the
.kb-doc-thumb-img: the fixed 48px box), so a size-less SVG
scales inside each cap exactly like a raster. Pins: neither
function sniffs a format at all (the deliberate non-change,
design D7), the phase-181 comments sit where they were made,
and the two rules keep their box declarations."""
app_js = _read(APP_JS)
assert app_js.count("function appendSourceImageFigure(") == 1
fig = _fn(app_js, "appendSourceImageFigure")
assert "img.src = s.image_url;" in fig
assert "format" not in fig, "no format sniffing on the chat figure (D7)"
fn_start = app_js.index("function appendSourceImageFigure(")
assert "phase 181" in app_js[max(0, fn_start - 2400) : fn_start].lower(), (
"the deliberate-non-change comment sits above the figure"
)
sources_js = _read(SOURCES_JS)
thumb_start = sources_js.index("function docThumb(d)")
thumb_end = sources_js.index("\n }", thumb_start)
thumb = sources_js[thumb_start:thumb_end]
assert "img.src = d.image_url;" in thumb
assert "format" not in thumb, "no format sniffing on the thumbnail (D7)"
branch_i = sources_js.index("if (d.is_image) {")
assert "phase 181" in sources_js[branch_i - 900 : branch_i].lower(), (
"the verified-non-change comment sits above the is_image branch"
)
css = _read(STYLES_CSS)
cap_i = css.find(".source-image-img {")
cap = css[cap_i : css.find("}", cap_i)]
assert "max-height: 96px" in cap and "object-fit: contain" in cap
box_i = css.find(".kb-doc-thumb-img {")
box = css[box_i : css.find("}", box_i)]
assert "width: 100%" in box and "height: 100%" in box
+29 -8
View File
@@ -106,19 +106,25 @@ def _settings(**kwargs: Any) -> Settings:
def test_image_defaults_match_locked_a3() -> None:
"""LOCKED A3: image support is OFF by default (enable only when the
chat model supports vision), the built-in extension set is the six
common formats, and the image bytes' persistent home is the
"""LOCKED A3 + phase 181 LOCKED A7: image support is OFF by default
(enable only when the chat model supports vision), the built-in
extension set is the six common raster formats PLUS ``svg`` (the one
VECTOR member — A1: it indexes from its own extracted text, never a
vision call), and the image bytes' persistent home is the
``~/bor-sources/images`` raw string (``expanduser`` is the
importer's job — the ``sources_dir``/``upload_dir`` convention)."""
importer's job — the ``sources_dir``/``upload_dir`` convention).
``images`` stays the SINGLE knob (A7): with it off an ``.svg`` is a
plain text document, no special-casing."""
s = _settings()
assert s.images is False
# Dotted, lowercased form — the shape the walk's matcher consumes
# (the ``import_extension_set`` precedent).
assert s.image_extension_set == {".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp"}
assert s.image_extension_set == {
".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp", ".svg",
}
assert s.image_dir == "~/bor-sources/images"
# The raw string is stored untouched (no expanduser here).
assert s.image_extensions == "png,jpg,jpeg,webp,gif,bmp"
assert s.image_extensions == "png,jpg,jpeg,webp,gif,bmp,svg"
def test_image_set_is_separate_from_the_import_set() -> None:
@@ -127,7 +133,9 @@ def test_image_set_is_separate_from_the_import_set() -> None:
the import set leaves the image set untouched, and vice versa."""
s = _settings(import_extensions="md")
assert s.import_extension_set == {".md"}
assert s.image_extension_set == {".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp"}
assert s.image_extension_set == {
".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp", ".svg",
}
s2 = _settings(image_extensions="md")
assert s2.image_extension_set == {".md"}
assert ".md" in s2.import_extension_set # the default import set is intact
@@ -762,7 +770,16 @@ def test_image_mimes_cover_the_default_family() -> None:
default image family — dotted lowercase keys, the
``image_extension_set`` shape — and the unlisted-token fallback is
the generic mime (a custom ``BOR_IMAGE_EXTENSIONS`` format fails
soft at the vision endpoint, never a guessed type)."""
soft at the vision endpoint, never a guessed type).
The map is pinned by IDENTITY (not membership), so a silent removal
or re-pointing of any entry fails here. Phase 181 (task 03) adds
``.svg`` as the ONE vector entry; the six raster entries are
untouched by it, and the vector import branch never reaches
:func:`image_data_url` (LOCKED A1: no vision call for an SVG), so
the addition cannot change a describe call — the value exists for
the BYTES route (``app.api.docs.get_document_image`` reads the same
map for its ``Content-Type`` and its sandboxed-CSP guard)."""
assert IMAGE_MIMES == {
".png": "image/png",
".jpg": "image/jpeg",
@@ -770,8 +787,12 @@ def test_image_mimes_cover_the_default_family() -> None:
".webp": "image/webp",
".gif": "image/gif",
".bmp": "image/bmp",
".svg": "image/svg+xml",
}
assert IMAGE_FALLBACK_MIME == "application/octet-stream"
# Every member of the default image set now has a mime — raster and
# vector alike (the vector one is serve-side only, per above).
assert ".svg" in _settings().image_extension_set
assert all(ext in IMAGE_MIMES for ext in _settings().image_extension_set)
+2 -1
View File
@@ -1008,6 +1008,7 @@ def test_import_summary_log_line_includes_summary_counters(
s.summary_backfilled = 1
s.dates_updated = 0
s.images_failed = 0
s.svg_indexed = 4
s.image_context_backfilled = 2
s.pdfs_screenshotted = 1
s.pdfs_pictorial = 2
@@ -1021,7 +1022,7 @@ def test_import_summary_log_line_includes_summary_counters(
assert line == (
"import: summary files=3 added=3 updated=0 unchanged=0 pruned=0 errors=0 "
"chunks=5 embed_batches=4 summaries=2 summary_errors=1 summary_backfilled=1 "
"dates_updated=0 images_failed=0 image_context_backfilled=2 "
"dates_updated=0 images_failed=0 svg_indexed=4 image_context_backfilled=2 "
"pdfs_screenshotted=1 pdfs_pictorial=2 pdf_pictorial_pages=5 "
"pdf_images_deduped=3 pdf_images_blank_skipped=1 "
"formats=yaml:2,md:1"
+482
View File
@@ -0,0 +1,482 @@
"""Unit: phase 181 task 02 — an ``.svg`` indexes as a VECTOR image document.
The pure extractor (:mod:`app.rag.svg_text`) is pinned in
``tests/unit/test_svg_text_extraction.py``; this module pins what the
IMPORTER does with it (LOCKED A1/A2/A7):
* the vector branch of :func:`app.rag.importer._index_image_file` — an
``.svg`` lands ``is_image=True`` with ``image_path`` under
``image_dir``, its ``content`` is the file's OWN extracted text, its
``summary`` mirrors it (the phase-122 convention), and the vision seam
:func:`app.rag.importer._describe_or_skip` is NEVER called (the seam is
patched with a recording stand-in that would return a sentinel — the
empty recording list + the content assertion are the proof, and a
call-counting seam cannot silently succeed);
* the log-only ``svg_indexed`` counter and the ``import: svg indexed``
auditability line (AGENTS rule 10), including ``structural=True`` for
a drawing whose only words are the LOCKED A2 fallback line;
* the fail-soft (LOCKED A2's boundary): a file the extractor REFUSES
(over the byte cap, malformed XML) skips exactly like an undescribed
raster — ``images_failed``, a warning, no row, no orphan copy — while
a merely textless SVG does NOT skip;
* the phase-125 ``image_context`` enrichment prepends for an SVG exactly
as for a raster;
* the phase-181 fast-path kind guard (:func:`_row_kind_mismatch`): a row
imported as TEXT before ``svg`` joined ``BOR_IMAGE_EXTENSIONS`` is
converted on the NEXT sync (same bytes, same ``file_hash`` — and for an
ASCII file the text digest equals the byte digest too, which is why the
image branch's own content-digest skip now requires ``is_image``), and
a row already of the right kind — raster or vector — keeps the
phase-147 fast path byte-for-byte (no second seam call, no second
``svg_indexed``).
"""
from __future__ import annotations
import asyncio
import hashlib
import logging
from pathlib import Path
from typing import Any
import pytest
from sqlalchemy import select
import app.rag.importer as importer
from app.config import Settings
from app.models import Chunk, Document
from app.rag.importer import import_sources
from app.rag.svg_text import MAX_SVG_BYTES
from tests.fakes import FakeEmbedder
#: A diagram that carries words: ``<title>`` + ``<desc>`` + a ``<text>``
#: inside a group (document order is the extractor's contract).
SVG_TOPO = (
b'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 120 80">'
b"<title>Server rack topology</title>"
b"<desc>Two switches uplink to the router.</desc>"
b'<g><path d="M0 0h10"/><text>rack-1</text></g>'
b"</svg>"
)
#: The text :func:`app.rag.svg_text.extract_svg_text` yields for
#: :data:`SVG_TOPO` (pinned literally here so a change to the extraction
#: contract breaks BOTH modules, not silently).
SVG_TOPO_TEXT = "Server rack topology\nTwo switches uplink to the router.\nrack-1"
#: A pure-shape drawing: no words at all (the LOCKED A2 class).
SVG_SHAPES_ONLY = (
b'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64">'
b'<path d="M0 0h10"/><path d="M0 20h10"/><circle cx="1" cy="1" r="2"/>'
b"</svg>"
)
#: The LOCKED A2 structural line for :data:`SVG_SHAPES_ONLY` — built only
#: from facts the file states (stem, shape census, viewBox).
SVG_SHAPES_ONLY_TEXT = (
'Vector diagram "gizmo" — 3 shapes (path×2, circle), viewBox 64×64'
)
#: Over :data:`MAX_SVG_BYTES`: the size guard refuses it BEFORE any parse.
SVG_OVER_CAP = b"<svg>" + b"\n" * (MAX_SVG_BYTES + 1) + b"</svg>"
#: Well-formed bytes, unparseable markup (the second refusal class).
SVG_MALFORMED = b'<svg xmlns="http://www.w3.org/2000/svg"><title>oops'
#: A real 1×1 PNG for the raster fast-path regression pins.
PNG_1X1 = b"\x89PNG\r\n\x1a\n" + b"\x00" * 24
#: What a describe seam would hand back if the vector branch ever reached
#: it — never a plausible diagram text, so it cannot pass unnoticed.
VISION_SENTINEL = "SHOULD NEVER BE DESCRIBED: a vision call ran for an SVG."
def _settings(**kwargs: Any) -> Settings:
"""Build Settings without reading a .env file (deterministic tests)."""
kwargs.setdefault("_env_file", None)
return Settings(**kwargs) # pyright: ignore[reportCallIssue] (kwarg exists at runtime)
def _image_llm(tmp_path: Path, **kwargs: Any) -> FakeEmbedder:
"""A fake LLM whose settings index images into *tmp_path*."""
kwargs.setdefault("_env_file", None)
kwargs.setdefault("images", True)
kwargs.setdefault("image_dir", str(tmp_path / "images"))
llm = FakeEmbedder()
llm.settings = Settings(**kwargs) # pyright: ignore[reportCallIssue]
return llm
def _record_description(monkeypatch: pytest.MonkeyPatch) -> list[str]:
"""Patch the vision seam with a recorder (never a real call): the
paths it is asked to describe, always answered with
:data:`VISION_SENTINEL`. An empty list + a non-sentinel ``content``
is the proof that the vector branch bypassed vision entirely."""
handed: list[str] = []
async def _fake(
llm: importer.Embedder,
*,
data: bytes,
source: str,
rel: str,
full_path: Path,
context: str | None = None,
) -> str | None:
handed.append(rel)
return VISION_SENTINEL
monkeypatch.setattr(importer, "_describe_or_skip", _fake)
return handed
def _doc(db, source: str, rel: str) -> Document:
doc = db.scalar(
select(Document).where(Document.source == source, Document.path == rel)
)
assert doc is not None, f"no documents row for ({source!r}, {rel!r})"
return doc
def _cleanup_source(db, source: str) -> None:
for doc in db.scalars(select(Document).where(Document.source == source)).all():
db.delete(doc)
db.commit()
# --- the vector branch ------------------------------------------------------
def test_svg_indexes_as_an_image_document_with_no_vision_call(
db, tmp_path: Path, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""LOCKED A1: the ``.svg`` takes the image branch (``is_image``,
``image_path``, the ``<doc-id>.svg`` copy, the ``summary = content``
convention, one embedded content chunk + the ``is_summary`` chunk)
and its searchable text is the file's own ``<title>``/``<desc>``/
``<text>`` — with the vision seam never invoked. ``svg_indexed`` and
the per-file ``import: svg indexed`` line make the run auditable."""
root = tmp_path / "svgsrc"
root.mkdir()
(root / "topo.svg").write_bytes(SVG_TOPO)
llm = _image_llm(tmp_path)
handed = _record_description(monkeypatch)
try:
with caplog.at_level(logging.INFO, logger="app.importer"):
summary = asyncio.run(import_sources([root], llm, session=db))
assert (summary.files, summary.added, summary.images_failed) == (1, 1, 0)
assert summary.errors == 0
assert summary.formats == {"svg": 1} # counted under its image token
assert summary.svg_indexed == 1
assert handed == [], "an SVG must never reach the vision seam"
doc = _doc(db, root.name, "topo.svg")
assert doc.is_image is True
assert doc.title == "topo" # the non-markdown stem rule (shared with raster)
assert doc.content == SVG_TOPO_TEXT
assert doc.summary == doc.content # the phase-122 image convention
assert doc.image_context is None # standalone: the pre-phase row shape
assert doc.image_path is not None
copy = Path(doc.image_path)
assert copy.parent == Path(llm.settings.image_dir).expanduser()
assert copy.name == f"{doc.id}.svg" # row and copy agree on the id
assert copy.read_bytes() == SVG_TOPO # the exact bytes, unserved here
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc.id)).all()
content_chunks = [c for c in chunks if not c.is_summary]
assert [c.content for c in content_chunks] == [doc.content]
summary_chunks = [c for c in chunks if c.is_summary]
assert len(summary_chunks) == 1 and summary_chunks[0].position == -1
assert summary_chunks[0].content == doc.content
for c in chunks:
assert c.embedding is not None and len(c.embedding) == 768
indexed = [
r.getMessage() for r in caplog.records if "import: svg indexed" in r.getMessage()
]
assert indexed == [
f"import: svg indexed source={root.name} path=topo.svg "
f"structural=False chars={len(SVG_TOPO_TEXT)}"
]
line = next(
r.getMessage()
for r in caplog.records
if r.getMessage().startswith("import: summary files=")
)
assert "svg_indexed=1" in line and "images_failed=0" in line
finally:
_cleanup_source(db, root.name)
def test_textless_svg_indexes_with_the_structural_line(
db, tmp_path: Path, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
) -> None:
"""LOCKED A2: a diagram of pure ``<path>``s is NOT skipped — it is
indexed with the structural line built from the file (stem, shape
census, viewBox), counted like any other vector doc and logged with
``structural=True`` so the owner can tell it from a worded diagram."""
root = tmp_path / "shapeonly"
root.mkdir()
(root / "gizmo.svg").write_bytes(SVG_SHAPES_ONLY)
llm = _image_llm(tmp_path)
handed = _record_description(monkeypatch)
try:
with caplog.at_level(logging.INFO, logger="app.importer"):
summary = asyncio.run(import_sources([root], llm, session=db))
assert (summary.files, summary.added, summary.images_failed) == (1, 1, 0)
assert summary.svg_indexed == 1
assert handed == []
doc = _doc(db, root.name, "gizmo.svg")
assert doc.is_image is True
assert doc.content == SVG_SHAPES_ONLY_TEXT
assert doc.summary == doc.content
line = next(
r.getMessage() for r in caplog.records if "import: svg indexed" in r.getMessage()
)
assert "structural=True" in line
finally:
_cleanup_source(db, root.name)
@pytest.mark.parametrize(
("name", "payload"),
[
pytest.param("huge.svg", SVG_OVER_CAP, id="over-the-byte-cap"),
pytest.param("broken.svg", SVG_MALFORMED, id="malformed-xml"),
],
)
def test_unusable_svg_skips_the_doc_counts_and_logs(
db,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
name: str,
payload: bytes,
) -> None:
"""The fail-soft stays LOCKED A3-shaped for the vector case: a file
the extractor refuses skips the doc ENTIRELY — no row, no copy (not
even the image dir), ``images_failed`` + a warning naming the vector
extraction, the sync continues. This is the ONLY skip class the phase
keeps for an SVG (:func:`app.rag.svg_text.extract_svg_text`)."""
root = tmp_path / "badsvg"
root.mkdir()
(root / name).write_bytes(payload)
llm = _image_llm(tmp_path)
handed = _record_description(monkeypatch)
try:
with caplog.at_level(logging.INFO, logger="app.importer"):
summary = asyncio.run(import_sources([root], llm, session=db))
assert (summary.files, summary.added, summary.images_failed) == (1, 0, 1)
assert summary.svg_indexed == 0
assert handed == []
assert db.scalar(select(Document).where(Document.source == root.name)) is None
assert not Path(llm.settings.image_dir).expanduser().exists(), (
"no copy — and not even the image dir — may be left behind"
)
warnings = [
r
for r in caplog.records
if "svg text extraction failed" in r.getMessage()
]
assert len(warnings) == 1 and warnings[0].levelno == logging.WARNING
assert "image description failed" not in caplog.text
line = next(
r.getMessage()
for r in caplog.records
if r.getMessage().startswith("import: summary files=")
)
assert "images_failed=1" in line and "svg_indexed=0" in line
finally:
_cleanup_source(db, root.name)
def test_markdown_reference_context_prepends_for_an_svg(
db, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Phase 125's enrichment is kind-agnostic: a ``![alt](topo.svg)``
reference in the same source still builds ``image_context`` (from the
file on disk, no vision call) and the doc's content becomes
``<context>\\n\\n<extracted text>`` — context first for retrieval, the
vector file's own words second."""
root = tmp_path / "svganchor"
root.mkdir()
(root / "guide.md").write_text(
"Why this diagram exists.\n\n![The rack plan](topo.svg)\n\nWhat follows it.\n",
encoding="utf-8",
)
(root / "topo.svg").write_bytes(SVG_TOPO)
llm = _image_llm(tmp_path)
handed = _record_description(monkeypatch)
try:
summary = asyncio.run(import_sources([root], llm, session=db))
assert (summary.files, summary.added, summary.images_failed) == (2, 2, 0)
assert summary.svg_indexed == 1
assert handed == []
doc = _doc(db, root.name, "topo.svg")
assert doc.is_image is True
assert doc.image_context is not None
assert doc.image_context.startswith(f"Referenced in {root.name}/guide.md:")
assert 'Alt text: "The rack plan"' in doc.image_context
assert doc.content == f"{doc.image_context}\n\n{SVG_TOPO_TEXT}"
assert doc.summary == doc.content
finally:
_cleanup_source(db, root.name)
# --- the fast-path kind guard (the text → image migration) ------------------
def test_text_indexed_svg_row_is_converted_on_the_next_sync(
db, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Design D4: the row an earlier run indexed as TEXT (``svg`` not yet
in the image set) is converted by the NEXT sync — same bytes, same
``file_hash`` — and the sync AFTER that is fast again.
The conversion needs BOTH halves of the guard: the fast path in
:func:`_index_file` must fall through, and the image branch's own
content-digest skip must require ``is_image`` — an ASCII file's text
digest equals its byte digest, so without it the row would report
``unchanged`` forever and never gain a copy (asserted below, because
it is the whole reason the guard exists)."""
root = tmp_path / "migrate"
root.mkdir()
(root / "topo.svg").write_bytes(SVG_TOPO)
handed = _record_description(monkeypatch)
# Stage A — the pre-phase state: images OFF, ``svg`` imported as text.
text_llm = _image_llm(
tmp_path,
images=False,
import_extensions=f"{_settings().import_extensions},svg",
)
try:
summary_a = asyncio.run(import_sources([root], text_llm, session=db))
assert (summary_a.files, summary_a.added, summary_a.svg_indexed) == (1, 1, 0)
before = _doc(db, root.name, "topo.svg")
assert before.is_image is False
assert before.image_path is None
assert before.content == SVG_TOPO.decode("utf-8") # the raw XML the owner saw
byte_digest = hashlib.sha256(SVG_TOPO).hexdigest()
assert before.file_hash == byte_digest
# The trap D4 closes: the text digest of a byte-clean file IS the
# byte digest, so no digest comparison can spot the mismatch.
assert before.content_hash == byte_digest
doc_id = before.id
# Stage B — the toggle on: the guard fires, the row converts.
image_llm = _image_llm(tmp_path)
summary_b = asyncio.run(import_sources([root], image_llm, session=db))
assert (summary_b.added, summary_b.updated, summary_b.unchanged) == (0, 1, 0)
assert (summary_b.images_failed, summary_b.svg_indexed) == (0, 1)
assert handed == [], "the conversion must not call vision either"
after = _doc(db, root.name, "topo.svg")
assert after.id == doc_id # converted in place — the id (and copy name) is stable
assert after.is_image is True
assert after.content == SVG_TOPO_TEXT
assert after.summary == after.content
assert after.file_hash == byte_digest # the bytes never changed
# The image branch stamps the BYTES digest as ``content_hash`` —
# the SAME value the text path had already stored, which is why
# that skip needed its own ``is_image`` guard.
assert after.content_hash == byte_digest
copy = Path(str(after.image_path))
assert copy == Path(image_llm.settings.image_dir).expanduser() / f"{doc_id}.svg"
assert copy.read_bytes() == SVG_TOPO
# The text chunks are replaced, not appended: one content chunk
# (the extracted text) + the is_summary mirror.
chunks = db.scalars(select(Chunk).where(Chunk.document_id == doc_id)).all()
assert [c.content for c in chunks if not c.is_summary] == [SVG_TOPO_TEXT]
assert [c.content for c in chunks if c.is_summary] == [SVG_TOPO_TEXT]
# Stage C — the guard fired once: the fast path resumes.
summary_c = asyncio.run(import_sources([root], image_llm, session=db))
assert (summary_c.added, summary_c.updated, summary_c.unchanged) == (0, 0, 1)
assert (summary_c.svg_indexed, summary_c.images_failed) == (0, 0)
assert handed == []
assert _doc(db, root.name, "topo.svg").is_image is True
finally:
_cleanup_source(db, root.name)
def test_correctly_classified_raster_row_still_takes_the_fast_path(
db, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""The phase-147 fast path survives the guard for every row whose
KIND already agrees: a raster image with a matching ``file_hash`` is
UNCHANGED on the next sync — the seam is not called a second time and
no vector counter moves."""
root = tmp_path / "rasterfast"
root.mkdir()
(root / "pic.png").write_bytes(PNG_1X1)
llm = _image_llm(tmp_path)
handed = _record_description(monkeypatch)
try:
first = asyncio.run(import_sources([root], llm, session=db))
assert (first.added, first.unchanged, first.svg_indexed) == (1, 0, 0)
assert handed == ["pic.png"] # the raster branch DID use the seam
doc = _doc(db, root.name, "pic.png")
assert doc.is_image is True
second = asyncio.run(import_sources([root], llm, session=db))
assert (second.added, second.updated, second.unchanged) == (0, 0, 1)
assert second.svg_indexed == 0
assert handed == ["pic.png"], "the fast path must not re-describe the image"
assert _doc(db, root.name, "pic.png").content == VISION_SENTINEL
finally:
_cleanup_source(db, root.name)
def test_correctly_classified_vector_row_still_takes_the_fast_path(
db, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Same pin for the vector row: once converted (or freshly indexed)
it is UNCHANGED — no second extraction, no second ``svg_indexed``,
no second copy write."""
root = tmp_path / "vectorfast"
root.mkdir()
(root / "topo.svg").write_bytes(SVG_TOPO)
llm = _image_llm(tmp_path)
handed = _record_description(monkeypatch)
try:
first = asyncio.run(import_sources([root], llm, session=db))
assert (first.added, first.svg_indexed) == (1, 1)
copy = Path(str(_doc(db, root.name, "topo.svg").image_path))
copy.unlink() # the fast path must not re-create it either
second = asyncio.run(import_sources([root], llm, session=db))
assert (second.added, second.updated, second.unchanged) == (0, 0, 1)
assert (second.svg_indexed, second.images_failed) == (0, 0)
assert handed == []
assert not copy.exists()
finally:
_cleanup_source(db, root.name)
def test_row_kind_mismatch_table() -> None:
"""The guard's whole truth table (pure — no DB, no filesystem):
it fires ONLY for a TEXT row whose file the CURRENT settings would
index as an image. The reverse direction (an ``is_image`` row while
the toggle is off, or a plain text file) stays on the fast path —
the phase-122 prune guard owns the off state (LOCKED A7)."""
svg = Path("/src/diagrams/topo.svg")
md = Path("/src/notes.md")
text_row = Document(source="s", path="topo.svg", is_image=False)
image_row = Document(source="s", path="topo.svg", is_image=True)
def _settings_(*, images: bool) -> Settings:
return _settings(images=images)
# The migration: the toggle is on, the file is an image, the row is not.
assert importer._row_kind_mismatch(svg, text_row, _settings_(images=True)) is True
# Already an image doc → nothing to migrate.
assert importer._row_kind_mismatch(svg, image_row, _settings_(images=True)) is False
# Toggle off → the walk never admits the file, the prune guard protects
# the row: never force the image path (LOCKED A7).
assert importer._row_kind_mismatch(svg, text_row, _settings_(images=False)) is False
assert importer._row_kind_mismatch(svg, image_row, _settings_(images=False)) is False
# A file outside the image set is a text file either way.
assert importer._row_kind_mismatch(md, text_row, _settings_(images=True)) is False
# An owner-narrowed image set (no ``svg``) is not a mismatch either.
narrow = _settings(images=True, image_extensions="png,jpg")
assert importer._row_kind_mismatch(svg, text_row, narrow) is False
+426
View File
@@ -0,0 +1,426 @@
"""Unit tests: SVG text extraction (phase 181, task 01).
:func:`app.rag.svg_text.extract_svg_text` is a PURE function — the text
that makes an SVG document retrievable WITHOUT a vision call (LOCKED
A1). No DB, no filesystem, no LLM: every case is a literal SVG string,
so the whole contract is pinned here before the importer branch (task
02) relies on it.
The matrix:
* the guards that run BEFORE any parse — empty / whitespace-only input,
the :data:`~app.rag.svg_text.MAX_SVG_BYTES` cap, the
``<!DOCTYPE``/``<!ENTITY`` refusal (case-insensitive, both a doctype
wrapper and a bare entity declaration), malformed XML → ``None``
(never a raise);
* extraction — ``<title>``/``<desc>``/``<text>`` in DOCUMENT order,
``<tspan>`` children joined with a space, namespace-prefixed and
namespaced tags treated alike, ``xlink:`` attributes ignored,
whitespace normalised, repeats dropped case-insensitively keeping the
first, ``&amp;``/``&#8212;`` decoded by the parser;
* the :data:`~app.rag.svg_text.MAX_SVG_TEXT_CHARS` cap — cut at the
last newline/period before the cap, hard-cut only when the window
holds no boundary at all;
* the LOCKED A2 structural fallback — stem, shape census (counts,
plural, top-3 selection with a document-order tie-break) and the
``viewBox`` clause, each omitted when the file does not state it.
Importer-level tests (the vector branch, the no-vision-call seam) live
in the phase's ingest module (tasks 02/05).
"""
from __future__ import annotations
from app.rag.svg_text import (
MAX_SVG_BYTES,
MAX_SVG_TEXT_CHARS,
SvgText,
extract_svg_text,
local_name,
)
SVG_NS = "http://www.w3.org/2000/svg"
XLINK_NS = "http://www.w3.org/1999/xlink"
# --- constants -------------------------------------------------------------
def test_module_constants_are_pinned() -> None:
"""The caps are part of the contract the importer's tests assume."""
assert MAX_SVG_BYTES == 2 * 1024 * 1024
assert MAX_SVG_TEXT_CHARS == 4000
# --- the local_name helper -------------------------------------------------
def test_local_name_strips_the_clark_notation_namespace() -> None:
assert local_name(f"{{{SVG_NS}}}title") == "title"
assert local_name("title") == "title"
def test_local_name_of_a_non_string_tag_is_empty() -> None:
"""Comment / PI markers are callables — they carry no name."""
assert local_name(object()) == ""
# --- guards (no parse attempted) -------------------------------------------
def test_empty_input_is_a_refusal_not_a_crash() -> None:
assert extract_svg_text(b"") is None
assert extract_svg_text(b"", stem="diagram") is None
def test_whitespace_only_input_is_a_refusal() -> None:
assert extract_svg_text(b" \n\t \r\n") is None
def test_bytes_over_the_size_cap_are_refused() -> None:
"""The memory guard: an over-cap file never reaches the parser."""
padded = b"<svg><title>" + b"x" * (MAX_SVG_BYTES + 1) + b"</title></svg>"
assert len(padded) > MAX_SVG_BYTES
assert extract_svg_text(padded, stem="huge") is None
def test_bytes_exactly_at_the_size_cap_are_parsed() -> None:
"""The cap is ``>``-based: exactly ``MAX_SVG_BYTES`` is in range."""
opener, closer = b"<svg><title>", b"</title></svg>"
filler = b"x" * (MAX_SVG_BYTES - len(opener) - len(closer))
result = extract_svg_text(opener + filler + closer)
assert result is not None
assert result.structural is False
def test_doctype_with_an_internal_entity_is_refused() -> None:
"""Stdlib ElementTree happily EXPANDS an internal entity — proving
why the doctype-bearing input is refused outright (billion-laughs
class): the reference must never be resolved."""
svg = b'<!DOCTYPE svg [<!ENTITY x "y">]><svg><title>&x;</title></svg>'
assert extract_svg_text(svg, stem="bomb") is None
def test_bare_entity_declaration_is_refused() -> None:
assert extract_svg_text(b'<!ENTITY x "y"><svg><title>x</title></svg>') is None
def test_entity_declaration_hidden_inside_content_is_refused() -> None:
"""The marker scan is over the raw bytes — a declaration anywhere
(even in a position the parser would reject) refuses the file."""
assert extract_svg_text(b"<svg>\n<!entity x 'y'>\n</svg>") is None
def test_guards_are_case_insensitive() -> None:
assert extract_svg_text(b'<!doctype svg SYSTEM "http://x/y.dtd"><svg/>') is None
assert extract_svg_text(b'<!entity x "y"><svg/>') is None
def test_malformed_xml_is_a_refusal_not_an_exception() -> None:
"""The caller's fail-soft is a skip (``images_failed``), so a parse
error must never escape."""
assert extract_svg_text(b"<svg><title>unclosed") is None
assert extract_svg_text(b"<svg><rect></svg>") is None
assert extract_svg_text(b"\x00\x01\x02") is None
assert extract_svg_text(b"not xml at all") is None
# --- extraction: title / desc / text ---------------------------------------
def test_title_desc_and_text_are_collected_in_document_order() -> None:
svg = (
b"<svg>"
b"<title>Top Title</title>"
b"<desc>What the diagram shows</desc>"
b"<text>first label</text>"
b"<text>second label</text>"
b"</svg>"
)
assert extract_svg_text(svg, stem="topo") == SvgText(
text="Top Title\nWhat the diagram shows\nfirst label\nsecond label",
structural=False,
)
def test_a_title_after_its_shape_still_follows_document_order() -> None:
svg = b"<svg><text>later</text><g><title>group title</title></g><desc>d</desc></svg>"
assert extract_svg_text(svg) == SvgText(
text="later\ngroup title\nd", structural=False
)
def test_title_and_desc_concatenate_nested_markup() -> None:
svg = b"<title>Deploy <tspan>flow</tspan> map</title>"
full = b"<svg>" + svg + b"</svg>"
assert extract_svg_text(full) == SvgText(text="Deploy flow map", structural=False)
def test_text_children_are_joined_with_a_space() -> None:
"""``<text>CPU<tspan>usage</tspan></text>`` reads "CPU usage" —
without the space the labels weld into one unsearchable token."""
svg = b'<svg><text>CPU<tspan>usage</tspan>7d</text></svg>'
assert extract_svg_text(svg) == SvgText(text="CPU usage 7d", structural=False)
def test_whitespace_is_normalised_in_every_item() -> None:
svg = b"<svg><title> rack\n\t layout map </title><text> a\n b </text></svg>"
assert extract_svg_text(svg) == SvgText(
text="rack layout map\na b", structural=False
)
def test_blank_title_is_dropped_and_never_stops_the_walk() -> None:
svg = b"<svg><title> </title><desc> </desc><text>real label</text></svg>"
assert extract_svg_text(svg) == SvgText(text="real label", structural=False)
def test_only_blank_text_falls_back_to_the_structural_line() -> None:
"""A blank ``<title>``/``<text>`` contributes no words, so the file
is still described by what it is (the empty ``<text>`` counts as a
drawn element)."""
svg = b'<svg viewBox="0 0 4 4"><title></title><text> </text></svg>'
assert extract_svg_text(svg, stem="blank") == SvgText(
text='Vector diagram "blank" — 1 shape (text), viewBox 4×4', structural=True
)
def test_repeats_are_dropped_case_insensitively_keeping_the_first() -> None:
svg = (
b"<svg>"
b"<desc>Power Draw</desc>"
b"<title>power draw</title>"
b"<text>Other</text>"
b"</svg>"
)
assert extract_svg_text(svg) == SvgText(text="Power Draw\nOther", structural=False)
def test_parser_decodes_entity_references_in_the_text() -> None:
"""``&amp;`` comes out decoded — the renderer re-escapes on display
(phase 180), so storing the decoded value is the correct direction."""
svg = b"<svg><title>Red &amp; Blue &#8212; map</title></svg>"
assert extract_svg_text(svg) == SvgText(text="Red & Blue — map", structural=False)
def test_attributes_are_never_text() -> None:
svg = (
b'<svg xmlns:xlink="'
+ XLINK_NS.encode()
+ b'"><title lang="en">Visible</title>'
b'<image xlink:href="secret-internal-name.png"/></svg>'
)
assert extract_svg_text(svg, stem="diagram") == SvgText(
text="Visible", structural=False
)
def test_an_image_only_svg_yields_no_words_from_its_href() -> None:
"""ASSUMPTION (task 01): an embedded ``<image>`` is not followed —
its ``href`` is not text, so the structural line describes the file."""
svg = (
b'<svg xmlns:xlink="'
+ XLINK_NS.encode()
+ b'" viewBox="0 0 10 10">'
b'<image xlink:href="inner.png"/>'
b"</svg>"
)
result = extract_svg_text(svg, stem="poster")
assert result is not None
assert result.structural is True
assert result.text == 'Vector diagram "poster" — 1 shape (image), viewBox 10×10'
# --- extraction: namespaces ------------------------------------------------
def test_the_default_svg_namespace_is_ignored() -> None:
svg = (
b'<svg xmlns="'
+ SVG_NS.encode()
+ b'"><title>Namespace title</title><desc>ns desc</desc>'
b'<text>ns label</text></svg>'
)
assert extract_svg_text(svg) == SvgText(
text="Namespace title\nns desc\nns label", structural=False
)
def test_a_declared_prefix_form_counts_like_the_plain_tag() -> None:
"""``ElementTree`` expands ``<svg:title>`` to Clark notation, so the
local-name match covers it for free."""
svg = (
b'<svg xmlns:svg="'
+ SVG_NS.encode()
+ b'"><svg:title>Prefixed</svg:title>'
b"<svg:desc>Prefixed desc</svg:desc></svg>"
)
assert extract_svg_text(svg) == SvgText(
text="Prefixed\nPrefixed desc", structural=False
)
def test_a_foreign_namespace_element_is_not_worded_text() -> None:
"""A ``<title>`` in a NON-SVG vocabulary is still matched by local
name (the walk is namespace-agnostic) while the foreign element
counts as a shape when it carries no words."""
svg = (
b'<svg xmlns:h="http://www.w3.org/1999/xhtml">'
b'<foreignObject><h:p>paragraph</h:p></foreignObject>'
b"</svg>"
)
result = extract_svg_text(svg, stem="mixed")
assert result is not None
assert result.structural is True
assert result.text == 'Vector diagram "mixed" — 2 shapes (foreignObject, p)'
def test_the_xml_declaration_encoding_is_honoured() -> None:
svg = b"<?xml version='1.0' encoding='UTF-8'?><svg><title>Caf\xc3\xa9</title></svg>"
assert extract_svg_text(svg) == SvgText(text="Café", structural=False)
def test_comments_are_not_text_and_not_shapes() -> None:
svg = b'<svg viewBox="0 0 2 2"><!-- a comment --><path d="M0 0"/></svg>'
result = extract_svg_text(svg, stem="clean")
assert result is not None
assert result.text == 'Vector diagram "clean" — 1 shape (path), viewBox 2×2'
# --- the MAX_SVG_TEXT_CHARS cap --------------------------------------------
def test_extraction_under_the_cap_is_untouched() -> None:
svg = b"<svg><title>" + b"x" * (MAX_SVG_TEXT_CHARS - 20) + b"</title></svg>"
result = extract_svg_text(svg)
assert result is not None
assert len(result.text) == MAX_SVG_TEXT_CHARS - 20
def test_over_cap_text_ends_on_a_sentence_boundary() -> None:
svg = b"<svg><title>" + b"x. " * 2000 + b"</title></svg>"
result = extract_svg_text(svg)
assert result is not None
assert result.structural is False
assert len(result.text) <= MAX_SVG_TEXT_CHARS
assert len(result.text) > MAX_SVG_TEXT_CHARS - 10
assert result.text.endswith(".")
assert result.text.count("x.") < 2000 # the cut is mid-item
def test_over_cap_text_prefers_the_last_newline_boundary() -> None:
"""Two items joined with ``\\n``: the newline is the LAST boundary
inside the window, so the second item is dropped whole (never a
partial word) and the trailing newline is stripped."""
first = b"a" * (MAX_SVG_TEXT_CHARS - 10)
svg = b"<svg><title>" + first + b"</title><text>" + b"b" * 200 + b"</text></svg>"
result = extract_svg_text(svg)
assert result is not None
assert result.text == first.decode()
def test_over_cap_text_without_any_boundary_is_hard_cut() -> None:
svg = b"<svg><title>" + b"x" * (MAX_SVG_TEXT_CHARS + 1000) + b"</title></svg>"
result = extract_svg_text(svg)
assert result is not None
assert result.text == "x" * MAX_SVG_TEXT_CHARS
# --- the structural fallback (LOCKED A2) -----------------------------------
def test_single_shape_line_matches_the_pinned_wording() -> None:
svg = b'<svg><path d="M0 0 L1 1"/></svg>'
assert extract_svg_text(svg, stem="topo") == SvgText(
text='Vector diagram "topo" — 1 shape (path)', structural=True
)
def test_shape_noun_pluralises_and_counts_are_listed() -> None:
svg = b"<svg><path/><path/><path/></svg>"
assert extract_svg_text(svg, stem="topo") == SvgText(
text='Vector diagram "topo" — 3 shapes (path×3)', structural=True
)
def test_top_three_kinds_with_document_order_tie_break() -> None:
svg = b"<svg><g/><g/><g/><path/><path/><circle/><rect/></svg>"
result = extract_svg_text(svg, stem="census")
assert result is not None
assert result.text == 'Vector diagram "census" — 7 shapes (g×3, path×2, circle)'
assert result.structural is True
def test_structure_elements_are_never_counted_as_shapes() -> None:
svg = b"<svg><title> </title><desc/><g><rect/></g></svg>"
result = extract_svg_text(svg, stem="nested")
assert result is not None
assert result.text == 'Vector diagram "nested" — 2 shapes (g, rect)'
def test_a_nested_svg_element_is_structure_too() -> None:
svg = b"<svg><g><svg><rect/></svg></g></svg>"
result = extract_svg_text(svg, stem="inset")
assert result is not None
assert result.text == 'Vector diagram "inset" — 2 shapes (g, rect)'
def test_viewbox_reports_width_and_height() -> None:
svg = b'<svg viewBox="0 0 128 64"><path/></svg>'
result = extract_svg_text(svg, stem="wide")
assert result is not None
assert result.text == 'Vector diagram "wide" — 1 shape (path), viewBox 128×64'
def test_viewbox_accepts_commas_and_integral_floats() -> None:
svg = b'<svg viewBox=" 0 , 0 , 8.0 , 6.0 "><path/></svg>'
result = extract_svg_text(svg, stem="comma")
assert result is not None
assert result.text == 'Vector diagram "comma" — 1 shape (path), viewBox 8×6'
def test_viewbox_keeps_fractional_sizes_without_padding() -> None:
svg = b'<svg viewBox="0 0 8.5 6.25"><g><circle/></g></svg>'
result = extract_svg_text(svg)
assert result is not None
assert result.text == "Vector diagram — 2 shapes (g, circle), viewBox 8.5×6.25"
def test_unparseable_viewbox_omits_the_clause() -> None:
for raw in (b"", b"0 0 8", b"0 0 8 6 4", b"0 0 wide tall", b"0 0 NaN NaN"):
svg = b'<svg viewBox="' + raw + b'"><path/></svg>'
result = extract_svg_text(svg, stem="odd")
assert result is not None, raw
assert result.text == 'Vector diagram "odd" — 1 shape (path)', raw
def test_missing_viewbox_omits_the_clause() -> None:
result = extract_svg_text(b"<svg><path/></svg>", stem="nobox")
assert result is not None
assert "viewBox" not in result.text
def test_viewbox_without_shapes_is_the_only_fact() -> None:
svg = b'<svg viewBox="0 0 16 9"></svg>'
assert extract_svg_text(svg, stem="canvas") == SvgText(
text='Vector diagram "canvas" — viewBox 16×9', structural=True
)
def test_empty_stem_never_renders_empty_quotes() -> None:
assert extract_svg_text(b"<svg></svg>") == SvgText(
text="Vector diagram", structural=True
)
assert extract_svg_text(b'<svg viewBox="0 0 1 1"><path/></svg>') == SvgText(
text="Vector diagram — 1 shape (path), viewBox 1×1", structural=True
)
def test_a_root_that_is_not_svg_still_yields_honest_text() -> None:
"""The walk is tag-based — a mis-named root is not rejected, and the
root is never counted among the shapes."""
result = extract_svg_text(b"<html><title>Legacy</title></html>", stem="legacy")
assert result == SvgText(text="Legacy", structural=False)
structural = extract_svg_text(b"<html><body/></html>", stem="legacy")
assert structural == SvgText(
text='Vector diagram "legacy" — 1 shape (body)', structural=True
)