phase: 181_svg_document_preview
**Phase 181 — final verification pass (all tasks already complete; verified + docs pass)**
- Verified: `app/rag/svg_text.py` (100 % cov), vector branch in `_index_image_file`, `_row_kind_mismatch` text→image migration, `IMAGE_MIMES[".svg"]`, sandboxed CSP on `GET /api/documents/{id}/image`, `.doc-image-img--vector` in `document.js`/`styles.css`, figure/thumbnail deliberate non-changes.
- Tests/lint (exact commands): `uv run pytest --cov=app --cov-report=term-missing` → **4265 passed, TOTAL 99 %** (exit 0) · `uv run pytest tests/e2e/test_svg_document_preview.py -v --no-cov` → **5 passed in isolation** · `uv run ruff check .` → clean · `uv run pyright` → **0 errors**.
- Regression (each run in isolation, `--no-cov`, all rc=0): `test_image_documents`, `test_chat_image_questions`, `test_answer_images`, `test_chat_answer_clusters`, `test_image_anchor_context`, `test_image_upload_feedback`, `test_markdown_inline_links_images`, `test_prose_word_wrap`, `test_pdf_support`, `test_pictorial_pdf`, `test_garbage_pdf`, `test_pdf_image_extraction`, `test_pdf_image_context`, `test_pdf_pages_in_sources`, `test_pdf_strip_additive_selfheal`, `test_sync_image_progress`.
- Completion criteria: ①SVG→`is_image`+`image_path`, no vision call, own `<title>/<desc>/<text>` — **met**; ②`image/svg+xml` + `default-src 'none'; sandbox` + `private, max-age=3600`, raster headers byte-unchanged — **met**; ③chat figure → modal `<img>`, no XML — **met**; ④`/document.html` full-width vector in `.doc-image` + text below — **met**; ⑤`` paints — **met**; ⑥raster/PDF unchanged — **met**; ⑦all gates green — **met**; ⑧commit + phase-dir move — harness-owned.
- Documentation: found and fixed README inconsistencies — stale forward reference "left to phase 181 / is phase 181's job" → completed-tense sizing note; "Updating the documents" standalone-image list missing `svg`; new **Raster only** bullet for question uploads (`.svg` → 422) documenting the `chat_images` guard; missing `svg_indexed=N` / `import: svg indexed` log contract; env-table heading `122/123/124` → `122/123/124/181`; new troubleshooting entry for an `.svg` showing raw XML. Full audit: every `Settings` field and every `.env.example` var is documented; pages/commands/scripts all exist. AGENTS.md verified accurate (phase-181 §5 clause present) — no changes needed beyond what the phase wrote.
- Notable deviation (kept from task 02): `app/api/chat_images.py` subtracts `_VECTOR_IMAGE_EXTENSIONS` from the question-image accept set — a required consequence of LOCKED A7 (`svg` in the default image set) since `_CHAT_IMAGE_FILENAME_RE` refuses to *serve* `.svg`; the regex itself is untouched, as the phase specified.
- Note for the commit: changes include `README.md`, `AGENTS.md` and `.env.example` — the phase file's `git add app/ frontend/ tests/ .env.example .agents/phases/` line omits the two markdown files; stage them too.
- Next pending phase: `182_verbatim_show_text`.
This commit is contained in:
+26
-7
@@ -22,7 +22,9 @@ unguessable ``uuid4().hex`` — no enumeration value).
|
||||
The extension is the source of truth (the Content-Type header is a
|
||||
hint — the archive-uploader precedent): it must be in the phase-122
|
||||
image set (``settings.image_extension_set`` — the same frozenset the
|
||||
image-document walk uses), lowercased; total bytes are streamed with a
|
||||
image-document walk uses) minus the vector members
|
||||
(:data:`_VECTOR_IMAGE_EXTENSIONS` — a question image is RASTER-only,
|
||||
phase 181), lowercased; total bytes are streamed with a
|
||||
``chat_image_max_mb`` cap (413, fixed detail naming the cap — never
|
||||
echoing the filename).
|
||||
"""
|
||||
@@ -55,6 +57,20 @@ _STREAM_CHUNK = 1 << 20
|
||||
#: upload endpoint's own naming).
|
||||
_CHAT_IMAGE_FILENAME_RE = re.compile(r"^[0-9a-fA-F]{32}\.(png|jpe?g|webp|gif|bmp)$")
|
||||
|
||||
#: The VECTOR members of ``settings.image_extension_set`` — excluded
|
||||
#: from the question-image accept set (phase 181: the vector family
|
||||
#: belongs to INDEXING, not to question attachments). Two reasons, both
|
||||
#: hard: an OpenAI-compatible vision endpoint rejects
|
||||
#: ``image/svg+xml`` (the phase-181 LOCKED A1 reason — the vector file
|
||||
#: never reaches the describer at all), and
|
||||
#: :data:`_CHAT_IMAGE_FILENAME_RE` above deliberately refuses to SERVE a
|
||||
#: ``.svg``, so an accepted upload would be stored, unanswerable and
|
||||
#: unservable (a broken figure in the user's own bubble). Subtracting
|
||||
#: instead of touching the serve regex keeps question attachments
|
||||
#: raster-only by design while ``svg`` rides the image set for the
|
||||
#: document walk.
|
||||
_VECTOR_IMAGE_EXTENSIONS: frozenset[str] = frozenset({".svg"})
|
||||
|
||||
|
||||
@router.post("/chat-images")
|
||||
async def upload_chat_image(
|
||||
@@ -66,10 +82,12 @@ async def upload_chat_image(
|
||||
Gates, in order:
|
||||
|
||||
1. **extension** — the file name's extension (lowercased) must be in
|
||||
the phase-122 image set (``settings.image_extension_set``); the
|
||||
Content-Type header is a hint, never a source of truth (the
|
||||
archive-uploader precedent). A missing/unknown extension is a
|
||||
422 naming the accepted set — a fixed detail, no filename echo.
|
||||
the phase-122 image set (``settings.image_extension_set``)
|
||||
MINUS the vector members (:data:`_VECTOR_IMAGE_EXTENSIONS` —
|
||||
question images are RASTER-only, phase 181); the Content-Type
|
||||
header is a hint, never a source of truth (the archive-uploader
|
||||
precedent). A missing/unknown/vector extension is a 422 naming
|
||||
the accepted set — a fixed detail, no filename echo.
|
||||
2. **size** — the bytes are streamed (1 MiB chunks) with the
|
||||
``chat_image_max_mb`` cap; over-cap is a 413 naming the cap
|
||||
(fixed detail — never echoing the filename), the temp file is
|
||||
@@ -85,8 +103,9 @@ async def upload_chat_image(
|
||||
settings = get_settings()
|
||||
filename = file.filename or ""
|
||||
ext = Path(filename).suffix.lower()
|
||||
if ext not in settings.image_extension_set:
|
||||
accepted = ", ".join(sorted(settings.image_extension_set))
|
||||
accepted_extensions = settings.image_extension_set - _VECTOR_IMAGE_EXTENSIONS
|
||||
if ext not in accepted_extensions:
|
||||
accepted = ", ".join(sorted(accepted_extensions))
|
||||
raise HTTPException(
|
||||
status_code=422, detail=f"only {accepted} images are accepted"
|
||||
)
|
||||
|
||||
+59
-5
@@ -52,10 +52,15 @@ BYTES route: the persistent copy behind an ``is_image`` document's
|
||||
``image_path``, served with the extension's ``Content-Type`` (the
|
||||
``app.rag.summarizer.IMAGE_MIMES`` map — one map, one truth) and a
|
||||
``Cache-Control: private, max-age=3600`` header (the bytes are
|
||||
content-hashed — long enough, bustable by re-upload). PUBLIC (phase
|
||||
124 — the phase-123 chat-image precedent: the image is part of a
|
||||
shared chat's rendered content, and the shared chat — the app's ONLY
|
||||
anonymous surface — must render the answer figure faithfully for
|
||||
content-hashed — long enough, bustable by re-upload), plus — for the
|
||||
vector ``image/svg+xml`` entry only, phase 181 LOCKED A3 — a
|
||||
``Content-Security-Policy: default-src 'none'; sandbox`` so a DIRECT
|
||||
navigation to the URL cannot run script from stored SVG markup (the
|
||||
``SecurityHeadersMiddleware`` preserves that inner CSP; raster
|
||||
responses gain nothing). PUBLIC (phase 124 — the phase-123 chat-image
|
||||
precedent: the image is part of a shared chat's rendered content, and
|
||||
the shared chat — the app's ONLY anonymous surface — must render the
|
||||
answer figure faithfully for
|
||||
guests; the unguessable ``uuid4`` document id IS the credential, no
|
||||
enumeration value): a missing doc, a non-image doc, a doc whose
|
||||
``image_path`` is NULL, or a row whose copy was lost all map to 404
|
||||
@@ -132,6 +137,23 @@ from app.schemas import (
|
||||
|
||||
router = APIRouter(tags=["kb"])
|
||||
|
||||
#: The vector media type (phase 181, task 03) — read out of
|
||||
#: :data:`app.rag.summarizer.IMAGE_MIMES` rather than restated, so the
|
||||
#: guard below can never drift from the ``Content-Type`` the route
|
||||
#: sends for an ``.svg`` (one map, one truth).
|
||||
SVG_MEDIA_TYPE = IMAGE_MIMES[".svg"]
|
||||
|
||||
#: The CSP an SVG bytes response carries (phase 181, task 03, LOCKED
|
||||
#: A3). ``sandbox`` — no scripting, and the document gets an OPAQUE
|
||||
#: origin (no same-origin fetch/cookie/form capability);
|
||||
#: ``default-src 'none'`` — no subresource of any kind. Scoped to the
|
||||
#: vector type ON PURPOSE: an ``<img>`` never runs SVG script, but this
|
||||
#: URL is directly navigable, and the bytes come from an untrusted
|
||||
#: source tree, so a direct visit must not become a script-capable
|
||||
#: same-origin document. Raster types are untouched (their header set
|
||||
#: stays byte-identical). See :func:`get_document_image`.
|
||||
SVG_SANDBOX_CSP = "default-src 'none'; sandbox"
|
||||
|
||||
|
||||
def doc_format(path: str, extensions: frozenset[str] = frozenset()) -> str:
|
||||
"""Lowercased path suffix without its dot (``kubernetes.md`` → ``md``,
|
||||
@@ -353,6 +375,33 @@ def get_document_image(
|
||||
``application/octet-stream``) and ``Cache-Control: private,
|
||||
max-age=3600`` (the bytes are content-hashed — long enough to be
|
||||
useful, bustable by re-upload).
|
||||
|
||||
Phase 181 (task 03, LOCKED A3): an ``image/svg+xml`` response — the
|
||||
vector entry of that same map, derived from the ROW's extension and
|
||||
NEVER from a client value — carries ONE extra header,
|
||||
:data:`SVG_SANDBOX_CSP` (``default-src 'none'; sandbox``). An
|
||||
``<img>`` can never execute an SVG's script, which is why the
|
||||
figure surfaces are safe as they stand; but this URL is also
|
||||
DIRECTLY NAVIGABLE (a guest who pastes it into the address bar),
|
||||
and without the policy a stored SVG would render as a
|
||||
script-capable SAME-ORIGIN document — the ingestion path takes
|
||||
untrusted files from a source tree, so that is a stored-XSS seam
|
||||
scoped to the app's origin. ``sandbox`` blocks scripting AND makes
|
||||
the response an opaque origin (no same-origin fetches, cookies or
|
||||
form submission from the document); ``default-src 'none'`` blocks
|
||||
every subresource load, so the SVG cannot pull a tracker or a
|
||||
remote image either. Deliberately NOT applied to the raster types:
|
||||
their header set stays byte-identical (a PNG needs no policy
|
||||
change, and adding one would risk the phase-123/124 image
|
||||
surfaces), and no ``Content-Disposition: attachment`` is set — the
|
||||
graphic must render inline, and ``sandbox`` is the stronger, less
|
||||
intrusive control. The route sets the value on the response
|
||||
itself; :class:`app.core.security_headers.SecurityHeadersMiddleware`
|
||||
PRESERVES a CSP an inner layer already set, so this is the policy
|
||||
the browser sees (the app-wide ``CSP`` string would be wrong here —
|
||||
``default-src 'self'`` would grant the SVG document the app's own
|
||||
origins), while the middleware's global
|
||||
``X-Content-Type-Options: nosniff`` still applies.
|
||||
"""
|
||||
try:
|
||||
uid = uuid.UUID(doc_id)
|
||||
@@ -368,10 +417,15 @@ def get_document_image(
|
||||
# onerror fallback renders the "image unavailable" note.
|
||||
raise HTTPException(status_code=404, detail="document not found")
|
||||
media_type = IMAGE_MIMES.get(image_file.suffix.lower(), IMAGE_FALLBACK_MIME)
|
||||
headers = {"Cache-Control": "private, max-age=3600"}
|
||||
if media_type == SVG_MEDIA_TYPE:
|
||||
# Phase 181 (LOCKED A3): vector bytes only — see the route
|
||||
# docstring. Raster responses keep the header set they had.
|
||||
headers["Content-Security-Policy"] = SVG_SANDBOX_CSP
|
||||
return FileResponse(
|
||||
image_file,
|
||||
media_type=media_type,
|
||||
headers={"Cache-Control": "private, max-age=3600"},
|
||||
headers=headers,
|
||||
)
|
||||
|
||||
|
||||
|
||||
+12
-1
@@ -433,7 +433,18 @@ class Settings(BaseSettings):
|
||||
#: knob). The validator rejects an empty list and malformed tokens,
|
||||
#: exactly like ``import_extensions`` (a typo would otherwise index
|
||||
#: zero images silently).
|
||||
image_extensions: str = "png,jpg,jpeg,webp,gif,bmp"
|
||||
#:
|
||||
#: ``svg`` (phase 181, LOCKED A1) is the one VECTOR member of the
|
||||
#: family: an ``.svg`` takes the same image branch (bytes copied to
|
||||
#: ``image_dir``, ``is_image``/``image_path``, served as
|
||||
#: ``image/svg+xml``) but NEVER the vision call — its searchable
|
||||
#: text comes from the file itself
|
||||
#: (:func:`app.rag.svg_text.extract_svg_text`, the ``<title>`` /
|
||||
#: ``<desc>`` / ``<text>`` words or an honest structural line).
|
||||
#: ``images`` remains the SINGLE knob (LOCKED A7): with it off an
|
||||
#: ``.svg`` stays a plain text document (raw XML in the viewer) and
|
||||
#: the walk never admits it as an image — no special-casing.
|
||||
image_extensions: str = "png,jpg,jpeg,webp,gif,bmp,svg"
|
||||
#: Where ingested image bytes are copied for serving (phase 122,
|
||||
#: ``BOR_IMAGE_DIR``). Raw string — ``Path.expanduser()`` is applied
|
||||
#: by the importer, not here (the ``sources_dir``/``upload_dir``
|
||||
|
||||
+125
-11
@@ -111,6 +111,20 @@ summary (stored verbatim, no ``lite`` call, no pointer line), so the
|
||||
``is_summary`` position −1 chunk mirrors ``Document.summary``, which
|
||||
equals ``Document.content``.
|
||||
|
||||
The image branch is RASTER (vision-described, the paragraph above) OR
|
||||
VECTOR (phase 181, LOCKED A1: an ``.svg``). A vector file takes the
|
||||
SAME branch — bytes digest, ``image_dir`` copy, ``is_image``/``image_path``,
|
||||
the ``summary = content`` convention, the phase-125 ``image_context``
|
||||
enrichment — but NEVER the model call: its searchable text is extracted
|
||||
from the file itself (:func:`app.rag.svg_text.extract_svg_text` — the
|
||||
``<title>``/``<desc>``/``<text>`` words, or LOCKED A2's structural line
|
||||
built from the stem, the shape census and the ``viewBox`` when the
|
||||
drawing carries no words at all). A file the extractor refuses (over the
|
||||
size cap, DOCTYPE/entity-bearing, malformed XML) fails soft exactly like
|
||||
an undescribed raster: ``images_failed`` + a warning, no row, no copy.
|
||||
Successful vector indexes count ``svg_indexed`` (log-only) and log one
|
||||
``import: svg indexed`` line.
|
||||
|
||||
PDF files (phase 126, LOCKED A1/A2/A3/A4): a ``.pdf`` file (the
|
||||
``pdf`` token of ``BOR_IMPORT_EXTENSIONS``) takes the PDF index path
|
||||
(:func:`_index_pdf_file` — the branch sits in :func:`_index_file`
|
||||
@@ -359,6 +373,7 @@ from app.rag.summarizer import (
|
||||
describe_pdf_page,
|
||||
generate_summary,
|
||||
)
|
||||
from app.rag.svg_text import SvgText, extract_svg_text
|
||||
|
||||
logger = logging.getLogger("app.importer")
|
||||
|
||||
@@ -433,6 +448,14 @@ class ImportSummary:
|
||||
#: Fail-soft: the sync continues, this counter + the warning line
|
||||
#: are the signal.
|
||||
images_failed: int = 0
|
||||
#: Vector image docs (phase 181, task 02 — LOCKED A1) indexed from
|
||||
#: their OWN extracted text (:func:`app.rag.svg_text.extract_svg_text`)
|
||||
#: with NO vision call. A subset of the image branch: it does NOT
|
||||
#: move ``added``/``updated`` (those count as for any other image).
|
||||
#: Log-only (the sync ``detail`` API shape is untouched — the stats
|
||||
#: log-only convention, phase 133): the owner reads it in the sync
|
||||
#: log beside the per-file ``import: svg indexed`` line.
|
||||
svg_indexed: int = 0
|
||||
#: PDFs (phase 133, task 03) whose text layer failed the garbage
|
||||
#: gate (:func:`app.rag.pdf.is_usable_pdf_text` — empty layer or
|
||||
#: XML/binary soup) and took the page-screenshot vision fallback
|
||||
@@ -498,7 +521,7 @@ class ImportSummary:
|
||||
logger.info(
|
||||
"import: summary files=%d added=%d updated=%d unchanged=%d pruned=%d "
|
||||
"errors=%d chunks=%d embed_batches=%d summaries=%d summary_errors=%d "
|
||||
"summary_backfilled=%d dates_updated=%d images_failed=%d "
|
||||
"summary_backfilled=%d dates_updated=%d images_failed=%d svg_indexed=%d "
|
||||
"image_context_backfilled=%d pdfs_screenshotted=%d "
|
||||
"pdfs_pictorial=%d pdf_pictorial_pages=%d pdf_images_deduped=%d "
|
||||
"pdf_images_blank_skipped=%d formats=%s",
|
||||
@@ -515,6 +538,7 @@ class ImportSummary:
|
||||
self.summary_backfilled,
|
||||
self.dates_updated,
|
||||
self.images_failed,
|
||||
self.svg_indexed,
|
||||
self.image_context_backfilled,
|
||||
self.pdfs_screenshotted,
|
||||
self.pdfs_pictorial,
|
||||
@@ -1325,6 +1349,38 @@ def _regate_pdf_strip(
|
||||
)
|
||||
|
||||
|
||||
def _row_kind_mismatch(path: Path, doc: Document, settings: Settings) -> bool:
|
||||
"""Phase 181 (task 02, design D4): does this row's KIND disagree with
|
||||
how *path* would be indexed NOW — the ONE case that must fall
|
||||
THROUGH the phase-147 unchanged fast path?
|
||||
|
||||
``True`` only while the ``images`` toggle is ON and the file matches
|
||||
the image extension set while the row is still a TEXT row
|
||||
(``is_image`` False). That is exactly the migration this phase needs:
|
||||
an ``.svg`` imported before ``svg`` joined
|
||||
:py:attr:`Settings.image_extension_set` is a text document whose
|
||||
``content`` is the raw XML — and because the text digest of an ASCII
|
||||
file equals the digest of its bytes, its ``file_hash`` (and even its
|
||||
``content_hash``) matches perfectly, so a hash-only fast path would
|
||||
freeze it as XML forever: no bytes route, no preview, no
|
||||
``image_url`` citation figure — the exact complaint this phase
|
||||
answers. Falling through runs the image branch, which rewrites the
|
||||
row (``is_image``/``image_path``/extracted ``content`` + the
|
||||
``image_dir`` copy) through the normal index write: no alembic
|
||||
migration, no backfill script, and the next sync is fast again.
|
||||
|
||||
The REVERSE direction is deliberately NOT forced: with the toggle
|
||||
OFF the walk never admits an image file (so an ``is_image`` row never
|
||||
reaches :func:`_index_file` at all) and the phase-122 prune guard
|
||||
keeps the row; requiring ``settings.images`` here keeps that
|
||||
contract byte-identical (``svg`` in ``BOR_IMPORT_EXTENSIONS`` with
|
||||
the toggle off still means "text document", LOCKED A7).
|
||||
"""
|
||||
if not settings.images or doc.is_image:
|
||||
return False
|
||||
return match_extension(path, settings.image_extension_set) is not None
|
||||
|
||||
|
||||
async def _index_file(
|
||||
session: Session,
|
||||
*,
|
||||
@@ -1400,10 +1456,17 @@ async def _index_file(
|
||||
# fast path (and the branches' unchanged path) refresh the
|
||||
# stored date from the same source.
|
||||
raw_date = file_mtime_datetime(full_path)
|
||||
if doc is not None and doc.file_hash == file_digest:
|
||||
if (
|
||||
doc is not None
|
||||
and doc.file_hash == file_digest
|
||||
and not _row_kind_mismatch(full_path, doc, settings)
|
||||
):
|
||||
# The fast path: the raw bytes are what was indexed last time
|
||||
# — the bookkeeping (backfill / D1 / D4) runs in the shared
|
||||
# helper, nothing else does.
|
||||
# helper, nothing else does. Phase 181 (task 02, D4): "the same
|
||||
# bytes" is only "the same document" when the row's KIND also
|
||||
# matches how the file would be indexed NOW — see
|
||||
# :func:`_row_kind_mismatch` (the SVG text→image migration).
|
||||
await _unchanged_doc(
|
||||
session, doc, source=source, rel=rel, llm=llm, summary=summary,
|
||||
raw_date=raw_date, file_digest=file_digest, full_path=full_path,
|
||||
@@ -3542,7 +3605,17 @@ async def _index_image_file(
|
||||
on_chunk: Callable[[int, int], None] | None = None,
|
||||
) -> None:
|
||||
"""The phase-122 image branch of :func:`_index_file` — a standalone
|
||||
image is indexed from its BYTES, never its text:
|
||||
image is indexed from its BYTES, never its text.
|
||||
|
||||
Phase 181 (task 02, LOCKED A1) makes the branch RASTER **or**
|
||||
VECTOR: an ``.svg`` takes this same path (digest, copy, ``is_image``,
|
||||
``image_path``, chunking, ``summary = content``, the phase-125
|
||||
``image_context`` enrichment) but its ``content`` comes from
|
||||
:func:`app.rag.svg_text.extract_svg_text` instead of the vision
|
||||
seam, and a success counts the log-only ``svg_indexed``. The bullets
|
||||
and the fail-soft below are shared by both kinds — the only
|
||||
difference is where the text comes from.
|
||||
|
||||
|
||||
* the sha256 digest is over the raw bytes (the digest rule is
|
||||
content identity — the same bytes are the same document);
|
||||
@@ -3600,13 +3673,20 @@ async def _index_image_file(
|
||||
digest = hashlib.sha256(data).hexdigest()
|
||||
doc = session.scalar(select(Document).where(Document.source == source, Document.path == rel))
|
||||
|
||||
if doc is not None and doc.content_hash == digest:
|
||||
if doc is not None and doc.is_image and doc.content_hash == digest:
|
||||
# Phase 147: the shared unchanged path — a pre-phase NULL-hash
|
||||
# row, or a bytes-changed/content-same image (the byte digest
|
||||
# still matches). The helper stamps the new file hash (the
|
||||
# next sync takes the fast path); for an image the summary is
|
||||
# the stored description (``doc.content``), so the image-aware
|
||||
# ``_store_summary`` backfill re-stores it verbatim.
|
||||
# Phase 181 (task 02, D4): ``doc.is_image`` joins the guard — a
|
||||
# NON-image row routed here by :func:`_row_kind_mismatch` is the
|
||||
# migration case, not an unchanged one. An ASCII SVG indexed as
|
||||
# TEXT digests to the SAME value here (the text path hashes
|
||||
# ``content.encode()``, which for a byte-clean file equals the
|
||||
# raw-bytes digest), so without it the text→image conversion
|
||||
# would report ``unchanged`` forever and never produce the copy.
|
||||
await _unchanged_doc(
|
||||
session, doc, source=source, rel=rel, llm=llm, summary=summary,
|
||||
raw_date=raw_date, file_digest=file_digest, full_path=full_path,
|
||||
@@ -3629,16 +3709,50 @@ async def _index_image_file(
|
||||
# it is generated BEFORE anything is written, so a failure skips the
|
||||
# doc with no row and no copy (the copy is only made after a
|
||||
# successful description — a failure never leaves an orphan).
|
||||
content = await _describe_or_skip(
|
||||
llm, data=data, source=source, rel=rel, full_path=full_path,
|
||||
context=image_context,
|
||||
)
|
||||
#
|
||||
# Phase 181 (task 02, LOCKED A1): the VECTOR branch. An ``.svg`` is
|
||||
# described by NOBODY — OpenAI-compatible vision endpoints reject
|
||||
# ``image/svg+xml`` and no rasteriser dependency is in scope — so
|
||||
# its text comes out of the file itself (its ``<title>``/``<desc>``/
|
||||
# ``<text>`` words, or LOCKED A2's structural line). Everything else
|
||||
# in this function is shared: the byte digest, the ``image_dir``
|
||||
# copy, ``is_image``/``image_path``, the title, the counters, the
|
||||
# phase-125 ``image_context`` prepend below, the chunk pipeline and
|
||||
# ``summary = content``. The seam below is not reached, so no vision
|
||||
# call happens for a vector file (pinned by the unit tests).
|
||||
is_vector = full_path.suffix.lower() == ".svg"
|
||||
svg: SvgText | None = None
|
||||
if is_vector:
|
||||
svg = extract_svg_text(data, stem=full_path.stem)
|
||||
content = None if svg is None else svg.text
|
||||
else:
|
||||
content = await _describe_or_skip(
|
||||
llm, data=data, source=source, rel=rel, full_path=full_path,
|
||||
context=image_context,
|
||||
)
|
||||
if content is None:
|
||||
# LOCKED A3 fail-soft: an undescribed image is unsearchable
|
||||
# noise — skip the doc entirely (no row, no copy).
|
||||
# noise — skip the doc entirely (no row, no copy). Phase 181
|
||||
# (LOCKED A2): the vector case skips ONLY when the extractor
|
||||
# refused the file (over the size cap, DOCTYPE/entity-bearing,
|
||||
# malformed XML) — a textless SVG is NOT a failure, it gets the
|
||||
# structural line.
|
||||
summary.images_failed += 1
|
||||
logger.warning("import: image description failed source=%s path=%s", source, rel)
|
||||
logger.warning(
|
||||
"import: %s source=%s path=%s",
|
||||
"svg text extraction failed" if is_vector else "image description failed",
|
||||
source, rel,
|
||||
)
|
||||
return
|
||||
if svg is not None:
|
||||
# Phase 181 (task 02): the log-only vector counter + the
|
||||
# auditability line (AGENTS rule 10) — ``structural=True`` marks
|
||||
# the docs whose text is the LOCKED A2 fallback line.
|
||||
summary.svg_indexed += 1
|
||||
logger.info(
|
||||
"import: svg indexed source=%s path=%s structural=%s chars=%d",
|
||||
source, rel, svg.structural, len(svg.text),
|
||||
)
|
||||
if image_context is not None:
|
||||
# Phase 125 (task 03): the enriched content — the reference
|
||||
# context FIRST (for retrieval), then the description. The
|
||||
|
||||
@@ -149,6 +149,17 @@ PICTORIAL_PROMPT = f"{PICTORIAL_PAGE_MODE}: {PICTORIAL_INSTRUCTION}"
|
||||
#: takes the :data:`IMAGE_FALLBACK_MIME` data-URL mime in the describe
|
||||
#: call — the vision endpoint may reject it, and the fail-soft skip
|
||||
#: (``images_failed``) is the honest outcome.
|
||||
#:
|
||||
#: Phase 181 adds ``.svg`` — the ONE **vector** entry in the map. It
|
||||
#: exists for the BYTES route only: the vector import branch
|
||||
#: (``app/rag/importer.py::_index_image_file``) takes
|
||||
#: :func:`app.rag.svg_text.extract_svg_text` INSTEAD of the vision
|
||||
#: describe call (LOCKED A1), so an ``.svg`` never reaches
|
||||
#: :func:`image_data_url` and this value is never put in a multimodal
|
||||
#: message. ``app/api/docs.py::get_document_image`` reads the same map
|
||||
#: for the ``Content-Type`` (one map, one truth), and that route is
|
||||
#: where the vector value earns its place — plus the sandboxed CSP the
|
||||
#: vector bytes need and the rasters do not (LOCKED A3, phase 181).
|
||||
IMAGE_MIMES: dict[str, str] = {
|
||||
".png": "image/png",
|
||||
".jpg": "image/jpeg",
|
||||
@@ -156,6 +167,7 @@ IMAGE_MIMES: dict[str, str] = {
|
||||
".webp": "image/webp",
|
||||
".gif": "image/gif",
|
||||
".bmp": "image/bmp",
|
||||
".svg": "image/svg+xml",
|
||||
}
|
||||
|
||||
#: The data-URL mime for an image extension :data:`IMAGE_MIMES` does not
|
||||
|
||||
@@ -0,0 +1,293 @@
|
||||
"""SVG text extraction — the words a vector drawing carries (phase 181).
|
||||
|
||||
Pure functions, no I/O — fully unit-testable. Stdlib only (the
|
||||
:mod:`app.rag.md_images` precedent, phase 125).
|
||||
|
||||
An ``.svg`` joins the image family (LOCKED A1, phase 181): its BYTES are
|
||||
served to the browser so every citation surface paints the graphic
|
||||
instead of XML. But OpenAI-compatible vision endpoints reject
|
||||
``image/svg+xml``, and adding a rasteriser dependency is out of scope —
|
||||
so the vector branch never calls the describer. The text that makes the
|
||||
document **retrievable** comes from the file itself:
|
||||
|
||||
* ``<title>``, ``<desc>`` and ``<text>`` (LOCKED D2) — collected in
|
||||
DOCUMENT order by LOCAL name (namespace-agnostic: ``<title>`` and
|
||||
``{http://www.w3.org/2000/svg}title`` both count, declared ``svg:``
|
||||
prefixes included), whitespace-normalised, de-duplicated
|
||||
case-insensitively keeping the FIRST occurrence, joined with ``\\n``.
|
||||
``<tspan>`` children are joined WITH a space so ``a<tspan>b</tspan>``
|
||||
reads "a b", not "ab". XML entity references inside the text
|
||||
(``&``) are decoded by the parser — the correct direction, the
|
||||
renderer re-escapes on display.
|
||||
* a textless diagram (pure ``<path>``s — extremely common) still gets
|
||||
honest, searchable text (LOCKED A2): one structural line built ONLY
|
||||
from facts the file states, never invented prose.
|
||||
|
||||
Guards, applied BEFORE any parse (untrusted markup, the sibling of
|
||||
LOCKED A3):
|
||||
|
||||
* empty / whitespace-only input → ``None``;
|
||||
* more than :data:`MAX_SVG_BYTES` → ``None`` (memory guard);
|
||||
* a ``<!DOCTYPE`` or ``<!ENTITY`` declaration anywhere in the bytes →
|
||||
``None``. Refusing the doctype outright is what makes stdlib
|
||||
:mod:`xml.etree.ElementTree` safe here: no external entities, no
|
||||
entity expansion (the billion-laughs class);
|
||||
* malformed XML (:class:`xml.etree.ElementTree.ParseError`) → ``None``
|
||||
— never re-raised, the caller's fail-soft is a skip, not a crash.
|
||||
|
||||
What is deliberately NOT followed: ``<image>`` elements embedding a
|
||||
raster inside the SVG (their ``href`` is not text), and
|
||||
``aria-label``/``role`` attributes — ``<title>``/``<desc>`` are the
|
||||
SVG-spec carriers of words and the fallback line covers the rest.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from xml.etree import ElementTree
|
||||
|
||||
#: Hard input cap for one SVG (phase 181, LOCKED A3's sibling) — bytes
|
||||
#: over this are refused without a parse attempt, so a hostile or
|
||||
#: accidentally gigantic file can never be the memory event of a sync.
|
||||
MAX_SVG_BYTES: int = 2 * 1024 * 1024
|
||||
|
||||
#: Cap on the EXTRACTED text (phase 181). It binds on the result, never
|
||||
#: silently mid-word: an over-cap extraction is cut at the last newline
|
||||
#: or period before the cap (a hard cut only when the window holds no
|
||||
#: boundary at all).
|
||||
MAX_SVG_TEXT_CHARS: int = 4000
|
||||
|
||||
#: A document type declaration in the raw bytes (lower-cased marker).
|
||||
#: Its presence — anywhere — refuses the input (billion-laughs /
|
||||
#: external-entity class).
|
||||
_DOCTYPE_MARKER = b"<!doctype"
|
||||
|
||||
#: An entity declaration in the raw bytes (lower-cased marker) — same
|
||||
#: refusal as :data:`_DOCTYPE_MARKER`, checked separately so a stray
|
||||
#: ``<!ENTITY`` inside a stray doctype-less prolog is still caught.
|
||||
_ENTITY_MARKER = b"<!entity"
|
||||
|
||||
#: Elements whose whole text block is one item, nested markup
|
||||
#: concatenated (:meth:`ElementTree.Element.itertext`).
|
||||
_BLOCK_TEXT_NAMES = frozenset({"title", "desc"})
|
||||
|
||||
#: Elements whose text is joined with a space so inline children
|
||||
#: (``<tspan>``) do not weld their neighbours together.
|
||||
_SPACED_TEXT_NAMES = frozenset({"text"})
|
||||
|
||||
#: Local names that are structure, not drawing: never counted as shapes
|
||||
#: in the fallback line (:func:`_shape_clause`).
|
||||
_NON_SHAPE_NAMES = frozenset({"svg", "title", "desc"})
|
||||
|
||||
#: How many element kinds the fallback line names (LOCKED D3: "the
|
||||
#: three most common element local names with counts").
|
||||
_TOP_SHAPE_NAMES = 3
|
||||
|
||||
#: ``viewBox`` components: four numbers separated by whitespace and/or
|
||||
#: commas (the SVG-spec grammar, which allows either).
|
||||
_VIEWBOX_SPLIT_RE = re.compile(r"[\s,]+")
|
||||
|
||||
#: The multiplication sign used between a shape name and its count, and
|
||||
#: between the viewBox width and height — the app's own typography.
|
||||
_TIMES = "×"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SvgText:
|
||||
"""The searchable text of one SVG document (phase 181, LOCKED A1/A2).
|
||||
|
||||
* ``text`` — what gets stored as the document's ``content`` (and
|
||||
``summary``, per the phase-122 image convention);
|
||||
* ``structural`` — ``True`` when the document carried no words and
|
||||
``text`` is the structural fallback line, so the importer can log
|
||||
and count it separately from a genuinely worded diagram.
|
||||
"""
|
||||
|
||||
text: str
|
||||
structural: bool
|
||||
|
||||
|
||||
def local_name(tag: object) -> str:
|
||||
"""The namespace-agnostic name of an element tag.
|
||||
|
||||
``{http://www.w3.org/2000/svg}title`` and ``title`` both give
|
||||
``"title"`` — ElementTree expands every declared prefix (including
|
||||
``<svg:title>``) into Clark notation, so stripping the ``{uri}`` is
|
||||
all "namespace-agnostic" needs. A non-string tag (comment /
|
||||
processing-instruction marker) has no name → ``""``.
|
||||
"""
|
||||
if not isinstance(tag, str):
|
||||
return ""
|
||||
return tag.rsplit("}", 1)[-1]
|
||||
|
||||
|
||||
def extract_svg_text(data: bytes, *, stem: str = "") -> SvgText | None:
|
||||
"""Turn raw SVG bytes into the document's searchable text.
|
||||
|
||||
Returns ``None`` when the input is unusable or unsafe: empty /
|
||||
whitespace-only, larger than :data:`MAX_SVG_BYTES`, carrying a
|
||||
``<!DOCTYPE``/``<!ENTITY`` declaration, or not well-formed XML. The
|
||||
caller's fail-soft (skip the document, count it) is the honest
|
||||
outcome — this function never raises for bad input.
|
||||
|
||||
*stem* is the file name without extension: it names the document in
|
||||
the structural fallback line and is never invented.
|
||||
|
||||
Otherwise a :class:`SvgText` whose ``text`` is the document-order
|
||||
``<title>``/``<desc>``/``<text>`` words (``structural=False``), or —
|
||||
when the file carries no words at all — the structural line built
|
||||
from the stem, the shape census and the ``viewBox``
|
||||
(``structural=True``, LOCKED A2).
|
||||
"""
|
||||
if not data or not data.strip():
|
||||
return None
|
||||
if len(data) > MAX_SVG_BYTES:
|
||||
return None
|
||||
lowered = data.lower()
|
||||
if _DOCTYPE_MARKER in lowered or _ENTITY_MARKER in lowered:
|
||||
return None
|
||||
try:
|
||||
root = ElementTree.fromstring(data)
|
||||
except ElementTree.ParseError:
|
||||
return None
|
||||
items = _dedupe(_walk_text(root))
|
||||
if items:
|
||||
return SvgText(text=_cap_text("\n".join(items)), structural=False)
|
||||
return SvgText(text=_structural_line(root, stem), structural=True)
|
||||
|
||||
|
||||
def _walk_text(root: ElementTree.Element) -> list[str]:
|
||||
"""Every worded ``title``/``desc``/``text`` block, in document order.
|
||||
|
||||
Items are whitespace-normalised; a block that normalises to nothing
|
||||
(an empty ``<title></title>``) is dropped so the fallback can still
|
||||
describe the file.
|
||||
"""
|
||||
items: list[str] = []
|
||||
for el in root.iter():
|
||||
name = local_name(el.tag)
|
||||
if name in _BLOCK_TEXT_NAMES:
|
||||
raw = "".join(el.itertext())
|
||||
elif name in _SPACED_TEXT_NAMES:
|
||||
raw = " ".join(el.itertext())
|
||||
else:
|
||||
continue
|
||||
normalised = " ".join(raw.split())
|
||||
if normalised:
|
||||
items.append(normalised)
|
||||
return items
|
||||
|
||||
|
||||
def _dedupe(items: list[str]) -> list[str]:
|
||||
"""Drop repeats case-insensitively, keeping the FIRST occurrence.
|
||||
|
||||
A ``<title>`` mirrored in a ``<desc>`` (exporters do this) is one
|
||||
sentence in the index, not two; the first position in document
|
||||
order is the one kept.
|
||||
"""
|
||||
seen: set[str] = set()
|
||||
unique: list[str] = []
|
||||
for item in items:
|
||||
key = item.casefold()
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
unique.append(item)
|
||||
return unique
|
||||
|
||||
|
||||
def _cap_text(text: str) -> str:
|
||||
"""Cap *text* at :data:`MAX_SVG_TEXT_CHARS` without cutting a word.
|
||||
|
||||
An over-cap text is cut at the LAST newline or period before the cap
|
||||
(whichever comes later) so the stored text ends on a boundary; only
|
||||
a window with no boundary at all (one colossal run of characters)
|
||||
is hard-cut.
|
||||
"""
|
||||
if len(text) <= MAX_SVG_TEXT_CHARS:
|
||||
return text
|
||||
window = text[:MAX_SVG_TEXT_CHARS]
|
||||
cut = max(window.rfind("\n"), window.rfind("."))
|
||||
if cut < 0:
|
||||
return window
|
||||
return text[: cut + 1].rstrip()
|
||||
|
||||
|
||||
def _structural_line(root: ElementTree.Element, stem: str) -> str:
|
||||
"""The LOCKED A2 fallback: one line of facts, nothing invented.
|
||||
|
||||
``Vector diagram "<stem>" — <N> shapes (<top-3 kinds>), viewBox
|
||||
<w>×<h>`` — each fact clause is omitted when the file does not
|
||||
state it, and the stem clause disappears entirely when empty (no
|
||||
empty quotes). With no facts and no stem the line is just
|
||||
``Vector diagram``.
|
||||
"""
|
||||
head = f'Vector diagram "{stem}"' if stem else "Vector diagram"
|
||||
facts = [c for c in (_shape_clause(root), _viewbox_clause(root)) if c]
|
||||
if not facts:
|
||||
return head
|
||||
return f"{head} — {', '.join(facts)}"
|
||||
|
||||
|
||||
def _shape_clause(root: ElementTree.Element) -> str:
|
||||
"""``<N> shapes (<kinds>)`` — the drawing's element census.
|
||||
|
||||
*N* counts every descendant element except ``svg``, ``title`` and
|
||||
``desc`` (structure, not drawing). The kinds are the
|
||||
:data:`_TOP_SHAPE_NAMES` most common local names, ordered by count
|
||||
(document order breaks ties) and annotated with their count — the
|
||||
count is left off a kind that appears exactly once, which is what
|
||||
the total already says. ``""`` when there are no shapes at all, so
|
||||
the clause is omitted rather than claiming "0 shapes".
|
||||
"""
|
||||
counts: dict[str, int] = {}
|
||||
first_seen: dict[str, int] = {}
|
||||
total = 0
|
||||
for index, el in enumerate(root.iter()):
|
||||
if el is root:
|
||||
continue
|
||||
name = local_name(el.tag)
|
||||
if not name or name in _NON_SHAPE_NAMES:
|
||||
continue
|
||||
counts[name] = counts.get(name, 0) + 1
|
||||
first_seen.setdefault(name, index)
|
||||
total += 1
|
||||
if not total:
|
||||
return ""
|
||||
top = sorted(counts, key=lambda name: (-counts[name], first_seen[name]))
|
||||
listing = ", ".join(
|
||||
f"{name}{_TIMES}{counts[name]}" if counts[name] > 1 else name
|
||||
for name in top[:_TOP_SHAPE_NAMES]
|
||||
)
|
||||
noun = "shape" if total == 1 else "shapes"
|
||||
return f"{total} {noun} ({listing})"
|
||||
|
||||
|
||||
def _viewbox_clause(root: ElementTree.Element) -> str:
|
||||
"""``viewBox <w>×<h>`` from the root's ``viewBox``, or ``""``.
|
||||
|
||||
The spec grammar is four whitespace-and/or-comma separated numbers;
|
||||
the last two are the width and height. Unparseable (wrong count,
|
||||
non-numeric, non-finite) → ``""`` — the clause is omitted rather
|
||||
than reporting a size the file does not state. Integral values
|
||||
print without a trailing ``.0``.
|
||||
"""
|
||||
raw = (root.get("viewBox") or "").strip()
|
||||
if not raw:
|
||||
return ""
|
||||
parts = [part for part in _VIEWBOX_SPLIT_RE.split(raw) if part]
|
||||
if len(parts) != 4:
|
||||
return ""
|
||||
try:
|
||||
_min_x, _min_y, width, height = (float(part) for part in parts)
|
||||
except ValueError:
|
||||
return ""
|
||||
if not (math.isfinite(width) and math.isfinite(height)):
|
||||
return ""
|
||||
return f"viewBox {_format_number(width)}{_TIMES}{_format_number(height)}"
|
||||
|
||||
|
||||
def _format_number(value: float) -> str:
|
||||
"""``8`` for 8.0, ``8.5`` for 8.5 — no trailing ``.0`` in a size."""
|
||||
return str(int(value)) if value.is_integer() else str(value)
|
||||
Reference in New Issue
Block a user