"""Document summarizer (phase 30, task 03) + image descriptions (phase 122, task 03). Builds the ``SUMMARY_MODE`` prompt for one document, calls the aipi ``lite`` model through the one-shot ``LLMClient.chat`` (phase 30, task 01), and returns the validated summary text with a **code-deterministic** pointer line back to the source:: Source: / The pointer is appended by this module, never model-generated — the model is told what to summarize, not to cite. Quality contracts enforced here: * **Capped input** — the document content is cut at ``BOR_SUMMARY_MAX_CHARS`` (default 12 000) before the single model call; overflow is cut exactly at the cap and the shared ``TRUNCATION_MARKER`` (``[…truncated…]``) is appended, so the model never sees more than the cap and the cut is visible. * **No silent summaries** — a reply that is empty after trimming raises :class:`LLMError` (the client already rejects empty content; the summarizer re-asserts defensively and never hands the importer a pointer-only row). Image descriptions (phase 122, LOCKED A3): :func:`describe_image` is this module's second one-shot generation path — a SINGLE CHAT-model (vision) call describing one image's bytes as a base64 data URL. The description becomes the image document's ``content`` AND ``summary`` (it is the ONLY embedded text of the doc — the embedding model never sees pixels, and the ``lite`` summary model is deliberately NOT used: it is not assumed vision-capable). Fail-soft by contract: any client error, empty reply, or non-2xx yields ``None`` — the importer skips the doc, counts ``images_failed``, and the sync continues. Pictorial PDF pages (phase 140, LOCKED A2): :func:`describe_pdf_page` is this module's third one-shot generation path — a SINGLE CHAT-model (vision) call describing ONE rendered sparse PDF page, with the page's OWN extracted text as context so the model describes what the text does NOT carry (the IKEA assembly-instruction class: vector pictograms around a 4–73-char text layer). The description is APPENDED to the page's indexed content block by the importer (the ``[Page N illustration: …]`` line, LOCKED A4) — the page's text stays; more information is better for the embeddings (the owner's L5 rationale). Fail-soft by contract: any client error or empty reply yields ``None`` — the importer skips that page with a warning and continues (one dead page never stalls a sync). The ``SUMMARY_MODE`` / ``IMAGE_DESCRIPTION_MODE`` / ``PICTORIAL_PAGE_MODE`` markers follow the ``DEFLECT_MODE`` convention: the deterministic E2E mock LLM keys on them (``tests/e2e/mock_llm.py`` — the image branch is wired by task 06's story suite, the pictorial branch by phase 140 task 02). """ from __future__ import annotations import base64 import logging from typing import Any, Protocol from app.config import Settings, get_settings from app.rag.llm import LLMError from app.rag.retriever import TRUNCATION_MARKER logger = logging.getLogger("app.summarizer") #: System-prompt marker for summary generation — the E2E mock LLM keys on #: it (same convention as ``DEFLECT_MODE``, PLAN §6). SUMMARY_MODE = "SUMMARY_MODE" #: Locked instruction for the ``lite`` model (phase 30): the summary is a #: natural-language retrieval target, so it must be plain, concrete, and #: strictly grounded in the document. SUMMARY_INSTRUCTION = ( "Write a 3-6 sentence plain-text summary of this document in natural " "language. Cover what it configures/defines and its most important " "values. Do not use markdown. Do not invent anything that is not in " "the document." ) #: Full system prompt: marker first (the mock's key), then the instruction. SYSTEM_PROMPT = f"{SUMMARY_MODE}: {SUMMARY_INSTRUCTION}" #: Image-description marker (phase 122, task 03) — the deterministic #: E2E mock LLM keys on it (same convention as ``SUMMARY_MODE`` / #: ``DEFLECT_MODE``, PLAN §6; the story suite wires the mock's branch #: in task 06). It heads the text part of the multimodal describe #: message, so it rides the user message, not a system prompt. IMAGE_DESCRIPTION_MODE = "IMAGE_DESCRIPTION_MODE" #: Locked instruction for the CHAT (vision) model (phase 122, LOCKED #: A3): the description is the ONLY retrievable text of the image #: document (the embedding model never sees pixels), so it must be a #: faithful, retrieval-oriented account that carries the image's full #: meaning — what is depicted, any visible text/labels/titles, #: diagram/table structure, salient details. DESCRIBE_INSTRUCTION = ( "Describe this image faithfully, in plain text, for a search index. " "State what is depicted, transcribe any visible text, labels, or " "titles, describe the structure of any diagram, table, or layout, and " "call out the most salient details. Write 2-4 sentences of substance. " "Do not use markdown. Do not invent anything that is not visible in " "the image. Your description is the ONLY text that will ever be " "retrieved for this image — it must carry the image's full meaning." ) #: The full describe prompt: marker first (the mock's key), then the #: instruction — the single text part of the multimodal user message. #: BYTE-IDENTICAL CONTRACT: the phase 122/126/137 suites pin this #: string untouched — pictorial-page descriptions go through their #: own marker (:data:`PICTORIAL_PAGE_MODE`), never a re-wrap of this #: one. DESCRIBE_PROMPT = f"{IMAGE_DESCRIPTION_MODE}: {DESCRIBE_INSTRUCTION}" #: Pictorial-page marker (phase 140, task 02, LOCKED A1/A2) — the #: deterministic E2E mock LLM keys on it (same convention as #: ``IMAGE_DESCRIPTION_MODE``, PLAN §6). It heads the text part of #: the multimodal describe message, so it rides the user message, not #: a system prompt. PICTORIAL_PAGE_MODE = "PICTORIAL_PAGE_MODE" #: Locked instruction for the CHAT (vision) model (phase 140, LOCKED #: A2): the description APPENDS to the page's indexed content block #: (``[Page N illustration: …]``) — the embeddings then carry what #: the thin text layer couldn't (the owner's L5 rationale) — so it #: must be a faithful, retrieval-oriented account of what the page #: SHOWS: the assembly steps, part names, arrows, tool icons, and #: before/after states — ESPECIALLY the information the extracted #: text does not contain. PICTORIAL_INSTRUCTION = ( "Describe in detail what this page of the PDF shows, in plain text, " "for a search index. State the assembly steps, part names, arrows, " "tool icons, and before/after states it illustrates — ESPECIALLY the " "information the extracted text does not contain. Write 2-4 sentences " "of substance. Do not use markdown. Do not invent anything that is " "not visible on the page." ) #: The full pictorial-page prompt: marker first (the mock's key), then #: the instruction — the head of the multimodal user message's text #: part (the page block is appended by :func:`describe_pdf_page`). PICTORIAL_PROMPT = f"{PICTORIAL_PAGE_MODE}: {PICTORIAL_INSTRUCTION}" #: Extension → MIME type for the image family (phase 122). Dotted, #: lowercase keys — the ``image_extension_set`` shape. Task 03 uses it #: for the describe call's data-URL mime; task 04's serve route reuses #: it for the image bytes' ``Content-Type`` (one map, one truth). A #: ``BOR_IMAGE_EXTENSIONS`` token outside this map (a custom format) #: takes the :data:`IMAGE_FALLBACK_MIME` data-URL mime in the describe #: call — the vision endpoint may reject it, and the fail-soft skip #: (``images_failed``) is the honest outcome. IMAGE_MIMES: dict[str, str] = { ".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".webp": "image/webp", ".gif": "image/gif", ".bmp": "image/bmp", } #: The data-URL mime for an image extension :data:`IMAGE_MIMES` does not #: name (phase 122) — the best-effort generic, never a guess at a #: specific type. IMAGE_FALLBACK_MIME = "application/octet-stream" def image_data_url(data: bytes, mime: str) -> str: """One image's bytes as a data URL for a multimodal message (phase 122; factored for phase 123, task 01). ``data:;base64,`` — the OpenAI-compatible ``image_url`` payload's ``url``. ONE helper, both call sites: :func:`describe_image` (document-image descriptions) and the chat question-image path (``app.api.chat``'s multimodal user message, phase 123). """ return f"data:{mime};base64,{base64.b64encode(data).decode('ascii')}" class SummaryLLM(Protocol): """The one-shot chat surface the summarizer needs. :class:`app.rag.llm.LLMClient` satisfies it; unit tests pass a duck-typed fake (``chat`` + ``settings``) instead — same pattern as the importer's ``Embedder`` protocol. ``content`` may be a string (text calls — ``generate_summary``) or a list of OpenAI-compatible parts (phase 122 multimodal image descriptions — ``{type: "text", …}`` + ``{type: "image_url", …}``); the client passes message dicts through untouched. """ settings: Settings async def chat( self, messages: list[dict[str, Any]], model: str | None = None ) -> str: ... def _capped_content(content: str, max_chars: int | None) -> str: """Document content for the user message, capped at *max_chars*. The default cap is ``BOR_SUMMARY_MAX_CHARS``. Overflow is cut exactly at the cap and the shared ``TRUNCATION_MARKER`` is appended on its own line; content that fits (length ≤ cap) passes through unchanged. """ limit = max_chars if max_chars is not None else get_settings().summary_max_chars if len(content) <= limit: return content return content[:limit] + "\n" + TRUNCATION_MARKER def build_summary_prompt( source: str, path: str, content: str, max_chars: int | None = None ) -> tuple[str, str]: """The ``(system, user)`` message pair for one summary call. * ``system`` — :data:`SYSTEM_PROMPT`: the ``SUMMARY_MODE`` marker + the locked instruction. * ``user`` — the document content, capped (see :func:`_capped_content`). *source* and *path* are part of the signature so the call site reads like the document it summarizes (and for :func:`generate_summary`'s pointer) — the pointer is built in code and deliberately **not** part of the prompt, so the model cannot echo or mangle it. """ return SYSTEM_PROMPT, _capped_content(content, max_chars) async def generate_summary( llm: SummaryLLM, *, source: str, path: str, content: str ) -> str: """One-shot ``lite`` summary of *content*, ending in the pointer line. Returns the model's text (trimmed) plus the deterministic ``Source: /`` line — the pointer is appended by code, never model-generated. Raises :class:`LLMError` when the model returns nothing usable after trimming, and propagates any :class:`LLMError` the client raises (the importer's fail-soft path turns that into a logged, counted ``summary_errors`` entry). """ system, user = build_summary_prompt(source, path, content) raw = await llm.chat( [{"role": "system", "content": system}, {"role": "user", "content": user}], model=llm.settings.llm_summary_model, ) summary = raw.strip() if not summary: raise LLMError( f"summary model returned empty content for {source}/{path} — " "refusing to store a silent summary" ) return f"{summary}\nSource: {source}/{path}" async def describe_image( llm: SummaryLLM, *, data: bytes, mime: str, settings: Settings | None = None, context: str | None = None, ) -> str | None: """One-shot CHAT-model (vision) description of one image (phase 122, LOCKED A3) — the text that becomes the image document's ``content`` AND ``summary`` (the ONLY embedded text of the doc; the embedding model never sees pixels). ONE chat-model call against ``settings.llm_chat_model`` (the vision model — the ``lite`` summary model is NOT assumed vision-capable) with the multimodal user message the OpenAI-compatible API expects: ``[{type: "text", text: DESCRIBE_PROMPT + context prefix}, {type: "image_url", image_url: {url: }}]`` — no system prompt, no tools, no app-level retries beyond the client's own (SDK-level + the house one-shot empty- content policy) — a description failure must not stall a sync. *context* (phase 125, task 03, LOCKED A3; generalized in phase 137, task 02): the optional DOCUMENT/REFERENCE context built by the importer — for a PDF-embedded image, the ``Extracted from / (page N):`` provenance block (the page's text, capped — WHERE the image came from) combined with any phase-125 markdown reference blocks (the ``![alt](path)`` alt text + surrounding paragraphs — WHAT it illustrates; document block first); for a standalone image, the markdown reference context alone (the importer's same-source scan). When provided, it is appended to the text part AFTER the locked :data:`DESCRIBE_PROMPT` (the marker + instruction stay byte-identical — the ``IMAGE_DESCRIPTION_MODE`` mock key is untouched): the vision model learns where the image comes from AND what it is used for, so it can write a context-aware description. ``context=None`` (the default — a reference-free standalone image) sends the bare prompt: the message is byte-identical to the pre-phase shape. Returns the stripped reply cut exactly at ``(settings or llm.settings).summary_max_chars`` (the phase-30 cap — the description IS the summary, so it keeps the same uniform ceiling). Returns ``None`` on any client error, empty reply, or non-2xx (the client raises :class:`LLMError` for all three classes) — the caller (the importer's ``_describe_or_skip`` seam) fails soft: the doc is skipped, counted in ``images_failed``, and the sync continues (LOCKED A3). """ data_url = image_data_url(data, mime) # Phase 137 (task 02, ASSUMPTION A5): the prefix wording is neutral # — it covers BOTH block kinds (the provenance's ``Extracted from # …`` document context and the phase-125 ``Referenced in …`` # markdown references). ``DESCRIBE_PROMPT`` (the marker + the locked # instruction) stays byte-identical; ``context=None`` sends the bare # prompt (byte-identical to the pre-phase-137 message). context_prefix = ( "\n\nThis image is part of a document. The following context " f"describes where it comes from and what it is used for:\n{context}" if context else "" ) return await _one_shot_vision_reply( llm, data_url=data_url, text=DESCRIBE_PROMPT + context_prefix, settings=settings, what="image description", ) async def _one_shot_vision_reply( llm: SummaryLLM, *, data_url: str, text: str, settings: Settings | None, what: str, ) -> str | None: """The shared one-chat-call multimodal contract (phase 122, reused by phase 140): ONE user message ``[{type: "text", text: }, {type: "image_url", image_url: {url: }}]`` against ``llm.settings.llm_chat_model`` — no system prompt, no tools, no app-level retry. Returns the stripped reply cut exactly at ``(settings or llm.settings).summary_max_chars`` (the phase-30 cap); ``None`` on any client error (the client raises :class:`LLMError` for a failure, an empty reply, or a non-2xx) or an empty/blank reply after trimming (the house one-shot empty-content policy — the duck-typed fakes may return one even though the real client rejects it). *what* names the caller for the warning log line. """ messages: list[dict[str, Any]] = [ { "role": "user", "content": [ {"type": "text", "text": text}, {"type": "image_url", "image_url": {"url": data_url}}, ], } ] model = llm.settings.llm_chat_model try: raw = await llm.chat(messages, model=model) except LLMError as e: logger.warning("%s failed (model=%s): %s", what, model, e) return None text_out = raw.strip() if not text_out: # The client already rejects empty content; this is the # defensive re-assert (the duck-typed fakes may return it). return None limit = (settings or llm.settings).summary_max_chars return text_out[:limit] async def describe_pdf_page( llm: SummaryLLM, *, data: bytes, mime: str, page: int, page_text: str | None, settings: Settings | None = None, ) -> str | None: """One-shot CHAT-model (vision) description of ONE rendered PDF page (phase 140, task 02, LOCKED A2) — the text the importer appends to that page's indexed content block as ``[Page N illustration: …]`` (the page's own text stays; the embeddings then carry what the thin text layer couldn't — the owner's L5 rationale). ONE chat-model call against ``llm.settings.llm_chat_model`` (the vision model — the ``lite`` summary model is NOT assumed vision-capable) with the same multimodal contract as :func:`describe_image` (via :func:`_one_shot_vision_reply`): no system prompt, no tools, no app-level retry. The message's text part is :data:`PICTORIAL_PROMPT` (the ``PICTORIAL_PAGE_MODE`` marker + the locked instruction) followed by the page block — the page's OWN extracted text as context, so the model describes what the text does NOT contain:: PICTORIAL_PROMPT + "\\n\\nPage {page} of the PDF. Text extracted from this page:\n" + (page_text.strip() or "(none)") ``data`` is the rendered page's PNG bytes (phase 133's ``render_pdf_pages`` output), ``mime`` its MIME type (``image/png`` for the rendered page). Returns the stripped reply cut exactly at ``(settings or llm.settings).summary_max_chars`` (the phase-30 cap). Returns ``None`` on any client error, empty reply, or non-2xx (fail-soft — the importer skips THAT page with a warning and continues: one dead page never stalls a sync, LOCKED A2). """ data_url = image_data_url(data, mime) extracted = page_text.strip() if page_text is not None else "" return await _one_shot_vision_reply( llm, data_url=data_url, text=( PICTORIAL_PROMPT + f"\n\nPage {page} of the PDF. Text extracted from this page:\n" + (extracted or "(none)") ), settings=settings, what=f"pictorial page description (page={page})", )