"""Static-frontend cache busting (phase 33). Two layers, one module: * **Version token** (``asset_version()``) — the value the HTML pages append to their asset URLs (``?v=``). * **Response middleware** (``CachingMiddleware`` / ``configure_caching``) — applies the caching behavior at the transport layer: the known HTML pages (``HTML_PAGES``) and the dynamic share page ``/shared/`` (phase 51) are always revalidated with a full 200 body (``no-cache``, never a 304) publishing no validators, and their local asset references are rewritten to carry ``?v=``; ``/assets/*`` is served ``immutable`` for a year; everything else — all of ``/api/*``, including the SSE chat stream — passes through byte-identical. Why pages must never 304 (phase 54): the static file's validators (``etag`` / ``last-modified``) describe the *file*, but the bytes the browser receives are the *rewritten body* this process built from its own token. A conditional request that matched those validators would 304 out of the rewrite — the browser keeps the HTML it already has, whose ``?v=`` points at the previous commit's CSS/JS, which are cached ``immutable`` for a year. So on page paths the middleware strips ``if-none-match`` / ``if-modified-since`` from the inbound request (before the downstream ``FileResponse`` can act on them) and drops the outbound validators. The asymmetry is deliberate: a 304 on ``/assets/*`` is safe because the URL itself encodes the version, while a 304 on an HTML page is never safe — the served body depends on the process token, which the validator ignores. Phase 91 (task 02) — the pre-paint theme tag: in the SAME rewrite branch, AFTER the ``?v=`` asset rewrite, the effective ``ui_settings`` row (task 01's :func:`app.core.theming.effective_settings` resolver — one short-lived session per response, NO process cache: the owner changes the theme at runtime from the admin tab, so the next request must see it without a restart, and a single-row SELECT is negligible at homelab page traffic) is rendered as an inline ```` and inserted immediately BEFORE the first ```` (:func:`app.core.theming.inject_theme`), so a themed deployment paints its palette on the FIRST paint — no red flash, no pop-in. An unset/defaults deployment gets ``tag == ""`` — the identity no-op — and serves the EXACT pre-phase-91 rewrite-only bytes (the byte-identical contract, B4); a DB blip (or a pre-migration boot) is the same no-op, the page never breaks. The asset rewrite is untouched, and ``/api/*`` / ``/assets/*`` still pass through byte-identical. Phase 91 (task 05, defect fix) — the CSP extension: the phase-82 policy (A1, ``default-src 'self'`` with no ``style-src``) BLOCKS the inline tag in every real browser, so a themed HTML page's response also carries ``style-src 'self' 'sha256-'`` appended to the A1 string, where ```` is the CSP3 hash of the EXACT tag content (:func:`app.core.theming.theme_csp_hash`) — the current theme is the only inline style ever permitted (no ``'unsafe-inline'``; a different palette or any other inline style is still blocked). The untagged response keeps the plain A1 string (the outer :class:`~app.core.security_headers.SecurityHeadersMiddleware` preserves a CSP an inner layer has already set), and no non-HTML response ever gets the extension. The token is computed **once per process** (``functools.cache``, i.e. ``lru_cache(maxsize=None)``) — zero per-request git/file cost. It changes when a new commit lands (git path) or the frontend tree's mtimes/sizes change (fallback path); either way the change takes effect on the next process start. Tests clear it with ``asset_version.cache_clear()``. Version-token sources: * **Git checkouts** (the homelab reality — the project root next to ``frontend/`` has a ``.git``): the token is ``git rev-parse --short HEAD``. A commit is a deploy, so the token flips on every deploy. * **Plain directories** (dev checkouts without a ``.git``, or a git failure): a stable content hash — first 12 hex chars of SHA-256 over the sorted ``relpath:mtime_ns:size`` list of every regular file under the static dir. Touching or editing a file (plus a process restart) busts the cache. * **Missing/empty directory**: ``"dev"``. """ from __future__ import annotations import functools import hashlib import logging import re import subprocess from pathlib import Path from fastapi import FastAPI from starlette.datastructures import MutableHeaders from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint from starlette.requests import Request from starlette.responses import Response from app.config import get_settings from app.core import theming from app.core.security_headers import CSP from app.db import SessionLocal logger = logging.getLogger("app") #: Token for a missing/empty static dir (e.g. a build that ships the API #: only — there is nothing on disk to version). DEV_TOKEN = "dev" #: Hard cap for the ``git rev-parse`` call — a deploy must not hang a boot. _GIT_TIMEOUT_S = 5 def _static_files(static_dir: Path) -> list[str]: """Sorted ``relpath:mtime_ns:size`` entries for every regular file.""" entries: list[str] = [] for path in static_dir.rglob("*"): if path.is_file(): st = path.stat() rel = path.relative_to(static_dir).as_posix() entries.append(f"{rel}:{st.st_mtime_ns}:{st.st_size}") return sorted(entries) def _git_short_sha(repo_root: Path) -> str: """``git -C rev-parse --short HEAD`` (raises on any failure).""" proc = subprocess.run( ["git", "-C", str(repo_root), "rev-parse", "--short", "HEAD"], capture_output=True, text=True, timeout=_GIT_TIMEOUT_S, check=True, ) sha = proc.stdout.strip() if not sha: raise ValueError("git returned an empty short SHA") return sha @functools.cache # == lru_cache(maxsize=None): one token per process def asset_version(static_dir: str | None = None) -> str: """Version token for the frontend's asset URLs (see module docstring). ``static_dir`` defaults to ``get_settings().static_dir`` (``frontend``); the git repo root is taken to be its parent. The result is cached for the process lifetime — clear it with ``asset_version.cache_clear()``. """ target = get_settings().static_dir if static_dir is None else static_dir directory = Path(target).expanduser().resolve() files = _static_files(directory) if directory.is_dir() else [] if not files: return DEV_TOKEN repo_root = directory.parent if (repo_root / ".git").exists(): try: return _git_short_sha(repo_root) except (OSError, subprocess.SubprocessError, ValueError): # git missing (OSError), timeout or non-zero exit # (SubprocessError — e.g. a repo with no commits), or empty # output → the stable content-hash fallback keeps dev checkouts # (and broken repos) cache-busting instead of breaking a boot. pass digest = hashlib.sha256("\n".join(files).encode("utf-8")).hexdigest() return digest[:12] # --------------------------------------------------------------------------- # Response-layer cache busting (task 02) # --------------------------------------------------------------------------- #: The known HTML pages — the ONLY paths whose body is rewritten. #: Phase 35 adds the admin git sources page: without this entry it #: would serve unversioned asset refs, which the immutable-for-a-year #: asset caching would pin to stale CSS after a deploy. HTML_PAGES: tuple[str, ...] = ( "/", "/index.html", "/sources.html", "/document.html", "/login.html", "/tuning.html", "/git-sources.html", # phase 35: the admin git sources page "/history.html", # phase 50: the admin saved-chats page "/tokens.html", # phase 79 task 06: the admin tokens page (shell route) "/theme.html", # phase 91 task 04: the admin theme page (shell route) # phase 51: the shared page's STATIC path (the static mount serves # shared.html at /shared.html as well as the real route serves the # dynamic /shared/ — both must carry the no-cache + ?v= # contract, so the direct URL can never pin stale assets). "/shared.html", # phase 59: the doc edit screen (the flow page task 06 ships). "/doc-edit.html", ) #: Prefix of the versioned static assets (header-only caching; the body is #: never read or modified). ASSETS_PREFIX = "/assets/" #: ``/assets/*`` — the URL carries ``?v=``, so long-term caching is #: safe (a deploy changes the token, hence the URL). ASSET_CACHE_CONTROL = "public, max-age=31536000, immutable" #: HTML pages are always revalidated — the page is the cheap redirector to #: the (long-cached) versioned assets. HTML_CACHE_CONTROL = "no-cache" #: Matches ``href="…assets/…"`` / ``src="…assets/…"`` references — leading #: slash optional (the pages mix ``/assets/…`` and ``assets/…``) — that do #: NOT already carry a query string or a fragment: ``[^"?#]+`` stops at #: ``?``/``#``, so an already-``?v=``-tagged reference can never match #: again (idempotent rewrite, never a double ``?v=``). _ASSET_REF_RE = re.compile(r'((?:href|src)="(?:/)?assets/[^"?#]+)(")') def _is_known_page(path: str) -> bool: """One source of truth for the known-page contract (phase 54). True for ``HTML_PAGES`` and for the dynamic share page ``/shared/`` (phase 51) by path prefix. Shared by the inbound conditional-header strip and the outbound rewrite branch so the two can never drift apart. """ return path in HTML_PAGES or path.startswith("/shared/") def rewrite_asset_refs(html: str, token: str) -> str: """Append ``?v=`` before the closing quote of every local ``assets/…`` ``href``/``src`` reference. Pure and idempotent: references that already carry a query string or a fragment (``?v=…``, ``#frag``) do not match the pattern and are left alone, and non-asset references (``href="/sources.html"``, ``href="data:…"``) pass through untouched. """ return _ASSET_REF_RE.sub(lambda m: f"{m.group(1)}?v={token}{m.group(2)}", html) async def _read_body(response: Response) -> bytes: """Buffer the whole response body. starlette 1.x's ``BaseHTTPMiddleware.call_next`` returns a streaming response (no ``body()`` helper), so drain ``body_iterator``; a plain buffered ``Response`` simply carries its ``body``. """ if hasattr(response, "body"): raw = response.body # bytes | memoryview (Response.render's output) if isinstance(raw, memoryview): return bytes(raw) return raw # starlette 1.x's BaseHTTPMiddleware.call_next wraps the app response # in a streaming shell (``body_iterator``, no ``body``) that is NOT a # StreamingResponse subclass — drain the iterator instead. iterator = response.body_iterator # pyright: ignore[reportAttributeAccessIssue] return b"".join([chunk async for chunk in iterator]) def _no_cache_headers(response: Response) -> dict[str, str]: """Copy the original headers, drop the stale ``content-length`` (the body size changes on rewrite), drop the upstream validators — ``etag`` / ``last-modified`` describe the static file, not the rewritten body, so a page response must never be revalidated against them later (phase 54) — and force ``Cache-Control: no-cache``.""" headers = {k: v for k, v in response.headers.items()} headers.pop("content-length", None) for name in ("etag", "last-modified"): headers.pop(name, None) headers["Cache-Control"] = HTML_CACHE_CONTROL return headers class CachingMiddleware(BaseHTTPMiddleware): """Transport-layer cache busting (phase 33, revalidation fix phase 54). Touches exactly two response shapes: * ``/assets/*`` — ``Cache-Control: public, max-age=31536000, immutable`` (header only — the body is never read; conditional requests may still 304, which is safe because the URL carries ``?v=``). * the known HTML pages (``HTML_PAGES``) plus the dynamic share page ``/shared/`` (phase 51, by path prefix) — always revalidated with a full 200 body (no 304), publishing no validators: the inbound ``if-none-match`` / ``if-modified-since`` are stripped before the downstream app runs, ``Cache-Control: no-cache`` is set, the outbound ``etag`` / ``last-modified`` are dropped, and (for ``text/html`` bodies) every local asset reference gains ``?v=``. Everything else — all of ``/api/*`` (including the SSE chat stream) — passes through byte-identical: no header changes, the body stream is never drained. """ async def dispatch( self, request: Request, call_next: RequestResponseEndpoint ) -> Response: path = request.url.path # Phase 54: on known page paths, strip the conditional request # headers BEFORE the downstream app runs. The 304 is produced # downstream (Starlette's ``FileResponse``/``StaticFiles`` honours # ``If-None-Match`` / ``If-Modified-Since`` before this middleware # sees a response), so a validator match would return a bodiless # 304 whose body can never be rewritten to carry the current # ``?v=`` refs — the browser would keep HTML pointing at # the previous commit's assets, immutable-cached for a year. # ``MutableHeaders`` mutates ``request.scope["raw_headers"]`` in # place, so the downstream app only ever sees the full 200. The # strip is scoped to page paths only — ``/assets/*`` 304s are safe # (the URL is versioned) and ``/api/*`` (incl. the SSE stream) # must stay byte-identical. if _is_known_page(path): headers = MutableHeaders(scope=request.scope) for name in ("if-none-match", "if-modified-since"): if name in headers: del headers[name] response = await call_next(request) if path.startswith(ASSETS_PREFIX): response.headers["Cache-Control"] = ASSET_CACHE_CONTROL return response if not _is_known_page(path): # /api/* (incl. SSE), /favicon.ico, unknown paths: untouched. return response # Phase 54: a bodiless downstream status (204/304) on a page # path is NEVER rewritten — starlette forbids a body on those # statuses, and the rewrite path would build exactly that. # Belt-and-braces: after the inbound conditional-header strip, # StaticFiles cannot 304 these paths; a future route or proxy # could still, so pass through with no-cache and no validators. if response.status_code in (204, 304): response.headers["Cache-Control"] = HTML_CACHE_CONTROL for name in ("etag", "last-modified"): if name in response.headers: del response.headers[name] return response content_type = response.headers.get("content-type", "") if content_type.split(";", 1)[0].strip().lower() != "text/html": # A page path with a non-HTML body (e.g. the 404 JSON served # when the static dir is missing): still revalidate, body as-is. response.headers["Cache-Control"] = HTML_CACHE_CONTROL return response # ``asset_version()`` is lru_cached, so only the very first call in # the process can fail — resolve it BEFORE draining the body stream # below. try: token = asset_version() body = await _read_body(response) except Exception: logger.exception( "cache busting: could not buffer %s — serving unmodified (no-cache)", path, ) response.headers["Cache-Control"] = HTML_CACHE_CONTROL return response # Phase 91 (task 02): the pre-paint theme tag. One short-lived # session per response (the sync-endpoint house pattern from # app/db.py — the middleware world is sync); NO process cache — # the theme changes at runtime from the admin tab, so the next # request must see it without a restart. A DB blip (or a # pre-migration boot) must never break the page: fall back to # ``tag == ""`` (the built-in palette) and keep the no-cache # contract (loadHealth house style). tag = "" try: db = SessionLocal() try: effective = theming.effective_settings(db) finally: db.close() tag = theming.theme_style_tag( {key: effective[key] for key in theming.COLOR_FIELDS} ) except Exception: logger.exception( "cache busting: theme read failed for %s — serving without the theme tag", path, ) tag = "" try: new_body = theming.inject_theme( rewrite_asset_refs(body.decode("utf-8"), token), tag ).encode("utf-8") except Exception: # The body IS buffered — re-serve the ORIGINAL bytes so a # rewrite hiccup never loses the page. logger.exception("cache busting: rewrite of %s failed — serving raw (no-cache)", path) return Response( content=body, status_code=response.status_code, headers=_no_cache_headers(response), ) headers = _no_cache_headers(response) if tag: # Phase 91 (task 05): the inline tag needs a style-src # exemption or the phase-82 CSP blocks it in the browser — # the strictest one: a sha256 hash of the EXACT tag content # (theming.theme_csp_hash), appended to the A1 string. The # outer SecurityHeadersMiddleware preserves this (it only # fills in a missing CSP); the untagged page keeps A1 # verbatim — byte- AND header-identical to pre-phase-91. headers["Content-Security-Policy"] = ( f"{CSP}; style-src 'self' '{theming.theme_csp_hash(tag)}'" ) return Response( content=new_body, status_code=response.status_code, headers=headers, ) def configure_caching(app: FastAPI) -> None: """Attach :class:`CachingMiddleware` to ``app`` (phase 33). Call inside ``create_app()`` after the routers are mounted; the middleware wraps the whole app, so the ``/`` static catch-all is covered too. ``/api/*`` responses (including the SSE chat stream) are never touched by it. """ app.add_middleware(CachingMiddleware)