mirror/.agents/PLAN.md

mirror — Master Plan

This file is the source of truth. Anything marked LOCKED may not be changed without an explicit decision recorded here first. Phases implement this plan; they do not revise it.

1. Vision

Mirror a screen from any device that has a browser, and watch every screen that anyone is mirroring, from any other device — with nothing to install and nothing to sign in to.

Day one, one command:

uv run mirror serve

• /mirror — pick a window, screen, or tab using the operating system's own picker, and mirror it.
• /watch — see every live mirror as a wall of tiles; click one to take it over the whole screen.
• N sharers and M watchers at the same time, in any combination.

The single thing that must never regress: a person on a Linux laptop, a person on a phone, and a person on a work machine can all join by opening a URL.

Design principles

1. The browser is the whole client. No native helper, no extension, no WebAssembly to fetch, no build step for the frontend.
2. Media never touches the server. The server is a roster and a signaling relay. It handles JSON, always small, never frame data.
3. Linux is the reference platform. X11/Xorg is the guaranteed path; the Wayland portal path is supported where the compositor provides it. Every capture-related change is verified on Linux before anything else.
4. Zero-configuration on a LAN. STUN/TURN are opt-in, not defaults: peers on one network reach each other with host candidates, and shipping STUN by default would leak internal addresses to a third party.
5. The prototype is the seed, not a spike. Code in src/mirror/ is production-shaped: typed, linted, covered, and gated by the same test suite future phases must keep green.

2. Architectural anchors

#	Component	Decision	Rationale	Status
A1	Language / runtime	Python 3.13+ (developed on 3.14), uv for everything	Matches the estate; uv is the only installer	LOCKED
A2	HTTP + signaling	FastAPI + uvicorn[standard], single process	Native WebSocket support, typed boundary, one dependency tree	LOCKED
A3	Media transport	WebRTC, mesh, one RTCPeerConnection per viewer	Hardware codecs, sub-100ms LAN latency, no server media handling; server stays a roster	LOCKED
A4	Who negotiates	The sharer is always the offerer; the viewer only answers	Removes renegotiation glare; the sharer owns the track set	LOCKED
A5	Wire format	JSON over one WebSocket at /ws, camelCase keys, discriminated on type	One connection per peer, trivially debuggable with websocat	LOCKED
A6	Validation	Pydantic models with alias_generator=to_camel and extra="ignore"	camelCase on the wire, snake_case in Python. Explicit aliases are mandatory: silent field-dropping is a bug we already shipped once	LOCKED
A7	State	In-memory roster only (mirror.registry.Registry). No database, no ORM, no Alembic	A mirror that outlives the process is a lie: the peers are gone. Persistence would add a failure mode and remove none	LOCKED
A8	Domain/transport split	Registry is pure and returns list[Envelope]; Hub owns sockets	The whole admission/teardown policy is unit-testable without a socket	LOCKED
A9	Frontend	Hand-written ES modules, no framework, no bundler	Two pages and one WebSocket do not earn a build pipeline; node --check is the whole toolchain	LOCKED
A10	Assets	No external CDNs, fonts, or analytics. Everything served from /static	Works offline on a LAN; nothing phones home; nothing to time out	LOCKED
A11	Authentication	None, deliberately	Trusted-LAN tool, per the product brief. Threat model lives in SECURITY.md	LOCKED
A12	Configuration	Environment variables (MIRROR_*) + optional .env	Twelve-factor, container-friendly, no config file format to invent	LOCKED
A13	E2E testing	Playwright + full Chromium (channel="chromium"), headless, with a real frame-content assertion	See §5. The assertion is on decoded pixels, never on log lines	LOCKED
A14	Linux-native capture test	Separate realdisplay marked suite: private Xvfb + headed Chromium pinned to --ozone-platform=x11	Proves the real capture backend without touching the developer's session	LOCKED
A15	CLI	argparse, single serve subcommand, flags also accepted bare	One command; no dependency for a two-option CLI	LOCKED
A16	Quality gates	ruff check + ruff format --check + pyright (0 errors) + pytest green, always	Non-negotiable before a phase is called complete	LOCKED
A17	Scale-out beyond a mesh	Proposed: WebSocket/MSE relay or WebCodecs SFU when a sharer's uplink, not the CPU, is the limit	Revisit only with a measured problem. Status: PROPOSED	Proposed
A18	TLS	Proposed: mirror serve --https with an auto-generated self-signed cert	Solves the secure-context problem (§6) for LAN IPs without a CA. Status: PROPOSED	Proposed

3. Architecture

   sharer browser                                   watcher browser(s)
  ┌───────────────┐                              ┌───────────────────────┐
  │ /mirror       │                              │ /watch                │
  │ getDisplayMedia│                             │ tile per mirror       │
  │  ↓ track      │                              │  ↑ ontrack → <video>  │
  │ sendonly      │◄════════ WebRTC (SRTP) ═════►│ recvonly              │
  │ transceivers  │      LAN, direct, per-viewer  │ transceivers          │
  └───────┬───────┘                              └───────────┬───────────┘
          │ ws /ws (JSON only)                    ws /ws     │
          ▼                                                ▼
  ┌────────────────────────────────────────────────────────────────┐
  │ FastAPI  ── Hub (sockets)  ── Registry (pure roster)           │
  │            /api/streams · /api/config · /healthz · /static     │
  └────────────────────────────────────────────────────────────────┘

Signaling sequence (the whole protocol)

sharer → hello{role:"sharer",name}      server → welcome{peerId,streams[]}
sharer → publish{label,width,height,displaySurface,audio}
                                        server → stream_added   (to every viewer)
viewer → subscribe{streamId}            server → peer_joined{viewerId}  (to sharer)
sharer → signal{to:viewer,sdp:"offer"}  server → signal{from:sharer}
viewer → signal{to:sharer,sdp:"answer"} server → signal{from:viewer}
          … ICE candidates both ways, same envelope …
sharer → unpublish                      server → stream_removed (to every viewer)

streamId == sharer peerId: one browser tab captures one surface, so a stream and its publisher share a lifetime and an identity. Teardown therefore cannot leak.

Roster rules (enforced in registry.py, tested in tests/test_registry.py)

• A stream exists exactly as long as its sharer's socket does.
• A signal frame is relayed only between a viewer and the sharer of a mirror that viewer subscribed to — in either direction. Viewer↔viewer and unsolicited sharer→viewer traffic is refused (not_subscribed, peer_to_peer).
• Teardown is broadcast to every viewer, not merely the subscribers: a tile is drawn on stream_added, so a viewer that has not yet subscribed still has something on screen that must be cleared.
• Every exit path (explicit unpublish, clean disconnect, crash, replaced stream) emits the same notification set. No path may leak an RTCPeerConnection.
• A repeat hello renames the peer and refreshes the mirrors that display it.

4. Data model

No schema, no migration: the state is a live graph.

Peer   { id, role ∈ {sharer, viewer}, name, device, joined_at }
Stream { id == sharer_id, label, display_surface, width, height, audio,
         viewers: set[viewer_id], created_at }

Stream/viewer state transitions:

mirror:   (absent) ──publish──► advertised ──subscribe──► advertised+watching
                          ▲                                    │
                     republish(same)                     republish(changed)
                          │                                    │
                     unchanged: metadata only            re-offer to each viewer
        any of: unpublish · sharer disconnect · capacity eviction ──► (absent)

viewer leg: idle ──subscribe──► negotiating ──► connected ──► closed
                                    │ failed (×3, backoff) ──► unreachable

Roles are exclusive per socket in the prototype: a tab is either sharing or watching. Making a peer able to do both is a phase, not an accident waiting to happen.

5. Verification workflow

The gate, in order, always:

uv run ruff check . && uv run ruff format --check .
uv run pyright                       # 0 errors
uv run pytest                        # unit + API + browser E2E
MIRROR_REALDISPLAY=1 uv run pytest -m realdisplay   # Linux-native capture

The E2E ladder — why each rung exists

Rung	Command	What it can prove nothing else can
1. Roster	pytest tests/test_registry.py	Admission, fan-out, authorization and teardown policy, without any I/O
2. Wire	pytest tests/test_protocol.py tests/test_app.py	camelCase round-trip, malformed-frame handling, the WS lifecycle
3. Media	pytest -m e2e	Real capture → encode → SRTP → decode → painted pixels, two Chromium processes
4. Real desktop	MIRROR_REALDISPLAY=1 pytest -m realdisplay	The actual X11 capture backend on a private Xvfb display

Browser-harness findings (do not rediscover these)

• navigator.mediaDevices is undefined on about:blank and on plain-HTTP non-localhost origins. E2E must go through http://127.0.0.1:<port>, which is a secure context.
• Playwright's default chromium headless shell has no media stack at all — navigator.mediaDevices is absent. Use channel="chromium" (full binary, new headless).
• --use-fake-ui-for-media-stream --use-fake-device-for-media-stream gives a deterministic 1280×720 displaySurface: monitor stream with genuinely moving pixels — no stubbing, and distinct frames > 1 is a real assertion.
• For real capture: --auto-select-desktop-capture-source="Entire screen" plus --ozone-platform=x11. Passing --use-fake-ui-for-media-stream instead breaks the X11 capturer (instant NotReadableError), because it approves the request without ever selecting a source.
• Chromium scales a monitor capture to the capturing window, so real-capture tests assert aspect ratio, not absolute pixels (1280×800 → 1200×750).
• A headed Chromium that inherits WAYLAND_DISPLAY or DBUS_SESSION_BUS_ADDRESS will route capture through xdg-desktop-portal: it mirrors the developer's real desktop and raises a permission dialog on their actual screen. browser_env() strips both so that is structurally impossible.

Rules for any new assertion about media

1. Assert on the viewer's <video> (videoWidth, advancing pixel hashes, framesDecoded), never on the sharer's preview — the preview proves nothing crossed the wire.
2. Prefer a content assertion (frames differ) over an event assertion (ontrack fired).
3. Where a test drives motion, drive it deliberately (mutate the captured page) rather than hoping a desktop animates on its own.

6. Known constraints (accepted, documented, not bugs)

Constraint	Why	Handling
getDisplayMedia needs a secure context	Browser rule. http://<lan-ip> is not one	/mirror detects it and prints the three ways out (localhost, HTTPS, the per-origin Chrome flag). Real fix is A18
Mesh upload = N × bitrate per sharer	WebRTC mesh	Per-leg maxBitrate cap (default 12 Mbit/s). A17 when it bites
Wayland capture depends on the portal	Compositor policy	Each share is confirmed by the user; X11 needs no such prompt. Documented in SECURITY.md
No reconnection of a sharer identity	Peer ids are per-socket	Viewer clears the tile and the sharer re-publishes on reconnect; the wall recovers by itself
Single process, in-memory	A11/A7	One roster per process. Multi-process needs a shared roster + a way to pin a stream to its peer — a real phase, not a config flag

7. Candidate phases (deliberately not yet written as phases)

Ordered by what a user would notice first. Each would become .agents/phases/todo/NN_name/{00_phase.md,NN_task.md} via the phase-authoring skill, with its own independently-runnable test suite.

1. HTTPS mode — self-signed cert + a "how to trust it" screen (A18). Unblocks capture from http://<lan-ip> on other devices, the single biggest papercut.
2. Share-page resilience — reconnect with roster resync, "source lost" states, surfaceSwitching UX, per-leg health on the sharer panel.
3. Wall UX — keyboard navigation (j/k/Enter/Esc), per-tile mute, layout memory, "auto-stage newest mirror", PiP.
4. Capacity & fairness — adaptive bitrate per leg by watcher count, MIRROR_MAX_MIRRORS surfaced in the UI, graceful degradation for a weak sharer uplink.
5. Scale-out media path — the relay/SFU evaluation in A17, decided with measurements from a 1-sharer/20-watcher LAN test.
6. Roles beyond sharer/viewer — one socket that shares and watches; a device naming/consent layer that keeps A11's threat model intact.
7. Ops — /metrics (Prometheus text), structured logging, Containerfile hardening, systemd unit for the always-on kiosk box.

8. Definition of done (per phase)

• All four gates in §5 green, including -m e2e.
• No new external asset, no CDN, no new dependency without a row in §2.
• Any new media behaviour has an assertion on the viewer side.
• AGENTS.md and this file updated if an anchor moved — with the reason.