phase: 03_audio_share
Build and Push Containers / test (push) Successful in 35s
Build and Push Containers / build-and-push-app (push) Successful in 11s

**Phase 03 final verification pass — complete, 100% green.**

- Verified all 4 tasks done in `complete/03_audio_share/`; confirmed implementation in `watch.js` (one composed `MediaStream` per tile, staged-only `setMuted()` policy, `data-audio` state), `share.js` (precise `NotAllowedError` rethrow, 3 honest `#stat-audio` states, mid-share audio republish), conftest flags, and the new `tests/e2e/test_audio_share.py` (11 tests, all **passed** — none skipped).
- **Criteria:** (1) viewer inbound audio `bytesReceived` strictly increases + composed stream 1 video/1 audio track while frames still decode — **passed**; (2) staged-only audio via keyboard `m`, mouse button, and `unstage()` re-mute — **passed**; (3) `shared`/`silent`/`not offered` + mid-share audio loss updates panel, roster and tiles — **passed**; (4) cancelled picker reports cancellation, single attempt — **passed**; (5) SECURITY.md §4 Audio, README platform notes, kiosk doc all describe shipped behaviour incl. Linux/PipeWire reality — **verified**; (6) full gate — **passed** (below).
- **Gates:** `uv run ruff check . && uv run ruff format --check .` → clean; `uv run pyright` → 0 errors; `uv run pytest` → 245 passed, 4 skipped (only the `MIRROR_REALDISPLAY`-gated X11 tests); `MIRROR_REALDISPLAY=1 uv run pytest -m realdisplay` → 3 passed, 1 skipped (real-capture audio: documented skip naming the exact host/flag reason, per task 04).
- **Docs:** README.md reviewed in full against code (env table, CLI flags, routes, kiosk key table, quickstart, audio notes) — consistent, no changes needed. AGENTS.md — all commands/paths/coverage claims verified accurate, no stale statements, no changes needed.
- **Defect fixed (pre-existing):** `docs/kiosk-silverblue.md` + `install-kiosk.sh`, `kiosk.conf`, `gnome-kiosk-script` used `mirror.reeselink.com` while README/SECURITY.md use `mirror.reeseapps.com` (the deployment domain, consistent with the fleet); corrected 7 occurrences in 4 files.
- No live-infrastructure changes; realdisplay suite ran on private Xvfb only.
- **Next pending phase:** `04_reachable_share_cta`.
This commit is contained in:
2026-11-01 22:40:30 -05:00
parent 1da466a412
commit d43eb407f8
39 changed files with 1554 additions and 36 deletions
+167
View File
@@ -32,14 +32,27 @@ sharer window requested delivered viewer decodes
not by the constraints. On real hardware ``screen`` is the true display
resolution, so a small window on a large display is the case the 90% floor
and the share page's 75% toast are for.
Audio on the real capture path is environment-gated, and the gap is written
down here rather than imagined away. ``--auto-select-desktop-capture-source=
Entire screen`` selects the screen without ticking the picker's "share audio"
box, and a host without an audio device has no track to offer at all — on
such hosts ``test_a_real_desktop_shares_audio_when_the_host_offers_any``
finds no audio track in the captured stream and *skips with that reason*
(never a stubbed peer, never an xfailed-into-green failure). Where a host
does expose an audio source, the test asserts the viewer's inbound-rtp audio
bytes increase across two samples and the picture still decodes distinct
frames, the same way the fake-device suite does.
"""
from __future__ import annotations
import os
import re
import shutil
import socket
import subprocess
import sys
import time
import httpx
@@ -139,6 +152,25 @@ PROBE = """() => {
}"""
# The one global a test is allowed to use (AGENTS.md §6): collect the
# RTCPeerConnection instances watch.js creates so a test can read the
# viewer's inbound-rtp stats. The e2e suite (test_audio_share.py) installs
# the same wrapper; it is repeated here because the e2e convention is one
# copy per suite file. On the module's shared watch page `window.__probe`
# is the frame-sampling promise instead — the two uses never share a page.
PC_PROBE_INIT = """
window.__probe = {pcs: []};
const RealPC = window.RTCPeerConnection;
window.RTCPeerConnection = function (config) {
const pc = new RealPC(config);
window.__probe.pcs.push(pc);
return pc;
};
window.RTCPeerConnection.prototype = RealPC.prototype;
window.RTCPeerConnection.static = RealPC.static;
"""
def _free_display() -> str:
for number in range(90, 120):
path = f"/tmp/.X11-unix/X{number}"
@@ -410,3 +442,138 @@ class TestRealX11Capture:
# And the sharer agrees that someone is watching its desktop.
expect(share_page.locator("#viewer-count")).to_have_text("1", timeout=20_000)
assert errors == [], f"uncaught browser errors: {errors}"
def test_a_real_desktop_shares_audio_when_the_host_offers_any(self, base_url, browsers):
"""The realdisplay counterpart of the audio-arrival assertion.
Where the host exposes an audio source to the real capture path, the
viewer's ``inbound-rtp`` audio bytes must strictly increase across two
samples (Opus frames over SRTP) and the picture must still decode
distinct frames. ``--auto-select-desktop-capture-source=Entire
screen`` selects the screen without ticking the picker's "share
audio" box, and a host without an audio device has no track to offer
at all — in that case the test *skips with the reason written down*
(module docstring), never asserts on an absent track, and never
stubs the peer connection.
"""
errors: list[str] = []
share_page = (
browsers["sharer"].new_context(viewport={"width": 1200, "height": 760}).new_page()
)
share_page.on("pageerror", lambda error: errors.append(str(error)))
share_page.goto(f"{base_url}/mirror", wait_until="domcontentloaded")
expect(share_page.locator("#net")).to_have_attribute("data-state", "online", timeout=20_000)
share_page.locator("#start").click()
expect(share_page.locator("body")).to_have_attribute("data-live", "on", timeout=40_000)
# The honest gate: does the captured stream actually carry an audio
# track? This is what the browser's real capture path delivered.
has_audio = share_page.evaluate(
"""() => {
const so = document.querySelector('#preview').srcObject;
return !!(so && so.getAudioTracks().length);
}"""
)
if not has_audio:
share_page.locator("#stop").click()
share_page.close()
pytest.skip(
"the host offers no audio source to the real capture path "
f"(platform={sys.platform}; --auto-select-desktop-capture-source="
"'Entire screen' does not tick 'share audio', and this host "
"delivers no audio track); see the module docstring"
)
watcher = browsers["watcher"].new_context()
watcher.add_init_script(PC_PROBE_INIT)
watch_page = watcher.new_page()
watch_page.on("pageerror", lambda error: errors.append(str(error)))
watch_page.goto(f"{base_url}/watch", wait_until="domcontentloaded")
expect(watch_page.locator("#net")).to_have_attribute("data-state", "online", timeout=20_000)
expect(watch_page.locator(".tile[data-state='live']")).to_have_count(1, timeout=60_000)
# The deliberate gesture: stage, then unmute.
watch_page.keyboard.press("Enter")
expect(watch_page.locator("body")).to_have_class(re.compile("staged"), timeout=5_000)
watch_page.keyboard.press("m")
assert (
watch_page.evaluate("() => !document.querySelector('#watch-stage video').muted") is True
)
def audio_inbound() -> dict:
"""One ``inbound-rtp`` audio sample from the viewer's leg."""
return watch_page.evaluate(
"""async () => {
const pcs = (window.__probe && window.__probe.pcs) || [];
const pc = pcs.find((p) => p.connectionState === 'connected');
if (!pc) return {pc: false};
const stats = await pc.getStats();
let audio = null;
stats.forEach((r) => {
if (r.type === 'inbound-rtp' && r.kind === 'audio') audio = r;
};
if (!audio) return {pc: true, entry: null};
const energy = audio.totalAudioEnergy;
const level = audio.audioLevel;
return {
pc: true,
entry: {
bytes: audio.bytesReceived || 0,
energy: Number.isFinite(energy) ? energy : null,
level: Number.isFinite(level) ? level : null,
},
};
}"""
)
first: dict = {}
deadline = time.time() + 30
while time.time() < deadline:
first = audio_inbound()
if first.get("entry"):
break
time.sleep(0.5)
assert first.get("entry"), (
f"the viewer's connected leg has no inbound-rtp audio entry: {first}"
)
time.sleep(4.0)
second = audio_inbound()
assert second.get("entry"), f"the inbound-rtp audio entry vanished: {second}"
assert second["entry"]["bytes"] > first["entry"]["bytes"], (
f"inbound audio bytes did not strictly increase: {first} -> {second}"
)
# The picture is asserted in the same test: audio arriving while the
# picture froze would be a different bug. One self-contained sample
# loop — no global (on this page `window.__probe` belongs to the
# peer-connection collector).
frames = watch_page.evaluate(
"""async () => {
const video = document.querySelector('#watch-stage video');
const canvas = document.createElement('canvas');
canvas.width = 96; canvas.height = 60;
const ctx = canvas.getContext('2d', {willReadFrequently: true});
const seen = new Set();
for (let i = 0; i < 10; i++) {
await new Promise((r) => setTimeout(r, 120));
try { ctx.drawImage(video, 0, 0, 96, 60); } catch {}
const d = ctx.getImageData(0, 0, 96, 60).data;
let h = 0;
for (let j = 0; j < d.length; j += 4)
h = ((h * 31) + d[j] + d[j + 1] * 3 + d[j + 2] * 7) >>> 0;
seen.add(h);
}
return {width: video.videoWidth, distinct: seen.size};
}"""
)
assert frames["width"] >= SHARP_FLOOR_PX, (
f"the watcher decoded a {frames['width']}px-wide picture: {frames}"
)
assert frames["distinct"] >= 2, f"the watcher received a frozen frame: {frames}"
# Clean up before the module-scoped fixtures serve the next test.
watch_page.close()
watcher.close()
share_page.locator("#stop").click()
share_page.close()
assert errors == [], f"uncaught browser errors: {errors}"