phase: 03_audio_share
**Phase 03 final verification pass — complete, 100% green.** - Verified all 4 tasks done in `complete/03_audio_share/`; confirmed implementation in `watch.js` (one composed `MediaStream` per tile, staged-only `setMuted()` policy, `data-audio` state), `share.js` (precise `NotAllowedError` rethrow, 3 honest `#stat-audio` states, mid-share audio republish), conftest flags, and the new `tests/e2e/test_audio_share.py` (11 tests, all **passed** — none skipped). - **Criteria:** (1) viewer inbound audio `bytesReceived` strictly increases + composed stream 1 video/1 audio track while frames still decode — **passed**; (2) staged-only audio via keyboard `m`, mouse button, and `unstage()` re-mute — **passed**; (3) `shared`/`silent`/`not offered` + mid-share audio loss updates panel, roster and tiles — **passed**; (4) cancelled picker reports cancellation, single attempt — **passed**; (5) SECURITY.md §4 Audio, README platform notes, kiosk doc all describe shipped behaviour incl. Linux/PipeWire reality — **verified**; (6) full gate — **passed** (below). - **Gates:** `uv run ruff check . && uv run ruff format --check .` → clean; `uv run pyright` → 0 errors; `uv run pytest` → 245 passed, 4 skipped (only the `MIRROR_REALDISPLAY`-gated X11 tests); `MIRROR_REALDISPLAY=1 uv run pytest -m realdisplay` → 3 passed, 1 skipped (real-capture audio: documented skip naming the exact host/flag reason, per task 04). - **Docs:** README.md reviewed in full against code (env table, CLI flags, routes, kiosk key table, quickstart, audio notes) — consistent, no changes needed. AGENTS.md — all commands/paths/coverage claims verified accurate, no stale statements, no changes needed. - **Defect fixed (pre-existing):** `docs/kiosk-silverblue.md` + `install-kiosk.sh`, `kiosk.conf`, `gnome-kiosk-script` used `mirror.reeselink.com` while README/SECURITY.md use `mirror.reeseapps.com` (the deployment domain, consistent with the fleet); corrected 7 occurrences in 4 files. - No live-infrastructure changes; realdisplay suite ran on private Xvfb only. - **Next pending phase:** `04_reachable_share_cta`.
This commit is contained in:
@@ -32,14 +32,27 @@ sharer window requested delivered viewer decodes
|
||||
not by the constraints. On real hardware ``screen`` is the true display
|
||||
resolution, so a small window on a large display is the case the 90% floor
|
||||
and the share page's 75% toast are for.
|
||||
|
||||
Audio on the real capture path is environment-gated, and the gap is written
|
||||
down here rather than imagined away. ``--auto-select-desktop-capture-source=
|
||||
Entire screen`` selects the screen without ticking the picker's "share audio"
|
||||
box, and a host without an audio device has no track to offer at all — on
|
||||
such hosts ``test_a_real_desktop_shares_audio_when_the_host_offers_any``
|
||||
finds no audio track in the captured stream and *skips with that reason*
|
||||
(never a stubbed peer, never an xfailed-into-green failure). Where a host
|
||||
does expose an audio source, the test asserts the viewer's inbound-rtp audio
|
||||
bytes increase across two samples and the picture still decodes distinct
|
||||
frames, the same way the fake-device suite does.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
import httpx
|
||||
@@ -139,6 +152,25 @@ PROBE = """() => {
|
||||
}"""
|
||||
|
||||
|
||||
# The one global a test is allowed to use (AGENTS.md §6): collect the
|
||||
# RTCPeerConnection instances watch.js creates so a test can read the
|
||||
# viewer's inbound-rtp stats. The e2e suite (test_audio_share.py) installs
|
||||
# the same wrapper; it is repeated here because the e2e convention is one
|
||||
# copy per suite file. On the module's shared watch page `window.__probe`
|
||||
# is the frame-sampling promise instead — the two uses never share a page.
|
||||
PC_PROBE_INIT = """
|
||||
window.__probe = {pcs: []};
|
||||
const RealPC = window.RTCPeerConnection;
|
||||
window.RTCPeerConnection = function (config) {
|
||||
const pc = new RealPC(config);
|
||||
window.__probe.pcs.push(pc);
|
||||
return pc;
|
||||
};
|
||||
window.RTCPeerConnection.prototype = RealPC.prototype;
|
||||
window.RTCPeerConnection.static = RealPC.static;
|
||||
"""
|
||||
|
||||
|
||||
def _free_display() -> str:
|
||||
for number in range(90, 120):
|
||||
path = f"/tmp/.X11-unix/X{number}"
|
||||
@@ -410,3 +442,138 @@ class TestRealX11Capture:
|
||||
# And the sharer agrees that someone is watching its desktop.
|
||||
expect(share_page.locator("#viewer-count")).to_have_text("1", timeout=20_000)
|
||||
assert errors == [], f"uncaught browser errors: {errors}"
|
||||
|
||||
def test_a_real_desktop_shares_audio_when_the_host_offers_any(self, base_url, browsers):
|
||||
"""The realdisplay counterpart of the audio-arrival assertion.
|
||||
|
||||
Where the host exposes an audio source to the real capture path, the
|
||||
viewer's ``inbound-rtp`` audio bytes must strictly increase across two
|
||||
samples (Opus frames over SRTP) and the picture must still decode
|
||||
distinct frames. ``--auto-select-desktop-capture-source=Entire
|
||||
screen`` selects the screen without ticking the picker's "share
|
||||
audio" box, and a host without an audio device has no track to offer
|
||||
at all — in that case the test *skips with the reason written down*
|
||||
(module docstring), never asserts on an absent track, and never
|
||||
stubs the peer connection.
|
||||
"""
|
||||
errors: list[str] = []
|
||||
share_page = (
|
||||
browsers["sharer"].new_context(viewport={"width": 1200, "height": 760}).new_page()
|
||||
)
|
||||
share_page.on("pageerror", lambda error: errors.append(str(error)))
|
||||
share_page.goto(f"{base_url}/mirror", wait_until="domcontentloaded")
|
||||
expect(share_page.locator("#net")).to_have_attribute("data-state", "online", timeout=20_000)
|
||||
share_page.locator("#start").click()
|
||||
expect(share_page.locator("body")).to_have_attribute("data-live", "on", timeout=40_000)
|
||||
|
||||
# The honest gate: does the captured stream actually carry an audio
|
||||
# track? This is what the browser's real capture path delivered.
|
||||
has_audio = share_page.evaluate(
|
||||
"""() => {
|
||||
const so = document.querySelector('#preview').srcObject;
|
||||
return !!(so && so.getAudioTracks().length);
|
||||
}"""
|
||||
)
|
||||
if not has_audio:
|
||||
share_page.locator("#stop").click()
|
||||
share_page.close()
|
||||
pytest.skip(
|
||||
"the host offers no audio source to the real capture path "
|
||||
f"(platform={sys.platform}; --auto-select-desktop-capture-source="
|
||||
"'Entire screen' does not tick 'share audio', and this host "
|
||||
"delivers no audio track); see the module docstring"
|
||||
)
|
||||
|
||||
watcher = browsers["watcher"].new_context()
|
||||
watcher.add_init_script(PC_PROBE_INIT)
|
||||
watch_page = watcher.new_page()
|
||||
watch_page.on("pageerror", lambda error: errors.append(str(error)))
|
||||
watch_page.goto(f"{base_url}/watch", wait_until="domcontentloaded")
|
||||
expect(watch_page.locator("#net")).to_have_attribute("data-state", "online", timeout=20_000)
|
||||
expect(watch_page.locator(".tile[data-state='live']")).to_have_count(1, timeout=60_000)
|
||||
|
||||
# The deliberate gesture: stage, then unmute.
|
||||
watch_page.keyboard.press("Enter")
|
||||
expect(watch_page.locator("body")).to_have_class(re.compile("staged"), timeout=5_000)
|
||||
watch_page.keyboard.press("m")
|
||||
assert (
|
||||
watch_page.evaluate("() => !document.querySelector('#watch-stage video').muted") is True
|
||||
)
|
||||
|
||||
def audio_inbound() -> dict:
|
||||
"""One ``inbound-rtp`` audio sample from the viewer's leg."""
|
||||
return watch_page.evaluate(
|
||||
"""async () => {
|
||||
const pcs = (window.__probe && window.__probe.pcs) || [];
|
||||
const pc = pcs.find((p) => p.connectionState === 'connected');
|
||||
if (!pc) return {pc: false};
|
||||
const stats = await pc.getStats();
|
||||
let audio = null;
|
||||
stats.forEach((r) => {
|
||||
if (r.type === 'inbound-rtp' && r.kind === 'audio') audio = r;
|
||||
};
|
||||
if (!audio) return {pc: true, entry: null};
|
||||
const energy = audio.totalAudioEnergy;
|
||||
const level = audio.audioLevel;
|
||||
return {
|
||||
pc: true,
|
||||
entry: {
|
||||
bytes: audio.bytesReceived || 0,
|
||||
energy: Number.isFinite(energy) ? energy : null,
|
||||
level: Number.isFinite(level) ? level : null,
|
||||
},
|
||||
};
|
||||
}"""
|
||||
)
|
||||
|
||||
first: dict = {}
|
||||
deadline = time.time() + 30
|
||||
while time.time() < deadline:
|
||||
first = audio_inbound()
|
||||
if first.get("entry"):
|
||||
break
|
||||
time.sleep(0.5)
|
||||
assert first.get("entry"), (
|
||||
f"the viewer's connected leg has no inbound-rtp audio entry: {first}"
|
||||
)
|
||||
time.sleep(4.0)
|
||||
second = audio_inbound()
|
||||
assert second.get("entry"), f"the inbound-rtp audio entry vanished: {second}"
|
||||
assert second["entry"]["bytes"] > first["entry"]["bytes"], (
|
||||
f"inbound audio bytes did not strictly increase: {first} -> {second}"
|
||||
)
|
||||
|
||||
# The picture is asserted in the same test: audio arriving while the
|
||||
# picture froze would be a different bug. One self-contained sample
|
||||
# loop — no global (on this page `window.__probe` belongs to the
|
||||
# peer-connection collector).
|
||||
frames = watch_page.evaluate(
|
||||
"""async () => {
|
||||
const video = document.querySelector('#watch-stage video');
|
||||
const canvas = document.createElement('canvas');
|
||||
canvas.width = 96; canvas.height = 60;
|
||||
const ctx = canvas.getContext('2d', {willReadFrequently: true});
|
||||
const seen = new Set();
|
||||
for (let i = 0; i < 10; i++) {
|
||||
await new Promise((r) => setTimeout(r, 120));
|
||||
try { ctx.drawImage(video, 0, 0, 96, 60); } catch {}
|
||||
const d = ctx.getImageData(0, 0, 96, 60).data;
|
||||
let h = 0;
|
||||
for (let j = 0; j < d.length; j += 4)
|
||||
h = ((h * 31) + d[j] + d[j + 1] * 3 + d[j + 2] * 7) >>> 0;
|
||||
seen.add(h);
|
||||
}
|
||||
return {width: video.videoWidth, distinct: seen.size};
|
||||
}"""
|
||||
)
|
||||
assert frames["width"] >= SHARP_FLOOR_PX, (
|
||||
f"the watcher decoded a {frames['width']}px-wide picture: {frames}"
|
||||
)
|
||||
assert frames["distinct"] >= 2, f"the watcher received a frozen frame: {frames}"
|
||||
|
||||
# Clean up before the module-scoped fixtures serve the next test.
|
||||
watch_page.close()
|
||||
watcher.close()
|
||||
share_page.locator("#stop").click()
|
||||
share_page.close()
|
||||
assert errors == [], f"uncaught browser errors: {errors}"
|
||||
|
||||
Reference in New Issue
Block a user