phase: 03_audio_share
**Phase 03 final verification pass — complete, 100% green.** - Verified all 4 tasks done in `complete/03_audio_share/`; confirmed implementation in `watch.js` (one composed `MediaStream` per tile, staged-only `setMuted()` policy, `data-audio` state), `share.js` (precise `NotAllowedError` rethrow, 3 honest `#stat-audio` states, mid-share audio republish), conftest flags, and the new `tests/e2e/test_audio_share.py` (11 tests, all **passed** — none skipped). - **Criteria:** (1) viewer inbound audio `bytesReceived` strictly increases + composed stream 1 video/1 audio track while frames still decode — **passed**; (2) staged-only audio via keyboard `m`, mouse button, and `unstage()` re-mute — **passed**; (3) `shared`/`silent`/`not offered` + mid-share audio loss updates panel, roster and tiles — **passed**; (4) cancelled picker reports cancellation, single attempt — **passed**; (5) SECURITY.md §4 Audio, README platform notes, kiosk doc all describe shipped behaviour incl. Linux/PipeWire reality — **verified**; (6) full gate — **passed** (below). - **Gates:** `uv run ruff check . && uv run ruff format --check .` → clean; `uv run pyright` → 0 errors; `uv run pytest` → 245 passed, 4 skipped (only the `MIRROR_REALDISPLAY`-gated X11 tests); `MIRROR_REALDISPLAY=1 uv run pytest -m realdisplay` → 3 passed, 1 skipped (real-capture audio: documented skip naming the exact host/flag reason, per task 04). - **Docs:** README.md reviewed in full against code (env table, CLI flags, routes, kiosk key table, quickstart, audio notes) — consistent, no changes needed. AGENTS.md — all commands/paths/coverage claims verified accurate, no stale statements, no changes needed. - **Defect fixed (pre-existing):** `docs/kiosk-silverblue.md` + `install-kiosk.sh`, `kiosk.conf`, `gnome-kiosk-script` used `mirror.reeselink.com` while README/SECURITY.md use `mirror.reeseapps.com` (the deployment domain, consistent with the fleet); corrected 7 occurrences in 4 files. - No live-infrastructure changes; realdisplay suite ran on private Xvfb only. - **Next pending phase:** `04_reachable_share_cta`.
This commit is contained in:
@@ -0,0 +1,90 @@
|
||||
# Phase 03 — Audio that actually arrives
|
||||
|
||||
**Source:** `TODO.md` L5 — "Need audio share to work."
|
||||
**Story:** n/a
|
||||
**Context:** `.agents/PLAN.md` §2 A3/A4 (WebRTC mesh, **the sharer always offers and owns the track set**), §3 (one `RTCPeerConnection` per viewer), `SECURITY.md` §4 "Audio", `src/mirror/static/watch.js` (`ensurePeer`, `pc.ontrack`), `share.js` (`pickSurface`, `publish`), `src/mirror/registry.py` L69/197/222–228 and `src/mirror/protocol.py` L57/118/129 (the `audio` flag already crosses the wire)
|
||||
|
||||
## Objective
|
||||
|
||||
When a share carries audio, a viewer can hear it — on the wall, on the stage, and
|
||||
with the keyboard alone — and the UI says honestly whether audio exists at all.
|
||||
|
||||
What is already true, and must stay true: the `audio` boolean is published,
|
||||
relayed and rendered (`registry.py` keeps it on the `Stream`, `stream_updated`
|
||||
carries changes, `paintTile()` shows the 🔇 button only when audio exists). The
|
||||
**server needs no change in this phase.** What is broken is on the client:
|
||||
|
||||
1. **A second `ontrack` throws the first track away.** `watch.js`'s `pc.ontrack`
|
||||
does `tile.video.srcObject = (event.streams && event.streams[0]) || new
|
||||
MediaStream([event.track])`. When the sharer's tracks arrive without a shared
|
||||
stream id (which happens with `addTransceiver(track)` and is common in
|
||||
Firefox), the audio callback builds a *new* one-track `MediaStream` and
|
||||
overwrites the video one. The video keeps playing from the still-attached
|
||||
transceiver, so nothing looks wrong — there is simply never any sound.
|
||||
2. **There is no way to unmute the thing you are watching.** The mute button lives
|
||||
in `.tile .actions`, hover-revealed in the grid (phase 01 makes it reachable),
|
||||
and the staged tile's caption dims to `0.28`.
|
||||
3. **The sharer does not know whether it offered audio.** `pickSurface()` retries
|
||||
without audio on *any* error except `NotAllowedError`, and `publish()` reads
|
||||
`getAudioTracks().length` once — so a source whose audio track ends mid-share
|
||||
leaves every viewer with a dead 🔇 button.
|
||||
|
||||
Policy for this phase (user-confirmed): **only the staged mirror can be audible.**
|
||||
`/watch` mutes everything else when one is unmuted, and unstaging re-mutes.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `01_keyboard_kiosk_control` (todo) — `setMuted(tile, muted)` and the `m` key
|
||||
come from there; this phase extends them rather than inventing a second mute path.
|
||||
- `02_sharp_capture` (todo) — the per-leg `RTCRtpSender` work: the audio leg must
|
||||
not be caught up in the video sender's parameters, and 25 Mbit/s of video headroom
|
||||
is what leaves room for Opus.
|
||||
|
||||
## Tasks
|
||||
|
||||
1. `01_audio_leg_plumbing.md` — one accumulating `MediaStream` per tile, so the
|
||||
audio and video tracks coexist; assert an `m=audio` line really is negotiated.
|
||||
2. `02_staged_audio_playback.md` — the staged-only unmute policy, applied to
|
||||
mouse, keyboard and unstage, with the tile's audio state readable at a glance.
|
||||
3. `03_audio_source_reporting.md` — a precise audio fallback in `pickSurface()`,
|
||||
three honest states on `/mirror`, and republish when an audio track ends.
|
||||
4. `04_audio_e2e_and_docs.md` — a viewer-side assertion that audio crossed, and
|
||||
the `SECURITY.md`/README text that tells users what a browser will actually
|
||||
capture on Linux.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- **Assert on the viewer** (`AGENTS.md` §5): inbound `bytesReceived` for
|
||||
`inbound-rtp`/`kind === "audio"`, `totalAudioEnergy`/`audioLevel`, and the
|
||||
composed `video.srcObject.getAudioTracks().length`. The sharer's panel proves
|
||||
nothing crossed the wire.
|
||||
- The E2E Chromium already starts with
|
||||
`--use-fake-device-for-audio-capture` (`tests/e2e/conftest.py`), but a headless
|
||||
container has **no audio device**, so a real sound assertion belongs to the
|
||||
`realdisplay`-class run and the fake path asserts the SDP and the composed
|
||||
`MediaStream`. Do not fake the WebRTC layer to make a green test.
|
||||
- `tests/e2e/conftest.py` also passes
|
||||
`--autoplay-policy=no-user-gesture-required`: the suite therefore cannot prove
|
||||
the *browser's* autoplay rule, only mirror's own `muted` state. Keep it that way
|
||||
and assert application state, not browser policy — and never add that flag to a
|
||||
test whose point is muting.
|
||||
- The deliberate, user-initiated unmute promised by `SECURITY.md` §4 is preserved:
|
||||
audio never starts without a click or a keypress.
|
||||
- Coverage **>90%** on new/modified code; full gate per task, plus `-m e2e` on
|
||||
every task that touches media.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] With an audio-bearing source, the viewer's inbound audio
|
||||
`bytesReceived` grows and `video.srcObject.getAudioTracks().length === 1`
|
||||
while the video still decodes frames.
|
||||
- [ ] Only the staged mirror can produce sound; unmuting mutes the others and
|
||||
`unstage()` re-mutes. Reachable by mouse **and** by `m`.
|
||||
- [ ] `/mirror` distinguishes `shared` / `silent` / `not offered`, and a source
|
||||
losing its audio mid-share updates every viewer's tile.
|
||||
- [ ] Cancelling the picker still reports a cancellation, not a silent-share
|
||||
fallback.
|
||||
- [ ] `SECURITY.md` §4 "Audio" and README's platform notes describe what shipped,
|
||||
including the Linux/PipeWire reality.
|
||||
- [ ] Full gate: `uv run ruff check . && uv run ruff format --check . && uv run
|
||||
pyright && uv run pytest && MIRROR_REALDISPLAY=1 uv run pytest -m realdisplay`.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Task 01 — Keep both tracks on one MediaStream
|
||||
|
||||
**Phase:** `03_audio_share` · **Source:** `TODO.md:5` — "Need audio share to work."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Fix the bug that makes audio impossible no matter what the sharer offers: the
|
||||
viewer replaces `srcObject` on every `ontrack`, so the second track to arrive
|
||||
erases the first. Then prove an audio section is actually negotiated.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/mirror/static/watch.js` — in `makeTile()`, add `stream: null` to the tile
|
||||
record (next to `pc`, `pending`, `attempts`), and in `ensurePeer()`'s
|
||||
`pc.ontrack` stop assigning a fresh `MediaStream`:
|
||||
|
||||
```js
|
||||
tile.stream = tile.stream || new MediaStream();
|
||||
tile.stream.addTrack(event.track);
|
||||
if (tile.video.srcObject !== tile.stream) tile.video.srcObject = tile.stream;
|
||||
```
|
||||
|
||||
Keep the existing `tile.video.play().catch(() => {})` and its comment. Using
|
||||
`event.streams[0]` when present is *not* kept: mixing "browser's stream" and
|
||||
"our stream" per track is exactly how one track ends up alone.
|
||||
2. `src/mirror/static/watch.js` — handle the track's life: on
|
||||
`event.track.addEventListener("ended", …)` remove it from `tile.stream` and
|
||||
`addTrack` again is unnecessary; the `<video>` follows the live `MediaStream`.
|
||||
Update `tile.audioBtn.hidden` from `tile.stream.getAudioTracks().length` so a
|
||||
source that stops offering audio loses its button.
|
||||
3. `src/mirror/static/watch.js` — reset `tile.stream = null` in `closePeer()` (a
|
||||
retry builds a fresh connection and must not reattach dead tracks) and in
|
||||
`dropTile()`.
|
||||
4. `src/mirror/static/watch.js` — **do not** add `addTransceiver("audio", …)` on
|
||||
the viewer. The viewer answers, and the sharer owns the track set (PLAN A4);
|
||||
answerer-side transceivers reorder m-lines and would fight the locked decision.
|
||||
Leave a comment saying so at `ensurePeer()` — this is the first thing a future
|
||||
reader will be tempted to try.
|
||||
5. `src/mirror/static/watch.js` — keep `muted` as the element's initial state
|
||||
(it is set in `watch.html`'s tile template, which
|
||||
`tests/test_app.py::…test_videos_are_muted_and_inline_so_autoplay_is_not_blocked`
|
||||
guards); this task only fixes *composition*, playback policy is task 02.
|
||||
6. No server change: `registry.py` / `protocol.py` already carry `audio`
|
||||
(`protocol.py` L57 on `publish`, L118/129 into the stream payload;
|
||||
`registry.py` L197/222–228 stores it and decides `stream_updated`). Add a
|
||||
regression test instead of code (below).
|
||||
|
||||
ASSUMPTION: one `MediaStream` per tile, built by us, rather than
|
||||
`event.streams[0]` — Chrome and Firefox disagree about whether `addTransceiver`
|
||||
tracks share a stream id, and the app must behave identically on both (the kiosk
|
||||
is Firefox, the sharers are usually Chrome).
|
||||
ASSUMPTION: a track that ends is removed from the composed stream rather than
|
||||
tearing down the peer connection — the video leg keeps working when a tab's audio
|
||||
goes away, which is the current (correct) behaviour for video-only sources.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Wire-level regression in `tests/test_app.py::TestSignalingSocket` (next to
|
||||
`test_capture_metadata_survives_the_wire`): a `publish` with `audio: true`
|
||||
reaches the viewer as `stream.stream.audio === true`, and a republish with
|
||||
`audio: false` produces a `stream_updated` that flips it — the policy half is
|
||||
already tested, this pins it for the audio work.
|
||||
- E2E in `tests/e2e/test_audio_share.py` (new file, `pytestmark =
|
||||
pytest.mark.e2e`, reuse the `device` fixture from `test_live_mirror.py`):
|
||||
* assert the negotiated SDP agrees about audio —
|
||||
`await pc.localDescription.sdp` on the viewer contains `m=audio` **iff** the
|
||||
sharer published `audio: true`;
|
||||
* assert the composed stream: `video.srcObject.getVideoTracks().length === 1`
|
||||
and (when audio was offered) `getAudioTracks().length === 1` on the
|
||||
**viewer** — the direct regression test for the overwrite bug;
|
||||
* assert the video still decoded distinct frames after the audio track was
|
||||
added (`frames_matching`), so a "fixed" audio path cannot break the picture.
|
||||
- If this browser/device combination offers no display audio at all, `pytest.skip`
|
||||
with a message naming the reason — never assert on an absent track.
|
||||
- Coverage: **>90%** on this task's new/modified code.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `grep -n "new MediaStream(\[event.track\])" src/mirror/static/watch.js`
|
||||
returns nothing.
|
||||
- [ ] The viewer's `<video>` holds one `MediaStream` containing both kinds while
|
||||
decoded frames keep advancing (E2E).
|
||||
- [ ] The SDP assertion passes in both directions (audio offered / not offered).
|
||||
- [ ] `uv run pytest -m e2e` and `uv run pytest tests/test_app.py` green.
|
||||
- [ ] `uv run ruff check . && uv run ruff format --check . && uv run pyright`
|
||||
clean.
|
||||
@@ -0,0 +1,77 @@
|
||||
# Task 02 — Only the mirror you are watching can make sound
|
||||
|
||||
**Phase:** `03_audio_share` · **Source:** `TODO.md:5` — "Need audio share to work."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Make the audio that now exists actually hearable, with the policy the user chose:
|
||||
**the staged mirror is the only one that can be audible.** One deliberate gesture
|
||||
turns it on — a click or the `m` key — which also keeps `SECURITY.md` §4's promise
|
||||
that sound is always a user-initiated action.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/mirror/static/watch.js` — extend the `setMuted(tile, muted)` helper
|
||||
created in phase 01:
|
||||
* `muted === false` → first `stage(tile.id, { fullscreen: true })` if this tile
|
||||
is not already staged (a wall tile may not become audible in the grid), then
|
||||
mute **every other** tile through the same helper, then unmute this one;
|
||||
* `muted === true` → mute only this tile; never unstages as a side effect.
|
||||
2. `src/mirror/static/watch.js` — `unstage()` sets the departing tile muted, so
|
||||
walking back to the wall is always silent (the kiosk in a shared room depends
|
||||
on this).
|
||||
3. `src/mirror/static/watch.js` — `stage()` never changes mute state by itself:
|
||||
joining a mirror silently is the safe default and matches the element's `muted`
|
||||
attribute in `watch.html`.
|
||||
4. `src/mirror/static/watch.js` — make the state readable without hovering:
|
||||
set `tile.el.dataset.audio = "on" | "off" | "none"` in `paintTile()` (from
|
||||
`stream.audio`) and in `setMuted()`, so CSS can show it and a screenshot from
|
||||
another machine (`docs/kiosk-silverblue/dim-probe.py`) shows what the wall is
|
||||
doing.
|
||||
5. `src/mirror/static/app.css` — a small always-visible speaker mark for a staged
|
||||
tile whose audio is on (`.tile[data-audio="on"]`), and a static dot on grid
|
||||
tiles that have audio available. Reveal by `opacity`/content, never by
|
||||
`display` (`tests/test_app.py::TestStylesheet`), and keep
|
||||
`.icon-btn`'s existing `[hidden]` behaviour intact.
|
||||
6. `src/mirror/static/watch.js` — the audio button keeps calling the same helper;
|
||||
`aria-pressed`, `title` and the 🔊/🔇 glyph all come from `setMuted()` so the
|
||||
button, the `m` key and the policy can never disagree.
|
||||
7. `src/mirror/static/watch.js` — when the staged tile's audio disappears (task 01
|
||||
ends the track), mute it and toast `"This mirror stopped sharing audio."`
|
||||
|
||||
ASSUMPTION: unmuting a grid tile stages it instead of refusing (rather than
|
||||
playing sound out of a small tile). The staged-only rule was chosen to keep a
|
||||
20-tile wall from becoming 20 sound sources; staging is also what a person who
|
||||
wants audio of a mirror is about to do anyway.
|
||||
ASSUMPTION: audio is never restored automatically after `unstage()` or after a
|
||||
reconnect — every audible state must trace back to a gesture (`SECURITY.md` §4).
|
||||
ASSUMPTION: `tile.video.volume` stays at 1; system volume is the operator's job on
|
||||
a kiosk with no mixer.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- E2E in `tests/e2e/test_audio_share.py`: stage mirror A, press `m`, assert
|
||||
`#watch-stage video.muted === false`; stage mirror B, press `m`, assert A's
|
||||
`<video>.muted === true` and B's is `false` — the exclusivity policy is the
|
||||
behaviour under test, and it is asserted on the two viewer elements.
|
||||
- E2E: `unstage()` (via `g` or the Exit button) leaves the tile muted, and a
|
||||
subsequent `stage()` does not resurrect sound on its own.
|
||||
- E2E: clicking a grid tile's 🔇 button stages it and unmutes it (the mouse path
|
||||
must obey the same rule as the keyboard).
|
||||
- E2E: `dataset.audio` reflects reality for a silent source (`"none"`), a staged
|
||||
silent-but-available source (`"off"`) and an audible one (`"on"`).
|
||||
- `uv run pytest tests/test_app.py` — element-id and stylesheet guards stay green
|
||||
with the new CSS and the `data-audio` attribute.
|
||||
- Coverage: **>90%** on this task's new/modified code.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] Exactly one mirror can ever be unmuted, enforced through `setMuted()` and
|
||||
verified with two live mirrors in a browser test.
|
||||
- [ ] `unstage()` always leaves the wall silent.
|
||||
- [ ] The audio state is legible without a pointer (`data-audio` + the CSS mark).
|
||||
- [ ] Nothing unmutes without a click or a keypress — asserted by staging and
|
||||
waiting, then checking `muted` is still `true`.
|
||||
- [ ] `uv run pytest -m e2e` green; `uv run ruff check . && uv run ruff format
|
||||
--check . && uv run pyright` clean.
|
||||
@@ -0,0 +1,88 @@
|
||||
# Task 03 — Say honestly whether audio exists
|
||||
|
||||
**Phase:** `03_audio_share` · **Source:** `TODO.md:5` — "Need audio share to work."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
The sharer page currently promises nothing and hides everything: `pickSurface()`
|
||||
retries without audio on *any* failure except `NotAllowedError`, and `publish()`
|
||||
samples `getAudioTracks().length` exactly once. Make the capture request precise,
|
||||
the panel truthful about which of the three states it is in, and the roster current
|
||||
when audio appears or disappears mid-share.
|
||||
|
||||
## Work
|
||||
|
||||
1. `src/mirror/static/share.js` — `pickSurface()`: narrow the fallback. Retry with
|
||||
`audio: false` only for the errors that genuinely mean "this source has no
|
||||
audio": `NotSupportedError`, `OverconstrainedError`, `NotFoundError`, `TypeError`
|
||||
(and a `DOMException` whose message mentions audio). **Rethrow
|
||||
`NotAllowedError`** — in Chrome that is what a user who unticked "Share tab
|
||||
audio" or pressed *Cancel* produces, and retrying would silently re-open the
|
||||
picker and share a screen the person meant not to share. Comment that reason at
|
||||
the `throw`.
|
||||
2. `src/mirror/static/share.js` — remember which attempt succeeded in a local
|
||||
variable (`audioRequested`, `audioAcceptedByBrowser`) so the panel can tell
|
||||
"the browser refused to offer audio" apart from "the source has none".
|
||||
3. `src/mirror/static/share.js` — `#stat-audio` (`ui.audio`) gets three states:
|
||||
`shared` (an audio track exists), `silent` (source offered none), `not offered`
|
||||
(the retry had to drop audio to capture at all). Keep the existing `—` for the
|
||||
idle state set by `renderIdle()`.
|
||||
4. `src/mirror/static/share.js` — track the audio track's life the way the video
|
||||
track's already is: on the audio track's `"ended"` listener call `publish()` so
|
||||
the `audio` flag in the roster flips and every viewer's tile updates (phase 03
|
||||
task 01 relies on this to hide a dead 🔇 button); on `mute`/`unmute` the
|
||||
existing `republish` already fires for the video — extend it to also refresh
|
||||
`#stat-audio`.
|
||||
5. `src/mirror/static/share.js` — `publish()` reads
|
||||
`state.stream.getAudioTracks().length > 0` at call time (it already does);
|
||||
confirm the value is recomputed on *every* publish and never cached in `state`,
|
||||
so there is exactly one source of truth for "is there audio".
|
||||
6. **No new wire field.** The three-way distinction stays in the sharer's panel;
|
||||
the roster keeps the single `audio: bool` it already has (`protocol.py` L57,
|
||||
`registry.py` L197). Adding a multi-word field would require a camelCase alias
|
||||
test (`AGENTS.md` §3) for a distinction the viewers do not need — viewers need
|
||||
"is there audio right now", which the boolean already answers via
|
||||
`stream_updated`.
|
||||
|
||||
ASSUMPTION: rethrowing `NotAllowedError` instead of falling back silently, even
|
||||
though it turns some "no audio" sources into a visible failure — a screen share
|
||||
the user did not consciously approve is worse than a share that did not start, and
|
||||
the toast tells them to pick again without audio.
|
||||
ASSUMPTION: `AudioContext`-based mixing (routing system audio through Web Audio to
|
||||
guarantee a track) is out of scope: it needs a user gesture to unlock, resamples
|
||||
the stream, and buys nothing on the platforms where Chrome already offers the
|
||||
track. Revisit only if a real deployment needs it.
|
||||
ASSUMPTION: no new environment variable gating audio (`MIRROR_DISABLE_AUDIO`); the
|
||||
browser's own picker is the consent mechanism and a second switch would only
|
||||
confuse it.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- Wire test in `tests/test_app.py::TestSignalingSocket`: publish `audio: true` →
|
||||
`unpublish`/republish `audio: false` produces `stream_updated` with
|
||||
`audio: false` for every subscribed viewer, and the empty-wall case is unaffected.
|
||||
- Guard in `tests/test_app.py::TestClientScripts` (regex over `share.js`, the
|
||||
existing style): `pickSurface` must rethrow `NotAllowedError` — assert the
|
||||
identifier appears in the catch block and that the audio-off retry is not
|
||||
unconditional (no bare `return await navigator.mediaDevices.getDisplayMedia`
|
||||
inside a catch-all).
|
||||
- E2E in `tests/e2e/test_audio_share.py`: with the fake device, assert `#stat-audio`
|
||||
is one of the three documented strings and that the tile's 🔇 button visibility
|
||||
agrees with `#stat-audio === "shared"` — the panel and the wall cannot disagree.
|
||||
- E2E: with a source that offers no audio, `/watch` shows no audio button at all
|
||||
(`paintTile` hides it) rather than a dead one.
|
||||
- Coverage: **>90%** on this task's new/modified code.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] `pickSurface()` retries without audio only for the named error names, and a
|
||||
cancelled picker still surfaces "Sharing was cancelled or blocked."
|
||||
- [ ] `#stat-audio` shows `shared` / `silent` / `not offered` correctly for all
|
||||
three real cases on the reference Linux host.
|
||||
- [ ] Ending the audio source mid-share updates `#stat-audio` and every viewer's
|
||||
tile within one `stream_updated` (E2E).
|
||||
- [ ] No new field on the wire; `uv run pytest tests/test_protocol.py
|
||||
tests/test_registry.py tests/test_app.py` green unchanged.
|
||||
- [ ] `uv run ruff check . && uv run ruff format --check . && uv run pyright`
|
||||
clean.
|
||||
@@ -0,0 +1,88 @@
|
||||
# Task 04 — Prove the sound crossed, and tell users what a browser will capture
|
||||
|
||||
**Phase:** `03_audio_share` · **Source:** `TODO.md:5` — "Need audio share to work."
|
||||
**Story:** n/a
|
||||
|
||||
## Objective
|
||||
|
||||
Two deliverables that belong together: a viewer-side assertion that audio really
|
||||
travelled over WebRTC, and documentation that tells a Linux user what the browser
|
||||
will and will not hand mirror — the part that made "audio doesn't work" feel
|
||||
undiagnosable.
|
||||
|
||||
## Work
|
||||
|
||||
1. `tests/e2e/test_audio_share.py` — the hard assertion, on the **viewer**:
|
||||
* after the sharer publishes with audio and the viewer unmutes the staged tile,
|
||||
poll `pc.getStats()` and require an `inbound-rtp` entry with
|
||||
`kind === "audio"` whose `bytesReceived` strictly increases across two samples;
|
||||
* where the browser reports it, also require `totalAudioEnergy > 0` or
|
||||
`audioLevel > 0` after the unmute (a fake device produces a tone, so energy
|
||||
exists when the track exists);
|
||||
* keep the existing content assertion (`frames_matching`) in the same test:
|
||||
audio arriving while the picture froze would be a different bug, and this
|
||||
phase must not create it.
|
||||
2. `tests/e2e/test_audio_share.py` — honest skipping, loud reporting: if the
|
||||
browser offers no display-audio track in this environment, `pytest.skip()` with
|
||||
a message naming the platform and the flags used. Never assert on a track that
|
||||
is not there, never `xfail` a real failure into green.
|
||||
3. `tests/e2e/test_x11_capture.py` — the real-desktop counterpart in the
|
||||
`realdisplay` suite (private Xvfb, headed Chromium, `browser_env()` stripping
|
||||
`WAYLAND_DISPLAY`/`DBUS_SESSION_BUS_ADDRESS`, **no**
|
||||
`--use-fake-ui-for-media-stream`): if the host exposes an audio source, assert
|
||||
the inbound audio bytes there too; otherwise record the skip reason in the
|
||||
module docstring so the gap is written down rather than imagined away.
|
||||
4. `tests/e2e/conftest.py` — do **not** add `--autoplay-policy=no-user-gesture-required`
|
||||
to any audio-muting test; it is already in `CHROMIUM_ARGS`, which is why those
|
||||
tests assert mirror's own `muted` state and never claim to prove the browser's
|
||||
autoplay rule. Say that in a comment next to the existing flag so nobody
|
||||
"fixes" it later.
|
||||
5. `SECURITY.md` §4 "Audio" — rewrite for what shipped: the picker offers audio and
|
||||
the app accepts it when the source has any; only the **staged** mirror can be
|
||||
audible; unmuting is always a click or a keypress; unstaging re-mutes; a mirror
|
||||
that stops offering audio says so instead of keeping a dead control. Keep the
|
||||
one-line statement that audio is user-initiated by design.
|
||||
6. `SECURITY.md` §4 — add the platform reality, stated plainly: Chrome on Linux
|
||||
captures **tab** audio reliably and **whole-screen/system** audio only where the
|
||||
platform (PipeWire/PulseAudio, and the picker's "share audio" tickbox) offers
|
||||
it; Firefox's desktop capture does not offer system audio at all on Linux.
|
||||
mirror neither works around nor fakes it — it reports `silent` / `not offered`
|
||||
(`03_audio_source_reporting.md`).
|
||||
7. `README.md` — "Platform notes" (~L73): a short *Audio* entry pointing at the
|
||||
`SECURITY.md` section, plus the operator-facing line "press `m` on the wall to
|
||||
hear the staged mirror".
|
||||
8. `docs/kiosk-silverblue.md` — one line for the wall box: the kiosk is silent
|
||||
until someone presses `m`, and audio depends on what that browser is given.
|
||||
|
||||
ASSUMPTION: audio assertions are environment-gated (`skip`, with a reason) in the
|
||||
fake-device and `realdisplay` suites rather than faked with a stubbed
|
||||
`RTCPeerConnection` — a stubbed peer connection proves the wiring and nothing else,
|
||||
and `AGENTS.md` §5 forbids "unit tests pretending to be WebRTC".
|
||||
ASSUMPTION: documentation names Firefox's Linux limitation instead of trying to
|
||||
detect and paper over it in JS, because the kiosk fleet is Firefox and an operator
|
||||
reading `SECURITY.md` needs to know the wall will be silent by the browser's rule,
|
||||
not mirror's.
|
||||
|
||||
## Testing & Quality
|
||||
|
||||
- `uv run pytest -m e2e` — the audio test either proves inbound audio bytes
|
||||
increased or skips with a named reason; the rest of the suite is unaffected.
|
||||
- `MIRROR_REALDISPLAY=1 uv run pytest -m realdisplay` — green, with the audio
|
||||
result (pass or documented skip) written into the commit message.
|
||||
- `uv run pytest tests/test_app.py` — the docs edits must not disturb the page /
|
||||
asset guards; no new asset, no CDN (`test_nothing_is_loaded_from_a_cdn`).
|
||||
- Coverage: **>90%** on new/modified code; this task's production-code delta is
|
||||
comments only, so the bar is met by the suites above.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
- [ ] A viewer-side assertion proves inbound audio bytes increase once the staged
|
||||
mirror is unmuted — or the test skips with a message naming the exact reason.
|
||||
- [ ] The picture still decodes distinct frames in the same test.
|
||||
- [ ] `SECURITY.md` §4 "Audio", README's platform notes and
|
||||
`docs/kiosk-silverblue.md` describe the shipped behaviour, including the
|
||||
Linux system-audio limitation and the `m` key.
|
||||
- [ ] Full gate: `uv run ruff check . && uv run ruff format --check . && uv run
|
||||
pyright && uv run pytest && MIRROR_REALDISPLAY=1 uv run pytest -m realdisplay`.
|
||||
- [ ] No change in completed phases: keyboard navigation, staging, fan-out,
|
||||
teardown and the address gate all behave as before.
|
||||
Reference in New Issue
Block a user