The four open items in TODO.md become an executable pipeline: 01 the verification gate (host suites + sanitiser + both firmware envs, and the hardware contract so no unit blocks on a board nobody is holding), 02 HTTPS and wss:// with an identity the board generates itself into NVS, 03 session takeover that always lands disarmed, 04 a responsive page whose safety statements are pinned before they are trimmed, 05 the phone as a controller (press-and-hold pad, soft keyboard, modifier latches over a new Q= frame). TODO.md is cleared to its empty title; the items live in .agents/phases/ now. AGENTS.md gains the five protocol rules and nothing else. LLM-Generated: true
6.2 KiB
Phase 05 — The phone is a controller: soft keyboard and touch pointer
Source: TODO.md L5 — "Needs to be mobile compatible - click arm and then mobile
keyboard pops up. Mobile trackpad for touch trackpad."
Story: n/a
Context: src/web_page.h (the mouse block, takeMouse()/flushMouse()/dropMouse(),
inPageField(), setArmed()), src/text.h (strokeFor()), src/proto.h, src/mouse.h,
src/config.h, tests/test_page.js, README "Controller page" / "Safety model"
Objective
Arm the bridge on a phone and the soft keyboard appears so you can actually type into the target; press and hold the pad and the target's pointer moves, let go and every button is up. Two capabilities the desktop gets from hardware — a physical keyboard and Pointer Lock — get honest software equivalents, with the same guarantees restated in touch terms rather than waived.
The two equivalences, stated up front
| desktop guarantee | touch equivalent |
|---|---|
Pointer Lock: the cursor cannot escape the tab, and Esc always frees it |
finger down on the pad is the capture, finger up is the release — movement stops and every button lets go the instant the touch ends, and the Release button is 44 px and on screen. The capture cannot outlive the finger that asked for it. |
| Modifiers are physical keys held with a second hand | an on-screen latch row whose state is lit, restated in every frame, and cleared by a tap — never a sticky Ctrl that silently modifies the next six keys. |
Both are more conservative than the desktop case, not less: a touch capture ends when you stop touching, which is stronger than a lock you have to remember to escape.
What must not be traded away
- Movement is relative, one report per frame, deltas capped at ±127 and split so the net displacement survives (invariant 7: splitting is the controller's job).
- No auto-click, no gesture, no macro, nothing scheduled. The button row is held-while-pressed.
- Capture requires armed and the board reporting
mouse:1, andE10/E11still refuse rather than ignore. - Characters are typed by the board's own map. The page must not grow a second
character→usage table: the board owns
strokeFor(), which is why the newQ=frame carries modifiers and hex bytes, not usages. (AGENTS.md: when you add a table that decodes something somebody else defined, test it — better still, do not add it.) - Anything a touch session can leave behind (a latch, a held button, a focused field) dies in the same funnel: disarm, blur, visibility change, link loss.
Dependencies
04_controller_page_responsive(todo) — the single-column layout, 44 px targets andtouch-action:manipulationare what make a touch pad usable; the folded prose is what makes room for the latch row and the button row.03_session_takeover(todo) —E12handling and thebusypill, because a phone is exactly the device that finds the laptop holding the bridge.
Tasks
01_touch_trackpad.md— the pad: press-and-hold capture, relative movement fromtouchmove, two-finger drag as the wheel, an explicit held-while-pressed button row, and release ontouchend/touchcancel/blur/disarm.02_soft_keyboard_capture.md— ARM focuses a forwarding field so the soft keyboard pops; printable characters leave asP=0:<hex>one per frame, named keys (Enter, Backspace, Tab, arrows…) as a press/release pair; the field is cleared as soon as it is read.03_modifier_latch_q_frame.md— the latch row and theQ=<mods>:<hex>frame that lets a latched modifier type a character without the page owning a second character map. A real four-place protocol change:proto.h,web_page.h,relayctl.py, README.04_mobile_tests_docs.md—test_page.jstouch/latch coverage, the named-key table pinned, README's Controller page and Safety model rewritten for touch,AGENTS.md's "three tables" row, and the phone observation recorded.
Testing & Quality
tests/test_page.js— the shim already records document/window listeners and canfire()them; extend it to dispatchtouchstart/touchmove/touchend/touchcancelwith multiple touch points andbeforeinputwithinputType, then assert: capture requires armed andmouse:1; one frame per report with nothing sent when idle; a 400×300-style drag split so no delta exceeds a signed byte and the net displacement is exact; two-finger drag becomes wheel/pan not movement; a finger up releases every button and stops movement; the button row's bits matchMOUSE_BTN; the latch row's byte matches the modifier names; a character becomes exactly oneP=0:frame and a latched one becomes oneQ=frame;E9/E10/E11land in the log and cancel nothing they should not. Every frame the page emits is parsed by the firmware's own parser — the suite rule that has already caught mislabelled chords.tests/test_proto.cpp:Q=parses, bounds-checks (mods byte, cap on hex length), refuses malformed input, round-trips.tests/test_relayctl.py: the tool buildsQ=identically, and the button/name tables andMOUSE_BUTTONSagreement extends to whatever the touch rows introduce.- Coverage >90% on the changed page script and the new parser path.
- Device (best effort, one attempt): a phone on the board's AP — arm, keyboard appears, type a
word and
watch_target.pyshows the characters; hold the pad, move, lift, and the target shows motion that stops and no button stuck.
Completion Criteria
- Arm on a phone brings up the soft keyboard and typed characters land on the target.
- Hold the pad → the pointer moves; lift → motion stops and every button is up, observed
with
watch_target.pyor recorded asskipped (no device)with the exact commands. Ctrl+C-equivalent reachable with the latch row and lands as Ctrl+C on the target.Q=in the README wire table,proto.h,web_page.handrelayctl.py, with the tests that pin all four agreeing.- No second character→usage table anywhere; the named-key table that does exist is pinned.
bash .agents/validate.shgreen; README andAGENTS.mddescribe the touch model.