diff --git a/AGENTS.md b/AGENTS.md index 92d5042..026ccae 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,7 +10,8 @@ The README is the single source of truth for behaviour, protocol, safety and mea This file is the operating manual: what must not break, how to build and verify, and where the traps are. **Do not fork the protocol or the safety model into `docs/`** — a second copy drifts, and two paths that can disagree about arming is the exact failure this project keeps -having to fix (see README, "Nine bugs"). +having to fix (see README, "Ten bugs"). + ## The invariants @@ -20,9 +21,20 @@ having to fix (see README, "Nine bugs"). stuck-key horror story in HID injection tooling is an event-protocol bug. 2. **Disarmed is the only default.** Boot, reload, reconnect, focus loss, socket loss, target detach, bus suspend, queue overflow and 8 s of controller silence all disarm, and disarming - releases every key. Arming needs the passphrase, the board button, or the serial console — - nothing else may gain the ability. -3. **Every fail-safe cancels work in flight; nothing "pauses".** `releaseEverything()` in + releases every key **and every mouse button**. Arming needs the passphrase, the board + button, or the serial console — nothing else may gain the ability. The mouse is captured, + never grabbed: Pointer Lock, so `Esc` (the browser's own, never forwarded) always frees + it. `mouse off` over serial removes the pointer interface at the descriptor level, which + needs a reboot and is the rollback. +3. **Every fail-safe cancels work in flight; nothing "pauses"** — and **no fail-safe may be + gated on the absence of the thing it detects.** `ControlServer::linked()` is false when a + controller is gone, so gating the silence-disarm on it made the backstop for a vanished + controller uncallable exactly when a controller vanished. Same shape: `closeLink()` + releasing only `if (linked())` skipped the release for every path that cleared `_authed` + first. If a rule fires when X stops existing, `X still exists` cannot be a precondition + of it. See README bug 10; `test_relayctl.py` pins the three sentences. + + `releaseEverything()` in `main.cpp` is the single funnel: it clears the report queue, raises the keys, ends the chord and discards a pasted run. Any new state that can outlive a disconnect (a queue, a timer, a run) must be cancelled there too, or it will keep acting after the operator stopped asking. @@ -47,6 +59,14 @@ having to fix (see README, "Nine bugs"). | `CODE_TO_USAGE` | `src/web_page.h` | physical position → usage | `tests/test_relayctl.py` | | `CODE_TO_USAGE` | `tools/relayctl.py` | same, for the terminal | `tests/test_relayctl.py` | | `strokeFor()` | `src/text.h` | character → modifier + usage | `tests/test_text.cpp` (whole printable range walked) + tool/board agreement on all 128 ASCII codes | +| mouse button bits | `src/mouse.h`, `src/web_page.h` (`MOUSE_BTN`), `tools/relayctl.py` (`MOUSE_BUTTONS`) | HID bit → DOM `MouseEvent.button` **and** name | `test_proto.cpp` (page↔config), `test_relayctl.py` (tool↔page↔config) | +| evdev axis codes | `tools/watch_target.py` (`REL_*`) | kernel code → label, for the *verifier* | `test_relayctl.py` — pinned to ``, **not** to the report's byte order | + +The last row exists because it was guessed from the HID report's field order and was wrong +(`REL_WHEEL` is 8, not 2): a working wheel looked dead and the summary said no scroll +happened. Codes the verifier has no name for are now counted and printed rather than +dropped — the same fix as giving F13–F24 names. **When you add a table that decodes +something somebody else defined, test the table, not your confidence in it.** Chord buttons carry their **wire bytes** in `data-chord`, not a name — which is why `tests/test_proto.cpp` asserts every labelled chord against the combination its label promises. @@ -58,18 +78,26 @@ the page and `TEXT_FRAME_MAX` / `MAX_PASTE_CHARS` in `relayctl.py` are hand-copi ## Build, test, flash ```bash -tests/run_tests.sh # 955 checks, no hardware +tests/run_tests.sh # 3406 checks, no hardware SANITIZE=1 tests/run_tests.sh # + ASan/UBSan — use it for anything touching buffers -/tmp/piovenv/bin/python -m platformio run -e bridge -e console +# PlatformIO. The venv below runs Python 3.15, which esptool REFUSES ("Python version must +# be 3.10..3.14") -- builds fine, dies at upload. Use uv's 3.14 for anything with -t upload: +uv run --python 3.14 --with platformio python -m platformio run -e bridge -e console python3 tools/to_bootloader.py # 1200-baud touch → ROM download mode -/tmp/piovenv/bin/python -m platformio run -e bridge -t upload +uv run --python 3.14 --with platformio python -m platformio run -e bridge -t upload ``` Environment facts that will otherwise cost you a cycle: -- **`pio` on PATH is broken** here (`ModuleNotFoundError: platformio`). Use the venv above. If - it is gone (`/tmp`), find another interpreter with the package: +- **`pio` on PATH is broken** here (`ModuleNotFoundError: platformio`), and + `/tmp/piovenv` is Python 3.15 so it can *build* but not *upload*. Use the uv invocation + above for both. If it is unavailable, find another interpreter with the package: `find / -maxdepth 6 -name platformio -type d -path '*site-packages*'`. +- **`pio run -t upload` reports SUCCESS on a stale binary if the build before it failed.** + This happened: a diagnostic build failed to compile, the upload line right after it said + SUCCESS, and the board was running the previous firmware. Never chain build and upload + with `;` — check the build's own exit status first, or the verification you run next is + verifying something that is not in the tree. - **No `g++` on the host.** `run_tests.sh` falls back to a local podman image listed in its `images[]`. The page suite deliberately runs on the *host* so it works without node in there. - **Build `console` too.** `ARDUINO_USB_MODE=1` compiles `KR_HAVE_HID == 0`, which is the only @@ -78,8 +106,18 @@ Environment facts that will otherwise cost you a cycle: until the board is reflashed — and a tab that is already open keeps the old page in memory until it is reloaded. Confirm with `curl -s http://keyrelay.local/ | wc -c` against the size of the raw string in `src/web_page.h`, and grep the served bytes for the element you added. -- This directory is **not a git repository**, so there is no diff or revert here. Back up - before sweeping edits. +- **This directory is a git repository** (one commit, `Initial import`). Use it: `git diff` + before and after any sweeping edit. Note that the commit predates the mouse and + `output.h` work, so `git checkout` reverts far more than you asked for — restore a + *region*, not a file. +- **Never do a multi-line `.*?` regex rewrite across a source file.** One did + `#if KR_DEBUG_LINK.*?#endif` with `DOTALL` to remove a diagnostic block and deleted ~500 + lines of `main.cpp` — `onMessage()`, `setup()`, the console, the USB handlers — because + the earliest `#if` in the file was elsewhere. The compiler was the only thing that + noticed. Remove instrumentation with exact-string edits, or per-function slices, and + compile immediately after. When it happens anyway: the last good `firmware.elf` still + has the `.rodata` strings (console text, `printf` formats) and the symbol table, which is + enough to rebuild faithfully against the surviving headers. ## Verification standard @@ -119,7 +157,7 @@ wired now — but ask, of every green check, *what would still pass if this were echo it; keep it that way. `/status` takes the token as a query parameter because `curl` is the point of that endpoint — do not add the token to any other URL. - `TODO.md` holds **open items only**; clear it when they are done. The durable record is the - README: a fixed bug belongs in "Nine bugs this verification caught" with its root cause and + README: a fixed bug belongs in "Ten bugs this verification caught" with its root cause and how it was proven, a new capability in the section that describes it, and a rule worth repeating here. A TODO that has accumulated resolved items tells the next reader nothing about what is left to do. diff --git a/README.md b/README.md index e5e7325..b17c07e 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,22 @@ -# keyrelay — a WiFi-remote USB keyboard on the M5Stack AtomS3 Lite +# keyrelay — a WiFi-remote USB keyboard and mouse on the M5Stack AtomS3 Lite Type on your laptop; the keystrokes land on a *different* machine, over the air, as -genuine USB HID input — including Enter and Ctrl+Alt+Del. The board enumerates on the -target as a plain boot-protocol keyboard, so the target needs no software, no driver -and no network. The controller is a web page served by the board itself. +genuine USB HID input — including Enter and Ctrl+Alt+Del. Capture the pointer and your +mouse moves the other machine's cursor. The board enumerates on the target as a +boot-protocol keyboard with a relative mouse on the same HID interface, so the target +needs no software, no driver and no network. The controller is a web page served by the +board itself. ``` your laptop target machine ┌──────────────────┐ ┌──────────────────┐ │ browser tab │ WiFi │ │ │ keydown/keyup ├────WebSocket──►│ AtomS3 Lite │ - │ of the page │ (token) │ keyrelay │ - └──────────────────┘ │ │ - └────────┬─────────┘ - │ USB HID boot keyboard + │ mousemove │ (token) │ keyrelay │ + │ (Pointer Lock) │ │ │ + └──────────────────┘ └────────┬─────────┘ + │ USB HID: boot keyboard + │ + relative mouse ▼ whatever has keyboard focus: a login prompt, a BIOS setup, @@ -21,21 +24,33 @@ and no network. The controller is a web page served by the board itself. ``` ``` -disarmed LED dark nothing you type goes anywhere +disarmed LED dark nothing you type or move goes anywhere armed LED solid red every keystroke in the tab goes to the target ``` +The mouse is opt-in twice over: the bridge has to be armed *and* the page has to be +handed the pointer, which the browser only does when you click. `Esc` gives it back — +that one is the browser's own guarantee, not something this firmware has to honour. + * **Nothing types until you arm it.** Boot, reload, reconnect, focus loss, socket loss - and 8 s of silence all disarm, and disarming sends "all keys up". + and 8 s of silence all disarm, and disarming sends "all keys up" **and "all mouse + buttons up"** — to the target, as reports, not just to the board's own bookkeeping. * **Arming needs a passphrase** (or a thumb on the board's button). The LED is readable from across the room, so the state is not a secret between you and the screen. +* **The mouse is captured, not grabbed.** The page asks the browser for Pointer Lock — + the same thing a web game asks for — so the cursor cannot escape the tab while it is + being forwarded, and `Esc` always returns it. Losing the lock, the tab or the link + releases every button. Movement is relative: the board says how far, never where, so + there is no position for the two machines to disagree about. * **The keystrokes are not stored, logged or replayable.** The board keeps counters and - the live key state, and nothing else — no sequence storage, no macro, no script, nothing - it can type later. Pasted text is typed from RAM and forgotten; the serial console has no - command that types anything at all. It is a keyboard, not a payload dropper. -* **955 checks** cover the key-state machine, the wire protocol, the text→keystroke map, - SHA-1/base64, the WebSocket codec and the controller page's own script before anything is - flashed, and the target-side verifier proves what actually arrived. + the live key and button state, and nothing else — no sequence storage, no macro, no + script, nothing it can type later, no gesture it can replay. Pasted text is typed from + RAM and forgotten; the serial console has no command that types or clicks anything at + all. It is a keyboard and a mouse, not a payload dropper. +* **3406 checks** cover the key-state machine, the mouse state, the report queues and the + release funnel, the wire protocol, the text→keystroke map, SHA-1/base64, the WebSocket + codec, the controller page's own script, the CLI's frames and the target-side verifier + itself — all before anything is flashed. --- @@ -73,13 +88,21 @@ armed LED solid red every keystroke in the tab goes to the target whose keyboard you would rather not wear out. - Keeping an agent session live on a machine you are not typing into (that is the sibling project [`../jiggler`](../jiggler), which is a simpler build of the same board). -- Anything where a KVM-over-IP would be right and you do not need video, mouse or - audio, and you have 8 dollars and a USB port to spare. +- Pointing at something on a machine you cannot reach — a BIOS menu item, a dialog in a + installer, a button in a kiosk app — with the cursor you can already see on that + machine's own monitor. +- Anything where a KVM-over-IP would be right and you do not need video or audio, and you + have 8 dollars and a USB port to spare. **Bad fits — use something else** -- **You need the screen.** There is no video path. PiKVM or TinyPilot - do video + keyboard + mouse; this is keyboard only. +- **You need the screen.** There is no video path. You steer the target's cursor without + being able to see it, which is fine for a monitor you are looking at and hopeless for one + you are not. PiKVM or TinyPilot do video + keyboard + mouse. +- **You need to know where the cursor is.** Movement is relative — see + [Layout caveats](#layout-caveats). There is no absolute pointer, so "click the OK + button" is not a thing this can express, deliberately: an absolute pointer would give + the board a position to be wrong about. - **You want unattended automation.** Deliberately impossible here (see above). If you want scripted input on a machine you own, `xdotool`/`ydotool`/`uinput` on that machine is the honest tool, and a Rubber Ducky is what this design is *not*. @@ -105,6 +128,16 @@ focus loss, a silent controller or a detached target cancels the rest of a run m What stays absent is the part that would change what this device *is* — something the board remembers, assembles on its own, or fires without a human in the loop. +**The mouse is the same line, drawn in the same place.** There is no auto-click, no +double-click macro, no drag recorded and replayed, no "move to (x, y)", no idle wiggle +that keeps a screen unlocked, and nothing that moves the pointer because a timer said so. +`relayctl mouse click` and `mouse drag` exist because verifying that the button bits +arrive is a thing the operator has to be able to do, and each needs `--yes`; a movement +(`mouse move`) does not, because moving a cursor is reversible by definition. If nobody is +moving a mouse, nothing moves. The pointer is captured with the browser's own Pointer Lock, +so `Esc` — which the browser keeps for itself and never forwards — is an escape hatch that +does not depend on the firmware, the network or this page being correct. + Two consequences hold either way. The CLI's ordinary commands still press one key per invocation, which is all verification needs; scripting 30 of them is possible and would still be 30 deliberate actions with the LED on. And a captured board is a keyboard, not a @@ -120,7 +153,7 @@ connector it already uses. ```bash cd keyrelay -tests/run_tests.sh # 955 checks, no hardware needed +tests/run_tests.sh # 3406 checks, no hardware needed pio run -e bridge # ~1.1 MB image, ~35% of the 3 MB app slot python3 tools/to_bootloader.py # 1200-baud touch -> ROM download mode pio run -e bridge -t upload # flash @@ -194,34 +227,63 @@ Enforced in this order, each independent of the others: 3. **Armed is displayed on the device.** Solid red LED while armed and attached. If the target is not attached the LED blinks red instead, because "armed" without a target means keystrokes are being dropped, and that deserves to look wrong. -4. **Silence disarms.** No inbound frame for `kNoClientTimeoutMs` (8 s) → all keys up, - disarmed. Covers the laptop sleeping, the browser being killed, the WiFi dying, the - network going down — every way a controller can vanish without saying goodbye. The - browser sends a keepalive every 2 s and answers the board's own 15 s WebSocket pings, +4. **Silence disarms.** No inbound frame for `kNoClientTimeoutMs` (8 s) → all keys up and + all buttons up, disarmed. Covers the laptop sleeping, the browser being killed, the WiFi + dying, the network going down — every way a controller can vanish without saying goodbye. + The browser sends a keepalive every 2 s and answers the board's own 15 s WebSocket pings, so a live tab holds it open indefinitely; `relayctl` does the same while it types. This - is checked in `loop()` against `ControlServer::lastInboundMs()`, and it is checked - *before* the typewriter is driven: it is the one rule that stops a run of pasted text - when the controller dies without a parting packet. + is checked in `loop()`, and it is checked *before* the typewriter is driven: it is the + one rule that stops a run of pasted text when the controller dies without a parting + packet. + + **What counts as silence, and what does not.** The clock is moved only by the + *authenticated controller*: by its authentication, and by bytes on its WebSocket. An + accepted TCP connection is not a live operator, and neither is anything reading the + board — a monitoring script, a health check, a second tab pointed at `/status`, or + `relayctl listen` used to hold this timer at zero and keep the bridge armed + indefinitely with the last key still pressed (bug 10). A board armed by its own button + with no controller at all is deliberately left running: it has no controller to go + quiet. This backstop is not gated on the link still existing, and must never be — see + [Ten bugs](#ten-bugs-this-verification-caught-all-fixed). 5. **Focus loss releases every key.** The tab blurring or being hidden sends "all keys up" and disarms (unless you tick *stay armed when this tab loses focus*, which still releases the keys — you cannot keep a held Ctrl alive by tabbing away). 6. **The target taking the bus away disarms.** USB detach or host suspend clears the report queue, releases the keys and disarms; `ARDUINO_USB_*` events drive it. -7. **A full queue is a stop, not a drop.** 48 queued state changes without a drain means - something is outrunning the USB path, so the bridge releases everything, reports `E7` - and disarms rather than typing out of step with what the operator sees. +7. **A full queue is a stop, not a drop.** 48 queued key states or 32 queued mouse + reports without a drain means something is outrunning the USB path, so the bridge + releases everything, reports `E7` and disarms rather than typing out of step with what + the operator sees. The queue is not a buffer to absorb loss: 255 queued reports once + sat through a disconnect, which is ~2 s of keystrokes delivered after the operator had + stopped asking. 8. **State, not events.** The controller restates the *whole* key state on every event (see [Wire protocol](#wire-protocol)), so a lost frame cannot leave a modifier stuck on the target. This is the single most important design decision in the project: every - "stuck Ctrl" horror story in HID injection tooling is an event-protocol bug. + "stuck Ctrl" horror story in HID injection tooling is an event-protocol bug. Mouse + buttons follow the same rule — every `M=` frame carries the complete button byte — and + movement is the one thing that is *not* state because a delta has no state to lose: a + dropped movement frame moves the cursor slightly less, never leaves anything stuck. 9. **A paste cancels; it never pauses.** `R` means "stop typing", not "hold on". The run is dropped on release-all, disarm, target detach or suspend, queue overflow, focus loss and silence — every fail-safe above routes through the same `releaseEverything()` - that clears the queue, raises the keys and discards what was still to type. The board - never holds a run that outlives the decision to send it. + that clears the queue, raises the keys, lets go of the mouse buttons and discards what + was still to type. The board never holds a run that outlives the decision to send it. +10. **Losing the pointer releases the pointer.** Pointer Lock going away, the tab + blurring, the tab being hidden, disarm, `R`, silence and target detach all send an + all-buttons-up mouse report. *stay armed when this tab loses focus* exempts the + keyboard only — a held mouse button is never something to keep across a tab switch. + The page stops capturing rather than queueing movement to send later. -The physical kill switch is the button: press it and the keys come up, no network, no -passphrase, no software required. +The physical kill switch is the button: press it and the keys come up and the buttons come +up, no network, no passphrase, no software required. `Esc` is a second one for the pointer +specifically, and it is the browser's, not ours. + +**One release funnel, and it has to actually send.** Every item above ends in +`releaseEverything()`, which now queues an all-keys-up *and* an all-buttons-up report to +the target rather than only clearing the board's own idea of what is down. That is the +fixed ninth bug: a held key stayed held on the target after silence-disarm, target loss or +a reload, because the report that would have released it was never queued (see +[Measured behaviour](#ten-bugs-this-verification-caught-all-fixed)). --- @@ -229,8 +291,9 @@ passphrase, no software required. | actor | can | cannot | |---|---|---| -| Anyone on the target (its own OS) | see a Logitech G815 keyboard in its device list; read the CDC console; type on itself | arm or drive the bridge remotely, detect that a second controller exists | -| Anyone on the board's network | fetch the page; open a socket; learn that a keyrelay exists | arm, send a key, send a chord, type a run of text, read `/status` — all need the passphrase | +| Anyone on the target (its own OS) | see a Logitech G815 keyboard and a pointer in its device list; read the CDC console; type on itself | arm or drive the bridge remotely, detect that a second controller exists | +| Anyone on the board's network | fetch the page; open a socket; learn that a keyrelay exists | arm, send a key, send a chord, type a run of text, move the pointer, click, read `/status` — all need the passphrase, and all but `/status` need the bridge armed | +| Whoever holds the laptop's mouse | the browser tab's pointer, only after a click on **Capture pointer**, and only while armed | move the target's pointer without the LED being red, or keep moving it after `Esc` — Pointer Lock is revoked by the browser and cannot be retaken without another click | | Someone guessing the passphrase | 5 attempts before a 30 s lockout during which the token is not evaluated at all | learn anything from the failure: same error, same close, no timing oracle (constant-time compare) | | The owner of the WiFi | join the network, then everything above | — the network *is* the perimeter; see below | | You, holding the board | arm/disarm (button), reconfigure or reflash (console), stop everything (unplug) | — | @@ -260,6 +323,15 @@ gated by what it reveals — `/status` needs the passphrase, the page needs it b arm or type. If that ever goes too far, the `ap off`/network config is the blunt instrument; there is no per-client ACL. +**The mouse widens what a compromised controller can do, so it is gated the same twice +over.** Arm plus Pointer Lock. It is also the reason there is a `mouse off` switch: the +composite descriptor is fixed when USB starts, so turning the pointer interface off is a +reboot, not a setting change — and a board that advertises no pointer cannot be made to +move one. When a target selects HID **boot protocol** (some BIOS setups, some KVMs) the +mouse is switched off by itself, because a 5-byte mouse report on a boot-protocol +interface would be parsed as keyboard data: the keyboard keeps working, the pointer stops, +and `E11` says why rather than the command silently doing nothing. + **Identity.** The board reports itself as `046d:c33f` "Logitech Gaming Keyboard G815", matching the sibling `jiggler` firmware so the two are indistinguishable from the host side and `/dev/input/by-id` names stay stable across reflashes. This makes the device @@ -312,6 +384,19 @@ by the compositor and never reach the page. Three answers, in order of preferenc not close the tab, `Ctrl+R` will not reload it and F11 will not fullscreen it. It also means you cannot type into the page's own fields while armed — disarm first. +**While the pointer is captured, `Esc` belongs to the browser.** That is the point: it +revokes Pointer Lock, the page sees `pointerlockchange`, and every button goes up. What it +cannot do is disarm the bridge — the keyboard is still armed and still typing, so the status +line says so and you can press `R` or the board's button. Nothing else the laptop's +compositor grabs takes the pointer away: `Super` and `Alt+Tab` are inert while locked, +which is another reason capture is the design and a plain `mousemove` relay would not be. + +**The laptop's own pointer settings apply to what gets forwarded.** Acceleration, +`libinput Accel Speed` and a high pointer speed on the laptop mean the same hand movement +sends different deltas going out, and the target then accelerates them again. Set the +laptop's pointer speed to the middle and disable acceleration while steering something you +cannot see the response curve of; the target's speed is what it is. + --- ## Layout caveats @@ -326,6 +411,22 @@ This is identical to what a physical KVM does, and the fixes are the same: literal; `tools/relayctl.py` carries the same table and `tests/test_relayctl.py` fails if the two drift). +**The pointer is relative, on purpose.** The board reports a boot mouse: buttons, ±127 per +axis per report, wheel and horizontal pan. It does not know where the target's cursor is, +and the target will not say, so there is no "move to" and no way to desynchronise — the +same property that makes `S=` the right model for keys. Two consequences to know before you +depend on it: + +- **Acceleration is applied twice**, once by the laptop's driver before the page sees the + movement and once by the target's driver after the board sends it. The deltas that leave + the page are the laptop's, not your hand's, and the cursor moves by the target's idea of + those. Fine for steering; not fine for a pixel-exact drag. +- **The target clamps at its screen edges**, and the board cannot see that happen. Moving + 5000 px right from the far right of the screen does nothing, and the board has no way to + know. This is why `mouse move` exists as a *verifiable* primitive and no macro of + movements is ever stored: relative motion is only meaningful while a human is watching + the screen it lands on. + ISO keyboards: `IntlBackslash` (0x32) is the extra key next to Left Shift on ISO and `Backslash` (0x31) is the ANSI one; both are in the table. `AltRight` is `AltGr` and travels as its own modifier bit, which is what layout engines on the target expect. @@ -350,7 +451,7 @@ invisible to every compiler and a typo there fails in a browser, with a target l |---|---| | big ARM/DISARM bar | the state must be unmissable; it only changes when the board confirms | | red page border + red LED while armed | two independent indicators of the same fact | -| status pills: network / target / link / armed / caps-num | "will my keystrokes land, and where" | +| status pills: network / target / link / armed / caps-num / **mouse** | "will my keystrokes land, and where", and whether the pointer is yours right now | | **Ctrl+Alt+Del** with two-press confirm | the OS eats the real one; on Linux it reboots | | VT1–VT7, Alt+Tab, Win, PrtSc, Menu | the chords the laptop's compositor steals | | **⇦ snap left / snap right ⇨ / ⛶ fullscreen** | Super+Left, Super+Right, Super+PageUp — KDE window management, sent by the board so your laptop does not answer it with its own windows | @@ -358,12 +459,25 @@ invisible to every compiler and a typo there fails in a browser, with a target l | *stay armed if this tab loses focus* | off by default; for the one workflow that needs it | | **Send text** box, its button and its *stop typing* | paste something and have it typed as keystrokes; the button says how many characters, refuses to be armed-into accidentally, and is the only path text takes | | *drop characters I cannot type* | off by default: a paste that quietly loses a character is worse than one that refuses | -| held-keys readout, keys/chords/text/chords counters | so "it is typing" is a fact you can see | +| **Capture pointer / Release pointer** and the mouse pad | the pointer half of the bridge. Capture asks the browser for Pointer Lock, so the cursor cannot escape the tab while forwarded; the pad says what a relative mouse means before you discover it mid-drag | +| **Release buttons** under the pad | panic for the pointer, separate from the keyboard's because the two fail independently — held-button-then-lost-link is the mouse's stuck-key, and it is the same one release funnel | +| held-keys readout and the mouse line under it, keys/chords/text/chords/mouse counters | so "it is typing" and "it is moving" are facts you can see; the mouse line names the buttons held and the button byte, because "left is down" and "the target has 0x01" are different claims and only the second one is on the wire | | log, *log each key sent*, **clear** | the board's errors verbatim plus one line per key pressed — never one per release, never a blank line, and a repeating line is counted rather than stacked | The caps/num indicators are read *from the target* (its HID output reports drive them), not guessed — a caps-lock desync between controller and target is a real failure mode of -remote keyboards and this design cannot have it. +remote keyboards and this design cannot have it. The mouse counters are read from the board +for the same reason: `mframes` counts the reports the board accepted, which is the number +`watch_target.py` on the target can be compared against. + +**What capture does and does not do.** The pointer is captured only while the bridge is +armed, and only after a click; `Esc`, blur, a hidden tab, disarm, `R` and the board +reporting it cannot do mouse reports (boot protocol, or `mouse off`) all end it, each +releasing the buttons. The page never accumulates movement to send later: if the link is +down, movement is dropped, because a queue of pointer motion delivered after you stopped +moving is the pointer version of a stuck key. `mouse frames 0` in the status line while you +are moving means the frames are not getting there — that is a fault to look at, not noise, +so it is shown rather than smoothed. --- @@ -381,7 +495,8 @@ and impossible for one malformed field to desynchronise what follows it. | c → d | `C` | Ctrl+Alt+Del (fixed in firmware, not read off the wire) | | c → d | `Y=:,…` | one synthetic chord, `kChordHoldMs` (40 ms) | | c → d | `P=:` | type pasted text, `` ms per character (0 = board default) | -| c → d | `R` | release every key, stay armed — **and drop a text run in progress** | +| c → d | `M=::::` | **the complete mouse button state** plus this report's relative movement | +| c → d | `R` | release every key *and every mouse button*, stay armed — **and drop a text run in progress** | | c → d | `K` | keepalive | | d → c | `OK` | accepted | | d → c | `E:` | error (codes below) | @@ -396,6 +511,21 @@ release are two frames and therefore two HID reports, so a fast tap cannot vanis identical consecutive reports collapse, which is what keeps the queue short. A single message resynchronises the target completely, which is why no key can get stuck. +**`M=` restates the buttons and carries one report's movement.** `` is the button byte +as two hex digits — `01` left, `02` right, `04` middle, `08` back, `10` forward, and any +combination — and is the *whole* button state every time, exactly like `S=`. A frame that +lifts one button restates the ones still held; there is no button-up event to lose, so no +button can get stuck on the target. The four signed decimal fields are this report's +movement, each `±127` (`kMouseDeltaMax`, the boot-mouse byte), and all five fields are +required — `M=01` is `E6`, not "left, no movement", because a controller that forgot three +fields is a controller whose next frame is also wrong. Movement is the one field that is +not state: losing a frame moves the cursor slightly less and leaves nothing behind. + +The page chunks movement to 127 per axis and keeps the diagonal shape (a 200×200 move +leaves as 127,127 then 73,73, never as two axis-parallel legs), so a fast flick cannot turn +into a silently clamped one, and it flushes the pending movement under the *old* button +state before restating a new one — that ordering is what makes a drag a drag. + **`P=` is text, typed by the board.** `` is the bytes to type, so one frame carries at most `kTextFrameMax` (120) characters and a longer paste is several frames; `` is the pace asked for, and the board keeps it — expansion and timing live on the USB side @@ -410,7 +540,13 @@ not a guess at timing. Error codes: `1` not authenticated · `2` locked out · `3` another controller holds the bridge · `4` not armed · `5` no USB host attached · `6` bad message · `7` queue overflow -(released and disarmed) · `8` text buffer full · `9` a character with no US-ANSI keystroke. +(released and disarmed) · `8` text buffer full · `9` a character with no US-ANSI keystroke · +`10` the mouse interface is off on this board · `11` the target chose HID boot protocol, so +no mouse report can be sent safely. + +`10` and `11` are two digits, which changed the encoder: `formatError` used to emit +`E` + `code % 10` and would have rendered both of them as `E0:` — a refusal that names +nothing. Codes now render in full, every single-digit code byte-identical to before. The same protocol is what a future OS-wide capture client would speak — see [Backlog](#backlog) — which is why it lives in `src/proto.h` instead of inside the server. @@ -435,10 +571,17 @@ python3 tools/console.py /dev/ttyACM0 status # one command, print the reply | `ap auto\|force\|off` | AP policy: when the station is down / always / never | | `token ` | control passphrase, 6–32 chars, never echoed back | | `arm` / `disarm` | same as the button | +| `mouse on\|off` | whether the bridge enumerates a pointer interface at all. Stored, and it needs a **reboot**: the composite descriptor is fixed when USB starts, so this cannot change under a live target. Off means the pointer cannot be moved by anything, ever, until it is turned back on | | `reboot`, `help` | | -There is deliberately **no command that types anything** — the console is configuration, -not an input path. Replies are printed with a blocking write (the global serial timeout is +`status` also says which of the two the running bridge is, and why: a line like +`mouse interface=yes target=report buttons=00 reports=41 queued=0`, where +`target=boot` means the target asked for boot protocol and the pointer is disabled by +itself until it re-selects report protocol. + +There is deliberately **no command that types or clicks anything** — the console is +configuration, not an input path. That is why `mouse on|off` is the only mouse command and +it changes a capability rather than performing an action. Replies are printed with a blocking write (the global serial timeout is 0 so logging into a port nobody reads can never stall `loop()`); without that, multi-line replies truncate at the 128-byte CDC TX ring, which is exactly what happened on the first build and is why `pollConsole()` bumps the timeout around each reply. @@ -451,22 +594,29 @@ build and is why `pollConsole()` bumps the timeout around each reply. src/ config.h defaults: network, timeouts, caps, buffers (NVS overrides) keystate.h pure modifier byte + pressed-set -> 8-byte HID report, rollover + mouse.h pure button byte + signed deltas -> 5-byte relative report; the + buttons are restated, not evented, exactly like keystate text.h pure pasted text -> US-ANSI keystrokes, paced; run buffer and cancel + output.h pure the one place keystrokes, pasted runs, chords and mouse reports + become two ordered queues, and the one place anything is released proto.h pure the wire protocol: parse/render commands, constant-time compare sha1.h pure SHA-1 + base64 for the WebSocket handshake ws_codec.h pure handshake request parsing + RFC 6455 frame codec - web_page.h the controller page, in PROGMEM + web_page.h the controller page, in PROGMEM (~48 KB now, with Pointer Lock) control_server.{h,cpp} sockets: HTTP routes, WebSocket lifecycle, auth, heartbeat - main.cpp glue: USB identity/HID, WiFi, report queue, chords, text run, + main.cpp glue: USB identity/HID composite, WiFi, chords, text run, LED, button, watchdogs, serial console tools/ to_bootloader.py 1200-baud touch into ROM download mode (stdlib termios) console.py serial console, one command or interactive (stdlib) - relayctl.py controller from a terminal: status/arm/disarm/key/chord/listen/paste - watch_target.py on the target: decode the board's evdev events, prove delivery + relayctl.py controller from a terminal: status/arm/disarm/key/chord/listen/ + paste/mouse + watch_target.py on the target: decode the board's evdev events (keyboard *and* + mouse), prove delivery tests/ run_tests.sh builds and runs everything (host g++ or a local container) - test_keystate.cpp test_proto.cpp test_text.cpp test_ws_codec.cpp + test_keystate.cpp test_mouse.cpp test_output.cpp test_proto.cpp test_text.cpp + test_ws_codec.cpp test_relayctl.py test_page.js (node, skipped when absent) AGENTS.md operating manual for whoever edits this next: the invariants, the three tables that must agree, the build traps, the verification standard @@ -476,7 +626,7 @@ TODO.md open items, kept with their root cause rather than deleted If you are an agent or a new contributor, read `AGENTS.md` before changing anything: several rules here are invisible to a compiler and to the test suite at the same time. -The five modules marked *pure* have no Arduino and no ESP-IDF include, so the identical +The seven modules marked *pure* have no Arduino and no ESP-IDF include, so the identical code compiles and runs on a host under `-fsanitize=address,undefined`. That is not convenience for its own sake: the sanitiser caught five real defects in the frame decoder before the first flash (a `reset()` that left the reader latched on the previous frame, an @@ -493,11 +643,31 @@ keydown/keyup -> full state S=.. -> Reader (RFC6455) -> proto::parse -> auth gat -> ReportQueue (dedupe identical, overflow = release + disarm) -> drainReports: tud_hid_n_ready(0) ? USBHIDKeyboard::sendReport : keep queued -> HID interrupt endpoint, boot protocol, 8-byte report -> target + +pointer move -> M=btn:dx:dy:wheel:pan -> same gate (armed + attached + mouse allowed) + -> MouseState::apply (button byte restated; idle frames produce no report at all) + -> a second ReportQueue, same dedupe and the same overflow rule + -> drainReports, after the keyboard queue -> report #2 on the *same* HID interface + +fail-safe (any of: disarm, R, blur, silence, detach, suspend, overflow) + -> releaseEverything() -> OutputStage::releaseAll() + clears both queues, both trackers, the chord and the text run, THEN queues + all-keys-up and (if the target can take it) all-buttons-up ``` `USBHID::SendReport` in the core is synchronous (it waits on a semaphore for the transfer to complete, up to 100 ms), so `drainReports()` takes at most `kMaxReportsPerLoop` = 8 -reports per pass — bounded time in the USB path, and the control plane never waits on it. +reports per pass across **both** devices — bounded time in the USB path, and the control +plane never waits on it. + +**Both devices live on one HID interface, which is a constraint with three consequences.** +The keyboard's `USBHIDKeyboard` is the first HID instance, so it claims the boot keyboard +subclass; the mouse is added as the second report on that same instance, which is why +`mouseDev` is created after `kbd` and never before. Because the descriptor is written when +`USB.begin()` runs, `mouse off` cannot take effect until a reboot. And because a +boot-protocol host parses every report on that interface as an 8-byte keyboard report, a +target that selects boot protocol gets the pointer switched off under it (`E11`) rather than +sending 5-byte mouse reports that would arrive as garbage keystrokes. Chords (Ctrl+Alt+Del, VT switches) are synthetic: the press is pushed, and after the hold the *tracked* state is pushed back, so keys that were genuinely held before a chord survive it and the tracker is never corrupted by something the operator did not type. @@ -529,12 +699,29 @@ SANITIZE=1 tests/run_tests.sh # + ASan/UBSan (recommended, and what found 5 b | suite | checks | what it pins down | |---|---|---| | `test_keystate` | 40 | idempotence, transition preservation (a fast tap is two reports), 6-key limit, ErrorRollOver and recovery from it, dedupe, `clear()`, the exact Ctrl+Alt+Del bytes `05 00 4c 00 00 00 00 00` | +| `test_mouse` | 48 | the 5-byte report layout, every button bit where the HID spec says it is, idle frames producing **no** report, a button change with no movement still producing one, the button byte being restated rather than evented, `releaseAll` being idempotent, and that no frame can put a value outside one signed byte on the wire | +| `test_output` | 153 | **the release funnel.** Queues hold their order and collapse duplicates; releasing with keys held queues an all-keys-up report *when there is a target to receive it*; releasing drops a press that was queued but never sent; releasing with a mouse button held queues all-buttons-up; boot protocol or `mouse off` means no mouse report is queued **but the key release still is**; a pasted run and an in-flight chord are both cancelled; and overflow is detected on both queues. Regression-proven: reinstating the old `releaseEverything()` fails 23 of these | | `test_proto` | 193 | every command form, strict rejection of malformed state specs, bounds (cap on keys, token length and text frames), render/parse round-trip, error framing, constant-time compare, **every chord button's bytes matched against the label printed on it**, and the page's text constants pinned to `config.h` | | `test_text` | 459 | the whole printable-ASCII map walked (nothing may be unmapped, nothing may claim a key it has no glyph for), shifted glyphs and the digit row, what is refused rather than guessed at, all-or-nothing frame acceptance, pace arithmetic and clamping, the exact press/release report sequence for two characters, overlay on the operator's held keys, ErrorRollOver propagation, cancel, buffer-full, `P=` render/parse round-trip including the largest legal frame, and one end-to-end case that takes the exact frame the page sends and asserts the HID report sequence the target ends up seeing | | `test_ws_codec` | 135 | SHA-1 against published vectors including the million-`a` case, base64 against RFC 4648, the RFC 6455 handshake example, frame encode/decode at the 125/126 and 64-bit boundaries, byte-at-a-time delivery, and rejection of unmasked, fragmented, reserved-bit, binary, over-long and oversized frames | -| `test_relayctl` | 81 | the Python client's codec, that its key table has not drifted from the page's, that its text validator agrees with the firmware's character map on all 128 ASCII codes, CRLF normalisation, and `TEXT_FRAME_MAX`/`MAX_PASTE_CHARS` pinned to `config.h` | +| `test_relayctl` | 160 | the Python client's codec, that its key table has not drifted from the page's, that its text validator agrees with the firmware's character map on all 128 ASCII codes, CRLF normalisation, `TEXT_FRAME_MAX`/`MAX_PASTE_CHARS` pinned to `config.h`, the mouse frame builder refusing a sixth button and any movement that does not fit one report, movement splitting that reproduces the original displacement exactly, click/drag/wheel frame *ordering*, every generated frame parseable by the firmware's own parser, and the button bits and frame pace agreeing between page, tool and `config.h` | +| `test_page` | 84 | the page's script, executed: blank and whitespace-only log lines never render and a repeating line is counted not stacked, character validation, CRLF, hex encoding, one frame for a short paste and exactly the right bytes, a busy board holding the next frame back, a 300-character paste reassembling to the text that went in, refusal of untypeable text and the opt-in drop, cancellation by stop/disarm, **the mouse: capture requires the bridge armed, movement becomes exactly one `M=` per report with nothing sent when idle, a 400×300 flick split so no frame exceeds one signed byte while the diagonal shape survives, the click ordering that makes a drag a drag, wheel pixels quantised to notches with the remainder carried, `Esc`/blur/disarm each releasing the buttons, `stay armed on blur` exempting the keyboard only, a board that reports `mouse:0` refusing capture and dropping one in progress — and every mouse frame the page emits parsed by the firmware's own parser** | | `test_page` | 47 | the page's script, executed: blank and whitespace-only log lines never render and a repeating line is counted not stacked, character validation, CRLF, hex encoding, one frame for a short paste and exactly the right bytes, a busy board holding the next frame back, a 300-character paste reassembling to the text that went in, refusal of untypeable text and the opt-in drop, and cancellation by stop/disarm, plus every element id the script reaches for checked against the markup | +The verifier is verified too, because twice now it was the verifier that was wrong: node +classification (EV_REL means a pointing device — not the device's name, not the order the +glob found them in), movement coalescing one report into one line, net displacement +arithmetic, hi-res wheel events not counted as extra notches, and a stuck button being +called out in those words. + +The last six checks in `test_relayctl` are **source pins**, and the suite says so out loud: +they assert that `loop()`'s silence disarm runs off `controllerSeen()` rather than +`linked()`, that `acceptNew()` never stamps the controller clock, and that `closeLink()` +stamps it only for a controller link. They cannot execute — that code needs a socket and a +board — but each one is a sentence a later edit can undo with no compiler error and no +behavioural test failing on the host, which is exactly how bug 10 survived. Reinstating +either half of it fails 3 of them. + Gaps, stated plainly: nothing here exercises real sockets, real WiFi associations, or real USB transactions — those are the [end-to-end procedure](#end-to-end-verification) below. The HTTP routing and auth-lifecycle code in `control_server.cpp` is the least-tested part @@ -563,8 +750,20 @@ printf 'Hello from keyrelay\n' | \ python3 tools/relayctl.py --host keyrelay.local paste --yes # 19 characters python3 tools/relayctl.py --host keyrelay.local listen 20 # heartbeats, live python3 tools/relayctl.py --host keyrelay.local disarm + +# 3. the mouse, same two terminals +python3 tools/relayctl.py --host keyrelay.local mouse move 120 -40 +python3 tools/relayctl.py --host keyrelay.local mouse click --yes +python3 tools/relayctl.py --host keyrelay.local mouse wheel -3 +python3 tools/relayctl.py --host keyrelay.local mouse drag 200 0 --button right --yes +python3 tools/relayctl.py --host keyrelay.local mouse release ``` +And in the browser, which is where the pointer is actually usable: open the page, arm it, +press **Capture pointer**, move the mouse (the target's cursor moves, the laptop's is +hidden), hold left and move for a drag, press `Esc`. Then arm it, capture, hold a button, +and **close the tab** — the target must stop clicking within 8 s of the silence timeout. + What a correct run proves, and how to tell: | claim | evidence | @@ -580,6 +779,13 @@ What a correct run proves, and how to tell: | target LEDs are read back | `LED CapsLock on` on the target, caps indicator on the page | | silence disarms | arm it, close the tab, wait 8 s, `status` says `armed: 0` | | auth is real | `relayctl` with a wrong token fails; the board logs `authfails=1`; 5 tries → `E2` | +| **movement reaches the target as movement** | `watch_target.py` prints a `movement dx … dy …` line per report and the summary's `net position` matches the direction and rough distance you moved — the keyboard half of the watcher cannot show this, which is why the mouse node is watched separately | +| **the button bits are the buttons they claim to be** | `mouse click --yes` gives `BTN Left press/release`; `--button right` gives `BTN Right`. A button that arrives as the wrong one is a bit-order bug and looks identical from the page | +| **a drag is down, move, up in that order** | `BTN Left press`, then `movement` lines, then `BTN Left release` — any other order is a drag that clicks at the start and drags nothing | +| **no stuck buttons** | `still held` absent from the mouse summary at the end. Check it the hard way: hold a button and close the tab | +| **the wheel is notches, not pixels** | `mouse wheel -3` produces `REL_WHEEL -3` three times, and hi-res events do not inflate the count | +| **boot protocol refuses the pointer, not the keyboard** | on a BIOS/KVM that selects boot protocol, keys keep working and `M=` answers `E11`; `console.py status` says `target=boot` | +| **`mouse off` really removes the capability** | `console.py mouse off` + reboot → `lsusb`/evdev show no pointing device, `watch_target.py` notes the missing interface, the page's Capture button is disabled, and keys still work | `watch_target.py` finds the board by USB id, not device name, so it keeps working on a different port or a renamed input node, and it accepts both the Logitech identity and the @@ -635,10 +841,17 @@ AtomS3 Lite → Linux laptop (US layout), board joined the LAN, firmware 1.0.0. | LED read-back | `LED CapsLock on` then `off`, coming from the *target's* HID output reports | | **silence disarms** | controller process killed → `armed:0` within 11 s, keys released. **Worth knowing:** that run passed *before* the silence timer existed — killing a process closes the socket, and the socket-close path disarmed. The FIN-less death (laptop slept, network vanished) is what the timer covers, and it is wired now. A check that passes for the wrong reason is still a check that passes, which is exactly the problem | | single controller | a second client got `E3` / `503` and the board logged `rejected=1` — no takeover | +| **mouse enumeration** | the target created **two** input devices from one HID interface — `keyboard=/dev/input/event19`, `mouse=/dev/input/event20` — with `caps rel: 1943`, i.e. REL_X, REL_Y, REL_WHEEL, REL_HWHEEL and both hi-res axes | +| **mouse movement** | `mouse move 120 -40` → `movement dx +120 dy -40` on the target, exact, one report | +| **mouse buttons** | `click` → `BTN Left press` + `BTN Left release`; `--button right` → `BTN Right`. The bit order the page claims is the bit order the target sees | +| **the wheel** | `mouse wheel -3` → `REL_WHEEL -3`. It took a wrong axis table in the *verifier* to prove this worked — see the note under Tests | +| **drag ordering** | `drag 200 0 --button right` → `BTN Right press`, `movement dx +127`, `movement dx +73`, `BTN Right release`. Both halves of the property at once: the ordering is down-then-move-then-up, and the 200 px left as two reports that stay inside one signed byte | +| **keyboard and mouse together** | keys, chords, a 14-character paste and mouse reports in one session, `drop=0 roll=0`, `keys=3` and `mframes` advancing independently | +| **a controller that dies holding a button** | `BTN Left press` → client `os._exit()` (no close, no disarm) → **`BTN Left release` 9.4 s later**, and a live controller holding the same link with keepalives stayed armed for the 24 s it was tested. This is bug 10, and it is the property the whole release funnel exists to provide | | lockout | 5 wrong passphrases → `E2`, for 30 s, from the page's own retry loop | | queues | `drop=0 roll=0` throughout; `queue 0 pending, 0 held, ok` at rest | -### Nine bugs this verification caught (all fixed) +### Ten bugs this verification caught (all fixed) Worth listing, because each one is a class of bug that HID-over-network tools ship with: @@ -689,6 +902,43 @@ Worth listing, because each one is a class of bug that HID-over-network tools sh type. The lesson is the one the whole project keeps relearning: a safety property that is only written down is not a safety property. +10. **A held mouse button stayed held, forever, and anything on the network could keep it + that way.** Found while verifying the mouse, with the mouse innocent: `releaseEverything()` + queued the release reports (bug 9's fix), and they still did not go out. + + Three things stacked up. `closeLink()` ran the release callback only `if (linked())`, + and `linked()` is `_phase == WebSocket && _authed` — so any path that cleared `_authed` + on its way to closing the socket closed it with the keys down and nothing downstream + able to notice. The board then sat `armed=1 mbtn=01` with `phase=none`. And the + backstop that should have caught it, `loop()`'s silence timer, was gated on + `control.linked()` — **a controller that has gone away is precisely the case in which + `linked()` is false**, so the fail-safe designed for a vanished controller could only + ever run while the controller was there. Measured: `BTN Left press`, controller killed, + target still holding the button **26 s later**, and the armed state never expired. + + The second half was worse. `acceptNew()` stamped `_lastInboundMs` for *any* incoming + TCP connection, so a read-only `GET /status` — which is served by design while a + controller holds the bridge — reset the silence clock. Any authenticated monitor, + second tab or health probe kept a dead controller's armed state open indefinitely. The + first fix traded that for stamping in `closeLink()`, which moved the reset from + *accept* to *every close* and changed nothing observable; `wasLinked` is the honest + test, because it is true exactly when the thing being closed can leave key state + behind. + + Fixed as: `wasLinked = (_phase == Phase::WebSocket)`; the silence clock moved only by + an authenticated controller's own traffic or by that link closing; `controllerSeen()` + distinguishes "no controller since boot" (button-armed, leave it alone) from "one came + and went" (never stay armed on its strength). Verified on hardware with the monitor + running the whole time: `BTN Left press` → `BTN Left release` at 9.4 s, armed 0 and + staying 0 for 37 s of continued polling, while a live controller holding the link with + keepalives stayed armed through the same test. + + What to take from it: **never gate a fail-safe on the absence of the thing its + condition detects.** And a backstop needs its own test, because this one passed every + keyboard suite for exactly the reason bug 9 did — the socket-close path happened to + cover the case being checked. `tests/test_relayctl.py` now pins all three sentences in + source, and reinstating either half of the bug fails 3 checks. + ### The hazard nobody warns you about **When the target is the machine you are working on, your keystrokes land on your own diff --git a/src/config.h b/src/config.h index 60de4fc..d5441f0 100644 --- a/src/config.h +++ b/src/config.h @@ -84,6 +84,22 @@ constexpr uint16_t kTextPeriodMsDefault = 12; // ms per character when asked fo constexpr uint16_t kTextPeriodMaxMs = 250; // slowest pace a controller may ask for constexpr uint16_t kTextHoldMaxMs = 6; // how long one character is held down +// --------------------------------------------------------------------------- +// Mouse +// --------------------------------------------------------------------------- +// The controller captures the pointer with the browser's own Pointer Lock and +// forwards it as relative HID mouse reports. One M= frame is one report: the button +// byte is restated every frame, and a movement delta is self-contained, so a frame +// that never arrives moves the cursor a little less and strands nothing. See mouse.h. +// +// The deltas are capped at one report's worth (a signed byte) because one frame is +// one command: splitting a large movement into several frames is the controller's +// job, exactly like splitting a long paste. The board does not split -- it would turn +// one command into a queue depth nobody asked for. +constexpr int8_t kMouseDeltaMax = 127; // one signed byte, the report's range +constexpr uint8_t kMouseButtonMask = 0x1F; // 5 buttons: all the report can carry +constexpr size_t kMouseReportQueueDepth = 32; // movement bursts, same fail-safe + // Safety: a bridge that has heard nothing from its controller for this long is // a bridge that should not be typing. Arming never survives it (laptop slept, // cable unplugged, browser killed). @@ -95,6 +111,11 @@ constexpr uint32_t kStatusIntervalMs = 1000; // --------------------------------------------------------------------------- // Control-server buffers // --------------------------------------------------------------------------- +// The status JSON, sized for every field in StatusSnapshot plus headroom. Sized in +// one place because three call sites build it and a truncated status lies about state, +// which is worse than no status. +constexpr size_t kStatusJsonCap = 384; + constexpr size_t kHttpHeaderCap = 1536; // reject a request line bigger than this constexpr size_t kWsMessageCap = 256; // one controller frame is a command, not a payload constexpr uint32_t kAuthTimeoutMs = 5000; // a new socket must authenticate this fast diff --git a/src/control_server.cpp b/src/control_server.cpp index 6b50243..2180f33 100644 --- a/src/control_server.cpp +++ b/src/control_server.cpp @@ -60,12 +60,14 @@ size_t statusJson(const StatusSnapshot &s, char *out, size_t cap) { out, cap, "{\"v\":\"%s\",\"up\":%lu,\"usb\":%u,\"armed\":%u,\"net\":\"%s\"," "\"ssid\":\"%s\",\"ip\":\"%s\",\"keys\":%lu,\"chords\":%lu,\"drop\":%lu," - "\"roll\":%u,\"leds\":%u,\"dflt\":%u,\"ap\":%u,\"typed\":%lu,\"typing\":%u}", + "\"roll\":%u,\"leds\":%u,\"dflt\":%u,\"ap\":%u,\"typed\":%lu,\"typing\":%u," + "\"mouse\":%u,\"mbtn\":%u,\"mframes\":%lu}", s.version, (unsigned long)s.uptimeS, s.usbHost ? 1u : 0u, s.armed ? 1u : 0u, s.net, ssid, s.ip, (unsigned long)s.keysSent, (unsigned long)s.chordsSent, (unsigned long)s.dropped, s.rollover ? 1u : 0u, (unsigned)s.ledState, s.defaultToken ? 1u : 0u, s.apForced ? 1u : 0u, - (unsigned long)s.typedChars, (unsigned)s.typing); + (unsigned long)s.typedChars, (unsigned)s.typing, + s.mouse ? 1u : 0u, (unsigned)s.mouseButtons, (unsigned long)s.mouseFrames); return (n < 0 || (size_t)n >= cap) ? 0 : (size_t)n; } @@ -118,6 +120,14 @@ void ControlServer::acceptNew(uint32_t nowMs) { WiFiClient fresh = _server.accept(); if (!fresh) return; + if (_phase != Phase::None && _client && !_client.connected()) { + // The incumbent controller is gone and its socket says so. Taking over without + // settling it would inherit its armed state and whatever keys it left down: the + // release funnel runs on a link *change*, and a silent handover to an unrelated + // socket is not one the operator would recognise. Say goodbye, then take the call. + closeLink("controller socket closed"); + } + if (_phase != Phase::None && _client && _client.connected()) { // The bridge is driven by somebody. The incumbent keeps the *keyboard*; this socket // is still allowed to look at the board. Only a second /ws upgrade -- a controller @@ -136,7 +146,11 @@ void ControlServer::acceptNew(uint32_t nowMs) { _httpLen = 0; _authed = false; _connectedAtMs = nowMs; - _lastInboundMs = nowMs; + // Deliberately NOT _lastInboundMs = nowMs. Someone connecting is not the operator being + // alive: a monitoring script, a health check, a second tab pointed at /status or a + // port-scan used to keep the silence timer at zero forever, which held the bridge armed + // -- with the last key the dead controller left still pressed -- for as long as anything + // kept dialling in. Only authenticated controller traffic counts (see readSome). _reader.reset(); } @@ -144,7 +158,11 @@ size_t ControlServer::readSome(uint8_t *dst, size_t cap) { if (!_client || !_client.available()) return 0; const int n = _client.read(dst, cap); if (n > 0) { - _lastInboundMs = millis(); + // "The controller is alive" means the *authenticated controller on the WebSocket* is + // sending. Bytes from a half-read HTTP request, an unauthenticated socket or a + // read-only /status poll must not count, or the fail-safe that is supposed to fire 8 s + // after a controller vanishes can be held open by anything else on the network. + if (_phase == Phase::WebSocket && _authed) _lastInboundMs = millis(); return (size_t)n; } return 0; @@ -253,7 +271,7 @@ void ControlServer::serveReadonly(WiFiClient &c, const ws::Request &req) { } StatusSnapshot s; if (_fillStatus) _fillStatus(_ctx, s); - char json[320]; + char json[kStatusJsonCap]; const size_t len = statusJson(s, json, sizeof(json)); httpResponse(c, "200 OK", "application/json", json, len); return; @@ -399,6 +417,8 @@ bool ControlServer::handleWsFrame(uint32_t nowMs) { _authed = true; _authFails = 0; _lockUntilMs = 0; + _controllerSeen = true; + _lastInboundMs = millis(); // the controller exists from here, and only from here writeTextFrame("OK", 2); sendStatus(); if (_onLinkChange) _onLinkChange(_ctx, true); @@ -428,10 +448,10 @@ void ControlServer::sendStatus() { if (_phase != Phase::WebSocket || !_authed) return; StatusSnapshot s; if (_fillStatus) _fillStatus(_ctx, s); - char json[320]; + char json[kStatusJsonCap]; const size_t len = statusJson(s, json, sizeof(json)); if (len == 0) return; - char framed[340]; + char framed[kStatusJsonCap + 8]; framed[0] = 'S'; // 'S' prefix, then the JSON body memcpy(framed + 1, json, len); writeTextFrame(framed, len + 1); @@ -445,8 +465,15 @@ void ControlServer::sendError(uint8_t code, const char *text) { } void ControlServer::closeLink(const char *why) { - (void)why; - const bool wasLinked = linked(); + (void)why; // named for the readers of this file, not printed: the CDC ring truncates + // "A controller could have left keys down on the target" is the question here, and the + // answer is yes for anything that reached the WebSocket phase -- not only for what is + // still marked authenticated. Checking linked() instead meant a path that cleared + // _authed before getting here closed the socket with the keys still held and nothing + // downstream able to notice: the silence timer, the backstop, asks control.linked(), + // and linked() is exactly what stops being true when a controller goes away. + // disarm() is a no-op when already disarmed, so this cannot over-release. + const bool wasLinked = _phase == Phase::WebSocket; if (_client && _client.connected()) { uint8_t frame[16]; const size_t n = ws::encodeFrame(ws::kOpClose, nullptr, 0, frame, sizeof(frame)); @@ -460,6 +487,15 @@ void ControlServer::closeLink(const char *why) { _authed = false; _httpLen = 0; _reader.reset(); + // The last instant at which we knew a controller existed. The silence timer counts from + // here once the link is gone, so a controller that vanishes disarms the board whether + // or not the socket close is ever noticed. + // Stamped only when it is a *controller* link being closed. Doing this on every close + // (the first attempt) reset the silence clock on each read-only /status request, so + // anything polling the board held a dead controller's armed state open indefinitely -- + // which is the same bug wearing a different hat. `wasLinked` is the honest test: it is + // true exactly when this close can leave key state behind. + if (wasLinked) _lastInboundMs = millis(); if (wasLinked && _onLinkChange) _onLinkChange(_ctx, false); } diff --git a/src/control_server.h b/src/control_server.h index d152334..ea121cb 100644 --- a/src/control_server.h +++ b/src/control_server.h @@ -45,11 +45,33 @@ struct StatusSnapshot { uint32_t typedChars = 0; // characters sent by pasted text (a count, never the text) uint16_t typing = 0; // characters still queued in the paste in progress bool rollover = false; + // The mouse: whether it can report at all, which buttons the board believes are + // down, and how many reports have gone out. `mouse` is false when the interface is + // switched off on the board or when the target has forced boot protocol -- in both + // cases the keyboard is untouched and M= frames are refused, not ignored. + bool mouse = false; + uint8_t mouseButtons = 0; + uint32_t mouseFrames = 0; uint8_t ledState = 0; // the target's caps/num/scroll LEDs, per its HID reports bool defaultToken = false; bool apForced = false; }; +// Error codes, named. The wire form is `E:`; see proto.h for the list. +enum ErrorCode : uint8_t { + kErrNotAuthed = 1, + kErrLockedOut = 2, + kErrBusy = 3, + kErrNotArmed = 4, + kErrNoTarget = 5, + kErrBadMessage = 6, + kErrQueueFull = 7, + kErrTextFull = 8, + kErrUntypeable = 9, + kErrMouseDisabled = 10, + kErrMouseBootProtocol = 11, +}; + /** * Renders the status frame. Split out of the class so it is callable from the serial * console too, and so the field names stay in one place. @@ -75,6 +97,14 @@ class ControlServer { void sendError(uint8_t code, const char *text); bool linked() const { return _phase == Phase::WebSocket && _authed; } + /** + * True from the first successful authentication until reboot. The silence timer needs + * this as well as lastInboundMs(): an armed board with no controller at all (armed by + * its own button, or by the serial console) has nothing to hear from, and must not be + * disarmed for it -- but a board whose controller came and went must never stay armed + * on the strength of a link that no longer exists. + */ + bool controllerSeen() const { return _controllerSeen; } uint32_t lastInboundMs() const { return _lastInboundMs; } uint32_t rejected() const { return _rejected; } uint32_t authFails() const { return _authFails; } @@ -111,6 +141,7 @@ class ControlServer { ws::Reader _reader{_wsBuf, sizeof(_wsBuf)}; bool _authed = false; + bool _controllerSeen = false; uint32_t _connectedAtMs = 0; uint32_t _lastInboundMs = 0; uint32_t _lastStatusMs = 0; diff --git a/src/main.cpp b/src/main.cpp index 46d54a8..6eb3dfb 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -3,11 +3,19 @@ * * laptop (browser tab) --WiFi--> AtomS3 Lite --USB HID--> target machine * - * The board enumerates on the target as a plain boot-protocol HID keyboard (and a - * CDC console), so the target needs no software, no network and no driver -- it - * thinks a keyboard is attached, because one is. The laptop is the controller: a - * page served by the board itself, which restates the complete key state on every - * keystroke, over a passphrase-authenticated WebSocket. + * The board enumerates on the target as a boot-protocol HID keyboard with a relative + * mouse on the same interface (plus a CDC console), so the target needs no software, + * no network and no driver -- it thinks a keyboard and a mouse are attached, because + * they are. The laptop is the controller: a page served by the board itself, which + * restates the complete key state on every keystroke and the complete mouse-button + * state on every mouse frame, over a passphrase-authenticated WebSocket. + * + * The keyboard and the mouse are two collections inside ONE TinyUSB HID interface: + * the Arduino core concatenates their report descriptors and keys them by report id, + * and every report leaves on the same endpoint. Two consequences are load-bearing and + * both are handled below -- the keyboard's constructor has to be the one that fixes + * the interface as a boot keyboard, and a target that puts the interface into BOOT + * protocol cannot be sent a mouse report at all (see onHidEvent). * * press button -> arm / disarm locally (works with no network at all) * hold 3 s at power-on -> ROM USB download mode, so the board can always be @@ -22,8 +30,11 @@ * 3. Armed is displayed on the device: solid red LED, and it blinks red instead if * there is no target attached to type into. * 4. Silence disarms. If the controller stops talking for kNoClientTimeoutMs (laptop - * slept, browser killed, network gone) the keys go up and the bridge disarms. - * Typing that continues after you stopped typing is the failure this prevents. + * slept, browser killed, network gone) the keys and the mouse buttons go up and the + * bridge disarms. Typing that continues after you stopped typing is the failure this + * prevents. Mouse capture is the controller's own extra gate: the browser releases + * the pointer on Escape, on blur and on tab hide, and each of those releases every + * button on the target before anything else is considered. * 5. The target taking the bus away (unplug, suspend) disarms and clears the queue. * A pasted run cancels on every one of these: "release every key" means "and stop * typing", not "pause". The characters live in RAM until they go out and are never @@ -65,6 +76,7 @@ #include "USBCDC.h" #include "esp32-hal-tinyusb.h" // usb_persist_restart() #include "USBHIDKeyboard.h" +#include "USBHIDMouse.h" #define KR_HAVE_HID 1 // This environment builds with ARDUINO_USB_CDC_ON_BOOT=0, so HardwareSerial.h has // mapped `Serial` to UART0 -- dead pins on this board, which has no USB-to-UART @@ -77,9 +89,13 @@ #include #include +#include // offsetof, for the report layout assertions below + #include "config.h" #include "control_server.h" #include "keystate.h" +#include "mouse.h" +#include "output.h" #include "proto.h" #include "text.h" @@ -89,7 +105,15 @@ // hid_device.h needs tinyusb's internal device struct visible, which drags the whole // private header set in for one boolean. At file scope, before the anonymous // namespace, so the code below can call it. +// +// Instance 0 is the one HID interface this device enumerates -- the Arduino core puts +// the keyboard and the mouse in the same interface, so this one boolean gates both. extern "C" bool tud_hid_n_ready(uint8_t instance); + +// hid_protocol_t from TinyUSB's hid_types.h, without including it: the values are part +// of the USB HID specification, not of the library's internals. +constexpr uint8_t kHidProtocolBoot = 0; +constexpr uint8_t kHidProtocolReport = 1; #endif namespace { @@ -115,10 +139,18 @@ constexpr uint32_t kSampleIntervalMs = 5; // button integrator constexpr uint8_t kDebounceSamples = 4; constexpr uint32_t kRecoveryHoldMs = 3000; // hold at boot -> ROM download mode constexpr uint32_t kLedBlinkPeriodMs = 300; // "armed but no target" urgency blink -constexpr size_t kMaxReportsPerLoop = 8; // bound the time in the USB send path +constexpr size_t kMaxReportsPerLoop = 8; // bound the time in the USB send path, + // keyboard and mouse sharing the budget constexpr size_t kConsoleLineCap = 96; #if KR_HAVE_HID +// Declaration order is not a style choice here. The first USBHID constructed is the one +// that fixes the shared HID interface's protocol, and the keyboard's constructor is the +// only one that claims HID_ITF_PROTOCOL_KEYBOARD -- the boot-protocol subclass the +// README measures as `class=03 sub=01 proto=01` and what Windows needs for secure +// attention. So the keyboard is a file-scope object (constructed before anything in +// setup() runs) and the mouse is deliberately NOT: it is built in setup(), which is the +// structural guarantee that it can never get in front of the keyboard. USBHIDKeyboard Keyboard; // ctor registers the HID interface with TinyUSB USBCDC Serial(0); // ctor registers the CDC interface, after Keyboard volatile bool usbStarted = false; @@ -127,8 +159,7 @@ uint8_t targetLeds = 0; // caps/num/scroll as the TARGET reports them Preferences prefs; ControlServer control; -kr::KeyboardState keys; -kr::text::Typewriter typer; // the paste in flight, paced here (see text.h) +kr::OutputStage stage; // both report queues, both device states, one fail-safe // --------------------------------------------------------------------------- // Persistent settings (NVS). Build-time defaults from config.h are only the @@ -139,18 +170,27 @@ constexpr char kKeyStaSsid[] = "sta_ssid"; constexpr char kKeyStaPass[] = "sta_pass"; constexpr char kKeyToken[] = "token"; constexpr char kKeyApMode[] = "ap_mode"; // 0 auto, 1 force on, 2 off +constexpr char kKeyMouse[] = "mouse"; // 1 = advertise the mouse collection char staSsid[33] = ""; char staPass[65] = ""; char token[kr::kTokenMax + 1] = ""; // filled from NVS, or kDefaultToken, in loadSettings uint8_t apMode = 0; +// Whether the target is offered a mouse at all. It decides what the target +// enumerates, so it is read before the USB descriptors are fixed (setup()) and a +// change only takes effect on the next boot -- which is also what makes it a +// rollback: `mouse off`, `reboot`, and the board is the keyboard-only device it used +// to be, with no reflash and the same Logitech identity. +bool mouseEnabled = true; + void loadSettings() { prefs.begin(kNvsNamespace, true); const String nvSsid = prefs.getString(kKeyStaSsid, ""); const String nvPass = prefs.getString(kKeyStaPass, ""); const String nvToken = prefs.getString(kKeyToken, ""); apMode = (uint8_t)prefs.getInt(kKeyApMode, 0); + mouseEnabled = prefs.getInt(kKeyMouse, 1) != 0; prefs.end(); if (nvSsid.length()) { @@ -188,64 +228,84 @@ void saveApMode(uint8_t mode) { apMode = mode; } +void saveMouseEnabled(bool on) { + prefs.begin(kNvsNamespace, false); + prefs.putInt(kKeyMouse, on ? 1 : 0); + prefs.end(); + mouseEnabled = on; +} + // --------------------------------------------------------------------------- -// Report queue +// The emit side // --------------------------------------------------------------------------- -// The controller can burst faster than the target's 1 ms poll interval, and every -// report the core sends is synchronous (USBHID::SendReport waits for the transfer -// to complete), so reports are queued here and drained a few per loop pass. -struct ReportQueue { - kr::KeyboardReport items[kr::kReportQueueDepth]; - size_t head = 0; - size_t count = 0; +// Both report queues, the keyboard and mouse device states, the pasted run and the +// chord in flight live in kr::OutputStage (src/output.h), because they all have to be +// cancelled together and that is a thing worth being unable to forget. The queues +// exist because the controller can burst faster than the target's 1 ms poll interval +// and every report the core sends is synchronous (USBHID::SendReport waits for the +// transfer to complete), so reports are queued and drained a few per loop pass. - bool push(const kr::KeyboardReport &r) { - if (count > 0) { - const size_t last = (head + count - 1) % kr::kReportQueueDepth; - // Two identical reports in a row are one report. Distinct transitions are - // always kept -- collapsing those is how a fast tap would vanish. - if (kr::reportsEqual(items[last], r)) return true; - } - if (count >= kr::kReportQueueDepth) return false; - items[(head + count) % kr::kReportQueueDepth] = r; - count++; - return true; - } - const kr::KeyboardReport *peek() const { return count ? &items[head] : nullptr; } - void pop() { - if (!count) return; - head = (head + 1) % kr::kReportQueueDepth; - count--; - } - void clear() { head = 0; count = 0; } -}; +#if KR_HAVE_HID +// The mouse object, built in setup() only when the board is configured to have one. +// The Arduino wrapper registers its report descriptor from its constructor and TinyUSB +// freezes the descriptors at the first tinyusb_init(), so "no mouse" has to mean +// "never constructed" -- a file-scope object could not be un-advertised, and `mouse +// off` would change nothing about what the target enumerates. +USBHIDRelativeMouse *mouseDev = nullptr; -ReportQueue queue; - -// A chord (Ctrl+Alt+Del, Ctrl+Alt+F3, ...) is a press that has to be followed by a -// release after a fixed hold, without disturbing the tracked key state -- the keys -// that were genuinely held before the chord must come back afterwards. -struct ChordInFlight { - bool active = false; - uint32_t releaseAtMs = 0; -}; -ChordInFlight chord; +// The report this firmware builds has to be the report TinyUSB's hid_mouse_report_t +// describes, byte for byte: buttons, then four signed bytes. Asserted rather than +// trusted, because a mismatch would not fail to build, it would move the target's +// cursor with the wheel byte. +static_assert(sizeof(kr::mouse::MouseReport) == 5, "5-byte mouse report"); +static_assert(offsetof(kr::mouse::MouseReport, dx) == 1 && + offsetof(kr::mouse::MouseReport, dy) == 2 && + offsetof(kr::mouse::MouseReport, wheel) == 3 && + offsetof(kr::mouse::MouseReport, pan) == 4, "mouse report layout"); +#endif bool armed = false; uint32_t keysSent = 0; uint32_t chordsSent = 0; uint32_t dropped = 0; uint32_t typedChars = 0; // a count only: the characters themselves are never kept +uint32_t mouseFrames = 0; // mouse reports accepted -- movement counts, never content uint32_t prevHeld = 0; -#if KR_HAVE_HID // True when a target is attached and the HID interface is live. Nothing can be // armed without it -- arming against nothing would only blink. +#if KR_HAVE_HID bool canType() { return usbStarted; } +bool mousePresent() { return mouseDev != nullptr; } #else bool canType() { return false; } +bool mousePresent() { return false; } #endif +// True unless the target has put the shared HID interface into BOOT protocol. TinyUSB's +// own default at attach is report protocol, and it only changes when the host says so +// (onHidEvent), so this starts true and is a fact about the target, not a guess. +bool hostWantsReportProtocol = true; + +// Whether a mouse report can actually be delivered: the interface has to exist, and +// the target must not be reading boot reports (see onHidEvent). When it is false the +// keyboard is unaffected -- it is what the device was before the mouse existed, and it +// carries on being exactly that. +bool mouseCanReport() { return mousePresent() && hostWantsReportProtocol; } + +void releaseEverything(); +void setArmed(bool on, const char *why); + +// Every producer of reports ends up here, so "the queue would not take it" is one +// behaviour instead of four copies of it that can drift apart. +void notePush(kr::OutputStage::Pushed p) { + if (p != kr::OutputStage::Pushed::Overflow) return; + dropped++; + releaseEverything(); + control.sendError(kr::kErrQueueFull, "report queue overflow -- released and disarmed"); + setArmed(false, "queue overflow"); +} + // --------------------------------------------------------------------------- // LED // --------------------------------------------------------------------------- @@ -294,25 +354,30 @@ void updateLed(uint32_t now) { // --------------------------------------------------------------------------- // Arming // --------------------------------------------------------------------------- +/** + * The one funnel every fail-safe goes through: `R`, disarm, the board's button, the + * controller vanishing, the target taking the bus away, silence, a queue that filled. + * + * It clears what has not gone out, raises the keys and the buttons, ends a chord + * mid-hold and cancels a pasted run -- and then queues the reports that say so, which + * is the part that used to be missing. Clearing the tracker is not the same statement + * to the target as releasing the keys, and disarm is exactly the moment the controller + * stops talking: no further frame was coming to carry the release, so whatever was held + * stayed held. src/output.h pins the behaviour in tests. + */ void releaseEverything() { -#if KR_HAVE_HID - queue.clear(); // drop any press still waiting to go out - keys.clear(); // all keys up, modifiers included - chord.active = false; - // "Release every key" has to mean "and stop typing", not "pause". Every fail-safe - // route (R, disarm, target detached, queue overflow) comes through here, so a - // pasted run cannot outlive any of them. - typer.cancel(); -#endif + // Which reports the target could receive is not the same question for the two + // devices: the keyboard needs a target attached, the mouse also needs report protocol + // (see onHidEvent). Passing both keeps the fail-safe from queueing a report that would + // arrive as corruption. + stage.releaseAll(canType(), mouseCanReport()); } -void setArmed(bool on, const char *why); - void disarm(const char *why) { setArmed(false, why); } void setArmed(bool on, const char *why) { if (on && !canType()) { - control.sendError(5, "no USB host attached -- nothing to type into"); + control.sendError(kr::kErrNoTarget, "no USB host attached -- nothing to type into"); Serial.println(F("[arm] refused: no USB host")); return; } @@ -449,8 +514,11 @@ void fillStatus(void *, kr::StatusSnapshot &s) { s.chordsSent = chordsSent; s.dropped = dropped; s.typedChars = typedChars; - s.typing = (uint16_t)typer.pending(); // flow control: how much is left to type - s.rollover = keys.rollover(); + s.typing = (uint16_t)stage.typer.pending(); // flow control: how much is left to type + s.rollover = stage.keys.rollover(); + s.mouse = mouseCanReport(); + s.mouseButtons = stage.mouseState.buttons(); + s.mouseFrames = mouseFrames; s.defaultToken = strcmp(token, kr::kDefaultToken) == 0; s.apForced = apMode == 1; } @@ -459,47 +527,73 @@ void linkChanged(void *, bool linked) { Serial.printf("[link] controller %s\n", linked ? "authenticated" : "gone"); flash(linked ? kColorBlue : kColorAmber, 180); // A controller that vanished must not leave its last key state latched on the - // target: whoever picks up the keyboard next starts from nothing pressed. + // target: whoever picks up the keyboard next starts from nothing pressed. This is + // the *fast* path; loop()'s silence timer is the backstop for the case where the + // socket close is never noticed, and it no longer asks whether a controller is + // linked -- see the comment there. if (!linked) disarm("controller disconnected"); } #if KR_HAVE_HID +// The controller's key state, queued if it changed. Overflow is a fail-safe, not a +// dropped key: stop typing rather than type out of step with what the operator sees. void pushTrackedState() { - if (!keys.dirty()) return; - keys.ackDirty(); if (!armed) return; - if (!queue.push(keys.report())) { - // Fail safe, not a dropped key: something is outrunning the queue, so stop - // typing rather than type out of step with what the operator sees. - dropped++; - releaseEverything(); - control.sendError(7, "report queue overflow -- released and disarmed"); - setArmed(false, "queue overflow"); - } + notePush(stage.pushKeys()); } -#endif + +/** + * The target choosing HID protocol for the shared interface. + * + * There is one HID interface and it carries both collections, so a BIOS setup or a KVM + * that issues SET_PROTOCOL(BOOT) will parse *every* report on it as an 8-byte boot + * keyboard report -- including a 5-byte mouse report, which would arrive as four bytes + * of garbage keystrokes followed by a truncated report. The keyboard cannot help that; + * the mouse can, by not sending. hostWantsReportProtocol is therefore a fact about the + * target, and mouseCanReport() is the only gate that consults it. + */ +void onHidProtocol(void *, esp_event_base_t, int32_t, void *event_data) { + auto *data = static_cast(event_data); + const bool report = data->set_protocol.protocol != kHidProtocolBoot; + if (report == hostWantsReportProtocol) return; + hostWantsReportProtocol = report; + Serial.printf("[usb] target selected %s protocol%s\n", + report ? "report" : "BOOT", + report ? "" : " -- mouse reports refused, keyboard unaffected"); + if (!report) { + // Anything queued would be delivered as corruption; the buttons themselves are + // released on the way through, because "we cannot tell the target to let go" is + // not the same as "the target may still be holding one". + stage.mouseQ.clear(); + stage.mouseState.releaseAll(); + } + control.sendStatus(); +} +#endif // KR_HAVE_HID void startChord(uint8_t mods, const uint8_t *usages, size_t n, uint32_t holdMs, const char *label) { #if KR_HAVE_HID - if (!armed) { control.sendError(4, "not armed"); return; } - if (!canType()) { control.sendError(5, "no USB host attached"); return; } - if (chord.active) { control.sendError(6, "a chord is still being held"); return; } - - const kr::KeyboardReport press = kr::makeReport(mods, usages, n); - if (!queue.push(press)) { - dropped++; - releaseEverything(); - setArmed(false, "queue overflow"); + if (!armed) { control.sendError(kr::kErrNotArmed, "not armed"); return; } + if (!canType()) { control.sendError(kr::kErrNoTarget, "no USB host attached"); return; } + if (stage.chord.active) { + control.sendError(kr::kErrBadMessage, "a chord is still being held"); return; } - chord.active = true; - chord.releaseAtMs = millis() + holdMs; + + const kr::KeyboardReport press = kr::makeReport(mods, usages, n); + const kr::OutputStage::Pushed p = stage.pushKeyboardReport(press); + if (p == kr::OutputStage::Pushed::Overflow) { + notePush(p); // release, disarm, and say so + return; + } + stage.chord.active = true; + stage.chord.releaseAtMs = millis() + holdMs; chordsSent++; Serial.printf("[hid] chord %s (%u ms)\n", label, (unsigned)holdMs); #else (void)mods; (void)usages; (void)n; (void)holdMs; (void)label; - control.sendError(5, "this build has no HID interface"); + control.sendError(kr::kErrNoTarget, "this build has no HID interface"); #endif } @@ -521,16 +615,16 @@ void onMessage(void *, const proto::Message &msg) { return; case proto::Cmd::ReleaseAll: -#if KR_HAVE_HID + // Keys up AND buttons up, plus the queue, the chord and a pasted run in + // flight. One funnel, so `R` cannot mean something different per device. releaseEverything(); -#endif return; case proto::Cmd::State: { - if (!armed) { control.sendError(4, "not armed"); return; } + if (!armed) { control.sendError(kr::kErrNotArmed, "not armed"); return; } #if KR_HAVE_HID - keys.apply(msg.mods, msg.usages, msg.nUsages); - const uint32_t held = (uint32_t)keys.held(); + stage.keys.apply(msg.mods, msg.usages, msg.nUsages); + const uint32_t held = (uint32_t)stage.keys.held(); if (held > prevHeld) keysSent += held - prevHeld; // count presses, not frames prevHeld = held; pushTrackedState(); @@ -538,6 +632,37 @@ void onMessage(void *, const proto::Message &msg) { return; } + case proto::Cmd::Mouse: { + // `M=` is the mouse's `S=`: the button byte is restated in full every frame, so + // a lost frame cannot leave a button stuck, and the movement fields are the one + // report's worth of relative motion -- which has no state to lose. + if (!armed) { control.sendError(kr::kErrNotArmed, "not armed"); return; } + if (!canType()) { control.sendError(kr::kErrNoTarget, "no USB host attached"); return; } + if (!mousePresent()) { + control.sendError(kr::kErrMouseDisabled, + "this board has no mouse interface (`mouse on`, then reboot)"); + return; + } + if (!hostWantsReportProtocol) { + // The honest refusal, and the reason for it: in boot protocol the target would + // read these five bytes as keyboard data. Keys keep working regardless. + control.sendError(kr::kErrMouseBootProtocol, + "target is in HID boot protocol, so a mouse report would " + "arrive as keystrokes -- keys still work"); + return; + } +#if KR_HAVE_HID + const kr::mouse::Frame f{msg.buttons, msg.dx, msg.dy, msg.wheel, msg.pan}; + const kr::OutputStage::Pushed p = stage.pushMouse(f); + // Nothing queued means the frame said nothing at all -- same buttons, no + // movement. That is a success, not an error: the controller is allowed to send + // an idle frame and the target simply does not see it. + if (p == kr::OutputStage::Pushed::Ok) mouseFrames++; + notePush(p); +#endif + return; + } + case proto::Cmd::SecureAttention: { // Fixed on the device side, not taken from the wire: the combination that // reboots a Linux VT or unlocks a Windows box is not something a malformed @@ -555,14 +680,14 @@ void onMessage(void *, const proto::Message &msg) { // A pasted run is still gated by everything else: armed, and a target to type // into. What it is NOT is stored -- the characters live in RAM until they have // gone out, and cancel() is reachable from every fail-safe. - if (!armed) { control.sendError(4, "not armed"); return; } + if (!armed) { control.sendError(kr::kErrNotArmed, "not armed"); return; } #if KR_HAVE_HID - if (!canType()) { control.sendError(5, "no USB host attached"); return; } + if (!canType()) { control.sendError(kr::kErrNoTarget, "no USB host attached"); return; } // Accepted as a unit: a frame that cannot be taken contributes nothing, so the // controller can retry exactly the same characters. - const kr::text::Outcome o = typer.feed(msg.payload, msg.nPayload); + const kr::text::Outcome o = stage.typer.feed(msg.payload, msg.nPayload); if (o.result == kr::text::Result::Full) { - control.sendError(8, "text buffer full -- wait for the run to drain"); + control.sendError(kr::kErrTextFull, "text buffer full -- wait for the run to drain"); return; } if (o.result == kr::text::Result::Untypeable) { @@ -572,22 +697,22 @@ void onMessage(void *, const proto::Message &msg) { snprintf(why, sizeof(why), "0x%02X at offset %u has no US-ANSI keystroke -- nothing typed", o.byte, (unsigned)o.index); - control.sendError(9, why); + control.sendError(kr::kErrUntypeable, why); return; } - typer.start(millis(), msg.periodMs); + stage.typer.start(millis(), msg.periodMs); // Length and pace only. Nothing here records what the characters were. Serial.printf("[text] %u chars queued, %u ms per character\n", - (unsigned)typer.pending(), (unsigned)typer.periodMs()); + (unsigned)stage.typer.pending(), (unsigned)stage.typer.periodMs()); control.sendStatus(); #else - control.sendError(5, "this build has no HID interface"); + control.sendError(kr::kErrNoTarget, "this build has no HID interface"); #endif return; } default: - control.sendError(6, "unrecognised command"); + control.sendError(kr::kErrBadMessage, "unrecognised command"); return; } } @@ -654,22 +779,50 @@ void onKeyboardLed(void *, esp_event_base_t, int32_t, void *event_data) { targetLeds = data->leds; } +/** + * Both queues, one endpoint, keyboard first. + * + * The order is a decision, not an accident: a key release that is waiting behind a + * burst of pointer movement is a key that stays down longer than the operator's + * finger did. The budget is shared, so one pass cannot spend all its time on the + * mouse and starve the keyboard. + * + * This runs whether or not the bridge is armed, and it has to: the release reports + * that disarm just queued are the whole point of it, and after a fail-safe there is + * no further command coming to carry them. + */ void drainReports() { for (size_t i = 0; i < kMaxReportsPerLoop; i++) { - const kr::KeyboardReport *r = queue.peek(); - if (!r) return; + if (stage.keyQ.empty()) break; if (!usbStarted) { // nobody to receive them - dropped += (uint32_t)queue.count; - queue.clear(); - return; + dropped += (uint32_t)stage.keyQ.count(); + stage.keyQ.clear(); + break; } - if (!tud_hid_n_ready(0)) return; // endpoint still busy; try next loop + if (!tud_hid_n_ready(0)) return; // endpoint busy; the mouse waits too + const kr::KeyboardReport *r = &stage.keyQ.front(); KeyReport wire; wire.modifiers = r->modifiers; wire.reserved = 0; memcpy(wire.keys, r->keys, sizeof(wire.keys)); Keyboard.sendReport(&wire); // synchronous: returns once transmitted - queue.pop(); + stage.keyQ.pop(); + } + + for (size_t i = 0; i < kMaxReportsPerLoop; i++) { + if (stage.mouseQ.empty()) break; + // Same rule as the keyboard, plus the two reasons a mouse report cannot go out at + // all. Dropping them is correct here: they are movement and button state that + // nobody can receive, and holding them until protocol changes would deliver a + // stale burst the operator never asked for. + if (!usbStarted || !mouseDev || !hostWantsReportProtocol) { + dropped += (uint32_t)stage.mouseQ.count(); + stage.mouseQ.clear(); + break; + } + if (!tud_hid_n_ready(0)) break; // endpoint busy; next loop + mouseDev->sendReport(stage.mouseQ.front()); + stage.mouseQ.pop(); } } #endif // KR_HAVE_HID @@ -742,25 +895,35 @@ void printHelp() { " ap auto|force|off access point policy (auto = when station is down)\n" " token control passphrase, max 32 chars\n" " arm | disarm same as the button, from the console\n" + " mouse on|off offer the target a pointer (needs a reboot)\n" " reboot\n" - "There is deliberately no command that types anything. Keystrokes come from\n" - "a human at a keyboard, through the controller page. Pasted text is a controller\n" - "action too: it needs the bridge armed, and nothing is remembered by the board.\n")); + "There is deliberately no command that types or clicks anything. Keystrokes come\n" + "from a human at a keyboard, through the controller page. Pasted text is a\n" + "controller action too: it needs the bridge armed, and nothing is remembered by\n" + "the board. `mouse` changes what the device IS, never what it does.\n")); } void printStatus() { kr::StatusSnapshot s; fillStatus(nullptr, s); - char json[320]; + char json[kr::kStatusJsonCap]; kr::statusJson(s, json, sizeof(json)); Serial.printf("status %s\n", json); Serial.printf("link controller=%s rejected=%u authfails=%u\n", control.linked() ? "yes" : "no", (unsigned)control.rejected(), (unsigned)control.authFails()); - Serial.printf("queue %u pending, %u held, %s\n", (unsigned)queue.count, - (unsigned)keys.held(), keys.rollover() ? "ROLLOVER" : "ok"); + Serial.printf("queue %u keys pending, %u held, %s\n", + (unsigned)stage.keyQ.count(), (unsigned)stage.keys.held(), + stage.keys.rollover() ? "ROLLOVER" : "ok"); Serial.printf("text %u chars typed, %u still to go\n", (unsigned)typedChars, - (unsigned)typer.pending()); + (unsigned)stage.typer.pending()); + // `target=BOOT protocol (refused)` is the line to look for when the page says the + // pointer is captured and nothing moves: it is the target's choice, not a bug here. + Serial.printf("mouse interface=%s target=%s buttons=0x%02X %u reports, %u queued\n", + mousePresent() ? "present" : "off", + hostWantsReportProtocol ? "report protocol" : "BOOT protocol (refused)", + (unsigned)stage.mouseState.buttons(), (unsigned)mouseFrames, + (unsigned)stage.mouseQ.count()); } // Split a line into `verb` and the rest of the line, trimmed. Returns false if the @@ -822,6 +985,28 @@ void handleConsoleLine(char *buf) { return; } + if (strcmp(verb, "mouse") == 0) { + if (!rest) { + Serial.printf("mouse interface: %s (target currently sees %s)\n", + mousePresent() ? "on" : "off", + hostWantsReportProtocol ? "report protocol" : "BOOT protocol"); + return; + } + const bool on = strcmp(rest, "on") == 0; + if (!on && strcmp(rest, "off") != 0) { + Serial.println(F("usage: mouse on|off")); + return; + } + saveMouseEnabled(on); + // Configuration, not an action: it changes whether the target is offered a pointer + // at all, and the composite descriptor is frozen when USB starts. Saying "reboot" + // rather than doing it keeps the surprise out of a console session that is halfway + // through something. + Serial.printf("[mouse] %s -- reboot to apply (the USB descriptor is fixed at start)." + " While off, nothing can move the pointer.\n", on ? "on" : "OFF"); + return; + } + if (strcmp(verb, "token") == 0) { if (!rest) { Serial.printf("token: %s\n", strcmp(token, kr::kDefaultToken) == 0 @@ -885,11 +1070,32 @@ void setup() { Serial.begin(115200); Serial.setTxTimeoutMs(0); // never stall the loop because nobody is reading + // Before USB.begin(), because `mouse off` has to change what the descriptors say and + // the first tinyusb_init() freezes them. Reading NVS has no USB dependency at all, so + // nothing is lost by doing it here rather than after the recovery hold. + loadSettings(); + #if KR_HAVE_HID USB.onEvent(onUsbEvent); setUsbIdentity(); // must precede USB.begin() - Keyboard.begin(); // descriptors freeze at USB.begin() + Keyboard.begin(); // ctor registered the boot keyboard; descriptors + // freeze at USB.begin() below Keyboard.onEvent(onKeyboardLed); + // The protocol event lives on USBHID's own event base, and USBHIDKeyboard only + // re-exports its keyboard subset of it -- so it is registered on the base directly, + // which is what the core posts to. Must be before USB.begin(): the host can issue + // SET_PROTOCOL the instant it enumerates, and a target that boots straight into boot + // protocol would otherwise get one mouse report before the gate exists. + esp_event_handler_register(ARDUINO_USB_HID_EVENTS, + ARDUINO_USB_HID_SET_PROTOCOL_EVENT, onHidProtocol, nullptr); + if (mouseEnabled) { + // Constructed here and not at file scope on purpose: the constructor is what + // registers the mouse report descriptor, so "no mouse" has to mean "never + // constructed". After Keyboard, always, or the interface stops being a boot + // keyboard and Windows stops honouring Ctrl+Alt+Del. + mouseDev = new USBHIDRelativeMouse(); + mouseDev->begin(); + } if (!USB.begin()) Serial.println(F("[usb] FAILED to start the TinyUSB device stack")); #endif @@ -899,7 +1105,6 @@ void setup() { for (uint32_t t = millis(); !usbStarted && millis() - t < 1000; delay(10)) {} #endif - loadSettings(); ledSelfTest(); Serial.println(); @@ -918,6 +1123,11 @@ void setup() { Serial.println(WiFi.status() == WL_CONNECTED ? WiFi.localIP().toString() : String(kr::kApGw)); Serial.println(F("button short press = arm / disarm. Console: `help`")); + if (mousePresent()) { + // Short on purpose: the CDC TX ring truncates a long line (README bug 1). + Serial.println(F("mouse: capture it in the page; Esc frees the cursor and releases " + "every button.")); + } } void loop() { @@ -933,8 +1143,18 @@ void loop() { // so this only ever fires when the controller is gone without having said goodbye: // laptop slept, browser killed, WiFi died without a FIN. That is exactly the moment // a pasted run must stop typing, which is why this is checked before the typewriter - // is driven rather than after. - if (armed && control.linked() && + // is driven rather than after. Captured mouse movement refreshes the same clock -- it + // is inbound traffic like anything else -- so holding a pointer still alive at 125 Hz + // counts as being there, and losing it counts as leaving. + // Note what this does NOT condition on: control.linked(). A controller that has gone + // away is precisely the case where linked() is false, so gating the backstop on it made + // the backstop uncallable in the one situation it exists for -- the board sat armed with + // a key or a mouse button still held, forever, whenever the link teardown failed to run + // the release itself. It now runs off "we have had an authenticated controller, and this + // is how long since we heard from it or watched it leave" (ControlServer::closeLink + // stamps the clock on the way out). A board armed by its own button with no controller + // at all is still left alone, because it has never had one to go quiet. + if (armed && control.controllerSeen() && static_cast(now - control.lastInboundMs()) > static_cast(kr::kNoClientTimeoutMs)) { disarm("controller silent"); @@ -943,34 +1163,29 @@ void loop() { #if KR_HAVE_HID // Release the synthetic chord after its hold, back to exactly what is tracked. // Nothing else touches the tracker, so the operator's held keys survive a chord. - if (chord.active && now >= chord.releaseAtMs) { - chord.active = false; - if (armed && usbStarted) { - if (!queue.push(keys.report())) { - dropped++; - releaseEverything(); - setArmed(false, "queue overflow"); - } - } + if (stage.chord.active && now >= stage.chord.releaseAtMs) { + stage.chord.active = false; + if (armed && usbStarted) notePush(stage.pushKeyboardReport(stage.keys.report())); } // A pasted run, typed on this clock and no other. text.h emits at most one report // transition per call, so a fast pace cannot deepen the queue, and the queue's own // overflow path releases and disarms if the target stops taking reports. - for (size_t i = 0; i < kMaxReportsPerLoop && typer.active(); i++) { - if (!armed || !usbStarted) { typer.cancel(); break; } + for (size_t i = 0; i < kMaxReportsPerLoop && stage.typer.active(); i++) { + if (!armed || !usbStarted) { stage.typer.cancel(); break; } kr::KeyboardReport r; bool pressed = false; - if (!typer.step(now, keys, r, pressed)) break; - if (!queue.push(r)) { - dropped++; - releaseEverything(); - control.sendError(7, "report queue overflow -- released and disarmed"); - setArmed(false, "queue overflow"); + if (!stage.typer.step(now, stage.keys, r, pressed)) break; + const kr::OutputStage::Pushed p = stage.pushKeyboardReport(r); + if (p == kr::OutputStage::Pushed::Overflow) { + notePush(p); break; } if (pressed) typedChars++; } + // Both queues, one endpoint, keyboard first. This runs whether or not the bridge is + // armed, and it has to: the release reports that disarm just queued are the whole + // point of it, and there will be no further command to send them. drainReports(); #endif diff --git a/src/mouse.h b/src/mouse.h new file mode 100644 index 0000000..8ef79a4 --- /dev/null +++ b/src/mouse.h @@ -0,0 +1,145 @@ +/** + * keyrelay -- the emulated mouse. Pure logic: no Arduino, no TinyUSB, so the exact + * code the board runs is compiled and unit-tested on a host (tests/test_mouse.cpp). + * + * Why a movement is not a violation of "state, not events" + * -------------------------------------------------------- + * Invariant 1 says the controller restates the whole desired state and never sends + * key up/down events, because one lost release is a key stuck forever. A mouse has + * two halves, and they behave differently: + * + * - **Buttons are state.** Every `M=` frame carries the complete button byte, not + * a press or a release, so a frame that never arrives did not click, and any one + * frame afterwards restates the truth. Same argument, same guarantee as `S=`. + * - **Movement is a delta, and a delta is self-contained.** There is no position + * for the board or the controller to be wrong about: a lost frame means the + * cursor moved a little less, which is what a relative mouse does when a real + * one is picked up and moved back down. Nothing is stranded, nothing needs to + * be reconciled, and no resynchronisation timer is needed because there is + * nothing to be out of step with. + * + * That is also why the pointer is relative and not absolute. An absolute pointer + * would give the board a position to disagree with the target about -- the target + * clips it at the screen edge, snaps it between monitors, and the next frame's + * "where the cursor is" would be a guess. Relative movement cannot be wrong, only + * incomplete. + * + * One frame is one report + * ----------------------- + * The report is five bytes: a button byte and four signed bytes of movement. A + * frame's deltas are therefore capped at one report's worth (config.h + * kMouseDeltaMax), and splitting a larger movement across frames is the + * controller's job, exactly as splitting a long paste is. The board does not + * split: it would turn one command into a queue depth nobody asked for, and a + * queue that fills is a fail-safe event. + * + * What the board never does + * ------------------------- + * No click automation, no double-click, no drag recorded and replayed, no + * "move to the OK button", nothing scheduled. Like the text run, this exists + * because a human is moving a mouse right now; the moment it could act on its own + * it would be a different and worse device. See README "What this deliberately is + * not". + */ +#pragma once + +#include +#include +#include + +namespace kr { +namespace mouse { + +// USB HID Usage Tables, Generic Desktop: the pointer page's button bits, in the +// order the report descriptor lays them out (see TinyUSB's TUD_HID_REPORT_DESC_MOUSE, +// which is what this firmware enumerates). Five buttons and three bits of padding -- +// a sixth would have nowhere to go, so the parser rejects one rather than dropping it. +enum Button : uint8_t { + kBtnNone = 0x00, + kBtnLeft = 0x01, + kBtnRight = 0x02, + kBtnMiddle = 0x04, + kBtnBackward = 0x08, + kBtnForward = 0x10, + kBtnAll = 0x1F, +}; + +/** + * The report on the wire: byte-identical to TinyUSB's `hid_mouse_report_t` + * (buttons, X delta, Y delta, wheel, horizontal pan), five bytes, no padding. + * + * `pan` is Consumer-page AC Pan, which is why it is in the descriptor at all: a + * trackpad's horizontal scroll has no other way out. + */ +struct MouseReport { + uint8_t buttons; + int8_t dx; + int8_t dy; + int8_t wheel; + int8_t pan; +}; + +static_assert(sizeof(MouseReport) == 5, "a relative mouse report is 5 bytes"); + +// "Nothing held, nothing moved." The report a fail-safe sends, and the one a real mouse +// sends whenever it presses a button without moving. +constexpr MouseReport kReportClear{0, 0, 0, 0, 0}; + +// One controller frame, before it becomes a report: the button byte restated, plus +// this frame's movement. Same five values the wire carries. +struct Frame { + uint8_t buttons = 0; + int8_t dx = 0; + int8_t dy = 0; + int8_t wheel = 0; + int8_t pan = 0; +}; + +inline bool reportsEqual(const MouseReport &a, const MouseReport &b) { + // No padding in a 5-byte struct of 1-byte members, so a byte compare is the + // whole comparison -- and it is the comparison the host itself makes. + return memcmp(&a, &b, sizeof(MouseReport)) == 0; +} + +inline bool moved(const Frame &f) { + return f.dx != 0 || f.dy != 0 || f.wheel != 0 || f.pan != 0; +} + +/** + * The mouse's tracked state, which is exactly one byte: which buttons are down. + * + * Deliberately small. Movement is not accumulated here -- a frame that reports 30 + * pixels of movement is a report, not a change in a value that has to stay in step + * with anything -- so there is no state on the board for a lost frame to corrupt. + */ +class MouseState { + public: + /** + * Turn one frame into the report to send. Returns false when the frame carries + * nothing at all: no button change and no movement. An idle report is not free + * (it is the whole endpoint for that frame, and the target has to parse it) and + * it says nothing, so it is dropped rather than sent. + */ + bool apply(const Frame &f, MouseReport &out) { + const uint8_t b = static_cast(f.buttons & kBtnAll); + if (!moved(f) && b == _buttons) return false; + _buttons = b; + out = MouseReport{b, f.dx, f.dy, f.wheel, f.pan}; + return true; + } + + // Every button up. The tracked byte is the only state there is, so this is the whole + // of it: the report that says so is the caller's to build (see OutputStage::releaseAll, + // which queues one whether or not anything was held -- a fail-safe should not depend + // on bookkeeping being right). + void releaseAll() { _buttons = kBtnNone; } + + uint8_t buttons() const { return _buttons; } + bool down() const { return _buttons != kBtnNone; } + + private: + uint8_t _buttons = kBtnNone; +}; + +} // namespace mouse +} // namespace kr diff --git a/src/output.h b/src/output.h new file mode 100644 index 0000000..0848ca8 --- /dev/null +++ b/src/output.h @@ -0,0 +1,185 @@ +/** + * keyrelay -- the emit side of the bridge: what the target is told, in what order, + * and what happens when it cannot be told. Pure logic, host-tested + * (tests/test_output.cpp): no Arduino, no TinyUSB. + * + * Why the queues and the devices live together + * -------------------------------------------- + * Before the mouse there was one queue, one device and one fail-safe, and the + * fail-safe was a function in main.cpp that knew about all three by name. That made + * the bug this module exists to prevent possible: `releaseEverything()` cleared the + * queue and the tracker and never queued the report that tells the target the keys + * went up -- and disarm is exactly the moment the controller stops talking, so no + * later frame was coming to carry it. A key stayed down on the target until the next + * keystroke, and after a disarm there need never be one. + * + * So the state and the funnel are one object now, and the funnel is the only way to + * reach it: + * + * - clearing means queueing the release. Both devices, in the same call. + * - a queue that cannot take a report is reported to the caller as an overflow, + * which the caller treats the way it always has: release, disarm, say so. There + * is no "drop the mouse frame, it is only a movement" special case, because the + * frame that does not fit may be the one releasing a held button, and a mouse + * button stuck down on a target is the same class of failure as a stuck key. + * - the pasted run is a member, so cancelling work in flight is not something a + * caller can forget to do. + * + * Order of consequence + * -------------------- + * Keyboard and mouse reports go out on one shared TinyUSB interface and one shared + * endpoint, so there is one queue per device and one drain. The keyboard queue is served + * first when both have something: it is what the device is for, and a burst of movement + * must not delay a key release. + */ +#pragma once + +#include +#include + +#include "config.h" +#include "keystate.h" +#include "mouse.h" +#include "text.h" + +namespace kr { + +/** + * A ring of reports waiting for the target's next 1 ms poll. + * + * Two identical reports in a row are one report: distinct transitions are always + * kept, and collapsing those is how a fast tap would vanish. + */ +template +class ReportQueue { + public: + bool push(const T &r) { + if (_count > 0) { + const size_t last = (_head + _count - 1) % DEPTH; + if (reportsEqual(_items[last], r)) return true; + } + if (_count >= DEPTH) return false; + _items[(_head + _count) % DEPTH] = r; + _count++; + return true; + } + const T &front() const { return _items[_head]; } + void pop() { + if (!_count) return; + _head = (_head + 1) % DEPTH; + _count--; + } + void clear() { _head = 0; _count = 0; } + size_t count() const { return _count; } + size_t capacity() const { return DEPTH; } + bool empty() const { return _count == 0; } + + private: + T _items[DEPTH]; + size_t _head = 0; + size_t _count = 0; +}; + +using KeyboardQueue = ReportQueue; +using MouseQueue = ReportQueue; + +/** + * A synthetic chord (Ctrl+Alt+Del, Super+Left, ...): a press that has to be followed + * by a release after a fixed hold without disturbing the tracked key state, so the + * keys that were genuinely held before the chord come back afterwards. + */ +struct ChordState { + bool active = false; + uint32_t releaseAtMs = 0; +}; + +/** + * Everything the bridge can put on the USB wire, and the one place that decides it. + * + * `canType` is the caller's answer to "is a target attached and is the HID interface + * live": the release reports are only queued when somebody could receive them, so a + * fail-safe that fires while detached does not fill a queue that the next drain will + * count as dropped. + */ +class OutputStage { + public: + enum class Pushed : uint8_t { Nothing, Ok, Overflow }; + + // ---- tracked state ------------------------------------------------------- + KeyboardState keys; + mouse::MouseState mouseState; + text::Typewriter typer; + ChordState chord; + + // ---- the queues ---------------------------------------------------------- + // Names, not `keyboard` / `mouse`: a member called `mouse` would shadow the + // kr::mouse namespace for every type name written inside this class. + KeyboardQueue keyQ; + MouseQueue mouseQ; + + /** + * Queue the tracked key state, if it changed since the last time. The dirty flag + * is cleared either way: an overflow is a fail-safe, not a report to retry. + */ + Pushed pushKeys() { + if (!keys.dirty()) return Pushed::Nothing; + keys.ackDirty(); + return keyQ.push(keys.report()) ? Pushed::Ok : Pushed::Overflow; + } + + Pushed pushKeyboardReport(const KeyboardReport &r) { + return keyQ.push(r) ? Pushed::Ok : Pushed::Overflow; + } + + /** + * One mouse frame: restated buttons plus this frame's movement. Nothing is queued + * when the frame said nothing (no button change, no movement) -- see mouse.h. + */ + Pushed pushMouse(const mouse::Frame &f) { + mouse::MouseReport r; + if (!mouseState.apply(f, r)) return Pushed::Nothing; + return mouseQ.push(r) ? Pushed::Ok : Pushed::Overflow; + } + + /** + * The funnel. Every fail-safe arrives here: `R`, disarm, the board's button, the + * controller vanishing, the target taking the bus away, silence, a queue that + * filled up. + * + * It does four things, and the fourth is the one that was missing once: + * 1. drop what has not gone out yet, for both devices; + * 2. raise the keys and the buttons, and end a chord mid-hold; + * 3. cancel a pasted run -- "release every key" means "and stop typing"; + * 4. queue the all-up reports, because a tracker with nothing pressed is not the + * same statement as a target with nothing pressed. + * + * The two arguments say which reports the target could actually receive: the keyboard + * when a target is attached at all, the mouse only when the interface exists AND the + * target is in report protocol. A report nobody can receive is not queued, because the + * drain counts whatever is left as dropped -- and in boot protocol a 5-byte mouse + * report is exactly the corruption the mouse refuses to send in the first place. + * + * The tracked state is cleared either way. "Nothing is held" has to be true of the + * board after a fail-safe whether or not anybody was told, or the next controller + * inherits a button its operator let go of. + * + * The reports themselves are unconditional rather than conditional on something having + * been held: a panic path should not depend on bookkeeping being right, and a real + * mouse sends an all-zero report every time it presses a button without moving. + */ + bool releaseAll(bool keysOut, bool mouseOut) { + keyQ.clear(); + mouseQ.clear(); + keys.clear(); + chord.active = false; + typer.cancel(); + mouseState.releaseAll(); // the byte, whatever the target is told about it + + bool queued = false; + if (keysOut && keyQ.push(keys.report())) queued = true; + if (mouseOut && mouseQ.push(mouse::kReportClear)) queued = true; + return queued; + } +}; + +} // namespace kr diff --git a/src/proto.h b/src/proto.h index 6014cc8..db7c71c 100644 --- a/src/proto.h +++ b/src/proto.h @@ -17,7 +17,10 @@ * c -> d Y=:,... one synthetic burst of a chord the laptop's * own OS steals (VT switch, Alt+Tab, PrtSc) * c -> d P=: type pasted text, ms per character - * c -> d R release every key (kept armed) + * c -> d M=:::: one mouse report: the whole button + * byte plus this frame's movement + * c -> d R release every key AND every mouse button + * (kept armed) * c -> d K keepalive * d -> c OK accepted, or the reply to H * d -> c E: error, e.g. E4:arm required @@ -27,6 +30,13 @@ * hex digits, lowercase or uppercase. Empty key lists are legal (`S=00:` means * "nothing pressed"), which is what makes the protocol self-healing. * + * `M=` carries `` as two hex digits (bits 0..4: left, right, middle, backward, + * forward -- five buttons is all a mouse report has, so a sixth is rejected rather + * than dropped) and the four movements as signed decimal, each within one report's + * worth of +/-127. All five fields are always present. Buttons are restated on every + * frame, exactly like `S=`; the movement is a delta and belongs to this frame only. + * See mouse.h for why that is as safe as a state restatement. + * * `P=` is the one command whose payload is text rather than a key state: `` * is the bytes to type (so at most kTextFrameMax characters per frame) and `` * is the pace in milliseconds per character, 0 for the board's default. The board @@ -57,6 +67,7 @@ enum class Cmd : uint8_t { SecureAttention, // Ctrl+Alt+Del Chord, // one-shot burst of an arbitrary chord Text, // pasted text to type, paced on the board + Mouse, // one mouse report: button byte plus movement ReleaseAll, KeepAlive, Bad, @@ -71,6 +82,8 @@ struct Message { uint8_t payload[kTextFrameMax]; // Text: the characters to type uint8_t nPayload; uint16_t periodMs; // Text: ms per character, 0 = board default + uint8_t buttons; // Mouse: the whole button byte, restated + int8_t dx, dy, wheel, pan; // Mouse: this frame's movement }; inline void initMessage(Message &m) { @@ -82,6 +95,8 @@ inline void initMessage(Message &m) { m.nPayload = 0; memset(m.payload, 0, sizeof(m.payload)); m.periodMs = 0; + m.buttons = 0; + m.dx = m.dy = m.wheel = m.pan = 0; } inline const char *name(Cmd c) { @@ -94,6 +109,7 @@ inline const char *name(Cmd c) { case Cmd::SecureAttention: return "secure-attention"; case Cmd::Chord: return "chord"; case Cmd::Text: return "text"; + case Cmd::Mouse: return "mouse"; case Cmd::ReleaseAll: return "release-all"; case Cmd::KeepAlive: return "keepalive"; default: return "bad"; @@ -189,6 +205,64 @@ inline Cmd parseText(const char *data, size_t len, Message &out) { return Cmd::Text; } +/** + * Parse one signed decimal movement: optional '-', one to three digits, and at most + * one report's worth (kMouseDeltaMax). Nothing else is legal -- no '+', no spaces, + * no value that would need splitting -- so a frame either describes one report + * exactly or is refused and changes nothing on the target. + * + * `-0` is accepted as 0; it is a hand-typed oddity, not an ambiguity. + */ +inline bool parseSignedDelta(const char *s, size_t len, int8_t &out) { + if (len == 0 || len > 4) return false; + size_t i = 0; + const bool neg = (s[0] == '-'); + if (neg) i = 1; + if (i >= len) return false; // '-' with nothing after it + int v = 0; + for (; i < len; i++) { + if (s[i] < '0' || s[i] > '9') return false; + v = v * 10 + (s[i] - '0'); + if (v > kMouseDeltaMax) return false; // more than one report is a typo + } + out = static_cast(neg ? -v : v); + return true; +} + +/** + * Parse `M=::::` -- five fields, always all five. + * + * The button byte is hex and must fit the five bits the report has; the movements are + * signed decimal. A sixth button bit is refused rather than quietly masked: the + * controller believed it pressed something this report cannot carry, and a click that + * silently never happens is the failure mode this protocol keeps refusing to have. + */ +inline Cmd parseMouse(const char *data, size_t len, Message &out) { + // `data` starts at the '=', so "M=00:0:0:0:0" arrives here as 11 bytes. + if (len < 11 || data[0] != '=') return Cmd::Bad; + + const char *field[5]; + size_t flen[5]; + size_t n = 0, start = 1; + for (size_t i = 1; i <= len; i++) { + if (i != len && data[i] != ':') continue; + if (n >= 5) return Cmd::Bad; // a sixth field + field[n] = data + start; + flen[n] = i - start; + n++; + start = i + 1; + } + if (n != 5) return Cmd::Bad; // missing field, or a trailing ':' + + if (!parseByte(field[0], flen[0], out.buttons)) return Cmd::Bad; + if (out.buttons & ~kMouseButtonMask) return Cmd::Bad; + if (!parseSignedDelta(field[1], flen[1], out.dx)) return Cmd::Bad; + if (!parseSignedDelta(field[2], flen[2], out.dy)) return Cmd::Bad; + if (!parseSignedDelta(field[3], flen[3], out.wheel)) return Cmd::Bad; + if (!parseSignedDelta(field[4], flen[4], out.pan)) return Cmd::Bad; + return Cmd::Mouse; +} + /** * Parse one command frame. `len` must be the frame's payload length; a trailing * CR or LF is tolerated (hand-typed test clients) but nothing else after the @@ -259,6 +333,9 @@ inline Cmd parse(const char *data, size_t len, Message &out) { case 'P': return out.cmd = parseText(data + 1, len - 1, out); + case 'M': + return out.cmd = parseMouse(data + 1, len - 1, out); + default: return out.cmd = Cmd::Bad; } @@ -301,6 +378,55 @@ inline size_t formatChord(char *out, size_t cap, uint8_t mods, const uint8_t *us return r; } +/** + * Render a signed movement the way the parser reads it: no '+', no padding. + * Returns the new offset, or 0 if `cap` would be exceeded. + */ +inline size_t putSigned(char *out, size_t cap, size_t o, int8_t v) { + char tmp[8]; + size_t n = 0; + const bool neg = (v < 0); + unsigned a = neg ? static_cast(-(v + 1)) + 1u : static_cast(v); + if (a == 0) tmp[n++] = '0'; + while (a) { tmp[n++] = static_cast('0' + (a % 10u)); a /= 10u; } + if (neg) tmp[n++] = '-'; + if (o + n > cap) return 0; + for (size_t i = 0; i < n; i++) out[o++] = tmp[n - 1 - i]; + return o; +} + +/** + * Render a mouse message, so the page, relayctl and the host tests all produce + * byte-identical frames for the same movement. Returns 0 and leaves the buffer empty + * rather than sending half a report; a movement larger than one report is the + * caller's to split before it gets here. + */ +inline size_t formatMouse(char *out, size_t cap, uint8_t buttons, int8_t dx, int8_t dy, + int8_t wheel, int8_t pan) { + static const char kHex[] = "0123456789abcdef"; + if (buttons & ~kMouseButtonMask) { out[0] = '\0'; return 0; } + if (dx > kMouseDeltaMax || dx < -kMouseDeltaMax) { out[0] = '\0'; return 0; } + if (dy > kMouseDeltaMax || dy < -kMouseDeltaMax) { out[0] = '\0'; return 0; } + if (wheel > kMouseDeltaMax || wheel < -kMouseDeltaMax) { out[0] = '\0'; return 0; } + if (pan > kMouseDeltaMax || pan < -kMouseDeltaMax) { out[0] = '\0'; return 0; } + if (cap < 12) { out[0] = '\0'; return 0; } // "M=00:0:0:0:0" plus its NUL + + size_t o = 0; + out[o++] = 'M'; + out[o++] = '='; + out[o++] = kHex[(buttons >> 4) & 0xF]; + out[o++] = kHex[buttons & 0xF]; + const int8_t v[4] = {dx, dy, wheel, pan}; + for (int i = 0; i < 4; i++) { + out[o++] = ':'; + const size_t next = putSigned(out, cap - 1, o, v[i]); // -1: room for the NUL + if (next == 0) { out[0] = '\0'; return 0; } + o = next; + } + out[o] = '\0'; + return o; +} + /** * Render a text message, so that every client -- the page, relayctl, the host * tests -- produces byte-identical frames for the same paste. `periodMs` 0 means @@ -347,15 +473,27 @@ inline size_t formatText(char *out, size_t cap, uint16_t periodMs, // 4 not armed 5 no USB host attached // 6 bad message 7 too many keys for one report // 8 text buffer full 9 a character this keyboard map cannot type +// 10 the mouse interface is off on this board +// 11 the target asked for boot protocol, so a mouse report cannot be sent +// +// The code is written in decimal at full width -- `code % 10` truncated 10 to `E0`, +// which is a code no client would recognise. Codes below 10 render byte-identically +// to the single-digit form, so every existing client keeps working unchanged; both +// clients read the code up to the first ':' rather than assuming one character. inline size_t formatError(char *out, size_t cap, uint8_t code, const char *text) { + char digits[4]; + size_t dn = 0; + if (code == 0) digits[dn++] = '0'; + for (uint8_t v = code; v; v /= 10) digits[dn++] = static_cast('0' + (v % 10)); + const size_t n = strlen(text); - if (cap < 4 || n + 4 > cap) return 0; // never a truncated frame + if (2 + dn + n >= cap) return 0; // never a truncated frame out[0] = 'E'; - out[1] = static_cast('0' + (code % 10)); - out[2] = ':'; - memcpy(out + 3, text, n); - out[3 + n] = '\0'; - return 3 + n; + for (size_t i = 0; i < dn; i++) out[1 + i] = digits[dn - 1 - i]; + out[1 + dn] = ':'; + memcpy(out + 2 + dn, text, n); + out[2 + dn + n] = '\0'; + return 2 + dn + n; } /** diff --git a/src/web_page.h b/src/web_page.h index e20a718..2ecbc7f 100644 --- a/src/web_page.h +++ b/src/web_page.h @@ -22,6 +22,14 @@ * and cancels a run of pasted text that is mid-flight. A keyboard that keeps * typing after you tabbed away is the failure mode this design cannot be allowed * to have. + * - The mouse is captured, never grabbed. "Capture the pointer" asks the browser for + * Pointer Lock -- the same thing a web game asks for -- so the cursor cannot escape + * the tab while the board is forwarding it, and Escape always gives it back. Capture + * is a second deliberate act on top of arming and it is the only way a mouse report + * ever leaves this tab. Every report restates the whole button byte, so a frame lost + * in the WiFi does not leave a button held: the next one says the truth. Losing the + * lock -- Escape, blur, tab hidden, the browser refusing -- releases every button on + * the target before anything else is considered. * - Text goes to the target from one place: the box below, by pressing its button, * while armed. It is not a macro facility -- there is no history, nothing is * scheduled or repeated, and the board holds the characters only until it has @@ -116,6 +124,12 @@ p.hint{color:var(--dim);font-size:11.5px;margin:8px 0 0} footer{padding:0 14px 18px;color:#5c6672;font-size:11.5px;max-width:980px} kbd{border:1px solid var(--line);border-bottom-width:2px;border-radius:4px;padding:0 5px; font-size:11.5px;background:#0b0f14} +.mousepad{height:118px;border:1px dashed var(--line);border-radius:10px;background:#0b0f14; + display:flex;align-items:center;justify-content:center;text-align:center;padding:0 14px; + color:var(--dim);font-size:12.5px;cursor:crosshair;user-select:none} +.mousepad:hover{border-color:var(--accent);color:var(--fg)} +.mousepad.captured{border-style:solid;border-color:var(--hot);color:var(--hot); + background:#1d0d10;cursor:none} body.armed{outline:3px solid var(--hot);outline-offset:-3px} body.off{opacity:.92} @@ -129,6 +143,7 @@ body.off{opacity:.92} link disarmed caps num + mouse free @@ -183,6 +198,7 @@ body.off{opacity:.92}
keys sent
0
text sent
0
chords
0
+
mouse reports
0
dropped
0

Held now

@@ -206,6 +222,33 @@ body.off{opacity:.92} +
+
+

Mouse

+ + + not captured +
+
+ click here to hand this tab the mouse +
+
+ buttons held +
none
+ 0 reports +
+

+ Movement is relative, like a real mouse: the board says how far the pointer + moved, never where it is, so the target's own pointer speed and acceleration decide + where that lands. Buttons are restated on every frame, which is why a frame lost in + the WiFi cannot leave one held down. Esc frees the cursor and lets go of + every button — the browser keeps that key for itself, so it never reaches the + target, and losing the tab does the same. No auto-click, no macro, no saved gesture, + nothing scheduled: if nobody is moving a mouse, nothing moves. Capture needs the + bridge armed and this board's mouse interface on (mouse on over serial). +

+
+

Send text