keyrelay/README.md

# keyrelay — a WiFi-remote USB keyboard and mouse on the M5Stack AtomS3 Lite

Type on your laptop; the keystrokes land on a *different* machine, over the air, as
genuine USB HID input — including Enter and Ctrl+Alt+Del. Capture the pointer and your
mouse moves the other machine's cursor. The board enumerates on the target as a
boot-protocol keyboard with a relative mouse on the same HID interface, so the target
needs no software, no driver and no network. The controller is a web page served by the
board itself.

```
   your laptop                          target machine
   ┌──────────────────┐                ┌──────────────────┐
   │ browser tab      │   WiFi         │                  │
   │  keydown/keyup   ├────WebSocket──►│  AtomS3 Lite     │
   │  mousemove       │   (token)      │  keyrelay        │
   │  (Pointer Lock)  │                │                  │
   └──────────────────┘                └────────┬─────────┘
                                                │ USB HID: boot keyboard
                                                │          + relative mouse
                                                ▼
                                       whatever has keyboard focus:
                                       a login prompt, a BIOS setup,
                                       a VT, a text editor
```

```
disarmed   LED dark      nothing you type or move goes anywhere
armed      LED solid red every keystroke in the tab goes to the target
```

The mouse is opt-in twice over: the bridge has to be armed *and* the page has to be
handed the pointer, which the browser only does when you click. `Esc` gives it back —
that one is the browser's own guarantee, not something this firmware has to honour.

* **Nothing types until you arm it.** Boot, reload, reconnect, focus loss, socket loss
  and 8 s of silence all disarm, and disarming sends "all keys up" **and "all mouse
  buttons up"** — to the target, as reports, not just to the board's own bookkeeping.
* **Arming needs a passphrase** (or a thumb on the board's button). The LED is readable
  from across the room, so the state is not a secret between you and the screen.
* **The mouse is captured, not grabbed.** The page asks the browser for Pointer Lock —
  the same thing a web game asks for — so the cursor cannot escape the tab while it is
  being forwarded, and `Esc` always returns it. Losing the lock, the tab or the link
  releases every button. Movement is relative: the board says how far, never where, so
  there is no position for the two machines to disagree about.
* **The keystrokes are not stored, logged or replayable.** The board keeps counters and
  the live key and button state, and nothing else — no sequence storage, no macro, no
  script, nothing it can type later, no gesture it can replay. Pasted text is typed from
  RAM and forgotten; the serial console has no command that types or clicks anything at
  all. It is a keyboard and a mouse, not a payload dropper.
* **3406 checks** cover the key-state machine, the mouse state, the report queues and the
  release funnel, the wire protocol, the text→keystroke map, SHA-1/base64, the WebSocket
  codec, the controller page's own script, the CLI's frames and the target-side verifier
  itself — all before anything is flashed.

---

## Contents

- [Is this the right tool](#is-this-the-right-tool)
- [Quick start](#quick-start)
- [The one USB port, and why the control path is WiFi](#the-one-usb-port-and-why-the-control-path-is-wifi)
- [Safety model](#safety-model)
- [Security model and trust boundaries](#security-model-and-trust-boundaries)
- [Linux notes (the target and the laptop)](#linux-notes-the-target-and-the-laptop)
- [Layout caveats](#layout-caveats)
- [Controller page](#controller-page)
- [Wire protocol](#wire-protocol)
- [Serial console](#serial-console)
- [Architecture](#architecture)
- [Tests](#tests)
- [End-to-end verification](#end-to-end-verification)
- [Measured behaviour](#measured-behaviour)
- [Troubleshooting](#troubleshooting)
- [Design decisions](#design-decisions)
- [Backlog](#backlog)
- [Restoring the sibling firmware](#restoring-the-sibling-firmware)

---

## Is this the right tool

**Good fits**

- Driving a machine that has no keyboard and should not get one: a headless box, a
  rack server at a console, a kiosk, an instrument, a second machine mid-install.
- Working on a machine you are allowed to touch but not sit at — BIOS/UEFI setup,
  bootloader and recovery screens, a login prompt, a full-screen game on a laptop
  whose keyboard you would rather not wear out.
- Keeping an agent session live on a machine you are not typing into (that is the
  sibling project [`../jiggler`](../jiggler), which is a simpler build of the same board).
- Pointing at something on a machine you cannot reach — a BIOS menu item, a dialog in a
  installer, a button in a kiosk app — with the cursor you can already see on that
  machine's own monitor.
- Anything where a KVM-over-IP would be right and you do not need video or audio, and you
  have 8 dollars and a USB port to spare.

**Bad fits — use something else**

- **You need the screen.** There is no video path. You steer the target's cursor without
  being able to see it, which is fine for a monitor you are looking at and hopeless for one
  you are not. PiKVM or TinyPilot do video + keyboard + mouse.
- **You need to know where the cursor is.** Movement is relative — see
  [Layout caveats](#layout-caveats). There is no absolute pointer, so "click the OK
  button" is not a thing this can express, deliberately: an absolute pointer would give
  the board a position to be wrong about.
- **You want unattended automation.** Deliberately impossible here (see above). If you
  want scripted input on a machine you own, `xdotool`/`ydotool`/`uinput` on that machine
  is the honest tool, and a Rubber Ducky is what this design is *not*.
- **A machine you are not authorised to input to.** Physical access is the whole
  security model of every USB HID tool, and "I could plug it in" is not authorisation.
  Use it on your own hardware or where you have explicit permission.

### What this deliberately is not

There is no macro store, no auto-arm at boot, no stored or scheduled sequence, no replay,
and no key logging — in firmware or in the tools. Not because it is hard (it is about ten
lines of `Keyboard.print()`), but because the difference between "a remote keyboard" and
"an unattended injection device" is exactly whether a human is deciding, live, with the
arming light on.

**Where the line sits.** Text you have just pasted and then deliberately sent is a
keyboard doing its job, so there is a command for it: the page's **Send text** box, or
`wl-paste | relayctl paste --yes`. It is gated like everything else — armed, a target
attached, one deliberate press of a button that says how many characters it will type —
and it cannot outlive that decision. The characters sit in RAM until they have gone out
and then they are gone: nothing is stored, nothing is scheduled, and release-all, disarm,
focus loss, a silent controller or a detached target cancels the rest of a run mid-word.
What stays absent is the part that would change what this device *is* — something the
board remembers, assembles on its own, or fires without a human in the loop.

**The mouse is the same line, drawn in the same place.** There is no auto-click, no
double-click macro, no drag recorded and replayed, no "move to (x, y)", no idle wiggle
that keeps a screen unlocked, and nothing that moves the pointer because a timer said so.
`relayctl mouse click` and `mouse drag` exist because verifying that the button bits
arrive is a thing the operator has to be able to do, and each needs `--yes`; a movement
(`mouse move`) does not, because moving a cursor is reversible by definition. If nobody is
moving a mouse, nothing moves. The pointer is captured with the browser's own Pointer Lock,
so `Esc` — which the browser keeps for itself and never forwards — is an escape hatch that
does not depend on the firmware, the network or this page being correct.

Two consequences hold either way. The CLI's ordinary commands still press one key per
invocation, which is all verification needs; scripting 30 of them is possible and would
still be 30 deliberate actions with the LED on. And a captured board is a keyboard, not a
recording — pull it out and there is nothing in it to find.

---

## Quick start

Needs [PlatformIO](https://platformio.org/) and the board's source on this machine.
The board is an ESP32-S3 with no USB-UART bridge, so all flashing is over the USB-C
connector it already uses.

```bash
cd keyrelay
tests/run_tests.sh                    # 3406 checks, no hardware needed
pio run -e bridge                     # ~1.1 MB image, ~35% of the 3 MB app slot
python3 tools/to_bootloader.py        # 1200-baud touch -> ROM download mode
pio run -e bridge -t upload           # flash
```

Build `console` as well as `bridge` before flashing: it compiles with `ARDUINO_USB_MODE=1`,
which is the only thing that checks the no-HID `#else` branches. And if `pio` on this machine is
a broken launcher rather than a working command (`ModuleNotFoundError: platformio`),
`python3 -m platformio` from an interpreter that has the package works instead — see
[`AGENTS.md`](AGENTS.md), which lists this and the other environment traps.

Then, with the board plugged into the **target**:

```bash
python3 tools/console.py wifi "My SSID" "the wifi passphrase"    # optional: join a LAN
python3 tools/console.py token "a-long-passphrase-you-pick"      # do this one, always
python3 tools/console.py status
```

Now, on the **laptop**:

| the board is on | open in a browser | because |
|---|---|---|
| your WiFi network | `http://keyrelay.local` (or the `ip=` from `status`) | mDNS, then direct IP |
| its own access point | `http://192.168.4.1` after joining `keyrelay-ctl` | no infrastructure needed |

Enter the passphrase, press **connect**, press **arm** — the LED goes solid red — and
type in that tab. Enter goes. Ctrl+Alt+Del is the button (twice), because your laptop's
own OS swallows the real combination before a browser can see it.

To send text instead of pressing keys, paste it into **Send text** and press its button:
it says how many characters it will type, the board types them as keystrokes at a pace the
board keeps, and **stop typing** abandons the rest. From a terminal it is
`wl-paste | relayctl.py paste --yes` (or `xclip -o -selection clipboard`, `pbpaste`).

To put the sibling project back on the board:
`cd ../jiggler && python3 .../to_bootloader.py && pio run -t upload`.

---

## The one USB port, and why the control path is WiFi

The AtomS3 Lite's USB-C is wired straight to the ESP32-S3's native USB pads (GPIO19/20),
and those pads belong to exactly one peripheral and exactly one host at a time. A board
that is a keyboard to machine A therefore cannot simultaneously be a serial modem to your
laptop. That single hardware fact decides the architecture:

| option | works? | why this project chose as it did |
|---|---|---|
| control over USB, HID to the same laptop | yes, pointless | the target *is* the laptop; that is the `jiggler` use case |
| second USB port / USB host shield | no | the board has one connector, and ESP32-S3 USB-host + device at once is not a thing |
| USB gadget on a Linux SBC (PiKVM-style) | yes | real answer for video + mouse; needs a Pi, a PoE HAT, and 10× the cost |
| **WiFi control, USB HID out** | **yes** | one connector stays free for the target, no host software at all |

The consequence worth stating plainly: the board draws its power and its USB identity
from the target, and its *instructions* from your network. Pulling it out stops everything
immediately — that is a feature.

---

## Safety model

Enforced in this order, each independent of the others:

1. **Disarmed is the only default.** Boot, page reload, WebSocket reconnect and page load
   all begin disarmed. ` armed` is a state the board holds, not something the page
   assumes — the page's own arming only becomes visible when the board confirms it.
2. **Arming requires authority**: the passphrase over the network, the button on the
   board, or the serial console. Nothing else can arm it, and nothing at all reaches the
   target while disarmed — not one key, not one chord, not even a "hello".
3. **Armed is displayed on the device.** Solid red LED while armed and attached. If the
   target is not attached the LED blinks red instead, because "armed" without a target
   means keystrokes are being dropped, and that deserves to look wrong.
4. **Silence disarms.** No inbound frame for `kNoClientTimeoutMs` (8 s) → all keys up and
   all buttons up, disarmed. Covers the laptop sleeping, the browser being killed, the WiFi
   dying, the network going down — every way a controller can vanish without saying goodbye.
   The browser sends a keepalive every 2 s and answers the board's own 15 s WebSocket pings,
   so a live tab holds it open indefinitely; `relayctl` does the same while it types. This
   is checked in `loop()`, and it is checked *before* the typewriter is driven: it is the
   one rule that stops a run of pasted text when the controller dies without a parting
   packet.

   **What counts as silence, and what does not.** The clock is moved only by the
   *authenticated controller*: by its authentication, and by bytes on its WebSocket. An
   accepted TCP connection is not a live operator, and neither is anything reading the
   board — a monitoring script, a health check, a second tab pointed at `/status`, or
   `relayctl listen` used to hold this timer at zero and keep the bridge armed
   indefinitely with the last key still pressed (bug 10). A board armed by its own button
   with no controller at all is deliberately left running: it has no controller to go
   quiet. This backstop is not gated on the link still existing, and must never be — see
   [Ten bugs](#ten-bugs-this-verification-caught-all-fixed).
5. **Focus loss releases every key.** The tab blurring or being hidden sends "all keys
   up" and disarms (unless you tick *stay armed when this tab loses focus*, which still
   releases the keys — you cannot keep a held Ctrl alive by tabbing away).
6. **The target taking the bus away disarms.** USB detach or host suspend clears the
   report queue, releases the keys and disarms; `ARDUINO_USB_*` events drive it.
7. **A full queue is a stop, not a drop.** 48 queued key states or 32 queued mouse
   reports without a drain means something is outrunning the USB path, so the bridge
   releases everything, reports `E7` and disarms rather than typing out of step with what
   the operator sees. The queue is not a buffer to absorb loss: 255 queued reports once
   sat through a disconnect, which is ~2 s of keystrokes delivered after the operator had
   stopped asking.
8. **State, not events.** The controller restates the *whole* key state on every event
   (see [Wire protocol](#wire-protocol)), so a lost frame cannot leave a modifier stuck
   on the target. This is the single most important design decision in the project: every
   "stuck Ctrl" horror story in HID injection tooling is an event-protocol bug. Mouse
   buttons follow the same rule — every `M=` frame carries the complete button byte — and
   movement is the one thing that is *not* state because a delta has no state to lose: a
   dropped movement frame moves the cursor slightly less, never leaves anything stuck.
9. **A paste cancels; it never pauses.** `R` means "stop typing", not "hold on". The run
   is dropped on release-all, disarm, target detach or suspend, queue overflow, focus
   loss and silence — every fail-safe above routes through the same `releaseEverything()`
   that clears the queue, raises the keys, lets go of the mouse buttons and discards what
   was still to type. The board never holds a run that outlives the decision to send it.
10. **Losing the pointer releases the pointer.** Pointer Lock going away, the tab
    blurring, the tab being hidden, disarm, `R`, silence and target detach all send an
    all-buttons-up mouse report. *stay armed when this tab loses focus* exempts the
    keyboard only — a held mouse button is never something to keep across a tab switch.
    The page stops capturing rather than queueing movement to send later.

The physical kill switch is the button: press it and the keys come up and the buttons come
up, no network, no passphrase, no software required. `Esc` is a second one for the pointer
specifically, and it is the browser's, not ours.

**One release funnel, and it has to actually send.** Every item above ends in
`releaseEverything()`, which now queues an all-keys-up *and* an all-buttons-up report to
the target rather than only clearing the board's own idea of what is down. That is the
fixed ninth bug: a held key stayed held on the target after silence-disarm, target loss or
a reload, because the report that would have released it was never queued (see
[Measured behaviour](#ten-bugs-this-verification-caught-all-fixed)).

---

## Security model and trust boundaries

| actor | can | cannot |
|---|---|---|
| Anyone on the target (its own OS) | see a Logitech G815 keyboard and a pointer in its device list; read the CDC console; type on itself | arm or drive the bridge remotely, detect that a second controller exists |
| Anyone on the board's network | fetch the page; open a socket; learn that a keyrelay exists | arm, send a key, send a chord, type a run of text, move the pointer, click, read `/status` — all need the passphrase, and all but `/status` need the bridge armed |
| Whoever holds the laptop's mouse | the browser tab's pointer, only after a click on **Capture pointer**, and only while armed | move the target's pointer without the LED being red, or keep moving it after `Esc` — Pointer Lock is revoked by the browser and cannot be retaken without another click |
| Someone guessing the passphrase | 5 attempts before a 30 s lockout during which the token is not evaluated at all | learn anything from the failure: same error, same close, no timing oracle (constant-time compare) |
| The owner of the WiFi | join the network, then everything above | — the network *is* the perimeter; see below |
| You, holding the board | arm/disarm (button), reconfigure or reflash (console), stop everything (unplug) | — |
| The target's user | notice a red LED next to their monitor, and pull it out | — |

**What the passphrase does and does not do.** It authenticates the *control* channel; it
is not confidentiality for keystrokes. The page is served over plain HTTP and the
WebSocket is `ws://`, so on a network you do not control, keystrokes are readable by
anyone who can see the traffic and the token is visible at first contact. This is a
deliberate v1 trade: WSS would need either a trusted certificate (no way to get one for
`keyrelay.local`) or certificate-pinning ceremony in a browser (which the target may not
even have). Documented in [Backlog](#backlog) with the two honest fixes.

Practical rule: **use it on a network you trust**, or on the board's own access point
whose WPA2 passphrase is a separate secret (`kr::kApPass`, change it in `src/config.h`).
Both are better than either alone: the AP means you never have to put the board on
someone else's LAN.

**Single controller, by design.** A second WebSocket upgrade is refused with `503` / `E3`
rather than queued, swapped or shared. Two controllers typing into one machine is precisely
the ambiguity a keyboard must not have, and "who owns this bridge" should never be a tie.

What is exclusive is the **keyboard**, not the board: `GET /` and `/status?token=` are
served even while a controller holds the bridge, so a page can be reloaded, a second window
can watch the counters, and `curl` keeps working mid-session. Read-only access is still
gated by what it reveals — `/status` needs the passphrase, the page needs it before it can
arm or type. If that ever goes too far, the `ap off`/network config is the blunt instrument;
there is no per-client ACL.

**The mouse widens what a compromised controller can do, so it is gated the same twice
over.** Arm plus Pointer Lock. It is also the reason there is a `mouse off` switch: the
composite descriptor is fixed when USB starts, so turning the pointer interface off is a
reboot, not a setting change — and a board that advertises no pointer cannot be made to
move one. When a target selects HID **boot protocol** (some BIOS setups, some KVMs) the
mouse is switched off by itself, because a 5-byte mouse report on a boot-protocol
interface would be parsed as keyboard data: the keyboard keeps working, the pointer stops,
and `E11` says why rather than the command silently doing nothing.

**Identity.** The board reports itself as `046d:c33f` "Logitech Gaming Keyboard G815",
matching the sibling `jiggler` firmware so the two are indistinguishable from the host
side and `/dev/input/by-id` names stay stable across reflashes. This makes the device
unremarkable in `lsusb`; it does not hide it from anyone who looks for a keyboard that
appeared at 14:32. It is your hardware and a real Logitech id, so: personal use only.
To make it honest instead, change the five `kUsb*` constants in `src/main.cpp` — nothing
else depends on them.

**What is never stored:** keystrokes, characters, key names, SSIDs beyond the configured
one, client addresses, or anything else about what was typed. Counters only (`keys`,
`chords`, `dropped`, and `typed`/`typing` for the paste path), which is what the status
display needs to be useful. A run of pasted text exists in RAM from the moment a frame is
accepted until the last character is typed or the run is cancelled, and nothing about it
survives that — no queue across a disconnect, no place to write it. The board has no clock
it trusts and no filesystem it writes to. Turning this into a logger would be a fork, not
a feature flag.

---

## Linux notes (the target and the laptop)

**Ctrl+Alt+Del on a Linux target can reboot it.** On the graphical session it is usually
inert; on a virtual terminal `systemd` maps it to `ctrl-alt-del.target`, which by default
**reboots the machine** (seven CAD presses in two seconds force it). This is why the
combination is a button with a two-press confirm and a 3-second window, never a forwarded
keypress, and why `relayctl cad` refuses to run without `--yes`.

**Ctrl+Alt+F1…F7 works** — which is a bigger deal than CAD: it switches the target's
virtual terminal, and a getty on an unused VT with **autologin** enabled will hand you a
shell as that user. This project puts the VT switches on the special-keys pad so they are
deliberate clicks rather than a chord you can fat-finger.

**Alt+SysRq+b etc.** are reachable too (the magic SysRq key is off by default on most
distros). The pad does not offer them; the keyboard table can produce them, which is what
a keyboard is.

**Keys the laptop's OS steals** (this is the laptop side, and it is the browser-capture
limitation, not a bug): `Ctrl+Alt+Del`, `Ctrl+Alt+F1…F7`, `Alt+Tab`, `Super`,
`PrintScreen`, and on GNOME/KDE a growing list of media and tiling shortcuts are grabbed
by the compositor and never reach the page. Three answers, in order of preference:

1. the **special-keys pad** on the page — each of those is one deliberate click, and the
   board synthesises the chord so the laptop never sees it;
2. run the laptop's compositor in a mode with fewer grabs while you work;
3. the **OS-wide capture client** (see [Backlog](#backlog)) — an evdev reader on the laptop
   that forwards everything before the compositor gets it. The protocol is already
   designed for it; it is a client-side addition, not a firmware change.

**While armed, the page swallows everything** with `preventDefault()`, so `Ctrl+W` will
not close the tab, `Ctrl+R` will not reload it and F11 will not fullscreen it. It also
means you cannot type into the page's own fields while armed — disarm first.

**While the pointer is captured, `Esc` belongs to the browser.** That is the point: it
revokes Pointer Lock, the page sees `pointerlockchange`, and every button goes up. What it
cannot do is disarm the bridge — the keyboard is still armed and still typing, so the status
line says so and you can press `R` or the board's button. Nothing else the laptop's
compositor grabs takes the pointer away: `Super` and `Alt+Tab` are inert while locked,
which is another reason capture is the design and a plain `mousemove` relay would not be.

**The laptop's own pointer settings apply to what gets forwarded.** Acceleration,
`libinput Accel Speed` and a high pointer speed on the laptop mean the same hand movement
sends different deltas going out, and the target then accelerates them again. Set the
laptop's pointer speed to the middle and disable acceleration while steering something you
cannot see the response curve of; the target's speed is what it is.

---

## Layout caveats

`KeyboardEvent.code` names a *physical position* on a US-ANSI keyboard, and those
positions are what travel as HID usage ids. If the **target** is set to a non-US layout,
`KeyQ` still means "the key left of A" and the target maps it to whatever its layout says.
This is identical to what a physical KVM does, and the fixes are the same:

- set the target's layout to English (US) — the one-line fix;
- or remap `CODE_TO_USAGE` in `src/web_page.h` for your laptop's board (it is one object
  literal; `tools/relayctl.py` carries the same table and
  `tests/test_relayctl.py` fails if the two drift).

**The pointer is relative, on purpose.** The board reports a boot mouse: buttons, ±127 per
axis per report, wheel and horizontal pan. It does not know where the target's cursor is,
and the target will not say, so there is no "move to" and no way to desynchronise — the
same property that makes `S=` the right model for keys. Two consequences to know before you
depend on it:

- **Acceleration is applied twice**, once by the laptop's driver before the page sees the
  movement and once by the target's driver after the board sends it. The deltas that leave
  the page are the laptop's, not your hand's, and the cursor moves by the target's idea of
  those. Fine for steering; not fine for a pixel-exact drag.
- **The target clamps at its screen edges**, and the board cannot see that happen. Moving
  5000 px right from the far right of the screen does nothing, and the board has no way to
  know. This is why `mouse move` exists as a *verifiable* primitive and no macro of
  movements is ever stored: relative motion is only meaningful while a human is watching
  the screen it lands on.

ISO keyboards: `IntlBackslash` (0x32) is the extra key next to Left Shift on ISO and
`Backslash` (0x31) is the ANSI one; both are in the table. `AltRight` is `AltGr` and
travels as its own modifier bit, which is what layout engines on the target expect.

**Pasted text goes through a third table**, `strokeFor()` in `src/text.h`: character →
modifier + usage, US-ANSI, and deliberately not position-based, because what is being
pasted is characters. Same caveat, same fix — the target must be on a US layout. A
character with no entry is refused (`E9`) rather than approximated, and `test_text` walks
the entire printable range so an omission cannot hide, while `test_relayctl` proves the
tool's validator and the board's map accept exactly the same 128 codes.

---

## Controller page

Served by the board from flash (`src/web_page.h`, ~35 KB, no CDN, no fonts, no
framework, nothing external — a keyboard bridge must work on an airplane and on a network
that is down). Its script is executed by `tests/test_page.js`, because a C++ raw string is
invisible to every compiler and a typo there fails in a browser, with a target live.

| element | why it is there |
|---|---|
| big ARM/DISARM bar | the state must be unmissable; it only changes when the board confirms |
| red page border + red LED while armed | two independent indicators of the same fact |
| status pills: network / target / link / armed / caps-num / **mouse** | "will my keystrokes land, and where", and whether the pointer is yours right now |
| **Ctrl+Alt+Del** with two-press confirm | the OS eats the real one; on Linux it reboots |
| VT1–VT7, Alt+Tab, Win, PrtSc, Menu | the chords the laptop's compositor steals |
| **⇦ snap left / snap right ⇨ / ⛶ fullscreen** | Super+Left, Super+Right, Super+PageUp — KDE window management, sent by the board so your laptop does not answer it with its own windows |
| **Release keys** and `Ctrl+Alt+Esc` | panic: every key up, disarm |
| *stay armed if this tab loses focus* | off by default; for the one workflow that needs it |
| **Send text** box, its button and its *stop typing* | paste something and have it typed as keystrokes; the button says how many characters, refuses to be armed-into accidentally, and is the only path text takes |
| *drop characters I cannot type* | off by default: a paste that quietly loses a character is worse than one that refuses |
| **Capture pointer / Release pointer** and the mouse pad | the pointer half of the bridge. Capture asks the browser for Pointer Lock, so the cursor cannot escape the tab while forwarded; the pad says what a relative mouse means before you discover it mid-drag | 
| **Release buttons** under the pad | panic for the pointer, separate from the keyboard's because the two fail independently — held-button-then-lost-link is the mouse's stuck-key, and it is the same one release funnel |
| held-keys readout and the mouse line under it, keys/chords/text/chords/mouse counters | so "it is typing" and "it is moving" are facts you can see; the mouse line names the buttons held and the button byte, because "left is down" and "the target has 0x01" are different claims and only the second one is on the wire |
| log, *log each key sent*, **clear** | the board's errors verbatim plus one line per key pressed — never one per release, never a blank line, and a repeating line is counted rather than stacked |

The caps/num indicators are read *from the target* (its HID output reports drive them),
not guessed — a caps-lock desync between controller and target is a real failure mode of
remote keyboards and this design cannot have it. The mouse counters are read from the board
for the same reason: `mframes` counts the reports the board accepted, which is the number
`watch_target.py` on the target can be compared against.

**What capture does and does not do.** The pointer is captured only while the bridge is
armed, and only after a click; `Esc`, blur, a hidden tab, disarm, `R` and the board
reporting it cannot do mouse reports (boot protocol, or `mouse off`) all end it, each
releasing the buttons. The page never accumulates movement to send later: if the link is
down, movement is dropped, because a queue of pointer motion delivered after you stopped
moving is the pointer version of a stuck key. `mouse frames 0` in the status line while you
are moving means the frames are not getting there — that is a fault to look at, not noise,
so it is shown rather than smoothed.

---

## Wire protocol

One WebSocket text frame = one command. ASCII, no JSON inbound, trivially hand-typed,
and impossible for one malformed field to desynchronise what follows it.

| dir | message | meaning |
|---|---|---|
| c → d | `T<token>` | authenticate; must be first (`kAuthTimeoutMs` = 5 s) |
| c → d | `H` | hello: status now |
| c → d | `A1` / `A0` | arm / disarm |
| c → d | `S=<mods>:<u>,<u>…` | **the complete key state** |
| c → d | `C` | Ctrl+Alt+Del (fixed in firmware, not read off the wire) |
| c → d | `Y=<mods>:<u>,<u>…` | one synthetic chord, `kChordHoldMs` (40 ms) |
| c → d | `P=<ms>:<hex>` | type pasted text, `<ms>` ms per character (0 = board default) |
| c → d | `M=<btn>:<dx>:<dy>:<wheel>:<pan>` | **the complete mouse button state** plus this report's relative movement |
| c → d | `R` | release every key *and every mouse button*, stay armed — **and drop a text run in progress** |
| c → d | `K` | keepalive |
| d → c | `OK` | accepted |
| d → c | `E<code>:<text>` | error (codes below) |
| d → c | `S{json}` | status, 1 Hz and on every change, incl. `typing` = characters of the run still to type |

`<mods>` is the modifier byte as two hex digits, `<u>` a HID usage id as two hex digits.
Both are strict: exactly two hex digits, no spaces, no trailing comma, usage `00` is not
a key, and a rejected message leaves the previous state in force — the safe outcome.

**`S=` carries state, not events.** `S=00:` means "nothing is pressed". A press and its
release are two frames and therefore two HID reports, so a fast tap cannot vanish;
identical consecutive reports collapse, which is what keeps the queue short. A single
message resynchronises the target completely, which is why no key can get stuck.

**`M=` restates the buttons and carries one report's movement.** `<btn>` is the button byte
as two hex digits — `01` left, `02` right, `04` middle, `08` back, `10` forward, and any
combination — and is the *whole* button state every time, exactly like `S=`. A frame that
lifts one button restates the ones still held; there is no button-up event to lose, so no
button can get stuck on the target. The four signed decimal fields are this report's
movement, each `±127` (`kMouseDeltaMax`, the boot-mouse byte), and all five fields are
required — `M=01` is `E6`, not "left, no movement", because a controller that forgot three
fields is a controller whose next frame is also wrong. Movement is the one field that is
not state: losing a frame moves the cursor slightly less and leaves nothing behind.

The page chunks movement to 127 per axis and keeps the diagonal shape (a 200×200 move
leaves as 127,127 then 73,73, never as two axis-parallel legs), so a fast flick cannot turn
into a silently clamped one, and it flushes the pending movement under the *old* button
state before restating a new one — that ordering is what makes a drag a drag.

**`P=` is text, typed by the board.** `<hex>` is the bytes to type, so one frame carries
at most `kTextFrameMax` (120) characters and a longer paste is several frames; `<ms>` is
the pace asked for, and the board keeps it — expansion and timing live on the USB side
because a controller's clock, arriving over WiFi in whatever order the network delivers
it, is not a keyboard timing source. Each character is pressed and released on top of
whatever the operator is physically holding, and released back to it, so a paste can
neither let go of a held key nor strand one. The board answers `E9` for a character the
US-ANSI map cannot express and types none of the frame, rather than sending a paste that
quietly lost one; `E8` means it already holds `kTextJobCap` (2048) characters. A
controller sends the next frame when the status says `typing: 0` — the board's own count,
not a guess at timing.

Error codes: `1` not authenticated · `2` locked out · `3` another controller holds the
bridge · `4` not armed · `5` no USB host attached · `6` bad message · `7` queue overflow
(released and disarmed) · `8` text buffer full · `9` a character with no US-ANSI keystroke ·
`10` the mouse interface is off on this board · `11` the target chose HID boot protocol, so
no mouse report can be sent safely.

`10` and `11` are two digits, which changed the encoder: `formatError` used to emit
`E` + `code % 10` and would have rendered both of them as `E0:` — a refusal that names
nothing. Codes now render in full, every single-digit code byte-identical to before.

The same protocol is what a future OS-wide capture client would speak — see
[Backlog](#backlog) — which is why it lives in `src/proto.h` instead of inside the server.

---

## Serial console

`/dev/ttyACM0` is the CDC half of the same composite USB device, so the console appears
on whatever machine the board is plugged into:

```bash
python3 tools/console.py                      # interactive, Ctrl-] to quit
python3 tools/console.py /dev/ttyACM0 status  # one command, print the reply
```

| command | effect |
|---|---|
| `status` | the same JSON the controller gets, plus link, queue and text-run state (`text  N chars typed, M still to go`) |
| `wifi <ssid> [passphrase]` | station network to join (stored in NVS, reconnects) |
| `wifi clear` | forget it; the access point becomes the only way back |
| `ap auto\|force\|off` | AP policy: when the station is down / always / never |
| `token <passphrase>` | control passphrase, 6–32 chars, never echoed back |
| `arm` / `disarm` | same as the button |
| `mouse on\|off` | whether the bridge enumerates a pointer interface at all. Stored, and it needs a **reboot**: the composite descriptor is fixed when USB starts, so this cannot change under a live target. Off means the pointer cannot be moved by anything, ever, until it is turned back on |
| `reboot`, `help` | |

`status` also says which of the two the running bridge is, and why: a line like
`mouse   interface=yes target=report buttons=00 reports=41 queued=0`, where
`target=boot` means the target asked for boot protocol and the pointer is disabled by
itself until it re-selects report protocol.

There is deliberately **no command that types or clicks anything** — the console is
configuration, not an input path. That is why `mouse on|off` is the only mouse command and
it changes a capability rather than performing an action. Replies are printed with a blocking write (the global serial timeout is
0 so logging into a port nobody reads can never stall `loop()`); without that, multi-line
replies truncate at the 128-byte CDC TX ring, which is exactly what happened on the first
build and is why `pollConsole()` bumps the timeout around each reply.

---

## Architecture

```
src/
  config.h            defaults: network, timeouts, caps, buffers   (NVS overrides)
  keystate.h    pure   modifier byte + pressed-set -> 8-byte HID report, rollover
  mouse.h       pure   button byte + signed deltas -> 5-byte relative report; the
                       buttons are restated, not evented, exactly like keystate
  text.h        pure   pasted text -> US-ANSI keystrokes, paced; run buffer and cancel
  output.h      pure   the one place keystrokes, pasted runs, chords and mouse reports
                       become two ordered queues, and the one place anything is released
  proto.h       pure   the wire protocol: parse/render commands, constant-time compare
  sha1.h        pure   SHA-1 + base64 for the WebSocket handshake
  ws_codec.h    pure   handshake request parsing + RFC 6455 frame codec
  web_page.h           the controller page, in PROGMEM (~48 KB now, with Pointer Lock)
  control_server.{h,cpp}  sockets: HTTP routes, WebSocket lifecycle, auth, heartbeat
  main.cpp             glue: USB identity/HID composite, WiFi, chords, text run,
                       LED, button, watchdogs, serial console
tools/
  to_bootloader.py  1200-baud touch into ROM download mode (stdlib termios)
  console.py        serial console, one command or interactive (stdlib)
  relayctl.py       controller from a terminal: status/arm/disarm/key/chord/listen/
                    paste/mouse
  watch_target.py   on the target: decode the board's evdev events (keyboard *and*
                    mouse), prove delivery
tests/
  run_tests.sh      builds and runs everything (host g++ or a local container)
  test_keystate.cpp test_mouse.cpp test_output.cpp test_proto.cpp test_text.cpp
  test_ws_codec.cpp
  test_relayctl.py  test_page.js (node, skipped when absent)
AGENTS.md           operating manual for whoever edits this next: the invariants, the three
                    tables that must agree, the build traps, the verification standard
TODO.md             open items, kept with their root cause rather than deleted
```

If you are an agent or a new contributor, read `AGENTS.md` before changing anything: several
rules here are invisible to a compiler and to the test suite at the same time.

The seven modules marked *pure* have no Arduino and no ESP-IDF include, so the identical
code compiles and runs on a host under `-fsanitize=address,undefined`. That is not
convenience for its own sake: the sanitiser caught five real defects in the frame decoder
before the first flash (a `reset()` that left the reader latched on the previous frame, an
unsequenced `_buf[_got++] = b ^ _mask[_got & 3]`, a missing FIN check that silently
accepted fragmentation, a mask key skipped on zero-length frames, and a
`"sec-websocket-key:"` prefix compared at the wrong length so the accept hash was
computed from `"ey: …"`).

The keystroke path, end to end:

```
keydown/keyup -> full state S=.. -> Reader (RFC6455) -> proto::parse -> auth gate
  -> KeyboardState::apply (dirty only if the report changed)
  -> ReportQueue (dedupe identical, overflow = release + disarm)
  -> drainReports: tud_hid_n_ready(0) ? USBHIDKeyboard::sendReport : keep queued
  -> HID interrupt endpoint, boot protocol, 8-byte report -> target

pointer move  -> M=btn:dx:dy:wheel:pan  -> same gate (armed + attached + mouse allowed)
  -> MouseState::apply (button byte restated; idle frames produce no report at all)
  -> a second ReportQueue, same dedupe and the same overflow rule
  -> drainReports, after the keyboard queue -> report #2 on the *same* HID interface

fail-safe (any of: disarm, R, blur, silence, detach, suspend, overflow)
  -> releaseEverything() -> OutputStage::releaseAll()
       clears both queues, both trackers, the chord and the text run, THEN queues
       all-keys-up and (if the target can take it) all-buttons-up
```

`USBHID::SendReport` in the core is synchronous (it waits on a semaphore for the transfer
to complete, up to 100 ms), so `drainReports()` takes at most `kMaxReportsPerLoop` = 8
reports per pass across **both** devices — bounded time in the USB path, and the control
plane never waits on it.

**Both devices live on one HID interface, which is a constraint with three consequences.**
The keyboard's `USBHIDKeyboard` is the first HID instance, so it claims the boot keyboard
subclass; the mouse is added as the second report on that same instance, which is why
`mouseDev` is created after `kbd` and never before. Because the descriptor is written when
`USB.begin()` runs, `mouse off` cannot take effect until a reboot. And because a
boot-protocol host parses every report on that interface as an 8-byte keyboard report, a
target that selects boot protocol gets the pointer switched off under it (`E11`) rather than
sending 5-byte mouse reports that would arrive as garbage keystrokes.
Chords (Ctrl+Alt+Del, VT switches) are synthetic: the press is pushed, and after the hold
the *tracked* state is pushed back, so keys that were genuinely held before a chord
survive it and the tracker is never corrupted by something the operator did not type.

The USB interface is confirmed boot-protocol on real hardware, which is the precondition
for Windows accepting Ctrl+Alt+Del as the secure attention sequence:

```
$ for i in /sys/bus/usb/devices/*:1.*/; do echo "$(basename $i) class=$(cat $i/bInterfaceClass) sub=$(cat $i/bInterfaceSubClass) proto=$(cat $i/bInterfaceProtocol)"; done
7-1:1.0 class=03 sub=01 proto=01        <- HID, boot subclass, keyboard protocol
7-1:1.1 class=02 sub=02 proto=00        <- CDC-ACM control
7-1:1.2 class=0a sub=00 proto=00        <- CDC-ACM data
```

Two build environments: `bridge` (the real thing, USB-OTG + TinyUSB) and `console`
(`ARDUINO_USB_MODE=1`: no HID, so it cannot type, but WiFi/page/WebSocket/LED/button all
live — for separating "the network side is broken" from "the HID side is broken" without
rebuilding. Arming refuses with `E5` there, on purpose).

---

## Tests

```bash
tests/run_tests.sh              # uses g++ if present, else a local container image
SANITIZE=1 tests/run_tests.sh   # + ASan/UBSan  (recommended, and what found 5 bugs)
```

| suite | checks | what it pins down |
|---|---|---|
| `test_keystate` | 40 | idempotence, transition preservation (a fast tap is two reports), 6-key limit, ErrorRollOver and recovery from it, dedupe, `clear()`, the exact Ctrl+Alt+Del bytes `05 00 4c 00 00 00 00 00` |
| `test_mouse` | 48 | the 5-byte report layout, every button bit where the HID spec says it is, idle frames producing **no** report, a button change with no movement still producing one, the button byte being restated rather than evented, `releaseAll` being idempotent, and that no frame can put a value outside one signed byte on the wire |
| `test_output` | 153 | **the release funnel.** Queues hold their order and collapse duplicates; releasing with keys held queues an all-keys-up report *when there is a target to receive it*; releasing drops a press that was queued but never sent; releasing with a mouse button held queues all-buttons-up; boot protocol or `mouse off` means no mouse report is queued **but the key release still is**; a pasted run and an in-flight chord are both cancelled; and overflow is detected on both queues. Regression-proven: reinstating the old `releaseEverything()` fails 23 of these |
| `test_proto` | 193 | every command form, strict rejection of malformed state specs, bounds (cap on keys, token length and text frames), render/parse round-trip, error framing, constant-time compare, **every chord button's bytes matched against the label printed on it**, and the page's text constants pinned to `config.h` |
| `test_text` | 459 | the whole printable-ASCII map walked (nothing may be unmapped, nothing may claim a key it has no glyph for), shifted glyphs and the digit row, what is refused rather than guessed at, all-or-nothing frame acceptance, pace arithmetic and clamping, the exact press/release report sequence for two characters, overlay on the operator's held keys, ErrorRollOver propagation, cancel, buffer-full, `P=` render/parse round-trip including the largest legal frame, and one end-to-end case that takes the exact frame the page sends and asserts the HID report sequence the target ends up seeing |
| `test_ws_codec` | 135 | SHA-1 against published vectors including the million-`a` case, base64 against RFC 4648, the RFC 6455 handshake example, frame encode/decode at the 125/126 and 64-bit boundaries, byte-at-a-time delivery, and rejection of unmasked, fragmented, reserved-bit, binary, over-long and oversized frames |
| `test_relayctl` | 160 | the Python client's codec, that its key table has not drifted from the page's, that its text validator agrees with the firmware's character map on all 128 ASCII codes, CRLF normalisation, `TEXT_FRAME_MAX`/`MAX_PASTE_CHARS` pinned to `config.h`, the mouse frame builder refusing a sixth button and any movement that does not fit one report, movement splitting that reproduces the original displacement exactly, click/drag/wheel frame *ordering*, every generated frame parseable by the firmware's own parser, and the button bits and frame pace agreeing between page, tool and `config.h` |
| `test_page` | 84 | the page's script, executed: blank and whitespace-only log lines never render and a repeating line is counted not stacked, character validation, CRLF, hex encoding, one frame for a short paste and exactly the right bytes, a busy board holding the next frame back, a 300-character paste reassembling to the text that went in, refusal of untypeable text and the opt-in drop, cancellation by stop/disarm, **the mouse: capture requires the bridge armed, movement becomes exactly one `M=` per report with nothing sent when idle, a 400×300 flick split so no frame exceeds one signed byte while the diagonal shape survives, the click ordering that makes a drag a drag, wheel pixels quantised to notches with the remainder carried, `Esc`/blur/disarm each releasing the buttons, `stay armed on blur` exempting the keyboard only, a board that reports `mouse:0` refusing capture and dropping one in progress — and every mouse frame the page emits parsed by the firmware's own parser** |
| `test_page` | 47 | the page's script, executed: blank and whitespace-only log lines never render and a repeating line is counted not stacked, character validation, CRLF, hex encoding, one frame for a short paste and exactly the right bytes, a busy board holding the next frame back, a 300-character paste reassembling to the text that went in, refusal of untypeable text and the opt-in drop, and cancellation by stop/disarm, plus every element id the script reaches for checked against the markup |

The verifier is verified too, because twice now it was the verifier that was wrong: node
classification (EV_REL means a pointing device — not the device's name, not the order the
glob found them in), movement coalescing one report into one line, net displacement
arithmetic, hi-res wheel events not counted as extra notches, and a stuck button being
called out in those words.

The last six checks in `test_relayctl` are **source pins**, and the suite says so out loud:
they assert that `loop()`'s silence disarm runs off `controllerSeen()` rather than
`linked()`, that `acceptNew()` never stamps the controller clock, and that `closeLink()`
stamps it only for a controller link. They cannot execute — that code needs a socket and a
board — but each one is a sentence a later edit can undo with no compiler error and no
behavioural test failing on the host, which is exactly how bug 10 survived. Reinstating
either half of it fails 3 of them.

Gaps, stated plainly: nothing here exercises real sockets, real WiFi associations, or real
USB transactions — those are the [end-to-end procedure](#end-to-end-verification) below.
The HTTP routing and auth-lifecycle code in `control_server.cpp` is the least-tested part
of the project; it is deliberately the thinnest layer, but it is not covered by a unit
test.

---

## End-to-end verification

Three machines' worth of roles on two boxes: the laptop is the controller, the board is
the bridge, and any Linux box (including the laptop itself) is the target.

```bash
# 1. on the target: what actually arrived, decoded from evdev, focus-independent
python3 tools/watch_target.py --seconds 60

# 2. on the laptop: drive it from the terminal
export KEYRELAY_TOKEN='the-passphrase-you-set'
python3 tools/relayctl.py --host keyrelay.local status
python3 tools/relayctl.py --host keyrelay.local arm
python3 tools/relayctl.py --host keyrelay.local key Enter
python3 tools/relayctl.py --host keyrelay.local chord 04 2b      # Alt+Tab, by hand
python3 tools/relayctl.py --host keyrelay.local cad --yes
printf 'Hello from keyrelay\n' | \
  python3 tools/relayctl.py --host keyrelay.local paste --yes    # 19 characters
python3 tools/relayctl.py --host keyrelay.local listen 20      # heartbeats, live
python3 tools/relayctl.py --host keyrelay.local disarm

# 3. the mouse, same two terminals
python3 tools/relayctl.py --host keyrelay.local mouse move 120 -40
python3 tools/relayctl.py --host keyrelay.local mouse click --yes
python3 tools/relayctl.py --host keyrelay.local mouse wheel -3
python3 tools/relayctl.py --host keyrelay.local mouse drag 200 0 --button right --yes
python3 tools/relayctl.py --host keyrelay.local mouse release
```

And in the browser, which is where the pointer is actually usable: open the page, arm it,
press **Capture pointer**, move the mouse (the target's cursor moves, the laptop's is
hidden), hold left and move for a drag, press `Esc`. Then arm it, capture, hold a button,
and **close the tab** — the target must stop clicking within 8 s of the silence timeout.

What a correct run proves, and how to tell:

| claim | evidence |
|---|---|
| HID reaches the host | `watch_target.py` prints every press/release with a timestamp |
| text is correct | its `typed` line, with case from the shift state |
| Enter works | `Enter press/release` and `<Enter>` in the typed text |
| Ctrl+Alt+Del works as a *combination* | `>>> Ctrl+Alt+Del <<<`, one line, not three |
| no stuck keys | `still held` is absent at the end of the run |
| **pasted text is keystrokes, not a shortcut** | the run's `typed` line reproduces the text, and the press count is roughly twice the character count — a paste that arrived as nothing, or as a single odd chord, means the character map or the pace, not the target |
| **Alt+Tab is Alt+Tab** | `LAlt press` then `Tab press` — and *not* `LCtrl`. This combination was `Ctrl+Alt+A` on the wire until it was caught by the label-vs-bytes test; `watch_target.py` is what settles any doubt about which keys the board actually sent |
| a paste stops | start a long one, press **stop typing** (or blur the tab, or press the board's button): the `typed` text stops mid-sentence and nothing further appears |
| target LEDs are read back | `LED CapsLock on` on the target, caps indicator on the page |
| silence disarms | arm it, close the tab, wait 8 s, `status` says `armed: 0` |
| auth is real | `relayctl` with a wrong token fails; the board logs `authfails=1`; 5 tries → `E2` |
| **movement reaches the target as movement** | `watch_target.py` prints a `movement dx … dy …` line per report and the summary's `net position` matches the direction and rough distance you moved — the keyboard half of the watcher cannot show this, which is why the mouse node is watched separately |
| **the button bits are the buttons they claim to be** | `mouse click --yes` gives `BTN Left press/release`; `--button right` gives `BTN Right`. A button that arrives as the wrong one is a bit-order bug and looks identical from the page |
| **a drag is down, move, up in that order** | `BTN Left press`, then `movement` lines, then `BTN Left release` — any other order is a drag that clicks at the start and drags nothing |
| **no stuck buttons** | `still held` absent from the mouse summary at the end. Check it the hard way: hold a button and close the tab |
| **the wheel is notches, not pixels** | `mouse wheel -3` produces `REL_WHEEL -3` three times, and hi-res events do not inflate the count |
| **boot protocol refuses the pointer, not the keyboard** | on a BIOS/KVM that selects boot protocol, keys keep working and `M=` answers `E11`; `console.py status` says `target=boot` |
| **`mouse off` really removes the capability** | `console.py mouse off` + reboot → `lsusb`/evdev show no pointing device, `watch_target.py` notes the missing interface, the page's Capture button is disabled, and keys still work |

`watch_target.py` finds the board by USB id, not device name, so it keeps working on a
different port or a renamed input node, and it accepts both the Logitech identity and the
Espressif default so it works on older builds.

A pasted run prints one line per press *and* release, so a few hundred characters is a
lot of scrolling: watch the `key presses` and `typed` summary lines at the end rather than
the stream, and give `--seconds` enough room for the pace (`2048` characters at the default
12 ms is about 25 s of typing, plus the run's own margins).

---

## Measured behaviour

_On real hardware: AtomS3 Lite → this laptop (Linux, US layout), board on the same WiFi,
firmware 1.0.0, `bridge` environment._ Fill in / re-run after any change to the keystroke
path; the numbers below are the shape a correct run has.

```
$ python3 tools/console.py status
status  {"v":"1.0.0","up":7,"usb":1,"armed":0,"net":"ap","ssid":"keyrelay-ctl",
         "ip":"192.168.4.1","keys":0,"chords":0,"drop":0,"roll":0,"leds":0,
         "dflt":1,"ap":0}
link    controller=no rejected=0 authfails=0
queue   0 pending, 0 held, ok

$ python3 tools/relayctl.py --host <ip> status        # armed, typing, then disarmed
  armed      YES -- keystrokes are being forwarded
  counters   keys=… chords=0 dropped=0

$ python3 tools/watch_target.py --seconds 30          # on the same laptop
  t=  0.42s  A press
  t=  0.48s  A release
  ...
  key presses     …
  typed           '...'
```

### Confirmed on hardware

AtomS3 Lite → Linux laptop (US layout), board joined the LAN, firmware 1.0.0.

| what | result |
|---|---|
| flash | 1 134 768 bytes written (706 661 compressed) in 8.1 s at 1 116 kbit/s, hash verified |
| footprint | RAM 26.7 % (87 400 / 327 680); Flash 35.5 % of the 3 MB app slot, measured with the paste path in |
| enumeration | `class=03 sub=01 proto=01` — boot-protocol keyboard, the Windows secure-attention precondition |
| reflashing | 1200-baud touch into ROM download mode works with this firmware running, so no button and no GPIO0 |
| page delivery | `GET /` → 200, `Content-Length: 35164`, zero external references; 404 on unknown paths; 401 on `/status` without the passphrase; `keyrelay.local` resolves over mDNS |
| arming from the browser | the page armed the board over `ws://` on first try (observed as `controller=yes armed=1`) |
| keystroke delivery | 6/6 inert keys arrived on the target with matched press **and** release: F13, F15, F17, ScrollLock, CapsLock ×2 |
| chord path | `chords=1` counted and delivered (Shift+F15) |
| LED read-back | `LED CapsLock on` then `off`, coming from the *target's* HID output reports |
| **silence disarms** | controller process killed → `armed:0` within 11 s, keys released. **Worth knowing:** that run passed *before* the silence timer existed — killing a process closes the socket, and the socket-close path disarmed. The FIN-less death (laptop slept, network vanished) is what the timer covers, and it is wired now. A check that passes for the wrong reason is still a check that passes, which is exactly the problem |
| single controller | a second client got `E3` / `503` and the board logged `rejected=1` — no takeover |
| **mouse enumeration** | the target created **two** input devices from one HID interface — `keyboard=/dev/input/event19`, `mouse=/dev/input/event20` — with `caps rel: 1943`, i.e. REL_X, REL_Y, REL_WHEEL, REL_HWHEEL and both hi-res axes |
| **mouse movement** | `mouse move 120 -40` → `movement dx +120 dy -40` on the target, exact, one report |
| **mouse buttons** | `click` → `BTN Left press` + `BTN Left release`; `--button right` → `BTN Right`. The bit order the page claims is the bit order the target sees |
| **the wheel** | `mouse wheel -3` → `REL_WHEEL -3`. It took a wrong axis table in the *verifier* to prove this worked — see the note under Tests |
| **drag ordering** | `drag 200 0 --button right` → `BTN Right press`, `movement dx +127`, `movement dx +73`, `BTN Right release`. Both halves of the property at once: the ordering is down-then-move-then-up, and the 200 px left as two reports that stay inside one signed byte |
| **keyboard and mouse together** | keys, chords, a 14-character paste and mouse reports in one session, `drop=0 roll=0`, `keys=3` and `mframes` advancing independently |
| **a controller that dies holding a button** | `BTN Left press` → client `os._exit()` (no close, no disarm) → **`BTN Left release` 9.4 s later**, and a live controller holding the same link with keepalives stayed armed for the 24 s it was tested. This is bug 10, and it is the property the whole release funnel exists to provide |
| lockout | 5 wrong passphrases → `E2`, for 30 s, from the page's own retry loop |
| queues | `drop=0 roll=0` throughout; `queue 0 pending, 0 held, ok` at rest |

### Ten bugs this verification caught (all fixed)

Worth listing, because each one is a class of bug that HID-over-network tools ship with:

1. **CDC replies truncated at 128 bytes** — `setTxTimeoutMs(0)` meant "drop what does not
   fit", and `serial status` came back cut mid-JSON. Replies now temporarily switch to a
   blocking write and flush. A diagnostic tool that lies about state is worse than none.
2. **The CLI lowercased the whole handshake header block** — base64 is case-sensitive, so
   every `Sec-WebSocket-Accept` compared wrong and the board looked broken while answering
   correctly. Header *names* are case-insensitive; values are not.
3. **The CLI reported stale status** — the board heartbeats at 1 Hz and the key helpers do
   not read while typing, so `disarm` printed `armed YES` from a heartbeat sent *before*
   the command. `arm`/`disarm` now drain, then poll until the board actually reports the
   state asked for — confirm, don't assume.
4. **The target-side verifier mislabelled every LED** — its table was copied from the HID
   bit numbering, but Linux `EV_LED` indices are zero-based (`LED_CAPSL` is 1, not 2).
   Verified against real events.
5. **Unnamed key codes looked like lost keystrokes** — F13–F24 had no names in the
   verifier's table, so a perfectly delivered run appeared to drop its first three keys.
   Now named, and unknown codes are counted rather than silently skipped.
6. **The single-controller gate was applied at `accept()`**, before the request was even
   read — so while a controller was connected, *anything* else that touched the board got
   a 503: `curl` of the page, a second browser, and worst of all **pressing reload on the
   page that already held the bridge**, since the board could still be holding the slot the
   instant the new request arrives. The fix moves the gate to what it actually protects: a
   second `/ws` upgrade is still refused (that is a controller claim), while read-only HTTP
   is served on a separate short-lived socket through the same routing code (`serveReadonly`)
   so the two paths cannot drift. Verified live: with your browser holding the bridge, the
   page returns 200 and `/status` returns 401 without the token, 200 with it, and a second
   upgrade still gets 503 while the incumbent stays `controller=yes` and healthy.
7. **The Alt+Tab button sent Ctrl+Alt+A.** Its `data-chord` was `05:04` — LCtrl|LAlt with
   usage `0x04`, which is the A key. It parsed cleanly, it was correctly gated by arming,
   it arrived intact and it was the wrong combination: Alt+Tab is `04:2b`. Reported by a
   user who could not work out why the Pi ignored it, and provable because the page stores
   the chord bytes on the button instead of a name to look up — so nothing on the browser
   side could ever disagree with a label. Every labelled chord is now pinned to the
   combination its label promises, by modifier byte and usage id.
8. **`relayctl`'s drain called two methods that do not exist** (`_send_frame`,
   `_check_error`) while answering a ping, so any session outliving the board's 15 s ping
   died on an `AttributeError` instead of reading its next frame. Only reachable on long
   sessions, which is exactly what typing text is; it answered pings the way `_poll` does
   and raises on error frames now.
9. **"Silence disarms" was documented and not implemented.** `kNoClientTimeoutMs` existed,
   `ControlServer::lastInboundMs()` existed, and nothing ever compared them. Harmless while
   the only input was a human keystroke (a vanished controller sends no more keystrokes),
   and not harmless once a run can type for twenty seconds on its own: a laptop that slept
   mid-paste, with a link that never got a FIN, would keep typing into the target.
   `loop()` now checks it, before driving the typewriter, and clients keep-alive while they
   type. The lesson is the one the whole project keeps relearning: a safety property that
   is only written down is not a safety property.

10. **A held mouse button stayed held, forever, and anything on the network could keep it
    that way.** Found while verifying the mouse, with the mouse innocent: `releaseEverything()`
    queued the release reports (bug 9's fix), and they still did not go out.

    Three things stacked up. `closeLink()` ran the release callback only `if (linked())`,
    and `linked()` is `_phase == WebSocket && _authed` — so any path that cleared `_authed`
    on its way to closing the socket closed it with the keys down and nothing downstream
    able to notice. The board then sat `armed=1 mbtn=01` with `phase=none`. And the
    backstop that should have caught it, `loop()`'s silence timer, was gated on
    `control.linked()` — **a controller that has gone away is precisely the case in which
    `linked()` is false**, so the fail-safe designed for a vanished controller could only
    ever run while the controller was there. Measured: `BTN Left press`, controller killed,
    target still holding the button **26 s later**, and the armed state never expired.

    The second half was worse. `acceptNew()` stamped `_lastInboundMs` for *any* incoming
    TCP connection, so a read-only `GET /status` — which is served by design while a
    controller holds the bridge — reset the silence clock. Any authenticated monitor,
    second tab or health probe kept a dead controller's armed state open indefinitely. The
    first fix traded that for stamping in `closeLink()`, which moved the reset from
    *accept* to *every close* and changed nothing observable; `wasLinked` is the honest
    test, because it is true exactly when the thing being closed can leave key state
    behind.

    Fixed as: `wasLinked = (_phase == Phase::WebSocket)`; the silence clock moved only by
    an authenticated controller's own traffic or by that link closing; `controllerSeen()`
    distinguishes "no controller since boot" (button-armed, leave it alone) from "one came
    and went" (never stay armed on its strength). Verified on hardware with the monitor
    running the whole time: `BTN Left press` → `BTN Left release` at 9.4 s, armed 0 and
    staying 0 for 37 s of continued polling, while a live controller holding the link with
    keepalives stayed armed through the same test.

    What to take from it: **never gate a fail-safe on the absence of the thing its
    condition detects.** And a backstop needs its own test, because this one passed every
    keyboard suite for exactly the reason bug 9 did — the socket-close path happened to
    cover the case being checked. `tests/test_relayctl.py` now pins all three sentences in
    source, and reinstating either half of the bug fails 3 checks.

### The hazard nobody warns you about

**When the target is the machine you are working on, your keystrokes land on your own
focused window.** The first live test sent `Escape`, and the target was the laptop running
this agent — so the keystroke interrupted the very session that sent it. That is not a bug
in the bridge; it is exactly the transparency the thing is designed for, pointed the wrong
way. So:

- verify with keys that nothing reacts to — **F13–F24 and the Lock keys** are inert almost
  everywhere, and they exercise the same code path as Enter;
- before arming against your own machine, decide what has focus; a terminal will *execute*
  what you send;
- the page releases keys and disarms when its tab loses focus, which is also how you get
  control back: click anything else, then click back.

---

## Troubleshooting

| symptom | cause, and what to do |
|---|---|
| the page does not have a feature the README describes | **the page is served from the board's flash**, so the image you built is not the image on the device. `curl -s http://keyrelay.local/ \| wc -c` and compare with the page in `src/web_page.h`; if they differ, reflash (`to_bootloader.py` then upload) and *reload the tab* — one already open still holds the old page in memory, passphrase notwithstanding |
| `pio run -t upload` sits at `Connecting…` | the app owns the USB pads. `python3 tools/to_bootloader.py`, or hold the button 3 s while powering up (LED solid red) |
| page will not connect | wrong address: `console.py status` for the real `ip=`; `net:"ap"` means you must join `keyrelay-ctl` |
| `keyrelay.local` does not resolve | mDNS is queried per-process with no cache on glibc (`mdns4_minimal [NOTFOUND=return]`), so a cold probe that misses returns "not known" immediately — **just retry**, or use the IP. Across VLANs/VPN it is often blocked outright |
| `arm` does nothing, LED blinks red | no USB host attached (`usb:0`); the board is not plugged into the target, or the target suspended the bus |
| typing does nothing, `armed` says yes | the tab lost focus: click into the page. Or the target's focus moved — that is the target's answer, not a bug |
| `E4 not armed` | it is not armed. `arm` first; chords and states are never free passes |
| `E3 another controller` | another tab/machine holds it. Close it; there is no takeover, deliberately |
| `E2 locked out` | 5 wrong passphrases; wait 30 s |
| keys arrive out of order or missing | check `dropped` (queue overflow) and that the laptop's signal is good; `WiFi.setSleep(false)` is already set. If a *verifier* says keys are missing, check it names the codes — see [the hazard nobody warns you about](#the-hazard-nobody-warns-you-about) |
| the CLI says `another controller holds this bridge` | your browser page has it, and that is the rule working. Close the tab; there is no takeover |
| pasted text arrives as different characters | the **target** is not on a US layout — characters travel as positions, exactly like the live keys. Or the run was sent with *drop characters I cannot type* / `--strip`, which reported how many it removed |
| a paste stops before the end | something cancelled it, on purpose: the tab lost focus, the bridge disarmed, the target suspended the bus, `E7`/`E8`, or 8 s with nothing inbound from the controller. The page's log says which, and how far it got |
| a chord button does something other than its label | the bytes on the button are wrong — the page carries the wire form itself, so nothing browser-side can catch it. `curl` the page and read `data-chord`; this was the case for Alt+Tab (`05:04`, Ctrl+Alt+A) until the label-vs-bytes test was written |
| a key is stuck on the target | `R` / **Release keys** / the button. Then check whether the tab lost focus without the page noticing (a hidden tab still gets blur on most browsers) |
| Ctrl+Alt+Del does nothing on the target | on Linux this is normal on a graphical session and *fatal* on a VT; on Windows the interface must be boot protocol (confirmed above) |
| target keeps suspending the bus | the target suspends, which disarms by design; disable suspend or plug into a port that stays powered |

---

## Design decisions

**Why a web page and not a host daemon.** Because "install something on the laptop" is
the difference between a tool you can use in ten minutes on a machine you are borrowing
and a project you set up once at your own desk. The page is served by the board, so the
only software that has to exist is the browser you already opened.

**Why full state instead of key events.** Covered under [Safety
model](#safety-model) point 8, and it is the decision that makes every other guarantee
here cheap: a lost message is corrected by the next one, so there is no resynchronisation
protocol, no sequence numbers, no stuck-key reconciliation timer.

**Why the board expands and paces text, not the browser.** It is tempting to turn a paste
into existing `S=` frames client-side and change no firmware at all. Timing is the reason
not to: a controller reaches the board over WiFi, in frames, with whatever gaps the network
inserts, and a character is a press and a release with a hold either side of it on a target
that samples every millisecond. A browser tab that stalls for a second would produce either
a burst of keys or a run that outran the 48-deep report queue and tripped the
overflow-disarm. Expanding on the board also means the page and the CLI cannot disagree
about what a character types, which is the same argument that put the protocol in
`proto.h`. Cost: the character map exists twice — positions in `web_page.h`/`relayctl.py`,
characters in `text.h` — and a test pins all three together.

**Why a paste refuses rather than sanitises.** Curly quotes, em dashes and accented
characters have no key on a US-ANSI keyboard. Dropping them quietly would send a passphrase,
a `rm` line or a config value that is *nearly* what was pasted, which is the failure mode no
one notices until it has landed. So the board answers `E9` naming the code point and types
none of the frame, the page says the same before it sends anything, and dropping is an
explicit, per-paste opt-in that says how many characters it removed. Fail-closed, and
loud.

**Why no WebSocket library.** The control channel needs text frames, ping/pong and close.
A library buys fragmentation, extensions, TLS plumbing and many-client handling — attack
surface and heap, none of it wanted. The hand-rolled codec is ~200 lines of pure logic
with 135 host checks behind it, and every deviation from the subset is a hard close.

**Why HTTP/WS without TLS.** See [Security model](#security-model-and-trust-boundaries).
The honest options in the backlog are a user-installed CA (works, annoying) or a
client-side proxy that terminates TLS with a pinned key. Pretending confidentiality here
would be worse than documenting its absence.

**Why NVS for configuration.** So the passphrase and network live in the board rather than
in a header you recompile, and survive reflashing — and so a passphrase never has to enter
a shell history, a git diff or a build log. The `token` command deliberately does not echo
it back.

**Why the sibling project's USB identity.** Two firmwares on one board that are
indistinguishable from the host side, and stable `/dev/input/by-id` names for the tools.
Cost: it is a borrowed VID/PID, so it is documented as personal-use and one edit away
from being honest.

---

## Backlog

Ordered by what would make the biggest difference first.

1. **OS-wide capture client** (`tools/keyrelay-capture.py`, laptop side): grab the
   keyboard with evdev, forward full state over the same protocol, `uinput` a shadow if
   needed. Solves every compositor-stolen key and is the missing piece for
   "transparently drive another machine". Firmware unchanged — the protocol was designed
   for it. Needs root or `input` group + a `udev` rule; documented as opt-in because it
   swallows the laptop's keyboard while running.
2. **TLS with a pinned certificate** (WSS): either an on-board CA the laptop trusts once,
   or a `localhost`-terminating proxy. Until then: trusted network or the board's AP.
3. **A second, independent key path for the panic release**: a GPIO or button
   double-click that force-releases without waiting for the loop's disarm path, so the
   panic release does not depend on the code that just failed.
4. **Per-connection rate limiting** — a token bucket on state messages, so a wedged or
   hostile client cannot pump the queue into the overflow-disarm path as a nuisance
   attack (currently it disarms: safe, but noisy), and a cap on page fetches now that the
   page is served even while a controller is live (each is ~35 KB, and `no-store` means a
   determined client will ask for it again and again).
5. **mDNS also on the AP interface**, verified on each chipset — some builds answer only
   on the station interface, which is why the docs say "IP if the name fails".
6. **WPA3/PMF on the SoftAP** and a per-board AP passphrase printed once at first boot,
   instead of one shared constant in `config.h`.
7. **Report-rate telemetry**: measure the achieved report interval against the host's
   poll rate, so the queue-depth choice is measured rather than reasoned.
8. **Mouse.** The descriptors and the queue would take a second HID interface, and every
   safety argument above still holds. It is out because the request was a keyboard, and
   half the care in this project went into *not* growing it.
9. **Pastes longer than one run.** `kTextJobCap` is 2048 characters and a controller that
   wants to send more has to split the job itself. The flow-control hook already exists —
   `typing` in the status frame is the board's own remaining count — so what is missing is
   only a client policy that refills as the board drains, plus the `E8` retry that goes
   with it. Deliberately not done yet: unbounded buffering on a keyboard bridge buys nothing
   and enlarges the window in which text sits in RAM waiting to be typed.
10. **A paste map that is not US-ANSI.** `strokeFor()` in `src/text.h` is the single table
    for characters; positions live in `web_page.h`/`relayctl.py`. A second layout means a
    second character table *and* a second `CODE_TO_USAGE`, with the existing refusal
    semantics intact (no entry → `E9`, never a guess). Until then the answer is "set the
    target to US", which is one line on the target and zero bytes on the board.

---

## Restoring the sibling firmware

```bash
cd ../jiggler && pio run -t upload          # after tools/to_bootloader.py, if needed
```

The `jiggler` source is untouched by this project; they are separate PlatformIO projects
for one board, and switching is one flash each way.