Experimental/keyrelay
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial import: keyrelay, a WiFi-controlled USB HID keyboard

Browse Source 
Browser or terminal controller drives an ESP32-S3 that acts as a Logitech
G815 into a target machine running no software. Disarmed by default; arming
requires the passphrase, the board button, or the serial console, and every
fail-safe path cancels in-flight work.

- src/: firmware with pure, host-testable modules (keystate, proto, text,
  ws_codec, sha1), the WebSocket control server, and the served page
- tests/: host suites for the pure modules, relayctl parity, and page JS
- tools/: relayctl terminal controller, serial console, bootloader helper,
  and an evdev target watcher
- platformio.ini: bridge (USB HID) and console (no-HID) environments
- README.md: single source of truth for behaviour, protocol and safety

LLM-Generated: true
This commit is contained in:
ducoterra
2026-11-08 00:20:33 -05:00
commit 9bc8d5213c
27 changed files with 8243 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+836
View File
@@ -0,0 +1,836 @@
# keyrelay — a WiFi-remote USB keyboard on the M5Stack AtomS3 Lite
Type on your laptop; the keystrokes land on a *different* machine, over the air, as
genuine USB HID input — including Enter and Ctrl+Alt+Del. The board enumerates on the
target as a plain boot-protocol keyboard, so the target needs no software, no driver
and no network. The controller is a web page served by the board itself.
```
your laptop target machine
┌──────────────────┐ ┌──────────────────┐
│ browser tab │ WiFi │ │
│ keydown/keyup ├────WebSocket──►│ AtomS3 Lite │
│ of the page │ (token) │ keyrelay │
└──────────────────┘ │ │
└────────┬─────────┘
│ USB HID boot keyboard
▼
whatever has keyboard focus:
a login prompt, a BIOS setup,
a VT, a text editor
```
```
disarmed LED dark nothing you type goes anywhere
armed LED solid red every keystroke in the tab goes to the target
```
* **Nothing types until you arm it.** Boot, reload, reconnect, focus loss, socket loss
and 8 s of silence all disarm, and disarming sends "all keys up".
* **Arming needs a passphrase** (or a thumb on the board's button). The LED is readable
from across the room, so the state is not a secret between you and the screen.
* **The keystrokes are not stored, logged or replayable.** The board keeps counters and
the live key state, and nothing else — no sequence storage, no macro, no script, nothing
it can type later. Pasted text is typed from RAM and forgotten; the serial console has no
command that types anything at all. It is a keyboard, not a payload dropper.
* **955 checks** cover the key-state machine, the wire protocol, the text→keystroke map,
SHA-1/base64, the WebSocket codec and the controller page's own script before anything is
flashed, and the target-side verifier proves what actually arrived.
---
## Contents
- [Is this the right tool](#is-this-the-right-tool)
- [Quick start](#quick-start)
- [The one USB port, and why the control path is WiFi](#the-one-usb-port-and-why-the-control-path-is-wifi)
- [Safety model](#safety-model)
- [Security model and trust boundaries](#security-model-and-trust-boundaries)
- [Linux notes (the target and the laptop)](#linux-notes-the-target-and-the-laptop)
- [Layout caveats](#layout-caveats)
- [Controller page](#controller-page)
- [Wire protocol](#wire-protocol)
- [Serial console](#serial-console)
- [Architecture](#architecture)
- [Tests](#tests)
- [End-to-end verification](#end-to-end-verification)
- [Measured behaviour](#measured-behaviour)
- [Troubleshooting](#troubleshooting)
- [Design decisions](#design-decisions)
- [Backlog](#backlog)
- [Restoring the sibling firmware](#restoring-the-sibling-firmware)
---
## Is this the right tool
**Good fits**
- Driving a machine that has no keyboard and should not get one: a headless box, a
rack server at a console, a kiosk, an instrument, a second machine mid-install.
- Working on a machine you are allowed to touch but not sit at — BIOS/UEFI setup,
bootloader and recovery screens, a login prompt, a full-screen game on a laptop
whose keyboard you would rather not wear out.
- Keeping an agent session live on a machine you are not typing into (that is the
sibling project [`../jiggler`](../jiggler), which is a simpler build of the same board).
- Anything where a KVM-over-IP would be right and you do not need video, mouse or
audio, and you have 8 dollars and a USB port to spare.
**Bad fits — use something else**
- **You need the screen.** There is no video path. PiKVM or TinyPilot
do video + keyboard + mouse; this is keyboard only.
- **You want unattended automation.** Deliberately impossible here (see above). If you
want scripted input on a machine you own, `xdotool`/`ydotool`/`uinput` on that machine
is the honest tool, and a Rubber Ducky is what this design is *not*.
- **A machine you are not authorised to input to.** Physical access is the whole
security model of every USB HID tool, and "I could plug it in" is not authorisation.
Use it on your own hardware or where you have explicit permission.
### What this deliberately is not
There is no macro store, no auto-arm at boot, no stored or scheduled sequence, no replay,
and no key logging — in firmware or in the tools. Not because it is hard (it is about ten
lines of `Keyboard.print()`), but because the difference between "a remote keyboard" and
"an unattended injection device" is exactly whether a human is deciding, live, with the
arming light on.
**Where the line sits.** Text you have just pasted and then deliberately sent is a
keyboard doing its job, so there is a command for it: the page's **Send text** box, or
`wl-paste | relayctl paste --yes`. It is gated like everything else — armed, a target
attached, one deliberate press of a button that says how many characters it will type —
and it cannot outlive that decision. The characters sit in RAM until they have gone out
and then they are gone: nothing is stored, nothing is scheduled, and release-all, disarm,
focus loss, a silent controller or a detached target cancels the rest of a run mid-word.
What stays absent is the part that would change what this device *is* — something the
board remembers, assembles on its own, or fires without a human in the loop.
Two consequences hold either way. The CLI's ordinary commands still press one key per
invocation, which is all verification needs; scripting 30 of them is possible and would
still be 30 deliberate actions with the LED on. And a captured board is a keyboard, not a
recording — pull it out and there is nothing in it to find.
---
## Quick start
Needs [PlatformIO](https://platformio.org/) and the board's source on this machine.
The board is an ESP32-S3 with no USB-UART bridge, so all flashing is over the USB-C
connector it already uses.
```bash
cd keyrelay
tests/run_tests.sh # 955 checks, no hardware needed
pio run -e bridge # ~1.1 MB image, ~35% of the 3 MB app slot
python3 tools/to_bootloader.py # 1200-baud touch -> ROM download mode
pio run -e bridge -t upload # flash
```
Build `console` as well as `bridge` before flashing: it compiles with `ARDUINO_USB_MODE=1`,
which is the only thing that checks the no-HID `#else` branches. And if `pio` on this machine is
a broken launcher rather than a working command (`ModuleNotFoundError: platformio`),
`python3 -m platformio` from an interpreter that has the package works instead — see
[`AGENTS.md`](AGENTS.md), which lists this and the other environment traps.
Then, with the board plugged into the **target**:
```bash
python3 tools/console.py wifi "My SSID" "the wifi passphrase" # optional: join a LAN
python3 tools/console.py token "a-long-passphrase-you-pick" # do this one, always
python3 tools/console.py status
```
Now, on the **laptop**:
| the board is on | open in a browser | because |
|---|---|---|
| your WiFi network | `http://keyrelay.local` (or the `ip=` from `status`) | mDNS, then direct IP |
| its own access point | `http://192.168.4.1` after joining `keyrelay-ctl` | no infrastructure needed |
Enter the passphrase, press **connect**, press **arm** — the LED goes solid red — and
type in that tab. Enter goes. Ctrl+Alt+Del is the button (twice), because your laptop's
own OS swallows the real combination before a browser can see it.
To send text instead of pressing keys, paste it into **Send text** and press its button:
it says how many characters it will type, the board types them as keystrokes at a pace the
board keeps, and **stop typing** abandons the rest. From a terminal it is
`wl-paste | relayctl.py paste --yes` (or `xclip -o -selection clipboard`, `pbpaste`).
To put the sibling project back on the board:
`cd ../jiggler && python3 .../to_bootloader.py && pio run -t upload`.
---
## The one USB port, and why the control path is WiFi
The AtomS3 Lite's USB-C is wired straight to the ESP32-S3's native USB pads (GPIO19/20),
and those pads belong to exactly one peripheral and exactly one host at a time. A board
that is a keyboard to machine A therefore cannot simultaneously be a serial modem to your
laptop. That single hardware fact decides the architecture:
| option | works? | why this project chose as it did |
|---|---|---|
| control over USB, HID to the same laptop | yes, pointless | the target *is* the laptop; that is the `jiggler` use case |
| second USB port / USB host shield | no | the board has one connector, and ESP32-S3 USB-host + device at once is not a thing |
| USB gadget on a Linux SBC (PiKVM-style) | yes | real answer for video + mouse; needs a Pi, a PoE HAT, and 10× the cost |
| **WiFi control, USB HID out** | **yes** | one connector stays free for the target, no host software at all |
The consequence worth stating plainly: the board draws its power and its USB identity
from the target, and its *instructions* from your network. Pulling it out stops everything
immediately — that is a feature.
---
## Safety model
Enforced in this order, each independent of the others:
1. **Disarmed is the only default.** Boot, page reload, WebSocket reconnect and page load
all begin disarmed. ` armed` is a state the board holds, not something the page
assumes — the page's own arming only becomes visible when the board confirms it.
2. **Arming requires authority**: the passphrase over the network, the button on the
board, or the serial console. Nothing else can arm it, and nothing at all reaches the
target while disarmed — not one key, not one chord, not even a "hello".
3. **Armed is displayed on the device.** Solid red LED while armed and attached. If the
target is not attached the LED blinks red instead, because "armed" without a target
means keystrokes are being dropped, and that deserves to look wrong.
4. **Silence disarms.** No inbound frame for `kNoClientTimeoutMs` (8 s) → all keys up,
disarmed. Covers the laptop sleeping, the browser being killed, the WiFi dying, the
network going down — every way a controller can vanish without saying goodbye. The
browser sends a keepalive every 2 s and answers the board's own 15 s WebSocket pings,
so a live tab holds it open indefinitely; `relayctl` does the same while it types. This
is checked in `loop()` against `ControlServer::lastInboundMs()`, and it is checked
*before* the typewriter is driven: it is the one rule that stops a run of pasted text
when the controller dies without a parting packet.
5. **Focus loss releases every key.** The tab blurring or being hidden sends "all keys
up" and disarms (unless you tick *stay armed when this tab loses focus*, which still
releases the keys — you cannot keep a held Ctrl alive by tabbing away).
6. **The target taking the bus away disarms.** USB detach or host suspend clears the
report queue, releases the keys and disarms; `ARDUINO_USB_*` events drive it.
7. **A full queue is a stop, not a drop.** 48 queued state changes without a drain means
something is outrunning the USB path, so the bridge releases everything, reports `E7`
and disarms rather than typing out of step with what the operator sees.
8. **State, not events.** The controller restates the *whole* key state on every event
(see [Wire protocol](#wire-protocol)), so a lost frame cannot leave a modifier stuck
on the target. This is the single most important design decision in the project: every
"stuck Ctrl" horror story in HID injection tooling is an event-protocol bug.
9. **A paste cancels; it never pauses.** `R` means "stop typing", not "hold on". The run
is dropped on release-all, disarm, target detach or suspend, queue overflow, focus
loss and silence — every fail-safe above routes through the same `releaseEverything()`
that clears the queue, raises the keys and discards what was still to type. The board
never holds a run that outlives the decision to send it.
The physical kill switch is the button: press it and the keys come up, no network, no
passphrase, no software required.
---
## Security model and trust boundaries
| actor | can | cannot |
|---|---|---|
| Anyone on the target (its own OS) | see a Logitech G815 keyboard in its device list; read the CDC console; type on itself | arm or drive the bridge remotely, detect that a second controller exists |
| Anyone on the board's network | fetch the page; open a socket; learn that a keyrelay exists | arm, send a key, send a chord, type a run of text, read `/status` — all need the passphrase |
| Someone guessing the passphrase | 5 attempts before a 30 s lockout during which the token is not evaluated at all | learn anything from the failure: same error, same close, no timing oracle (constant-time compare) |
| The owner of the WiFi | join the network, then everything above | — the network *is* the perimeter; see below |
| You, holding the board | arm/disarm (button), reconfigure or reflash (console), stop everything (unplug) | — |
| The target's user | notice a red LED next to their monitor, and pull it out | — |
**What the passphrase does and does not do.** It authenticates the *control* channel; it
is not confidentiality for keystrokes. The page is served over plain HTTP and the
WebSocket is `ws://`, so on a network you do not control, keystrokes are readable by
anyone who can see the traffic and the token is visible at first contact. This is a
deliberate v1 trade: WSS would need either a trusted certificate (no way to get one for
`keyrelay.local`) or certificate-pinning ceremony in a browser (which the target may not
even have). Documented in [Backlog](#backlog) with the two honest fixes.
Practical rule: **use it on a network you trust**, or on the board's own access point
whose WPA2 passphrase is a separate secret (`kr::kApPass`, change it in `src/config.h`).
Both are better than either alone: the AP means you never have to put the board on
someone else's LAN.
**Single controller, by design.** A second WebSocket upgrade is refused with `503` / `E3`
rather than queued, swapped or shared. Two controllers typing into one machine is precisely
the ambiguity a keyboard must not have, and "who owns this bridge" should never be a tie.
What is exclusive is the **keyboard**, not the board: `GET /` and `/status?token=` are
served even while a controller holds the bridge, so a page can be reloaded, a second window
can watch the counters, and `curl` keeps working mid-session. Read-only access is still
gated by what it reveals — `/status` needs the passphrase, the page needs it before it can
arm or type. If that ever goes too far, the `ap off`/network config is the blunt instrument;
there is no per-client ACL.
**Identity.** The board reports itself as `046d:c33f` "Logitech Gaming Keyboard G815",
matching the sibling `jiggler` firmware so the two are indistinguishable from the host
side and `/dev/input/by-id` names stay stable across reflashes. This makes the device
unremarkable in `lsusb`; it does not hide it from anyone who looks for a keyboard that
appeared at 14:32. It is your hardware and a real Logitech id, so: personal use only.
To make it honest instead, change the five `kUsb*` constants in `src/main.cpp` — nothing
else depends on them.
**What is never stored:** keystrokes, characters, key names, SSIDs beyond the configured
one, client addresses, or anything else about what was typed. Counters only (`keys`,
`chords`, `dropped`, and `typed`/`typing` for the paste path), which is what the status
display needs to be useful. A run of pasted text exists in RAM from the moment a frame is
accepted until the last character is typed or the run is cancelled, and nothing about it
survives that — no queue across a disconnect, no place to write it. The board has no clock
it trusts and no filesystem it writes to. Turning this into a logger would be a fork, not
a feature flag.
---
## Linux notes (the target and the laptop)
**Ctrl+Alt+Del on a Linux target can reboot it.** On the graphical session it is usually
inert; on a virtual terminal `systemd` maps it to `ctrl-alt-del.target`, which by default
**reboots the machine** (seven CAD presses in two seconds force it). This is why the
combination is a button with a two-press confirm and a 3-second window, never a forwarded
keypress, and why `relayctl cad` refuses to run without `--yes`.
**Ctrl+Alt+F1…F7 works** — which is a bigger deal than CAD: it switches the target's
virtual terminal, and a getty on an unused VT with **autologin** enabled will hand you a
shell as that user. This project puts the VT switches on the special-keys pad so they are
deliberate clicks rather than a chord you can fat-finger.
**Alt+SysRq+b etc.** are reachable too (the magic SysRq key is off by default on most
distros). The pad does not offer them; the keyboard table can produce them, which is what
a keyboard is.
**Keys the laptop's OS steals** (this is the laptop side, and it is the browser-capture
limitation, not a bug): `Ctrl+Alt+Del`, `Ctrl+Alt+F1…F7`, `Alt+Tab`, `Super`,
`PrintScreen`, and on GNOME/KDE a growing list of media and tiling shortcuts are grabbed
by the compositor and never reach the page. Three answers, in order of preference:
1. the **special-keys pad** on the page — each of those is one deliberate click, and the
board synthesises the chord so the laptop never sees it;
2. run the laptop's compositor in a mode with fewer grabs while you work;
3. the **OS-wide capture client** (see [Backlog](#backlog)) — an evdev reader on the laptop
that forwards everything before the compositor gets it. The protocol is already
designed for it; it is a client-side addition, not a firmware change.
**While armed, the page swallows everything** with `preventDefault()`, so `Ctrl+W` will
not close the tab, `Ctrl+R` will not reload it and F11 will not fullscreen it. It also
means you cannot type into the page's own fields while armed — disarm first.
---
## Layout caveats
`KeyboardEvent.code` names a *physical position* on a US-ANSI keyboard, and those
positions are what travel as HID usage ids. If the **target** is set to a non-US layout,
`KeyQ` still means "the key left of A" and the target maps it to whatever its layout says.
This is identical to what a physical KVM does, and the fixes are the same:
- set the target's layout to English (US) — the one-line fix;
- or remap `CODE_TO_USAGE` in `src/web_page.h` for your laptop's board (it is one object
literal; `tools/relayctl.py` carries the same table and
`tests/test_relayctl.py` fails if the two drift).
ISO keyboards: `IntlBackslash` (0x32) is the extra key next to Left Shift on ISO and
`Backslash` (0x31) is the ANSI one; both are in the table. `AltRight` is `AltGr` and
travels as its own modifier bit, which is what layout engines on the target expect.
**Pasted text goes through a third table**, `strokeFor()` in `src/text.h`: character →
modifier + usage, US-ANSI, and deliberately not position-based, because what is being
pasted is characters. Same caveat, same fix — the target must be on a US layout. A
character with no entry is refused (`E9`) rather than approximated, and `test_text` walks
the entire printable range so an omission cannot hide, while `test_relayctl` proves the
tool's validator and the board's map accept exactly the same 128 codes.
---
## Controller page
Served by the board from flash (`src/web_page.h`, ~35 KB, no CDN, no fonts, no
framework, nothing external — a keyboard bridge must work on an airplane and on a network
that is down). Its script is executed by `tests/test_page.js`, because a C++ raw string is
invisible to every compiler and a typo there fails in a browser, with a target live.
| element | why it is there |
|---|---|
| big ARM/DISARM bar | the state must be unmissable; it only changes when the board confirms |
| red page border + red LED while armed | two independent indicators of the same fact |
| status pills: network / target / link / armed / caps-num | "will my keystrokes land, and where" |
| **Ctrl+Alt+Del** with two-press confirm | the OS eats the real one; on Linux it reboots |
| VT1–VT7, Alt+Tab, Win, PrtSc, Menu | the chords the laptop's compositor steals |
| **⇦ snap left / snap right ⇨ / ⛶ fullscreen** | Super+Left, Super+Right, Super+PageUp — KDE window management, sent by the board so your laptop does not answer it with its own windows |
| **Release keys** and `Ctrl+Alt+Esc` | panic: every key up, disarm |
| *stay armed if this tab loses focus* | off by default; for the one workflow that needs it |
| **Send text** box, its button and its *stop typing* | paste something and have it typed as keystrokes; the button says how many characters, refuses to be armed-into accidentally, and is the only path text takes |
| *drop characters I cannot type* | off by default: a paste that quietly loses a character is worse than one that refuses |
| held-keys readout, keys/chords/text/chords counters | so "it is typing" is a fact you can see |
| log, *log each key sent*, **clear** | the board's errors verbatim plus one line per key pressed — never one per release, never a blank line, and a repeating line is counted rather than stacked |
The caps/num indicators are read *from the target* (its HID output reports drive them),
not guessed — a caps-lock desync between controller and target is a real failure mode of
remote keyboards and this design cannot have it.
---
## Wire protocol
One WebSocket text frame = one command. ASCII, no JSON inbound, trivially hand-typed,
and impossible for one malformed field to desynchronise what follows it.
| dir | message | meaning |
|---|---|---|
| c → d | `T<token>` | authenticate; must be first (`kAuthTimeoutMs` = 5 s) |
| c → d | `H` | hello: status now |
| c → d | `A1` / `A0` | arm / disarm |
| c → d | `S=<mods>:<u>,<u>…` | **the complete key state** |
| c → d | `C` | Ctrl+Alt+Del (fixed in firmware, not read off the wire) |
| c → d | `Y=<mods>:<u>,<u>…` | one synthetic chord, `kChordHoldMs` (40 ms) |
| c → d | `P=<ms>:<hex>` | type pasted text, `<ms>` ms per character (0 = board default) |
| c → d | `R` | release every key, stay armed — **and drop a text run in progress** |
| c → d | `K` | keepalive |
| d → c | `OK` | accepted |
| d → c | `E<code>:<text>` | error (codes below) |
| d → c | `S{json}` | status, 1 Hz and on every change, incl. `typing` = characters of the run still to type |
`<mods>` is the modifier byte as two hex digits, `<u>` a HID usage id as two hex digits.
Both are strict: exactly two hex digits, no spaces, no trailing comma, usage `00` is not
a key, and a rejected message leaves the previous state in force — the safe outcome.
**`S=` carries state, not events.** `S=00:` means "nothing is pressed". A press and its
release are two frames and therefore two HID reports, so a fast tap cannot vanish;
identical consecutive reports collapse, which is what keeps the queue short. A single
message resynchronises the target completely, which is why no key can get stuck.
**`P=` is text, typed by the board.** `<hex>` is the bytes to type, so one frame carries
at most `kTextFrameMax` (120) characters and a longer paste is several frames; `<ms>` is
the pace asked for, and the board keeps it — expansion and timing live on the USB side
because a controller's clock, arriving over WiFi in whatever order the network delivers
it, is not a keyboard timing source. Each character is pressed and released on top of
whatever the operator is physically holding, and released back to it, so a paste can
neither let go of a held key nor strand one. The board answers `E9` for a character the
US-ANSI map cannot express and types none of the frame, rather than sending a paste that
quietly lost one; `E8` means it already holds `kTextJobCap` (2048) characters. A
controller sends the next frame when the status says `typing: 0` — the board's own count,
not a guess at timing.
Error codes: `1` not authenticated · `2` locked out · `3` another controller holds the
bridge · `4` not armed · `5` no USB host attached · `6` bad message · `7` queue overflow
(released and disarmed) · `8` text buffer full · `9` a character with no US-ANSI keystroke.
The same protocol is what a future OS-wide capture client would speak — see
[Backlog](#backlog) — which is why it lives in `src/proto.h` instead of inside the server.
---
## Serial console
`/dev/ttyACM0` is the CDC half of the same composite USB device, so the console appears
on whatever machine the board is plugged into:
```bash
python3 tools/console.py # interactive, Ctrl-] to quit
python3 tools/console.py /dev/ttyACM0 status # one command, print the reply
```
| command | effect |
|---|---|
| `status` | the same JSON the controller gets, plus link, queue and text-run state (`text N chars typed, M still to go`) |
| `wifi <ssid> [passphrase]` | station network to join (stored in NVS, reconnects) |
| `wifi clear` | forget it; the access point becomes the only way back |
| `ap auto\|force\|off` | AP policy: when the station is down / always / never |
| `token <passphrase>` | control passphrase, 6–32 chars, never echoed back |
| `arm` / `disarm` | same as the button |
| `reboot`, `help` | |
There is deliberately **no command that types anything** — the console is configuration,
not an input path. Replies are printed with a blocking write (the global serial timeout is
0 so logging into a port nobody reads can never stall `loop()`); without that, multi-line
replies truncate at the 128-byte CDC TX ring, which is exactly what happened on the first
build and is why `pollConsole()` bumps the timeout around each reply.
---
## Architecture
```
src/
config.h defaults: network, timeouts, caps, buffers (NVS overrides)
keystate.h pure modifier byte + pressed-set -> 8-byte HID report, rollover
text.h pure pasted text -> US-ANSI keystrokes, paced; run buffer and cancel
proto.h pure the wire protocol: parse/render commands, constant-time compare
sha1.h pure SHA-1 + base64 for the WebSocket handshake
ws_codec.h pure handshake request parsing + RFC 6455 frame codec
web_page.h the controller page, in PROGMEM
control_server.{h,cpp} sockets: HTTP routes, WebSocket lifecycle, auth, heartbeat
main.cpp glue: USB identity/HID, WiFi, report queue, chords, text run,
LED, button, watchdogs, serial console
tools/
to_bootloader.py 1200-baud touch into ROM download mode (stdlib termios)
console.py serial console, one command or interactive (stdlib)
relayctl.py controller from a terminal: status/arm/disarm/key/chord/listen/paste
watch_target.py on the target: decode the board's evdev events, prove delivery
tests/
run_tests.sh builds and runs everything (host g++ or a local container)
test_keystate.cpp test_proto.cpp test_text.cpp test_ws_codec.cpp
test_relayctl.py test_page.js (node, skipped when absent)
AGENTS.md operating manual for whoever edits this next: the invariants, the three
tables that must agree, the build traps, the verification standard
TODO.md open items, kept with their root cause rather than deleted
```
If you are an agent or a new contributor, read `AGENTS.md` before changing anything: several
rules here are invisible to a compiler and to the test suite at the same time.
The five modules marked *pure* have no Arduino and no ESP-IDF include, so the identical
code compiles and runs on a host under `-fsanitize=address,undefined`. That is not
convenience for its own sake: the sanitiser caught five real defects in the frame decoder
before the first flash (a `reset()` that left the reader latched on the previous frame, an
unsequenced `_buf[_got++] = b ^ _mask[_got & 3]`, a missing FIN check that silently
accepted fragmentation, a mask key skipped on zero-length frames, and a
`"sec-websocket-key:"` prefix compared at the wrong length so the accept hash was
computed from `"ey: …"`).
The keystroke path, end to end:
```
keydown/keyup -> full state S=.. -> Reader (RFC6455) -> proto::parse -> auth gate
-> KeyboardState::apply (dirty only if the report changed)
-> ReportQueue (dedupe identical, overflow = release + disarm)
-> drainReports: tud_hid_n_ready(0) ? USBHIDKeyboard::sendReport : keep queued
-> HID interrupt endpoint, boot protocol, 8-byte report -> target
```
`USBHID::SendReport` in the core is synchronous (it waits on a semaphore for the transfer
to complete, up to 100 ms), so `drainReports()` takes at most `kMaxReportsPerLoop` = 8
reports per pass — bounded time in the USB path, and the control plane never waits on it.
Chords (Ctrl+Alt+Del, VT switches) are synthetic: the press is pushed, and after the hold
the *tracked* state is pushed back, so keys that were genuinely held before a chord
survive it and the tracker is never corrupted by something the operator did not type.
The USB interface is confirmed boot-protocol on real hardware, which is the precondition
for Windows accepting Ctrl+Alt+Del as the secure attention sequence:
```
$ for i in /sys/bus/usb/devices/*:1.*/; do echo "$(basename $i) class=$(cat $i/bInterfaceClass) sub=$(cat $i/bInterfaceSubClass) proto=$(cat $i/bInterfaceProtocol)"; done
7-1:1.0 class=03 sub=01 proto=01 <- HID, boot subclass, keyboard protocol
7-1:1.1 class=02 sub=02 proto=00 <- CDC-ACM control
7-1:1.2 class=0a sub=00 proto=00 <- CDC-ACM data
```
Two build environments: `bridge` (the real thing, USB-OTG + TinyUSB) and `console`
(`ARDUINO_USB_MODE=1`: no HID, so it cannot type, but WiFi/page/WebSocket/LED/button all
live — for separating "the network side is broken" from "the HID side is broken" without
rebuilding. Arming refuses with `E5` there, on purpose).
---
## Tests
```bash
tests/run_tests.sh # uses g++ if present, else a local container image
SANITIZE=1 tests/run_tests.sh # + ASan/UBSan (recommended, and what found 5 bugs)
```
| suite | checks | what it pins down |
|---|---|---|
| `test_keystate` | 40 | idempotence, transition preservation (a fast tap is two reports), 6-key limit, ErrorRollOver and recovery from it, dedupe, `clear()`, the exact Ctrl+Alt+Del bytes `05 00 4c 00 00 00 00 00` |
| `test_proto` | 193 | every command form, strict rejection of malformed state specs, bounds (cap on keys, token length and text frames), render/parse round-trip, error framing, constant-time compare, **every chord button's bytes matched against the label printed on it**, and the page's text constants pinned to `config.h` |
| `test_text` | 459 | the whole printable-ASCII map walked (nothing may be unmapped, nothing may claim a key it has no glyph for), shifted glyphs and the digit row, what is refused rather than guessed at, all-or-nothing frame acceptance, pace arithmetic and clamping, the exact press/release report sequence for two characters, overlay on the operator's held keys, ErrorRollOver propagation, cancel, buffer-full, `P=` render/parse round-trip including the largest legal frame, and one end-to-end case that takes the exact frame the page sends and asserts the HID report sequence the target ends up seeing |
| `test_ws_codec` | 135 | SHA-1 against published vectors including the million-`a` case, base64 against RFC 4648, the RFC 6455 handshake example, frame encode/decode at the 125/126 and 64-bit boundaries, byte-at-a-time delivery, and rejection of unmasked, fragmented, reserved-bit, binary, over-long and oversized frames |
| `test_relayctl` | 81 | the Python client's codec, that its key table has not drifted from the page's, that its text validator agrees with the firmware's character map on all 128 ASCII codes, CRLF normalisation, and `TEXT_FRAME_MAX`/`MAX_PASTE_CHARS` pinned to `config.h` |
| `test_page` | 47 | the page's script, executed: blank and whitespace-only log lines never render and a repeating line is counted not stacked, character validation, CRLF, hex encoding, one frame for a short paste and exactly the right bytes, a busy board holding the next frame back, a 300-character paste reassembling to the text that went in, refusal of untypeable text and the opt-in drop, and cancellation by stop/disarm, plus every element id the script reaches for checked against the markup |
Gaps, stated plainly: nothing here exercises real sockets, real WiFi associations, or real
USB transactions — those are the [end-to-end procedure](#end-to-end-verification) below.
The HTTP routing and auth-lifecycle code in `control_server.cpp` is the least-tested part
of the project; it is deliberately the thinnest layer, but it is not covered by a unit
test.
---
## End-to-end verification
Three machines' worth of roles on two boxes: the laptop is the controller, the board is
the bridge, and any Linux box (including the laptop itself) is the target.
```bash
# 1. on the target: what actually arrived, decoded from evdev, focus-independent
python3 tools/watch_target.py --seconds 60
# 2. on the laptop: drive it from the terminal
export KEYRELAY_TOKEN='the-passphrase-you-set'
python3 tools/relayctl.py --host keyrelay.local status
python3 tools/relayctl.py --host keyrelay.local arm
python3 tools/relayctl.py --host keyrelay.local key Enter
python3 tools/relayctl.py --host keyrelay.local chord 04 2b # Alt+Tab, by hand
python3 tools/relayctl.py --host keyrelay.local cad --yes
printf 'Hello from keyrelay\n' | \
python3 tools/relayctl.py --host keyrelay.local paste --yes # 19 characters
python3 tools/relayctl.py --host keyrelay.local listen 20 # heartbeats, live
python3 tools/relayctl.py --host keyrelay.local disarm
```
What a correct run proves, and how to tell:
| claim | evidence |
|---|---|
| HID reaches the host | `watch_target.py` prints every press/release with a timestamp |
| text is correct | its `typed` line, with case from the shift state |
| Enter works | `Enter press/release` and `<Enter>` in the typed text |
| Ctrl+Alt+Del works as a *combination* | `>>> Ctrl+Alt+Del <<<`, one line, not three |
| no stuck keys | `still held` is absent at the end of the run |
| **pasted text is keystrokes, not a shortcut** | the run's `typed` line reproduces the text, and the press count is roughly twice the character count — a paste that arrived as nothing, or as a single odd chord, means the character map or the pace, not the target |
| **Alt+Tab is Alt+Tab** | `LAlt press` then `Tab press` — and *not* `LCtrl`. This combination was `Ctrl+Alt+A` on the wire until it was caught by the label-vs-bytes test; `watch_target.py` is what settles any doubt about which keys the board actually sent |
| a paste stops | start a long one, press **stop typing** (or blur the tab, or press the board's button): the `typed` text stops mid-sentence and nothing further appears |
| target LEDs are read back | `LED CapsLock on` on the target, caps indicator on the page |
| silence disarms | arm it, close the tab, wait 8 s, `status` says `armed: 0` |
| auth is real | `relayctl` with a wrong token fails; the board logs `authfails=1`; 5 tries → `E2` |
`watch_target.py` finds the board by USB id, not device name, so it keeps working on a
different port or a renamed input node, and it accepts both the Logitech identity and the
Espressif default so it works on older builds.
A pasted run prints one line per press *and* release, so a few hundred characters is a
lot of scrolling: watch the `key presses` and `typed` summary lines at the end rather than
the stream, and give `--seconds` enough room for the pace (`2048` characters at the default
12 ms is about 25 s of typing, plus the run's own margins).
---
## Measured behaviour
_On real hardware: AtomS3 Lite → this laptop (Linux, US layout), board on the same WiFi,
firmware 1.0.0, `bridge` environment._ Fill in / re-run after any change to the keystroke
path; the numbers below are the shape a correct run has.
```
$ python3 tools/console.py status
status {"v":"1.0.0","up":7,"usb":1,"armed":0,"net":"ap","ssid":"keyrelay-ctl",
"ip":"192.168.4.1","keys":0,"chords":0,"drop":0,"roll":0,"leds":0,
"dflt":1,"ap":0}
link controller=no rejected=0 authfails=0
queue 0 pending, 0 held, ok
$ python3 tools/relayctl.py --host <ip> status # armed, typing, then disarmed
armed YES -- keystrokes are being forwarded
counters keys=… chords=0 dropped=0
$ python3 tools/watch_target.py --seconds 30 # on the same laptop
t= 0.42s A press
t= 0.48s A release
...
key presses …
typed '...'
```
### Confirmed on hardware
AtomS3 Lite → Linux laptop (US layout), board joined the LAN, firmware 1.0.0.
| what | result |
|---|---|
| flash | 1 134 768 bytes written (706 661 compressed) in 8.1 s at 1 116 kbit/s, hash verified |
| footprint | RAM 26.7 % (87 400 / 327 680); Flash 35.5 % of the 3 MB app slot, measured with the paste path in |
| enumeration | `class=03 sub=01 proto=01` — boot-protocol keyboard, the Windows secure-attention precondition |
| reflashing | 1200-baud touch into ROM download mode works with this firmware running, so no button and no GPIO0 |
| page delivery | `GET /` → 200, `Content-Length: 35164`, zero external references; 404 on unknown paths; 401 on `/status` without the passphrase; `keyrelay.local` resolves over mDNS |
| arming from the browser | the page armed the board over `ws://` on first try (observed as `controller=yes armed=1`) |
| keystroke delivery | 6/6 inert keys arrived on the target with matched press **and** release: F13, F15, F17, ScrollLock, CapsLock ×2 |
| chord path | `chords=1` counted and delivered (Shift+F15) |
| LED read-back | `LED CapsLock on` then `off`, coming from the *target's* HID output reports |
| **silence disarms** | controller process killed → `armed:0` within 11 s, keys released. **Worth knowing:** that run passed *before* the silence timer existed — killing a process closes the socket, and the socket-close path disarmed. The FIN-less death (laptop slept, network vanished) is what the timer covers, and it is wired now. A check that passes for the wrong reason is still a check that passes, which is exactly the problem |
| single controller | a second client got `E3` / `503` and the board logged `rejected=1` — no takeover |
| lockout | 5 wrong passphrases → `E2`, for 30 s, from the page's own retry loop |
| queues | `drop=0 roll=0` throughout; `queue 0 pending, 0 held, ok` at rest |
### Nine bugs this verification caught (all fixed)
Worth listing, because each one is a class of bug that HID-over-network tools ship with:
1. **CDC replies truncated at 128 bytes** — `setTxTimeoutMs(0)` meant "drop what does not
fit", and `serial status` came back cut mid-JSON. Replies now temporarily switch to a
blocking write and flush. A diagnostic tool that lies about state is worse than none.
2. **The CLI lowercased the whole handshake header block** — base64 is case-sensitive, so
every `Sec-WebSocket-Accept` compared wrong and the board looked broken while answering
correctly. Header *names* are case-insensitive; values are not.
3. **The CLI reported stale status** — the board heartbeats at 1 Hz and the key helpers do
not read while typing, so `disarm` printed `armed YES` from a heartbeat sent *before*
the command. `arm`/`disarm` now drain, then poll until the board actually reports the
state asked for — confirm, don't assume.
4. **The target-side verifier mislabelled every LED** — its table was copied from the HID
bit numbering, but Linux `EV_LED` indices are zero-based (`LED_CAPSL` is 1, not 2).
Verified against real events.
5. **Unnamed key codes looked like lost keystrokes** — F13–F24 had no names in the
verifier's table, so a perfectly delivered run appeared to drop its first three keys.
Now named, and unknown codes are counted rather than silently skipped.
6. **The single-controller gate was applied at `accept()`**, before the request was even
read — so while a controller was connected, *anything* else that touched the board got
a 503: `curl` of the page, a second browser, and worst of all **pressing reload on the
page that already held the bridge**, since the board could still be holding the slot the
instant the new request arrives. The fix moves the gate to what it actually protects: a
second `/ws` upgrade is still refused (that is a controller claim), while read-only HTTP
is served on a separate short-lived socket through the same routing code (`serveReadonly`)
so the two paths cannot drift. Verified live: with your browser holding the bridge, the
page returns 200 and `/status` returns 401 without the token, 200 with it, and a second
upgrade still gets 503 while the incumbent stays `controller=yes` and healthy.
7. **The Alt+Tab button sent Ctrl+Alt+A.** Its `data-chord` was `05:04` — LCtrl|LAlt with
usage `0x04`, which is the A key. It parsed cleanly, it was correctly gated by arming,
it arrived intact and it was the wrong combination: Alt+Tab is `04:2b`. Reported by a
user who could not work out why the Pi ignored it, and provable because the page stores
the chord bytes on the button instead of a name to look up — so nothing on the browser
side could ever disagree with a label. Every labelled chord is now pinned to the
combination its label promises, by modifier byte and usage id.
8. **`relayctl`'s drain called two methods that do not exist** (`_send_frame`,
`_check_error`) while answering a ping, so any session outliving the board's 15 s ping
died on an `AttributeError` instead of reading its next frame. Only reachable on long
sessions, which is exactly what typing text is; it answered pings the way `_poll` does
and raises on error frames now.
9. **"Silence disarms" was documented and not implemented.** `kNoClientTimeoutMs` existed,
`ControlServer::lastInboundMs()` existed, and nothing ever compared them. Harmless while
the only input was a human keystroke (a vanished controller sends no more keystrokes),
and not harmless once a run can type for twenty seconds on its own: a laptop that slept
mid-paste, with a link that never got a FIN, would keep typing into the target.
`loop()` now checks it, before driving the typewriter, and clients keep-alive while they
type. The lesson is the one the whole project keeps relearning: a safety property that
is only written down is not a safety property.
### The hazard nobody warns you about
**When the target is the machine you are working on, your keystrokes land on your own
focused window.** The first live test sent `Escape`, and the target was the laptop running
this agent — so the keystroke interrupted the very session that sent it. That is not a bug
in the bridge; it is exactly the transparency the thing is designed for, pointed the wrong
way. So:
- verify with keys that nothing reacts to — **F13–F24 and the Lock keys** are inert almost
everywhere, and they exercise the same code path as Enter;
- before arming against your own machine, decide what has focus; a terminal will *execute*
what you send;
- the page releases keys and disarms when its tab loses focus, which is also how you get
control back: click anything else, then click back.
---
## Troubleshooting
| symptom | cause, and what to do |
|---|---|
| the page does not have a feature the README describes | **the page is served from the board's flash**, so the image you built is not the image on the device. `curl -s http://keyrelay.local/ \| wc -c` and compare with the page in `src/web_page.h`; if they differ, reflash (`to_bootloader.py` then upload) and *reload the tab* — one already open still holds the old page in memory, passphrase notwithstanding |
| `pio run -t upload` sits at `Connecting…` | the app owns the USB pads. `python3 tools/to_bootloader.py`, or hold the button 3 s while powering up (LED solid red) |
| page will not connect | wrong address: `console.py status` for the real `ip=`; `net:"ap"` means you must join `keyrelay-ctl` |
| `keyrelay.local` does not resolve | mDNS is queried per-process with no cache on glibc (`mdns4_minimal [NOTFOUND=return]`), so a cold probe that misses returns "not known" immediately — **just retry**, or use the IP. Across VLANs/VPN it is often blocked outright |
| `arm` does nothing, LED blinks red | no USB host attached (`usb:0`); the board is not plugged into the target, or the target suspended the bus |
| typing does nothing, `armed` says yes | the tab lost focus: click into the page. Or the target's focus moved — that is the target's answer, not a bug |
| `E4 not armed` | it is not armed. `arm` first; chords and states are never free passes |
| `E3 another controller` | another tab/machine holds it. Close it; there is no takeover, deliberately |
| `E2 locked out` | 5 wrong passphrases; wait 30 s |
| keys arrive out of order or missing | check `dropped` (queue overflow) and that the laptop's signal is good; `WiFi.setSleep(false)` is already set. If a *verifier* says keys are missing, check it names the codes — see [the hazard nobody warns you about](#the-hazard-nobody-warns-you-about) |
| the CLI says `another controller holds this bridge` | your browser page has it, and that is the rule working. Close the tab; there is no takeover |
| pasted text arrives as different characters | the **target** is not on a US layout — characters travel as positions, exactly like the live keys. Or the run was sent with *drop characters I cannot type* / `--strip`, which reported how many it removed |
| a paste stops before the end | something cancelled it, on purpose: the tab lost focus, the bridge disarmed, the target suspended the bus, `E7`/`E8`, or 8 s with nothing inbound from the controller. The page's log says which, and how far it got |
| a chord button does something other than its label | the bytes on the button are wrong — the page carries the wire form itself, so nothing browser-side can catch it. `curl` the page and read `data-chord`; this was the case for Alt+Tab (`05:04`, Ctrl+Alt+A) until the label-vs-bytes test was written |
| a key is stuck on the target | `R` / **Release keys** / the button. Then check whether the tab lost focus without the page noticing (a hidden tab still gets blur on most browsers) |
| Ctrl+Alt+Del does nothing on the target | on Linux this is normal on a graphical session and *fatal* on a VT; on Windows the interface must be boot protocol (confirmed above) |
| target keeps suspending the bus | the target suspends, which disarms by design; disable suspend or plug into a port that stays powered |
---
## Design decisions
**Why a web page and not a host daemon.** Because "install something on the laptop" is
the difference between a tool you can use in ten minutes on a machine you are borrowing
and a project you set up once at your own desk. The page is served by the board, so the
only software that has to exist is the browser you already opened.
**Why full state instead of key events.** Covered under [Safety
model](#safety-model) point 8, and it is the decision that makes every other guarantee
here cheap: a lost message is corrected by the next one, so there is no resynchronisation
protocol, no sequence numbers, no stuck-key reconciliation timer.
**Why the board expands and paces text, not the browser.** It is tempting to turn a paste
into existing `S=` frames client-side and change no firmware at all. Timing is the reason
not to: a controller reaches the board over WiFi, in frames, with whatever gaps the network
inserts, and a character is a press and a release with a hold either side of it on a target
that samples every millisecond. A browser tab that stalls for a second would produce either
a burst of keys or a run that outran the 48-deep report queue and tripped the
overflow-disarm. Expanding on the board also means the page and the CLI cannot disagree
about what a character types, which is the same argument that put the protocol in
`proto.h`. Cost: the character map exists twice — positions in `web_page.h`/`relayctl.py`,
characters in `text.h` — and a test pins all three together.
**Why a paste refuses rather than sanitises.** Curly quotes, em dashes and accented
characters have no key on a US-ANSI keyboard. Dropping them quietly would send a passphrase,
a `rm` line or a config value that is *nearly* what was pasted, which is the failure mode no
one notices until it has landed. So the board answers `E9` naming the code point and types
none of the frame, the page says the same before it sends anything, and dropping is an
explicit, per-paste opt-in that says how many characters it removed. Fail-closed, and
loud.
**Why no WebSocket library.** The control channel needs text frames, ping/pong and close.
A library buys fragmentation, extensions, TLS plumbing and many-client handling — attack
surface and heap, none of it wanted. The hand-rolled codec is ~200 lines of pure logic
with 135 host checks behind it, and every deviation from the subset is a hard close.
**Why HTTP/WS without TLS.** See [Security model](#security-model-and-trust-boundaries).
The honest options in the backlog are a user-installed CA (works, annoying) or a
client-side proxy that terminates TLS with a pinned key. Pretending confidentiality here
would be worse than documenting its absence.
**Why NVS for configuration.** So the passphrase and network live in the board rather than
in a header you recompile, and survive reflashing — and so a passphrase never has to enter
a shell history, a git diff or a build log. The `token` command deliberately does not echo
it back.
**Why the sibling project's USB identity.** Two firmwares on one board that are
indistinguishable from the host side, and stable `/dev/input/by-id` names for the tools.
Cost: it is a borrowed VID/PID, so it is documented as personal-use and one edit away
from being honest.
---
## Backlog
Ordered by what would make the biggest difference first.
1. **OS-wide capture client** (`tools/keyrelay-capture.py`, laptop side): grab the
keyboard with evdev, forward full state over the same protocol, `uinput` a shadow if
needed. Solves every compositor-stolen key and is the missing piece for
"transparently drive another machine". Firmware unchanged — the protocol was designed
for it. Needs root or `input` group + a `udev` rule; documented as opt-in because it
swallows the laptop's keyboard while running.
2. **TLS with a pinned certificate** (WSS): either an on-board CA the laptop trusts once,
or a `localhost`-terminating proxy. Until then: trusted network or the board's AP.
3. **A second, independent key path for the panic release**: a GPIO or button
double-click that force-releases without waiting for the loop's disarm path, so the
panic release does not depend on the code that just failed.
4. **Per-connection rate limiting** — a token bucket on state messages, so a wedged or
hostile client cannot pump the queue into the overflow-disarm path as a nuisance
attack (currently it disarms: safe, but noisy), and a cap on page fetches now that the
page is served even while a controller is live (each is ~35 KB, and `no-store` means a
determined client will ask for it again and again).
5. **mDNS also on the AP interface**, verified on each chipset — some builds answer only
on the station interface, which is why the docs say "IP if the name fails".
6. **WPA3/PMF on the SoftAP** and a per-board AP passphrase printed once at first boot,
instead of one shared constant in `config.h`.
7. **Report-rate telemetry**: measure the achieved report interval against the host's
poll rate, so the queue-depth choice is measured rather than reasoned.
8. **Mouse.** The descriptors and the queue would take a second HID interface, and every
safety argument above still holds. It is out because the request was a keyboard, and
half the care in this project went into *not* growing it.
9. **Pastes longer than one run.** `kTextJobCap` is 2048 characters and a controller that
wants to send more has to split the job itself. The flow-control hook already exists —
`typing` in the status frame is the board's own remaining count — so what is missing is
only a client policy that refills as the board drains, plus the `E8` retry that goes
with it. Deliberately not done yet: unbounded buffering on a keyboard bridge buys nothing
and enlarges the window in which text sits in RAM waiting to be typed.
10. **A paste map that is not US-ANSI.** `strokeFor()` in `src/text.h` is the single table
for characters; positions live in `web_page.h`/`relayctl.py`. A second layout means a
second character table *and* a second `CODE_TO_USAGE`, with the existing refusal
semantics intact (no entry → `E9`, never a guess). Until then the answer is "set the
target to US", which is one line on the target and zero bytes on the board.
---
## Restoring the sibling firmware
```bash
cd ../jiggler && pio run -t upload # after tools/to_bootloader.py, if needed
```
The `jiggler` source is untouched by this project; they are separate PlatformIO projects
for one board, and switching is one flash each way.