Files
mirror/docs/kiosk-silverblue.md
T
ducoterra 8fefd83ef8 feat(kiosk): self-recover a hung wall and page when it goes quiet
An unattended wall failed in two ways on 2026-11-02. It hard-hung with the kernel
alive but userspace unable to start a process: ssh completed the handshake and
closed 6-13 ms later with zero bytes and no banner, while 62 GiB RAM and zero OOM
lines rule out memory. Separately, the service knew at 14:03Z that nothing was
watching (healthz viewers:0) and nothing was reading it. It was discovered by
walking over to the panel.

Hang recovery, two independent layers (install step 7/9, asserted by
check-kiosk.sh):

- /etc/sysctl.d/90-mirror-wall-selfheal.conf: soft lockup, hard lockup and any
  oops escalate to a panic, and kernel.panic=30 reboots instead of parking.
- /etc/systemd/system.conf.d/90-mirror-wall-watchdog.conf: PID 1 must keep kicking
  the iTCO hardware watchdog or the chipset resets the box, which is the layer that
  covers a kernel too wedged to notice. Clamped to the driver's own 30s because
  systemd kicks at a third of whatever value it is handed.

Presence alerting: docs/wall-watch/ runs on the server host, probes loopback
/healthz (exempt from the address gate) every minute, debounces three checks, pages
over ntfy and then announces the recovery. WALL_DRILL=1 labels a test page in the
title, body and priority so a drill can never be mistaken for an incident.
Verified against a fake ntfy: caller environment wins over the config file, both
transitions fire, and the debounce holds.

Firewall (install step 8/9): drop FedoraWorkstation's 1025-65535/tcp. The UDP half
is load-bearing for peer-to-peer media (mirror deliberately uses no STUN/TURN) and
stays open; the TCP half was every high port on a box whose only access control is
its address. Measured, not assumed: a live share kept flowing at 459.6 kbit/s
inbound with the TCP range closed, and tcp/8080 stopped answering from a
same-subnet host.

Also fixed: install-kiosk.sh overwrote an existing client's kiosk.conf with its
built-in default URL, so re-running it to repair a wall silently repointed it at
the production default. An existing URL now wins and KIOSK_URL= is an explicit
change.

Documented with the evidence: why gnome-remote-desktop cannot share a gnome-kiosk
session (gnome-kiosk exports no org.gnome.Mutter.* names, so grd has nothing to
capture, and this image has grdctl rather than grdc), and why no userspace RDP
would have survived the hang either.

LLM-Generated: true
2026-11-02 09:53:13 -05:00

566 lines
28 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Kiosk client — Fedora Silverblue
The target client for a wall display is **Fedora Silverblue running GNOME Kiosk,
auto-logged in as `kiosk`, showing `https://mirror.reeseapps.com/watch`
fullscreen in Firefox.** One box takes about five minutes over SSH and needs no
keyboard, no monitor access, and no console.
Everything is in this directory and is idempotent — re-run it to repair a kiosk
or to change its URL.
| File | Lands at | Why |
|---|---|---|
| `install-kiosk.sh` | run on the client as root | the whole install, one pass |
| `check-kiosk.sh` | run on the client as root | read-only report; nothing needs the console |
| `gnome-kiosk-script` | `~kiosk/.local/bin/gnome-kiosk-script` | the kiosk application + watchdog loop |
| `kiosk.conf` | `~kiosk/.config/mirror/kiosk.conf` | **the one per-client value: the URL** |
| `policies.json` | `/etc/firefox/policies/policies.json` | no updates, accounts, telemetry, first-run pages |
| `window-config.ini` | `/usr/local/share/gnome-kiosk/window-config.ini` | force the Firefox window fullscreen |
| `gnome-kiosk-script-wayland.desktop` | `/usr/local/share/wayland-sessions/` | **stops GDM covering the wall with a greeter** |
| `mirror-kiosk-console-guard` | `/usr/local/libexec/` | reclaims the console if the wall loses it |
| `mirror-kiosk-console-guard.{service,timer}` | `/etc/systemd/system/` | runs that check every minute |
| `dim-probe.py` | run on any reachable host | measures what the kiosk actually renders |
| `90-mirror-wall-selfheal.conf` | `/etc/sysctl.d/` | lockups and oopses become a reboot, not a frozen frame |
| `90-mirror-wall-watchdog.conf` | `/etc/systemd/system.conf.d/` | PID 1 must kick the hardware watchdog or the chipset resets the box |
Presence alerting lives beside this, not inside it: [`../wall-watch/`](../wall-watch/)
runs on the **server** host and pages you when a wall goes quiet.
---
## Add a client
```bash
# 1. A vanilla Silverblue install. Enable Settings → Sharing → Remote Login.
# Create an admin user (the first one you make) — that is your SSH account.
# 2. Layer the kiosk packages. This is the only step that needs a reboot.
ssh client 'sudo rpm-ostree install gnome-kiosk gnome-kiosk-script-session && sudo reboot'
# 3. Wait ~60s, then install everything else. Stage it in /var/tmp, not /tmp:
# /tmp is wiped at every boot, so a checker staged there is gone exactly when
# you want it. The installer keeps a permanent copy of the payload in
# /usr/local/share/mirror-kiosk.
scp -r docs/kiosk-silverblue client:/var/tmp/ && ssh client \
'cd /var/tmp/kiosk-silverblue && sudo sh install-kiosk.sh'
# 4. Hand the console to the kiosk.
ssh client 'sudo systemctl restart gdm'
# 5. Prove it, without touching the machine.
ssh client 'sudo sh /usr/local/share/mirror-kiosk/check-kiosk.sh'
```
`install-kiosk.sh` creates the `kiosk` user if it does not exist, so step 2 can
be skipped when the packages are already layered. To point a client somewhere
other than the production wall, pass the URL:
```bash
sudo KIOSK_URL=https://mirror.reeseapps.com/watch sh install-kiosk.sh
```
Only `kiosk.conf` differs between clients; every other file stays byte-identical
so the fleet is auditable.
---
## The two facts that make this work
**Silverblue's mutable seams.** `/usr` is read-only and belongs to the OSTree
commit, but `/etc`, `/var` and `/usr/local` (which *is* `/var/usrlocal`) are
writable and survive upgrades. That is why the payload goes to `/etc/firefox`,
`/usr/local/share/gnome-kiosk` and `/var/home/kiosk` and nowhere else. Never
`ostree admin unlock` to shove files into `/usr` — the next upgrade deletes them.
**GNOME Kiosk's script session.** `gnome-kiosk-script-session` starts a locked
down mutter session (no top bar, no activities, no dock) and runs
`~/.local/bin/gnome-kiosk-script` as the kiosk application. Two consequences
drive the design of that file:
* It **re-executes itself forever**, so an unguarded launch opens a new window
every few seconds. The script checks `pgrep -u $(id -u) -x firefox` first.
* It **must never exit**. `org.gnome.Kiosk.Script.service` carries
`OnSuccess=gnome-session-shutdown.target`, so the script returning logs the
kiosk out and the screen falls back to the GDM greeter. Exiting is the failure
mode, not crashing — which is exactly why a browser that dies is a feature:
the loop comes back and starts it again.
The loop also **re-sources `kiosk.conf` on every relaunch**, so changing the URL
is:
```bash
sudo sed -i 's|^MIRROR_KIOSK_URL=.*|MIRROR_KIOSK_URL="https://…/watch"|' \
/var/home/kiosk/.config/mirror/kiosk.conf
sudo pkill -u kiosk firefox # loop notices within 5s and relaunches
```
---
## Operating the wall: keys
The whole wall works from a keyboard — and the box needs no keyboard attached
to be operated *or* debugged: everything about it is checkable over SSH (see
[Verifying a kiosk you cannot see](#verifying-a-kiosk-you-cannot-see)). The keys
are there when one is plugged in. On `/watch`:
| Key | Action |
|---|---|
| `j` / `↓` · `k` / `↑` | select the next / previous mirror (wraps) |
| `Enter` / `f` | stage the selected mirror fullscreen (`f` re-enters fullscreen) |
| `g` | back to the wall |
| `n` / `p` | stage the next / previous mirror |
| `1`–`9` | stage the Nth mirror |
| `m` | mute / unmute the staged mirror |
| `?` | show / hide the key list on screen |
| `Esc` | close the key list, then leave the stage |
On `/mirror`: `s` starts a share, `x` stops one, `Esc` stops while live. Keys
with `Ctrl`/`Cmd`/`Alt` are never intercepted, and a key typed in the
display-name field is text, not a command. `?` is the only key that needs
Shift, and it is matched on the character itself, so on a layout where `?` is a
dead key the list is not reachable — keep this table to hand.
**Sound.** The wall is silent until someone presses `m` (or clicks a tile's
audio button): only the staged mirror can be audible, and leaving the stage
re-mutes it. What that sound is depends on what the *sharer's* browser was
given — Chrome on Linux picks up a tab's audio reliably, system audio only
where PipeWire offers it, and Firefox's desktop capture does not offer system
audio at all (SECURITY.md §4 "Audio").
---
## Why each piece is where it is
**Auto-login — `/etc/gdm/custom.conf`**
```ini
[daemon]
AutomaticLoginEnable=True
AutomaticLogin=kiosk
TimedLoginEnable=False
```
`TimedLoginEnable=False` matters: a timed login would hand the console to the
greeter on its own schedule and you get a login form on your wall.
**Default session — `/var/lib/AccountsService/users/kiosk`**
```ini
[User]
Session=gnome-kiosk-script-wayland
```
This is what GDM logs the kiosk user *into*. Without it auto-login drops the
user into a normal GNOME session with a top bar. The installer restarts
`accounts-daemon` afterwards, because the daemon caches these files and happily
writes the stale value back over your edit.
**gsettings — applied inside the session, not at install time**
The screen must never blank or lock. This cannot be done from your SSH session:
`gsettings` only commits when it can reach a D-Bus session bus, and over SSH it
prints `Cannot autolaunch D-Bus without X11` and silently leaves the default. It
also cannot live in `/etc/dconf/db/local.d`, because `/usr/bin/gnome-kiosk` sets
`DCONF_PROFILE=gnomekiosk` and that profile is `user-db:user` plus the shipped
compiled defaults — it never reads a system db.
So the kiosk script applies them itself on first run each boot, stamped by a
file in `$XDG_RUNTIME_DIR` (which is per-login, so it re-applies if someone
changes a setting by hand):
```sh
gsettings set org.gnome.desktop.session idle-delay 0
gsettings set org.gnome.desktop.screensaver lock-enabled false
gsettings set org.gnome.desktop.interface enable-animations false
```
**Firefox window rules — `/usr/local/share/gnome-kiosk/window-config.ini`**
gnome-kiosk reads the **first** `window-config.ini` it finds in
`$XDG_CONFIG_HOME/gnome-kiosk`, then `/var/lib/flatpak/exports/share`,
`/usr/local/share`, `/usr/share`. Because ours wins, the shipped file is not
read at all — anything it defines is lost unless repeated. Ours therefore copies
the shipped `[notifications]` section verbatim and adds:
```ini
[mirror-browser]
match-class=org.mozilla.*
set-fullscreen=true
```
`firefox --kiosk` already goes fullscreen; this is the belt so a window that
comes back from a resume still fills the output.
**Firefox policy — `/etc/firefox/policies/policies.json`**
No update prompts, no Firefox Account or Sync, no studies, no telemetry, no
first-run/restore pages, no default bookmarks, no saved-password prompts.
`/usr/lib64/firefox/distribution/` is the other documented location and it is
inside the read-only `/usr`.
> Verified on this box: the file installs, is world-readable with a sane SELinux
> label, and Firefox runs happily with it. What is *not* verified headlessly is
> that each policy is applied — Firefox's policy engine does not initialise in
> the `--headless --screenshot`/`--dump-dom` modes, so a probe there proves
> nothing either way. Open `about:policies` on a real kiosk display to confirm;
> it lists every policy Firefox read and any it rejected.
**Initial-setup guard — `~kiosk/.config/gnome-initial-setup-done`**
Without it the GNOME "welcome" wizard claims the session on first login and the
kiosk never starts. The journal confirms the guard working:
`gnome-initial-setup-first-login.service … skipped, unmet condition check
ConditionPathExists=!/var/home/kiosk/.config/gnome-initial-setup-done`.
---
## The greeter that eats the wall
The shipped `gnome-kiosk-script-wayland.desktop` declares:
```ini
X-GDM-SessionRegisters=false
```
That is true of the session itself — gnome-kiosk is a bare mutter kiosk, so
nothing ever registers with GDM. But GDM reads "never registered" as "this
display is still mine", and about ten seconds after a perfectly successful
auto-login it **autospawns a full greeter session on the same GPU**. A login
screen covers the wall.
The part that makes this maddening: the kiosk keeps running underneath. The
session stays `Active`, Firefox keeps streaming, the mirror roster keeps
counting the wall as a viewer, `loginctl` reports the kiosk on `seat0` — every
remote check says nothing is wrong while a password prompt is on the physical
screen. This is what "it didn't auto-login" actually looks like.
From a real boot, `journalctl -b`:
```
19:07:28 PAM:authentication grantors=pam_permit acct="kiosk" ... res=success
19:07:28 New session '1' of user 'kiosk' with class 'user' and type 'wayland'
19:07:30 Started org.gnome.Kiosk@wayland.service ← the wall is up
19:07:38 New session 'c1' of user 'gdm-greeter' with class 'greeter'
19:07:39 gnome-shell: Registering session with GDM ← the greeter eats it
19:08:46 Stopped org.gnome.Shell@gdm.service ← 68s of login screen
```
**The fix** is the local override of that one file, changing only
`X-GDM-SessionRegisters` to `true`, so GDM treats the kiosk session as the owner
of the display and never brings up a greeter. Two things about it are traps:
* **Keep the file name and session id identical to the shipped one.** Inventing
`mirror-kiosk.desktop` and pointing AccountsService at it does *not* work —
GDM silently falls back to the plain GNOME session, and you get a normal
desktop with a top bar instead of a kiosk. Verified the hard way.
* A normal GNOME session already sets `true`, which is why this bug only ever
shows up on the kiosk.
After the override, the same boot looks like this: `gdm-greeter` appears **zero**
times, the session registers, and the kiosk compositor and Firefox come up
undisturbed.
## And if the wall loses the console anyway
Because the failure above is invisible from the outside, the install also drops
in a guard: once a minute it checks that the kiosk's `seat0` session is still the
foreground VT, and if not it runs `loginctl activate` on it and writes one line
to the journal.
It is deliberately timid — it never restarts gdm and never logs anyone out, and
if there is no kiosk session at all it only reports that, because a display
manager is a bad thing to fight automatically. To stop it fighting you while you
debug the console by hand:
```bash
sudo systemctl stop mirror-kiosk-console-guard.timer
sudo systemctl start mirror-kiosk-console-guard.timer # when finished
sudo journalctl -u mirror-kiosk-console-guard.service # what it has done
```
Tested by stealing the console with `chvt 1`; the wall came back on its own:
```
mirror-kiosk-guard[4598]: kiosk session 1 brought back to the foreground
```
## Verifying a kiosk you cannot see
The console is not an interface. Every claim below is checkable over SSH.
| Claim | Evidence |
|---|---|
| Auto-login works | `loginctl list-sessions` → a `seat0` session whose user is `kiosk` |
| The right session | `loginctl show-session N -p Type` → `wayland`; `pgrep -u kiosk -a gnome-kiosk` |
| The browser is up in kiosk mode | `pgrep -u kiosk -a firefox` → `--kiosk … <url>` |
| It restarted after a crash | `~kiosk/.local/state/mirror-kiosk.log` shows `exited rc=143` then `launching:` |
| The page loaded | the **server's** `GET /api/streams` shows the kiosk in `viewers` and its socket in `ss -tn` |
| Media really flows | the sharer's own page renders each viewer's `RTCPeerConnection.connectionState`; `connected` = ICE+DTLS+SRTP up |
| It is truly fullscreen | `dim-probe.py`: `viewport == window == screen == available` |
| The URL is serving *mirror* | `check-kiosk.sh` greps the page for `<title>Watch · mirror` |
| The kiosk holds the screen | `/sys/class/tty/tty0/active` equals the kiosk session's TTY and the session is `Active` — the one check that catches a covered wall |
| No greeter is fighting it | `journalctl -b \| grep -c "New session.*gdm-greeter"` is `0`; any non-zero number is a login screen on your wall |
| It survives a power cut | `sudo systemctl reboot`, then re-run `check-kiosk.sh` |
Measured on the reference client, which is attached to a 4K output:
```
viewport=3840x2160 window=3840x2160 screen=3840x2160 available=3840x2160
```
Window equal to viewport means no browser chrome is left over; screen equal to
available means no panel or bar took any of it.
After a cold boot the wall re-joins on its own — reference client back up and
subscribed to a live mirror in under 60 seconds:
```
{"mirrors": 1, "sharers": 1, "viewers": 1, "connections": 2, "watching": 1}
Firefox · Linux | Primary Monitor | 2560 x 1600 | viewers 1
```
### One trap worth knowing
`/watch` works over plain `http://<lan-ip>`, but **`/mirror` does not**:
`getDisplayMedia()` needs a secure context, so the Share button is disabled
there. Pointing a kiosk at a bare-LAN-IP mirror gives you a wall that watches
fine and cannot share. Production is HTTPS at the edge, so both work — which is
the main practical reason the kiosk URL should be the `https://` one and not a
convenient IP.
---
## When the wall hangs: making it come back by itself
On 2026-11-02 the reference wall hung solid at 14:03Z. Nothing said so, nothing
recovered it, and it stayed that way until someone walked over and powered it off.
Everything in this section exists so the next one is a 90-second blink instead of
a drive.
**What the hang actually looked like** — worth reading, because every easy
diagnosis was wrong:
| Signal | Reading |
|---|---|
| `ping` | answered, 0% loss, RTT jittering 1.9–8.2 ms on a LAN |
| `ssh` | **`connect()` succeeded**, then the socket closed 6–13 ms later with **zero bytes**. No SSH banner ever, and `sshd` never logged an attempt. |
| the service | the wall's `/ws` dropped at 14:03:04Z after 1212 s and never reconnected; `/healthz` → `viewers:0` (it is normally `viewers:1, watching:1`) |
| journal | every unit goes silent at 09:01:59 and stays silent for the 13 minutes until the power cut |
| memory | 62 GiB installed, 2.5 GiB in use, **zero OOM lines** in the previous boot |
| `pstore` | empty; the next boot reported `system.journal corrupted or uncleanly shut down` — the hard reset, and the reason the journal's tail is lost |
The trap: **ICMP replies and TCP handshakes are kernel work.** "It pings" and
"port 22 is open" say nothing about userspace. Fedora starts `sshd` per
connection, so a master that cannot fork answers the SYN from the listen socket
and then hangs up without a byte — which is exactly what happened. Put together:
the kernel was alive and no userspace process could be started. Not memory, not a
full disk, not a fail2ban ban (a ban refuses the SYN, it does not complete the
handshake) — a hard hang that left no trace.
**Two layers, because neither one covers the other:**
| Layer | File | Catches |
|---|---|---|
| kernel escalates to a reboot | `/etc/sysctl.d/90-mirror-wall-selfheal.conf` | soft lockup, hard lockup (NMI watchdog), any oops → panic; `kernel.panic = 30` reboots 30 s later |
| the chipset reboots the kernel | `/etc/systemd/system.conf.d/90-mirror-wall-watchdog.conf` | PID 1 stops kicking `/dev/watchdog` — i.e. the case above, where the kernel is too wedged to notice it is wedged |
Verify all of it (or just run `check-kiosk.sh`, which asserts every line):
```bash
ssh mirror 'systemctl show -p RuntimeWatchdogUSec --value' # 30s
ssh mirror 'cat /sys/class/watchdog/watchdog0/status' # 0x8100 on the reference box
ssh mirror 'sysctl kernel.softlockup_panic kernel.hardlockup_panic kernel.panic_on_oops kernel.panic'
# 1 1 1 30
```
`status` is a bitmask of driver features *and* current state, so do not compare it
for equality: the reference box reports `0x8100` = `WDIOF_KEEPALIVEPING` (PID 1 is
currently kicking it) plus `WDIOF_MAGICCLOSE` (a clean shutdown stops it, so the
watchdog cannot reset a box that is deliberately powered off). The bit that matters
is `0x8000`. `check-kiosk.sh` asserts `RuntimeWatchdogUSec` instead, which is not
ambiguous.
**Why 30 seconds.** It is the timeout `iTCO_wdt` initialises itself with
(`heartbeat=30 sec`), and systemd kicks at a third of whatever it is handed. Asking
a device for more than it supports would make systemd kick *less* often than the
hardware expires — a reboot loop of our own making — so `install-kiosk.sh` clamps
to the driver's own value. Boxes with no watchdog at all (most VMs) get the sysctls
and a line saying so.
**The cost, stated plainly.** Any stall of PID 1 longer than 30 s now reboots the
wall — a genuinely stuck disk, say. On an unattended display that is the intended
trade, but it is a trade; to back off raise the value or set it to 0 and rely on
the sysctls alone (the drop-in documents both).
**What it still cannot do.** `kernel.hung_task_panic` does not exist in Fedora's
kernel build (`CONFIG_DETECT_HUNG_TASK` is off), so a pure blocked-on-I/O hang is
caught by the watchdog rather than by a targeted panic — and a machine that is
*off* (dead PSU, tripped plug, firmware that never reaches the bootloader) cannot
watchdog anything. That remains an ops decision.
---
## The wall's firewall
Fedora's `FedoraWorkstation` zone — the default on Silverblue — ships with:
```
ports: 1025-65535/udp 1025-65535/tcp 443/tcp 80/tcp
```
The **UDP** half is load-bearing: media is peer-to-peer and mirror deliberately
uses no STUN/TURN, so a wall that cannot receive UDP on high ports shows
`unreachable` tiles. The **TCP** half is pure exposure on a box whose only access
control is its address (A19) — it made every high port on the wall connectable
from anywhere the wall was reachable, and a wall never serves anything.
`install-kiosk.sh` step 8/9 removes `1025-65535/tcp`, leaves the UDP alone, and
says which of the two it found; `check-kiosk.sh` FAILs if either half moves.
This was measured, not assumed. With the TCP range gone, the wall's WebRTC socket
process still had ICE candidates bound on LAN IPv4 and public IPv6, and media kept
flowing inbound:
```
wall NIC over 6s: rx 459.6 kbit/s tx 16.6 kbit/s
```
while inbound probes from a same-subnet host went `tcp/22` reachable and
`tcp/8080`, `tcp/9999` silent. (To see the ICE sockets, grep for
`Socket Process` — Firefox's WebRTC sockets are not owned by a process named
`firefox`.)
Still open and not needed by a wall: `80/tcp`, `443/tcp`, `samba-client`. Left
alone because they are not this change.
---
## The wall went quiet and nobody noticed
`/healthz` knew at 14:03:04Z that nothing was watching. It is a JSON endpoint and
nothing was reading it, so the outage was discovered by looking at the panel. The
other half of running an unattended wall is being told about it —
[`../wall-watch/`](../wall-watch/) checks presence on the server host every minute
and pages over ntfy, then announces the recovery.
```bash
tar czf - -C docs wall-watch | ssh server \
'cd /var/tmp && tar xzf - && cd wall-watch && sudo sh install-wall-watch.sh'
```
Test it the honest way — a **drill**, labelled as one in the title, body and
priority so it can never be mistaken for an incident:
```bash
sudo sh -c '. /etc/mirror-wall-watch.env; WALL_DRILL=1 WALL_HEALTH_URL=http://127.0.0.1:1/healthz \
WALL_THRESHOLD=1 WALL_STATE_DIR=/tmp/wall-watch-test sh /usr/local/libexec/mirror-wall-watch.sh'
```
Two things worth knowing: it probes loopback `/healthz`, which works only because
that path is exempt from the address gate (see `AGENTS.md` §3); and it cannot
detect the **server** being down, because it lives there. `scp` to some of these
hosts dies with `lost connection` (no sftp subsystem) — the tar pipe above works
everywhere.
---
## Why there is no RDP on the wall
You cannot see the panel, so of course the idea is to look at it remotely.
`gnome-remote-desktop` is already in the Silverblue base image, so it looks free.
It does not work here, for a structural reason:
**`grd` does not capture a screen.** It is a *client* of mutter's remote-desktop
D-Bus API, and gnome-kiosk embeds libmutter without exporting it. Ask the wall's
own session bus:
```bash
sudo -u kiosk XDG_RUNTIME_DIR=/run/user/1001 \
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1001/bus \
busctl call org.gnome.Mutter.RemoteDesktop /org/gnome/Mutter/RemoteDesktop \
org.freedesktop.DBus.Peer Ping
# Call failed: The name is not activatable (same for .ScreenCast and .DisplayConfig)
```
Not a missing package: gnome-shell 50.6 *is* installed and is what exports those
names; gnome-kiosk 50.1 does not. So `grdctl rdp enable` and
`grdctl rdp set-credentials` would give you a running daemon with nothing to
share. For whoever tries next:
- this image ships **`grdctl`, not `grdc`**, and has **no `org.gnome.RemoteDesktop`
gsettings schema** — configuration is grdctl/JSON, so the `DCONF_PROFILE` trap
above does not even apply to it.
- the only way to really see the panel over RDP is to move the wall onto a
gnome-shell-based session. That replaces the script session this whole runbook
is built on — the never-exit loop, `window-config.ini`, the greeter trap — and is
a `PLAN.md` decision with a full re-validation, not a config tweak.
- **and it would not have survived the hang above.** An RDP server is userspace;
the failure was userspace being unable to start. RDP is a way to *look*, never a
rescue path. The rescue path is the watchdog.
---
## Troubleshooting
| Symptom | Cause and fix |
|---|---|
| **Login screen on the wall, but every remote check passes** | The session is not registered with GDM, so GDM autospawns a greeter over it. See [The greeter that eats the wall](#the-greeter-that-eats-the-wall); confirm with `ls /usr/local/share/wayland-sessions/` |
| Wall vanished, console on a blank VT | Someone or something switched VT — the guard reclaims it within a minute; `journalctl -u mirror-kiosk-console-guard.service` |
| Greeter on screen after boot and `seat0` is another user | `sudo systemctl restart gdm`. GDM reads `custom.conf` at start only |
| Kiosk logs in to a normal desktop | `Session=` empty in `/var/lib/AccountsService/users/kiosk`; re-run the installer (it restarts `accounts-daemon`) |
| Wall shows a non-mirror page | the URL serves something else. `check-kiosk.sh` calls this out; a stale vhost/DNS looks like a working kiosk showing a login form |
| Black screen, no browser at all | the URL is unreachable so the loop is waiting. `curl -fsSL <url>` from the client; then look in `~/.local/state/mirror-kiosk.log` |
| A new window every few seconds | someone removed the `pgrep` guard from the kiosk script |
| Blank on wake from idle | the gsettings did not apply (they must run inside the session) — `journalctl _UID=<kiosk uid> \| grep -i permission` |
| `Permission denied` about keyrings/ICC | `~kiosk/.local` is not owned by kiosk. `sudo chown -R kiosk:kiosk /var/home/kiosk/.local` and re-run the installer |
| Firefox shows a restore-session or first-run page | the policy file is missing or unreadable — `sudo cat /etc/firefox/policies/policies.json` |
| Tile says `unreachable` | the sharer and the kiosk could not find each other over ICE. Both must be on a network where they can send UDP directly; mirror uses no STUN/TURN by design |
| Wall shows the "not allowed" page | the kiosk's address is outside `MIRROR_ALLOW_CIDRS` — this is admission control working, see `SECURITY.md` |
| **`ssh` connects, then closes in milliseconds with no banner** | Not a firewall ban — a ban refuses the SYN. The kernel is answering from the listen socket while `sshd` cannot spawn a session: the box is **hung**, and nothing userspace on it, including this checker, will run. The hardware watchdog should reset it within a minute. |
| It pings and the port is open, so why is it broken? | ICMP replies and TCP handshakes are produced by the kernel. They say nothing about userspace. Trust `viewers` on `/healthz`, not `ping`. |
| Wall reboots on its own | Either the kernel panicked or PID 1 missed the watchdog — that is the design working. Find out which before it happens again: `journalctl -b -1 -p err`, `journalctl -b -1 -k \| tail`, and `ls /sys/fs/pstore` (this box has no ERST backend, so a panic trace may not survive). |
| `install-kiosk.sh` pointed the wall somewhere else | It no longer can: an existing `kiosk.conf` URL wins over the built-in default. Pass `KIOSK_URL=...` to change it deliberately. |
Debugging rule for this project applies: reproduce at the lowest layer that can
fail. `curl` the URL from the client, check `/api/streams` on the server, and
only then reach for the browser.
---
## Deliberately not done here
These are real hardening steps for a kiosk in a public space; each is a
decision, not an oversight.
* **The `kiosk` user has no password.** Auto-login does not need one and a
password does not stop anyone with a keyboard. If you want to be able to log in
as `kiosk`, set one with `sudo passwd kiosk`.
* **TTYs are live.** Ctrl-Alt-F3 still gets a console, which is your rescue
path. `mask`-ing `getty@tty3..6` is the usual lockdown — do it only once the
kiosk is known good, since it removes the way back in without SSH.
* **No GRUB password**, so anyone with physical access can boot something else.
* **No automatic OS updates.** `rpm-ostree` is transactional and the previous
deployment is kept, so a bad upgrade is undone with `rpm-ostree rollback` (or
chosen from the GRUB menu if it will not boot). Unattended upgrades of a box
nobody looks at is its own risk — decide it deliberately, via
`rpm-ostreed-automatic.timer`.
* **SELinux is left enforcing, unmodified.** Nothing here needed a policy tweak;
treat a new AVC as a bug in this runbook.
* **No power cycle.** A hang is now recovered by the kernel and the chipset
watchdogs above. A machine that is *off* — dead PSU, tripped plug, firmware that
never reaches the bootloader — cannot reboot itself, and if the wall is remote
enough for that to matter it is still an ops decision (greenboot, or a smart
PDU), not a client one.
## Naming on the reference network
| Name | Address | What it is |
|---|---|---|
| `mirror.reeseapps.com` | 10.1.1.2 | the mirror service (nginx → container), valid TLS cert |
| `mirror-client.reeseapps.com` | 10.2.1.183 | the reference kiosk client |
| `mirror` | — | SSH alias for the kiosk client in `~/.ssh/config` |
The client's address is **DHCP**. A kiosk wants a reservation: the address is
what admits it to the service (`MIRROR_ALLOW_CIDRS`, nginx `allow`/`deny`), so a
renewal into a different range can lock a wall out by itself.