ai-sandbox/PLAN.md

# PLAN

Execution plan for `ai-sandbox`, with the state each phase leaves behind.
Every phase ends with a gate that must pass before the next one starts.
Status as of this writing: **phases 1-9 complete**, follow-ups listed at the end.

| # | Phase | Gate | Status |
|---|-------|------|--------|
| 0 | Measure the host before designing | `tests/spike_network.sh` results recorded in `docs/networking.md` | ✅ done — killed the sidecar-on-internal-network design, found `pasta --map-host-loopback` + in-container nftables |
| 1 | Skeleton: CLI, config, preview modes | `make test` (args, config, precedence) | ✅ done |
| 2 | Image build | build, cache, content-hash tag, contents asserted | ✅ done (`tests/e2e/10_image.sh`) |
| 3 | Enforcement in the container | generated ruleset shape + real-container capability checks | ✅ done (`tests/unit/06_entrypoint.sh`, `tests/e2e/11_net.sh`) |
| 4 | Egress gate | allowlisted host reachable, everything else refused, gate lifecycle | ✅ done (`tests/e2e/11_net.sh`) |
| 5 | Environment passing | deny-by-default, `0600` delivery, redaction, fail-closed on unset | ✅ done (`tests/unit/04_env.sh`, `tests/e2e/12_sandbox.sh`) |
| 6 | Limits, lifecycle, observability | exit codes, cleanup, caps, session log, `--keep` | ✅ done (`tests/e2e/12_sandbox.sh`) |
| 7 | Harness defaults, docs, portability sweep | alias UX, README/threat model, shellcheck clean, no personal paths | ✅ done |
| 8 | Supply chain: pinned base, TTL refresh, audit trail | digest in `FROM` and in the content hash; `update`/`--check`; labels + in-image package manifest | ✅ done (`tests/unit/07_base.sh`, `tests/e2e/10_image.sh`) |
| 9 | Pi config bridge: same provider inside, credential by env | host `models.json`/`settings.json` rewritten not copied; canary secret nowhere in state; seeding, refresh and edit-presence asserted in a real container | ✅ done (`tests/unit/08_pi_config.sh`, `tests/e2e/12_sandbox.sh`) |

## Phase detail

### 0 — Measure first (this is why the design looks unusual)
Ran `tests/spike_network.sh`. Three results decided the architecture:
containers on one user-defined network cannot talk to each other rootless here
(so no sidecar proxy on an internal network); `--network=host` works rootless and
has internet (so a host-loopback gate is possible); `pasta:--map-host-loopback,<addr>`
makes a container's connection to `<addr>` land on the host's loopback; and
`--cap-add=NET_ADMIN` lets root *inside* the container install nftables that
genuinely gate the pasta path. Also learned bind mounts need `:z` under SELinux.

### 1 — Skeleton
`parse_args` with the alias-friendly rule (sandbox flags until the first
positional/unknown flag; those become arguments to the harness; `--` replaces the
command), config files at two levels, `AISB_*` overrides, `--dry-run` /
`--print-policy` / `--print-entrypoint` / `--print-containerfile`.
Precedence is built-in < config file < env < flag, and the built-in defaults live
only in `resolve_opts()` — materialising them at the top of the script silently
made the environment beat config files, which is a bug worth remembering.

### 2 — Image
Generated Containerfile, tag = content hash of base image + versions + install
list + entrypoint + Containerfile itself, so a policy change never reuses a stale
image and variants coexist. Node tarballs are checksum-verified against
nodejs.org. `pi` is baked in so the `pi` alias works after one build.

### 3 — Enforcement
`entrypoint_script()`: read policy → render `inet aisb` with `policy drop` →
`nft -f` → verify the default-drop rule exists → `setpriv` drop to uid 1000 with
`--inh-caps=-all --bounding-set=-all --no-new-privs`. Refuses to start (77/78)
rather than run unenforced.

### 4 — Gate
Per-session squid container on `--network=host`, bound to `127.0.0.1:<free port>`,
config generated and bind-mounted read-only, `http_access deny all` last.
Sandbox gets proxy env + a ruleset that allows nothing else out. Torn down with
the session unless `gate up` created a named one.

### 5-7 — Environment, lifecycle, polish
`0600` env file delivery and redaction everywhere; limits and caps asserted from
inside; `--keep` leaves the container *and* the session dir for forensics;
`clean` reaps (`--all` for running ones, `--volumes[=GLOB]` for unused home caches,
never a volume a container still references); `doctor` verifies the host including a
real bind/uid round-trip; docs; shellcheck-clean in a container.

`/home/sandbox` is a named volume so npm and tool caches survive between runs. It
outlives the image, which is the one kind of staleness this design cannot hash away:
change the image's home layout and existing volumes keep the old files, so
`ensure_home_volume()` creates it labelled and `clean --volumes` is the documented way
to drop it.

### 8 — Supply chain
The hash made *our* changes traceable but said nothing about the base: `FROM
ubuntu:24.04` is a moving pointer, `podman build`'s pull policy is *missing*, and
`ensure_image()` returns early when the tag exists — so even `build --rebuild` reused
the cached base. Now the tag is resolved to a digest (`skopeo`, falling back to
`podman pull` unless `AISB_BASE_PULL_FALLBACK=0`), recorded per-ref under
`$AISB_CACHE_DIR/base/`, and used in `FROM` *and* in `image_hash()`: a refreshed base
yields a new image rather than mutating one that may be running.

Two rules make it survivable rather than merely strict. **A stale pin beats a mutable
tag** — offline, or with `AISB_BASE_AUTO_UPDATE=0`, the recorded digest is used, and
only a build with no digest at all is allowed, loudly marked `UNPINNED` in the output,
in the image labels and in `doctor`. **Freshness is a TTL** (`AISB_BASE_TTL_DAYS=7`,
`0` every build, `-1` never) rather than a background job, so nothing surprises you
mid-session and `update` stays an explicit, dated event. Both images carry
`aisb.base-digest`/`aisb.node`/`aisb.pi-version` labels and
`/etc/aisb/manifest.txt` with resolved apt versions, which is how you answer "which
squid was enforcing this?" a month later. `--print-*` stays cache-only and
deterministic; `inspect` is the surface that asks the machine now.

One bug worth remembering: `POL[base_digest]="$(base_pin …)"` looked right and
silently produced `unpinned` labels, because the subshell's assignment was discarded.
Helpers return text; callers assign.

### 9 — Pi configuration bridge
The image ships `pi` with no provider and no preferences, so the first run asked you to
configure the model twice. It now reads `~/.pi/agent/{models,settings}.json` and
rewrites them: every provider `apiKey` becomes `"$AI_API_TOKEN"` (pi resolves an env
reference itself, so no value is ever written to disk), `!command` header values are
dropped rather than executed, settings pointing into `$HOME` are dropped because that
path is not mounted, and `auth.json`/`models-store.json`/sessions/extensions are never
read. The pair lands in `/home/sandbox/.pi/agent` at container start, owned by uid 1000,
mode 600, and is a managed file: a host change is re-seeded, a sandbox edit is kept and
reported, a deletion takes the host version again.

Two decisions worth keeping. The token follows the existing env rules — one variable,
passed by name through the `0600` file, announced when auto-passed — and a bridged run
where that variable is unset *or empty* is refused instead of starting a pi that cannot
reach its model, because "sandbox is broken" is a worse bug report than an explicit
refusal. And the bridge never widens the allowlist; it warns with the exact `--allow`
to add, since an agent silently unable to answer looks like a network failure.

### 10 — The `.agents` share
A sandboxed harness found none of your skills, prompt templates or standing `AGENTS.md`,
because those live in `~/.agents` and only the project is mounted. It is now bind-mounted
read-only at `/home/sandbox/.agents` by default (`agents`, `agents_dir`, `agents_dest`;
`--agents`, `--no-agents`, `--agents-dir`), which is where pi looks for user-level skills.
Bound, never copied, so the sandbox always sees the live tree and nothing of it lands in
ai-sandbox state.

Read-only is the load-bearing part and there is deliberately no way to lift it: a skill is
an instruction the model follows *later*, on the host, so a sandbox that could write
`~/.agents` had a persistence path out of itself, planted as a helpful-looking
`SKILL.md`. It holds even under `--insecure-keep-root` (e2e-checked), the mount is nested
inside the home volume *after* it so it cannot be shadowed, and an explicit `--mount` on
the same destination wins because podman refuses duplicates. A missing or unreadable
directory is a silent skip like the pi bridge's; an explicit `--agents-dir` that resolves
to nothing is a hard error.

### 11 — Thinking level as policy, and a settings merge that does not freeze
`defaultThinkingLevel` is now `xhigh` by default (`thinking`, `--thinking`,
`AISB_THINKING`; pi's own default is `medium`), written into the bridged settings and
re-applied by the entrypoint on every run, so a run cannot lower the reasoning of the runs
after it. `--thinking host` hands the key back to `settings.json`, `off` is pi's own
"no reasoning" level, and an unknown value is refused with the list to pick from. Nothing
is written for codex, claude or a command after `--`, and `--no-pi-config` declines the
policy along with the bridge.

Getting there found a worse bug. `settings.json` was managed as one managed file, and pi
rewrites its own settings (a theme change, "seen the changelog"), so the first such write
matched nothing and froze *every* host preference out of the sandbox for good -- provider, model,
thinking budgets and the level included, with a real session left holding
`{"lastChangelogVersion": "1.4.0"}`. It is merged per key instead, against a copy of what
we last wrote (`.aisb-settings.json.wrote`): sandbox edits win per key, untouched keys
follow the host again, untouched keys the host dropped are dropped, keys pi invented
survive, and a file that is missing a base copy takes the host values -- which is what
repairs the volumes already frozen. Unparseable JSON is re-seeded with a warning (pi
cannot use it either); a file the container cannot *read* is reported with the fix, because
nothing inside the sandbox can repair it.

One trap worth remembering: the seeding script is passed to `bash -c` in single quotes, so
a single quote anywhere inside it closes the argument early and the remainder becomes code
in the outer shell -- valid bash, wrong program, invisible to `bash -n` and shellcheck.
`tests/unit/06_entrypoint.sh` now extracts the seeding script and asserts it is quote-free,
and extracts the merge program to test its semantics without a container.

### 12 — The pipeline driver: agent in, git out
`bin/ai-sandbox-autophase` runs a `.agents/phases/` pipeline (the shape the
phased-execution skill drives) with the split that the sandbox forces: executor agents
*and* `.agents/validate.sh` inside, commits and pushes outside. Two reasons, both about
the writable mount. `validate.sh` lives in it, so running the gate on the host is
running agent-editable bash on the host (`--validate-in host` for a project whose
toolchain is only on the machine, warned every run). And the remote is deliberately not
allowlisted -- an agent that can push can ship what you are about to review -- so
somebody else has to deliver, and that somebody is this script, between unit runs: after
each phase commit, again before the next phase starts, again at the end, and a re-push of
whatever an earlier run could not deliver.

The launcher is the user's own command, because the allowlist and the token are theirs.
A script cannot see an interactive shell's functions, so `sandbox-pi` (default,
`--pi-cmd`/`AISB_AP_PI_CMD`) is looked up in `~/.bashrc.d/*`, `~/.bashrc`,
`~/.bash_aliases`, `~/.profile`: the rc file is sourced in a throwaway `bash -ic`
with `exit` neutered and the *definition* returned -- never called, which is what lets
`--status`/`--dry-run` promise no token fetch, no image build, no container. That probe
is a `setsid`, `</dev/null`, `timeout -k` read into a **file**: an interactive bash
started from a terminal takes that terminal, is stopped for trying, and a stopped process
does not take the SIGTERM meant for it, so a piped probe could wedge the whole script in
silence -- and any grandchild the rc files leave behind holds a pipe's EOF open long
after the shell it came from is gone. A function launcher is called *with* the run
(`NAME --no-tty --no-interactive -- pi <args> <prompt>`, stdin on `/dev/null`), so it has
to forward its arguments; one that ends at `--env NAME` drops the command, and is named
by `--status` and refused by the preflight rather than running the image default with no
prompt. Each child then re-sources and runs it, so `pass` runs per unit rather than once
a night.

Integration is by shadowing five functions of the sourced library (`pi`,
`first_prompt`, `fix_prompt`, `run_validation`, `push_phase`) behind a `declare -F`
contract check, so the library keeps scheduling, retries, provider backoff, report
recovery and the scoped commit, and a library that changes shape is refused rather than
half-wired.

What it found on the way, in order of how much it mattered:

* **A repository is what git runs, not just what it records.** Committing on the host
  after an agent had the tree writable means `.git/config` and `.git/hooks/` are now
  input to a host process. Host git therefore runs with an empty `core.hooksPath`, no
  pager, no system config and `protocol.ext.allow=never`, and delivery is *refused* if a
  unit changed either (`--trust-git-changes` to override). `tests/unit/10_autophase.sh`
  plants a `pre-commit` from inside the fake sandbox and asserts it never runs, in either
  mode.
* **Shadowing `pi` and resolving a launcher called `pi` are the same name.** Resolve
  first, install the shims after; a PATH program is captured as an absolute path, or the
  function wins and the child recurses.
* **A preview can leak a body.** The captured launcher definition contains the `pass`
  call, and the child command is that text plus arguments -- so previews name the
  launcher instead of printing it, and `ap_redact` covers `--env NAME=value` and
  credentials inside remote URLs, which is exactly what a failing `git push` quotes back.
* **A report is a gate too.** Running the library's `push_phase` around the guard, rather
  than letting it push on its own, is the difference between "the harness pushes" and
  "every push is checked".

### 13 — Naming the directory the agent works in
Until now `pwd` was `/project` in every sandbox on the machine, which is two problems
wearing one costume: paths an agent prints (`/project/src/api.py:41`) have to be translated
back to the host by whoever reads them, and a harness that titles its window after the cwd
makes every tab of a session read the same. So the cwd is mounted at the path it has on the
host, and `--workdir` now defaults to `auto`, which means precisely that.

Nothing about the *mount* changed -- same directory, same `rw`, same `:z`, still the only
writable one. What changed is the destination, and the destination is where a bind mount
can do damage: it hides whatever the image already had at that path. Hence
`mirror_block_of()`, whose list is not "places people keep code" but "paths the image or
this sandbox owns": `/usr`, `/etc`, `/var`, `/opt`, `/bin`, `/sbin`, `/lib*`, `/boot`,
`/proc`, `/sys`, `/dev`, `/root`, `/run`, `/tmp`, `/home` and `/home/sandbox*`. `/home`
needs its own rule because it is the one standard hierarchy where work lives:
`/home/<user>/…` mirrors, while `/home` and the container's own home do not -- that is
where the home volume and the read-only `.agents` share are mounted.

`/tmp` decided the shape of the function, and it was measured rather than assumed
(podman's behaviour *is* the question): with `--tmpfs /tmp` and `-v /tmp/x:/tmp/x` in one
run, podman puts the tmpfs on top. `pwd` prints the right path, writes succeed, and the
project is not there. A failure that quiet, in the one place the sandbox promises not to
fail quietly, earns a refusal plus a line of output: `workdir_note` in `--print-policy`, a
comment line in `--dry-run`, one log line on a real run. `/project` -- the one directory
baked into the image -- is what everything falls back to, so a run always has a working
directory that exists, `--no-cwd` included.

The tests could not create most of the deny list, so `tests/unit/03_argv.sh` extracts
`mirror_block_of` from the script and asks it about `/usr`, `/home/sandbox` and `/`
directly (the pattern `06_entrypoint.sh` already uses for the entrypoint), and drives both
real branches through the CLI with a scratch directory under `$HOME` -- `lib.sh:
mirrorable_dir`, because `$TMPDIR` is always the fallback branch, which is why the whole
suite had only ever seen `/project`. `tests/e2e/12_sandbox.sh` then holds a real container
up to the same promise: `pwd`, writes landing on the host, and `/project`, `/home/sandbox`
and a home-volume marker all still present through a mirrored run.

Two things the naming turned up on the way:

* The first version took the first path component with `${top%%/*}`, which on any path with
  a second slash yields the empty string -- so every deny rule matched nothing, everything
  mirrored, and `bash -n` and shellcheck were both clean. The fix is the table of paths in
  *both* directions in `03_argv.sh`; one assertion on the happy path would not have seen it.
* A cwd containing `:` cannot be handed to podman in a `-v` spec at all (`invalid option
  type "/project"`), and mirroring doubles the exposure by making that string the
destination too. It is refused now, with the reason, instead of podman's. (`--mount
type=bind,src=…,dst=…` parses it happily; switching the mount form for one character is not
worth the relabel-option churn that comes with it.)

### 14 — Publishing a port: the one control that runs the other way

Everything up to here gates egress, which left an everyday request unanswered: the agent starts a
dev server *inside* the sandbox and a human wants to look at it. The documented answer used to be
"run the server on the host", which is good advice until the agent has to drive the server — seed the
database it starts, migrate then serve, hit its own endpoints, read its logs. So: `--publish` /
`publish`, ingress, with the cost named in the flag rather than discovered later.

Two measurements decided the shape. A bare podman `-p 8000:8000` binds **every interface** on the
host, so a port committed in a repo config would also be a port opened on the LAN — hence the default
bind address is `127.0.0.1`, an address in the spec is written as `addr:host:container` and warns,
and `publish_addr` exists as a machine-wide override nobody should put in a project file. And the
`input` chain, which reads as `policy drop` with no room for a forward, turns out not to block it:
`nft -a list chain inet aisb input` while a published port serves traffic shows the
`aisb-publish` rule at **0 packets**, because pasta forwards into the namespace over loopback and
`iifname "lo" accept` carries it. The explicit rule is therefore not what makes publishing work here;
it is what makes it work on a host that delivers on the interface, and what makes
`nft list table inet aisb` answer "what can reach this container" instead of leaving that question to
a flag on the host.

A third measurement became the documentation, because it is what makes a configured preview look
broken: a server bound to `127.0.0.1` *inside* answers 200 from inside and refuses the published port
from the host. pasta delivers to the container's own address, so the project's dev command has to bind
`0.0.0.0` — `--host 0.0.0.0`, `runserver 0.0.0.0:8000` — and the skill is told to say so in the same
change that writes the port.

The grammar is deliberately smaller than podman's: `port`, `host:container`, `addr:host:container`,
one port per entry, no ranges and no `/udp`, because this list is reviewed line by line and a range is
a line nobody reads. An address requires all three parts, so `127.0.0.1:8000` is refused with the
grammar in the message instead of silently meaning whichever of the two things podman would have
picked. No `-p` alias: `-p` belongs to the harness (`claude -p` prints), and an unknown flag already
passes through. Refusals follow the usual rule — `--net none` cannot forward a port and podman
discards the mapping while starting the container anyway, so "it ran" would mean "nothing is
listening"; and a host port already in use is named before the container starts, not after a build.
`pick_port()` skips ports this run publishes, or the gate would bind the port the sandbox asked for
first and lose the run.

The unit tests check the shape (`--publish 127.0.0.1:8000:8000`, the `# ingress:` header, which chain
the rule lands in, a non-numeric port never reaching nft as code) and the e2e file checks the only
question that matters: a real container with `enforce=nft`, a host `curl` that gets 200, **the same
run still refusing a non-allowlisted domain at 403**, the server running as uid 1000 with no
capabilities so it cannot widen its own port, and the host listener gone once the container exits.
That test paid for itself immediately: the first version of the rule put `comment` before `accept`,
nft refused it, and the sandbox exited 77 — correctly refusing to run unenforced, and silently
killing the feature, with `--dry-run` looking perfect.

## Follow-ups (not yet done)

* **`--net unix` mode**: `--network=none` plus a unix-socket proxy and an in-container
  TCP→socket forwarder. Structurally stronger (no IP stack at all); needs a story
  for tools that resolve names themselves.
* **Re-resolve `--no-gate` domains** on a timer, or resolve through the gate and pin
  the returned answers, so long-running sessions do not go stale against CDNs.
* **Idle gate reaping** beyond `clean`: a `--gate-ttl` label check on start.
* **`--allow-from-lockfile`**: derive a first-cut allowlist from
  `package.json`/`requirements.txt` registries so the first run is not a blank page.
* **Quarantine mode for untrusted repos**: `--net none` + read-only mount +
  `--ephemeral-home` as a single flag, plus a warning if `.git` hooks exist.
* **`--record`**: optional pcap/CONNECT-log capture of a session for review.
* Packaging: a `quadlet`/systemd unit for a long-lived named gate.
* Signature verification for the base where the registry signs it (cosign/sigstore),
  so a pin is authenticated rather than merely immutable; and an optional local
  mirror/`AISB_SKOPEO`-compatible offline source for air-gapped hosts.
* `update --auto` as a timer unit, if TTL-on-next-run turns out to be too lazy for
  security response — deliberately not done yet, since a background rebuild that
  surprises a running session is its own hazard.
* **Scoped sandbox tokens**: if the gateway can mint a short-lived, per-sandbox key,
  `--pi-token-env` should ask for one rather than forwarding the long-lived host
  credential. The bridge deliberately does not guess a minting endpoint.
* **`--thinking` for the other harnesses**: codex and claude have their own reasoning
  knobs; a policy that spans harnesses needs to know each one's config shape rather than
  assume pi's.
* **Bridging extensions/skills on request**: `--pi-mount-ext` or similar for the
  extension paths named in `enabledExtensionPaths`, which are host paths and therefore
  currently dropped; needs the same explicit-grant shape as `--mount`.
* **Published ports in the driver**: `.ai-sandbox/config` already reaches every child unit, so a
  pipeline can publish — but two units running in sequence want the same port, and a long run holding
  one is a decision the driver should make out loud (`--publish` per unit, or a range handed out from
  a pool) rather than inherit.
* **`publish = :8000`** (podman's "pick a free host port"): useful for parallel sandboxes, and it
  needs the chosen port reported back to the terminal, since nothing in the config says what to open
  in the browser.
* **UDP publishing**: deliberately left out of v1. It drags in the protocol suffix the parser
  refuses, and the sandboxed use cases (dev servers, APIs, hot-reload) are all TCP.
* **`models.json` review surface**: `inspect` could print the exact diff between host
  config and bridged config, so a change is reviewable before a run rather than after.