ai-sandbox/PLAN.md

PLAN

Execution plan for ai-sandbox, with the state each phase leaves behind. Every phase ends with a gate that must pass before the next one starts. Status as of this writing: phases 1-9 complete, follow-ups listed at the end.

#	Phase	Gate	Status
0	Measure the host before designing	tests/spike_network.sh results recorded in docs/networking.md	✅ done — killed the sidecar-on-internal-network design, found pasta --map-host-loopback + in-container nftables
1	Skeleton: CLI, config, preview modes	make test (args, config, precedence)	✅ done
2	Image build	build, cache, content-hash tag, contents asserted	✅ done (tests/e2e/10_image.sh)
3	Enforcement in the container	generated ruleset shape + real-container capability checks	✅ done (tests/unit/06_entrypoint.sh, tests/e2e/11_net.sh)
4	Egress gate	allowlisted host reachable, everything else refused, gate lifecycle	✅ done (tests/e2e/11_net.sh)
5	Environment passing	deny-by-default, 0600 delivery, redaction, fail-closed on unset	✅ done (tests/unit/04_env.sh, tests/e2e/12_sandbox.sh)
6	Limits, lifecycle, observability	exit codes, cleanup, caps, session log, --keep	✅ done (tests/e2e/12_sandbox.sh)
7	Harness defaults, docs, portability sweep	alias UX, README/threat model, shellcheck clean, no personal paths	✅ done
8	Supply chain: pinned base, TTL refresh, audit trail	digest in FROM and in the content hash; update/--check; labels + in-image package manifest	✅ done (tests/unit/07_base.sh, tests/e2e/10_image.sh)
9	Pi config bridge: same provider inside, credential by env	host models.json/settings.json rewritten not copied; canary secret nowhere in state; seeding, refresh and edit-presence asserted in a real container	✅ done (tests/unit/08_pi_config.sh, tests/e2e/12_sandbox.sh)

Phase detail

0 — Measure first (this is why the design looks unusual)

Ran tests/spike_network.sh. Three results decided the architecture: containers on one user-defined network cannot talk to each other rootless here (so no sidecar proxy on an internal network); --network=host works rootless and has internet (so a host-loopback gate is possible); pasta:--map-host-loopback,<addr> makes a container's connection to <addr> land on the host's loopback; and --cap-add=NET_ADMIN lets root inside the container install nftables that genuinely gate the pasta path. Also learned bind mounts need :z under SELinux.

1 — Skeleton

parse_args with the alias-friendly rule (sandbox flags until the first positional/unknown flag; those become arguments to the harness; -- replaces the command), config files at two levels, AISB_* overrides, --dry-run / --print-policy / --print-entrypoint / --print-containerfile. Precedence is built-in < config file < env < flag, and the built-in defaults live only in resolve_opts() — materialising them at the top of the script silently made the environment beat config files, which is a bug worth remembering.

2 — Image

Generated Containerfile, tag = content hash of base image + versions + install list + entrypoint + Containerfile itself, so a policy change never reuses a stale image and variants coexist. Node tarballs are checksum-verified against nodejs.org. pi is baked in so the pi alias works after one build.

3 — Enforcement

entrypoint_script(): read policy → render inet aisb with policy drop → nft -f → verify the default-drop rule exists → setpriv drop to uid 1000 with --inh-caps=-all --bounding-set=-all --no-new-privs. Refuses to start (77/78) rather than run unenforced.

4 — Gate

Per-session squid container on --network=host, bound to 127.0.0.1:<free port>, config generated and bind-mounted read-only, http_access deny all last. Sandbox gets proxy env + a ruleset that allows nothing else out. Torn down with the session unless gate up created a named one.

5-7 — Environment, lifecycle, polish

0600 env file delivery and redaction everywhere; limits and caps asserted from inside; --keep leaves the container and the session dir for forensics; clean reaps (--all for running ones, --volumes[=GLOB] for unused home caches, never a volume a container still references); doctor verifies the host including a real bind/uid round-trip; docs; shellcheck-clean in a container.

/home/sandbox is a named volume so npm and tool caches survive between runs. It outlives the image, which is the one kind of staleness this design cannot hash away: change the image's home layout and existing volumes keep the old files, so ensure_home_volume() creates it labelled and clean --volumes is the documented way to drop it.

8 — Supply chain

The hash made our changes traceable but said nothing about the base: FROM ubuntu:24.04 is a moving pointer, podman build's pull policy is missing, and ensure_image() returns early when the tag exists — so even build --rebuild reused the cached base. Now the tag is resolved to a digest (skopeo, falling back to podman pull unless AISB_BASE_PULL_FALLBACK=0), recorded per-ref under $AISB_CACHE_DIR/base/, and used in FROM and in image_hash(): a refreshed base yields a new image rather than mutating one that may be running.

Two rules make it survivable rather than merely strict. A stale pin beats a mutable tag — offline, or with AISB_BASE_AUTO_UPDATE=0, the recorded digest is used, and only a build with no digest at all is allowed, loudly marked UNPINNED in the output, in the image labels and in doctor. Freshness is a TTL (AISB_BASE_TTL_DAYS=7, 0 every build, -1 never) rather than a background job, so nothing surprises you mid-session and update stays an explicit, dated event. Both images carry aisb.base-digest/aisb.node/aisb.pi-version labels and /etc/aisb/manifest.txt with resolved apt versions, which is how you answer "which squid was enforcing this?" a month later. --print-* stays cache-only and deterministic; inspect is the surface that asks the machine now.

One bug worth remembering: POL[base_digest]="$(base_pin …)" looked right and silently produced unpinned labels, because the subshell's assignment was discarded. Helpers return text; callers assign.

9 — Pi configuration bridge

The image ships pi with no provider and no preferences, so the first run asked you to configure the model twice. It now reads ~/.pi/agent/{models,settings}.json and rewrites them: every provider apiKey becomes "$AI_API_TOKEN" (pi resolves an env reference itself, so no value is ever written to disk), !command header values are dropped rather than executed, settings pointing into $HOME are dropped because that path is not mounted, and auth.json/models-store.json/sessions/extensions are never read. The pair lands in /home/sandbox/.pi/agent at container start, owned by uid 1000, mode 600, and is a managed file: a host change is re-seeded, a sandbox edit is kept and reported, a deletion takes the host version again.

Two decisions worth keeping. The token follows the existing env rules — one variable, passed by name through the 0600 file, announced when auto-passed — and a bridged run where that variable is unset or empty is refused instead of starting a pi that cannot reach its model, because "sandbox is broken" is a worse bug report than an explicit refusal. And the bridge never widens the allowlist; it warns with the exact --allow to add, since an agent silently unable to answer looks like a network failure.

10 — The .agents share

A sandboxed harness found none of your skills, prompt templates or standing AGENTS.md, because those live in ~/.agents and only the project is mounted. It is now bind-mounted read-only at /home/sandbox/.agents by default (agents, agents_dir, agents_dest; --agents, --no-agents, --agents-dir), which is where pi looks for user-level skills. Bound, never copied, so the sandbox always sees the live tree and nothing of it lands in ai-sandbox state.

Read-only is the load-bearing part and there is deliberately no way to lift it: a skill is an instruction the model follows later, on the host, so a sandbox that could write ~/.agents had a persistence path out of itself, planted as a helpful-looking SKILL.md. It holds even under --insecure-keep-root (e2e-checked), the mount is nested inside the home volume after it so it cannot be shadowed, and an explicit --mount on the same destination wins because podman refuses duplicates. A missing or unreadable directory is a silent skip like the pi bridge's; an explicit --agents-dir that resolves to nothing is a hard error.

11 — Thinking level as policy, and a settings merge that does not freeze

defaultThinkingLevel is now xhigh by default (thinking, --thinking, AISB_THINKING; pi's own default is medium), written into the bridged settings and re-applied by the entrypoint on every run, so a run cannot lower the reasoning of the runs after it. --thinking host hands the key back to settings.json, off is pi's own "no reasoning" level, and an unknown value is refused with the list to pick from. Nothing is written for codex, claude or a command after --, and --no-pi-config declines the policy along with the bridge.

Getting there found a worse bug. settings.json was managed as one managed file, and pi rewrites its own settings (a theme change, "seen the changelog"), so the first such write matched nothing and froze every host preference out of the sandbox for good -- provider, model, thinking budgets and the level included, with a real session left holding {"lastChangelogVersion": "1.4.0"}. It is merged per key instead, against a copy of what we last wrote (.aisb-settings.json.wrote): sandbox edits win per key, untouched keys follow the host again, untouched keys the host dropped are dropped, keys pi invented survive, and a file that is missing a base copy takes the host values -- which is what repairs the volumes already frozen. Unparseable JSON is re-seeded with a warning (pi cannot use it either); a file the container cannot read is reported with the fix, because nothing inside the sandbox can repair it.

One trap worth remembering: the seeding script is passed to bash -c in single quotes, so a single quote anywhere inside it closes the argument early and the remainder becomes code in the outer shell -- valid bash, wrong program, invisible to bash -n and shellcheck. tests/unit/06_entrypoint.sh now extracts the seeding script and asserts it is quote-free, and extracts the merge program to test its semantics without a container.

12 — The pipeline driver: agent in, git out

bin/ai-sandbox-autophase runs a .agents/phases/ pipeline (the shape the phased-execution skill drives) with the split that the sandbox forces: executor agents and .agents/validate.sh inside, commits and pushes outside. Two reasons, both about the writable mount. validate.sh lives in it, so running the gate on the host is running agent-editable bash on the host (--validate-in host for a project whose toolchain is only on the machine, warned every run). And the remote is deliberately not allowlisted -- an agent that can push can ship what you are about to review -- so somebody else has to deliver, and that somebody is this script, between unit runs: after each phase commit, again before the next phase starts, again at the end, and a re-push of whatever an earlier run could not deliver.

The launcher is the user's own command, because the allowlist and the token are theirs. A script cannot see an interactive shell's functions, so sandbox-pi (default, --pi-cmd/AISB_AP_PI_CMD) is looked up in ~/.bashrc.d/*, ~/.bashrc, ~/.bash_aliases, ~/.profile: the rc file is sourced in a throwaway bash -ic with exit neutered and the definition returned -- never called, which is what lets --status/--dry-run promise no token fetch, no image build, no container. That probe is a setsid, </dev/null, timeout -k read into a file: an interactive bash started from a terminal takes that terminal, is stopped for trying, and a stopped process does not take the SIGTERM meant for it, so a piped probe could wedge the whole script in silence -- and any grandchild the rc files leave behind holds a pipe's EOF open long after the shell it came from is gone. A function launcher is called with the run (NAME --no-tty --no-interactive -- pi <args> <prompt>, stdin on /dev/null), so it has to forward its arguments; one that ends at --env NAME drops the command, and is named by --status and refused by the preflight rather than running the image default with no prompt. Each child then re-sources and runs it, so pass runs per unit rather than once a night.

Integration is by shadowing five functions of the sourced library (pi, first_prompt, fix_prompt, run_validation, push_phase) behind a declare -F contract check, so the library keeps scheduling, retries, provider backoff, report recovery and the scoped commit, and a library that changes shape is refused rather than half-wired.

What it found on the way, in order of how much it mattered:

• A repository is what git runs, not just what it records. Committing on the host after an agent had the tree writable means .git/config and .git/hooks/ are now input to a host process. Host git therefore runs with an empty core.hooksPath, no pager, no system config and protocol.ext.allow=never, and delivery is refused if a unit changed either (--trust-git-changes to override). tests/unit/10_autophase.sh plants a pre-commit from inside the fake sandbox and asserts it never runs, in either mode.
• Shadowing pi and resolving a launcher called pi are the same name. Resolve first, install the shims after; a PATH program is captured as an absolute path, or the function wins and the child recurses.
• A preview can leak a body. The captured launcher definition contains the pass call, and the child command is that text plus arguments -- so previews name the launcher instead of printing it, and ap_redact covers --env NAME=value and credentials inside remote URLs, which is exactly what a failing git push quotes back.
• A report is a gate too. Running the library's push_phase around the guard, rather than letting it push on its own, is the difference between "the harness pushes" and "every push is checked".

13 — Naming the directory the agent works in

Until now pwd was /project in every sandbox on the machine, which is two problems wearing one costume: paths an agent prints (/project/src/api.py:41) have to be translated back to the host by whoever reads them, and a harness that titles its window after the cwd makes every tab of a session read the same. So the cwd is mounted at the path it has on the host, and --workdir now defaults to auto, which means precisely that.

Nothing about the mount changed -- same directory, same rw, same :z, still the only writable one. What changed is the destination, and the destination is where a bind mount can do damage: it hides whatever the image already had at that path. Hence mirror_block_of(), whose list is not "places people keep code" but "paths the image or this sandbox owns": /usr, /etc, /var, /opt, /bin, /sbin, /lib*, /boot, /proc, /sys, /dev, /root, /run, /tmp, /home and /home/sandbox*. /home needs its own rule because it is the one standard hierarchy where work lives: /home/<user>/… mirrors, while /home and the container's own home do not -- that is where the home volume and the read-only .agents share are mounted.

/tmp decided the shape of the function, and it was measured rather than assumed (podman's behaviour is the question): with --tmpfs /tmp and -v /tmp/x:/tmp/x in one run, podman puts the tmpfs on top. pwd prints the right path, writes succeed, and the project is not there. A failure that quiet, in the one place the sandbox promises not to fail quietly, earns a refusal plus a line of output: workdir_note in --print-policy, a comment line in --dry-run, one log line on a real run. /project -- the one directory baked into the image -- is what everything falls back to, so a run always has a working directory that exists, --no-cwd included.

The tests could not create most of the deny list, so tests/unit/03_argv.sh extracts mirror_block_of from the script and asks it about /usr, /home/sandbox and / directly (the pattern 06_entrypoint.sh already uses for the entrypoint), and drives both real branches through the CLI with a scratch directory under $HOME -- lib.sh: mirrorable_dir, because $TMPDIR is always the fallback branch, which is why the whole suite had only ever seen /project. tests/e2e/12_sandbox.sh then holds a real container up to the same promise: pwd, writes landing on the host, and /project, /home/sandbox and a home-volume marker all still present through a mirrored run.

Two things the naming turned up on the way:

• The first version took the first path component with ${top%%/*}, which on any path with a second slash yields the empty string -- so every deny rule matched nothing, everything mirrored, and bash -n and shellcheck were both clean. The fix is the table of paths in both directions in 03_argv.sh; one assertion on the happy path would not have seen it.
• A cwd containing : cannot be handed to podman in a -v spec at all (invalid option type "/project"), and mirroring doubles the exposure by making that string the destination too. It is refused now, with the reason, instead of podman's. (--mount type=bind,src=…,dst=… parses it happily; switching the mount form for one character is not worth the relabel-option churn that comes with it.)

14 — Publishing a port: the one control that runs the other way

Everything up to here gates egress, which left an everyday request unanswered: the agent starts a dev server inside the sandbox and a human wants to look at it. The documented answer used to be "run the server on the host", which is good advice until the agent has to drive the server — seed the database it starts, migrate then serve, hit its own endpoints, read its logs. So: --publish / publish, ingress, with the cost named in the flag rather than discovered later.

Two measurements decided the shape. A bare podman -p 8000:8000 binds every interface on the host, so a port committed in a repo config would also be a port opened on the LAN — hence the default bind address is 127.0.0.1, an address in the spec is written as addr:host:container and warns, and publish_addr exists as a machine-wide override nobody should put in a project file. And the input chain, which reads as policy drop with no room for a forward, turns out not to block it: nft -a list chain inet aisb input while a published port serves traffic shows the aisb-publish rule at 0 packets, because pasta forwards into the namespace over loopback and iifname "lo" accept carries it. The explicit rule is therefore not what makes publishing work here; it is what makes it work on a host that delivers on the interface, and what makes nft list table inet aisb answer "what can reach this container" instead of leaving that question to a flag on the host.

A third measurement became the documentation, because it is what makes a configured preview look broken: a server bound to 127.0.0.1 inside answers 200 from inside and refuses the published port from the host. pasta delivers to the container's own address, so the project's dev command has to bind 0.0.0.0 — --host 0.0.0.0, runserver 0.0.0.0:8000 — and the skill is told to say so in the same change that writes the port.

The grammar is deliberately smaller than podman's: port, host:container, addr:host:container, one port per entry, no ranges and no /udp, because this list is reviewed line by line and a range is a line nobody reads. An address requires all three parts, so 127.0.0.1:8000 is refused with the grammar in the message instead of silently meaning whichever of the two things podman would have picked. No -p alias: -p belongs to the harness (claude -p prints), and an unknown flag already passes through. Refusals follow the usual rule — --net none cannot forward a port and podman discards the mapping while starting the container anyway, so "it ran" would mean "nothing is listening"; and a host port already in use is named before the container starts, not after a build. pick_port() skips ports this run publishes, or the gate would bind the port the sandbox asked for first and lose the run.

The unit tests check the shape (--publish 127.0.0.1:8000:8000, the # ingress: header, which chain the rule lands in, a non-numeric port never reaching nft as code) and the e2e file checks the only question that matters: a real container with enforce=nft, a host curl that gets 200, the same run still refusing a non-allowlisted domain at 403, the server running as uid 1000 with no capabilities so it cannot widen its own port, and the host listener gone once the container exits. That test paid for itself immediately: the first version of the rule put comment before accept, nft refused it, and the sandbox exited 77 — correctly refusing to run unenforced, and silently killing the feature, with --dry-run looking perfect.

Follow-ups (not yet done)

• --net unix mode: --network=none plus a unix-socket proxy and an in-container TCP→socket forwarder. Structurally stronger (no IP stack at all); needs a story for tools that resolve names themselves.
• Re-resolve --no-gate domains on a timer, or resolve through the gate and pin the returned answers, so long-running sessions do not go stale against CDNs.
• Idle gate reaping beyond clean: a --gate-ttl label check on start.
• --allow-from-lockfile: derive a first-cut allowlist from package.json/requirements.txt registries so the first run is not a blank page.
• Quarantine mode for untrusted repos: --net none + read-only mount + --ephemeral-home as a single flag, plus a warning if .git hooks exist.
• --record: optional pcap/CONNECT-log capture of a session for review.
• Packaging: a quadlet/systemd unit for a long-lived named gate.
• Signature verification for the base where the registry signs it (cosign/sigstore), so a pin is authenticated rather than merely immutable; and an optional local mirror/AISB_SKOPEO-compatible offline source for air-gapped hosts.
• update --auto as a timer unit, if TTL-on-next-run turns out to be too lazy for security response — deliberately not done yet, since a background rebuild that surprises a running session is its own hazard.
• Scoped sandbox tokens: if the gateway can mint a short-lived, per-sandbox key, --pi-token-env should ask for one rather than forwarding the long-lived host credential. The bridge deliberately does not guess a minting endpoint.
• --thinking for the other harnesses: codex and claude have their own reasoning knobs; a policy that spans harnesses needs to know each one's config shape rather than assume pi's.
• Bridging extensions/skills on request: --pi-mount-ext or similar for the extension paths named in enabledExtensionPaths, which are host paths and therefore currently dropped; needs the same explicit-grant shape as --mount.
• Published ports in the driver: .ai-sandbox/config already reaches every child unit, so a pipeline can publish — but two units running in sequence want the same port, and a long run holding one is a decision the driver should make out loud (--publish per unit, or a range handed out from a pool) rather than inherit.
• publish = :8000 (podman's "pick a free host port"): useful for parallel sandboxes, and it needs the chosen port reported back to the terminal, since nothing in the config says what to open in the browser.
• UDP publishing: deliberately left out of v1. It drags in the protocol suffix the parser refuses, and the sandboxed use cases (dev servers, APIs, hot-reload) are all TCP.
• models.json review surface: inspect could print the exact diff between host config and bridged config, so a change is reviewable before a run rather than after.