ducoterra
a08a69f143
Two changes, one bug found through the other.
Thinking is policy now: defaultThinkingLevel is xhigh unless you say otherwise
(thinking / --thinking / AISB_THINKING, with pi's levels plus 'host' to decline
and 'off' meaning pi's own no-reasoning level). It is written into the bridged
settings and re-applied by the entrypoint on every run, so a run cannot set its
own reasoning down for the runs after it. Unknown levels are refused with the
list to pick from. Nothing is written for codex, claude or a command after --,
and --no-pi-config declines it along with the bridge, because pi's own defaults
are what that flag asks for.
The reason xhigh had never reached a sandbox is worse than a missing default:
settings.json was managed as one whole file, and pi rewrites its own settings
(a theme change, "seen the changelog"), so the first such write matched nothing
and froze every host preference out permanently -- provider, model, thinking
budgets and level included, a real session left holding just
{"lastChangelogVersion": "1.4.0"}. It is merged per key instead, against a copy
of what we last wrote: a key edited in the sandbox stays edited, a key never
touched follows the host again, an untouched key the host dropped is dropped, a
key pi invented survives, and a file with no base copy takes the host values --
which repairs the volumes already frozen. Unparseable JSON is re-seeded with a
warning since pi cannot use it either; a file the container cannot read is
reported with the fix, because nothing inside the sandbox can repair one.
One trap worth remembering: the seeding script is passed to bash -c in single
quotes, so a single quote inside it closes the argument early and the rest runs
in the outer shell -- valid bash, wrong program, clean bash -n and clean
shellcheck. tests/unit/06_entrypoint.sh extracts the seeding script and asserts
it is quote-free, and extracts the merge program to test its semantics without
a container; tests/e2e/12_sandbox.sh does the same in a real run, including
forcing the level back after the sandbox sets it to off.
LLM-Generated: true
307 lines
14 KiB
Markdown
307 lines
14 KiB
Markdown
# ai-sandbox
|
|
|
|
Run an autonomous coding agent against your real project, with the escape routes
|
|
closed: only the current directory is mounted, no ambient environment variables,
|
|
no root inside, hard resource caps, and outbound network limited to an explicit
|
|
allowlist that the agent cannot lift.
|
|
|
|
Single bash script, rootless podman, Ubuntu 24.04. No daemon, no k8s, no config
|
|
language.
|
|
|
|
```
|
|
$ cd ~/Projects/billing-api
|
|
$ pi --allow aipi.reeseapps.com --env AI_API_TOKEN
|
|
[aisb] egress gate aisb-gate-7f3c1a -> 127.0.0.1:18084 (1 domains, 0 ip/cidr entries)
|
|
[aisb] egress net=allow enforce=nft proxy=[100.64.0.1:18084] direct=0 dns=0
|
|
pi
|
|
```
|
|
|
|
`pi` there is an alias for `ai-sandbox`. Inside, the agent sees `/project` (your
|
|
cwd, read-write), nothing else on your disk, and the internet only where you
|
|
pointed it.
|
|
|
|
## Install
|
|
|
|
```bash
|
|
git clone <this repo> && cd ai-sandbox
|
|
./install.sh --alias pi # symlinks to ~/.local/bin and adds the alias to ~/.bashrc
|
|
ai-sandbox doctor # checks podman, rootless, subuid, pasta, SELinux, uid mapping
|
|
ai-sandbox build # optional: build the image now instead of on first run
|
|
make test # 390+ assertions (unit needs no containers; --e2e builds and runs)
|
|
```
|
|
|
|
Requirements: podman ≥ 5 running rootless (with a `/etc/subuid` range for your
|
|
user), and either `pasta` (default in podman 5) or `slirp4netns`. Nothing else —
|
|
shellcheck, tests and the image build all run through podman.
|
|
|
|
## The alias, and how arguments are read
|
|
|
|
Sandbox flags are consumed until the first positional argument or unknown flag;
|
|
those become **arguments to the harness**:
|
|
|
|
```bash
|
|
alias pi='ai-sandbox'
|
|
|
|
pi # runs pi inside the sandbox
|
|
pi --allow aipi.reeseapps.com --env AI_API_TOKEN --model turbo
|
|
# ^^^^^ sandbox ^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^ for pi
|
|
pi --shell # a bash shell in the same sandbox
|
|
pi --net none -- make test # -- replaces the command entirely
|
|
```
|
|
|
|
`--strict-args` turns an unrecognised flag into an error instead of a pass-through,
|
|
which is what you want in scripts.
|
|
|
|
## Network policy
|
|
|
|
| mode | what it means |
|
|
|---|---|
|
|
| `--net allow` *(default)* | Only the allowlist is reachable. A netfilter ruleset inside the container plus a domain-filtering proxy on the host loopback. If the ruleset cannot be installed, the sandbox **refuses to start** (exit 77). |
|
|
| `--net none` | No network at all. |
|
|
| `--net full` | Nothing restricted. Prints a warning. Never the default. |
|
|
|
|
```bash
|
|
ai-sandbox --allow aipi.reeseapps.com --allow registry.npmjs.org --env AI_API_TOKEN
|
|
ai-sandbox --allow 10.20.0.5:5432 # a database: pinned IP and port
|
|
ai-sandbox --allow files.pythonhosted.org --allow pypi.org
|
|
ai-sandbox --no-gate --allow example.com # no proxy container: pin resolved IPs instead
|
|
ai-sandbox --proxy corp-proxy.internal:3128 # route through a proxy you already run
|
|
```
|
|
|
|
Domains match that name and anything below it, and are enforced by the gate at
|
|
CONNECT time, so CDNs and rotating IPs are not a problem. IP/CIDR entries become
|
|
direct netfilter rules for non-HTTP destinations (databases, etc.).
|
|
|
|
An allowlist is **required** in `allow` mode: an empty list is a hard error, never
|
|
a silent free-for-all. Keep it in the repo so the policy is reviewable:
|
|
|
|
```
|
|
# .ai-sandbox/allow (one entry per line, # comments fine)
|
|
aipi.reeseapps.com
|
|
registry.npmjs.org
|
|
```
|
|
|
|
If you bridge your pi config (below), ai-sandbox tells you when the model endpoint
|
|
your own `models.json` points at is not in the allowlist -- pi would otherwise start
|
|
and sit there unable to answer, which reads like a broken sandbox.
|
|
|
|
## Environment
|
|
|
|
Nothing is inherited except `TERM LANG LC_ALL LC_CTYPE TZ`.
|
|
|
|
```bash
|
|
--env AI_API_TOKEN # from your environment
|
|
--env FOO=bar # literal (shows up in ps; prefer the two below)
|
|
--env-file ~/.config/secrets # a 0600 NAME=VALUE file
|
|
```
|
|
|
|
Values are handed to podman through a `0600` file that is removed when the run
|
|
ends, so they never appear in `ps`, in the printed dry-run, or in the session log.
|
|
Asking for a variable that is not set fails the run closed.
|
|
|
|
## Filesystem
|
|
|
|
Your cwd is mounted at `/project` and is the working directory. That is the only
|
|
*writable* mount, and nothing else on the host is visible except the one exception
|
|
below. Everything else is opt-in:
|
|
|
|
```bash
|
|
--mount ~/.ssh:ro # if the agent must be able to push
|
|
--mount ../docs:/docs:ro # read-only reference material
|
|
--no-cwd # a scratch sandbox with nothing mounted
|
|
--read-only # read-only root filesystem
|
|
--ephemeral-home # do not persist /home/sandbox (default: persist in a volume)
|
|
```
|
|
|
|
Files the agent writes land on your disk owned by **your** uid, not root
|
|
(`--userns=keep-id`, verified by `doctor`).
|
|
|
|
### Your `.agents` is shared, read-only
|
|
|
|
A harness with no skills is a harness you have to re-brief on every run, so `~/.agents`
|
|
— skills, prompt templates, your standing `AGENTS.md` — is bind-mounted at
|
|
`/home/sandbox/.agents`, which is where pi looks for user-level skills. Nothing is
|
|
copied; it is one mount, and it is **read-only**:
|
|
|
|
```bash
|
|
pi # your skills are there
|
|
pi --no-agents # a run where the harness sees none of them
|
|
pi --agents-dir ~/work/.agents # a different set, e.g. per-team
|
|
```
|
|
|
|
Read-only is not a convenience limit. A skill is an instruction the model follows
|
|
*later*, on the host, outside the sandbox — so a sandbox that could edit `~/.agents`
|
|
had a persistence path out of itself, planted as a helpful-looking `SKILL.md`. There is
|
|
no flag to make it writable, and the `ro` holds even with `--insecure-keep-root`.
|
|
|
|
A project's own `.agents/` needs no setting at all: it is inside the project mount, and
|
|
harnesses that read both (`pi` reads `~/.agents/skills` and `.agents/skills`) see the
|
|
project's next to yours. What that costs in exposure is spelled out in
|
|
[`docs/threat-model.md`](docs/threat-model.md): whatever is in those files — including
|
|
any token someone pasted into a skill — is now readable by the agent you are sandboxing.
|
|
Run `--no-agents` if a set of skills is not one you would hand to a stranger.
|
|
|
|
## Pi inside the sandbox
|
|
|
|
The image has `pi` baked in, and its home is a volume, so out of the box the sandboxed
|
|
pi has no provider and no preferences. Rather than making you configure it twice,
|
|
`ai-sandbox` **bridges** your host `~/.pi/agent/{models,settings}.json` -- rewriting
|
|
them, because copying them would ship your credential into a container whose entire
|
|
purpose is to be unable to use it:
|
|
|
|
| your host config | what the sandbox gets |
|
|
|---|---|
|
|
| `apiKey: "sk-..."` or `apiKey: "!pass show ..."` | `apiKey: "$AI_API_TOKEN"` -- pi resolves it itself, no value is written to disk |
|
|
| a header like `X-Trace: "!date +%s"` | dropped, and reported: a host command is not run in the sandbox |
|
|
| `baseUrl: "https://aipi.reeseapps.com/v1"` | unchanged, along with every model, limit and thinking level |
|
|
| settings: provider, model, theme, thinking budgets, telemetry | unchanged, merged per key |
|
|
| a settings value pointing into `~` | dropped, and reported: that path is not mounted |
|
|
| `auth.json`, `models-store.json`, sessions, extensions | never touched |
|
|
|
|
The rewritten pair is seeded into `/home/sandbox/.pi/agent` at container start, owned by
|
|
your uid, mode `600`. The two files are managed differently, because they are different
|
|
kinds of thing:
|
|
|
|
| file | rule | why |
|
|
|---|---|---|
|
|
| `models.json` | replaced when missing or unchanged, left alone once you edit it in the sandbox | it is a catalog; a partial merge of a provider list is worse than either choice |
|
|
| `settings.json` | **merged per key** | pi rewrites its own settings — a theme change, "seen the changelog" — and managing the file whole meant that one write froze *every* host preference out of the sandbox permanently |
|
|
|
|
So in `settings.json`: a key you change inside the sandbox stays yours, a key you never
|
|
touched follows the host again, a key the host drops disappears unless you had changed
|
|
it, and a key pi added (`lastChangelogVersion`) survives.
|
|
|
|
### Thinking is xhigh, and it is policy
|
|
|
|
`defaultThinkingLevel` is the one setting `ai-sandbox` insists on, because pi's own
|
|
default is `medium` and a sandbox run is usually the kind of work you want reasoned
|
|
about hard:
|
|
|
|
```bash
|
|
pi --allow aipi.reeseapps.com # xhigh, whatever your host settings.json says
|
|
pi --thinking max ... # this run only
|
|
pi --thinking off ... # no reasoning, e.g. a mechanical refactor at scale
|
|
pi --thinking host ... # stop forcing it; your settings.json decides
|
|
```
|
|
|
|
`--thinking` is a policy, not a preference: it is written over the merged settings on
|
|
every run, so a run cannot set its own thinking level down for the runs after it. It is
|
|
`pi`-only — codex, claude and anything you put after `--` read their own config, and
|
|
ai-sandbox leaves that alone. `--no-pi-config` declines the policy too, since "pi starts
|
|
with its own defaults" is what that flag is for.
|
|
|
|
The token itself follows the normal rule -- it is the one variable the bridged config
|
|
names, so it is passed by name (never by value, and never into `ps`):
|
|
|
|
```bash
|
|
pi --allow aipi.reeseapps.com # AI_API_TOKEN is picked up and announced
|
|
pi --no-pi-token-env ... # don't pass it; the run then fails closed
|
|
pi --pi-token-env GW_TOKEN ... # a different variable name
|
|
pi --no-pi-config --shell # no bridge at all
|
|
```
|
|
|
|
Refusing rather than launching a pi with no credential is deliberate -- the alternative
|
|
is an agent that silently cannot reach its model.
|
|
|
|
## Image
|
|
|
|
Both images come from one base, `docker.io/library/ubuntu:24.04` — including the gate,
|
|
which is squid from Ubuntu's apt rather than a third-party image. Built once per
|
|
distinct policy and tagged by content hash: git, curl, jq, build tools, python3, Node
|
|
22 (tarball verified against nodejs.org `SHASUMS256.txt`) and the `pi` CLI. `--install
|
|
PKG` adds packages and produces a different tag, so variants coexist, and
|
|
`--base-image` swaps the base.
|
|
|
|
### The base is pinned by digest
|
|
|
|
A tag is a moving pointer: `FROM ubuntu:24.04` means "whatever Canonical points at
|
|
this week", which makes a build unreproducible and a security patch invisible. So the
|
|
tag is resolved to a digest, recorded under `~/.cache/ai-sandbox/base/`, and the
|
|
digest goes into the `FROM` line *and* into the content hash — refreshing the base
|
|
produces a new image instead of mutating one a sandbox may be running right now.
|
|
|
|
```bash
|
|
ai-sandbox update --check # what upstream has, what you are pinned to, nothing built
|
|
ai-sandbox update # re-resolve, rebuild sandbox + gate, keep the old image
|
|
ai-sandbox build --refresh-base
|
|
```
|
|
|
|
Freshness is a TTL: every `AISB_BASE_TTL_DAYS` (default 7) the digest is re-resolved,
|
|
which is a manifest read (`skopeo`), not a download. Offline, or with
|
|
`AISB_BASE_AUTO_UPDATE=0`, the recorded digest is kept — a stale pin beats a mutable
|
|
one, and `AISB_BASE_TTL_DAYS=-1` pins until you say otherwise. A build with no digest
|
|
is announced as `UNPINNED`, labelled that way, and warned about by `doctor`. Preview
|
|
modes (`--dry-run`, `--print-*`) never touch the network, so they stay scriptable and
|
|
report the pin you already have; `inspect` is the one that asks the machine now.
|
|
|
|
### What an image admits to being
|
|
|
|
`podman image inspect` on any ai-sandbox image shows `aisb.base-digest`, `aisb.node`,
|
|
`aisb.pi-package`, `aisb.pi-version` (plus `aisb.role=gate` for the gate), and
|
|
`/etc/aisb/manifest.txt` inside it lists every resolved apt package version — the
|
|
answer to "which squid was doing the filtering" once the build log is long gone. The
|
|
previous image is always kept after an update, so rollback is a tag away.
|
|
|
|
## Inspecting a run
|
|
|
|
```bash
|
|
ai-sandbox --dry-run --allow example.com --env KEY # exact podman invocation, secrets redacted
|
|
ai-sandbox --print-policy # effective policy as key=value lines
|
|
ai-sandbox --print-entrypoint # the enforcement script baked into the image
|
|
ai-sandbox doctor [--deep] # host checks; --deep tests egress for real
|
|
ai-sandbox ps | clean [--all] [--volumes[=GLOB]] | inspect | gate up|status|logs|down
|
|
```
|
|
|
|
State lives under `~/.local/state/ai-sandbox` (session dirs, `log` of every run
|
|
with no secret values, build logs). Config in `~/.config/ai-sandbox`.
|
|
|
|
`/home/sandbox` is a named volume (`aisb-home`) so npm and tool caches survive
|
|
between runs. It outlives the image, so after changing the image you may be looking
|
|
at a stale home: `ai-sandbox clean --volumes` reclaims unused ones. It only deletes
|
|
volumes in the `aisb-` namespace that no container references, and `--volumes=GLOB`
|
|
narrows it to one name.
|
|
|
|
## Why it is hard to undo from inside
|
|
|
|
The container starts as root with `CAP_NET_ADMIN`, the entrypoint installs the
|
|
egress ruleset, and then it execs your command through
|
|
|
|
```
|
|
setpriv --reuid=1000 --regid=1000 --clear-groups --inh-caps=-all --bounding-set=-all --no-new-privs
|
|
```
|
|
|
|
No root, no capabilities, `no-new-privileges`. Flushing the ruleset needs
|
|
`CAP_NET_ADMIN`; `unshare -n` needs `CAP_SYS_ADMIN`; setuid helpers cannot hand
|
|
either back. Both are asserted against a real container in
|
|
[`tests/e2e/11_net.sh`](tests/e2e/11_net.sh). Details and the measurements behind
|
|
the design: [docs/networking.md](docs/networking.md).
|
|
|
|
**Read [docs/threat-model.md](docs/threat-model.md)** — it says plainly what this
|
|
does not protect you from (allowlisted destinations, container escapes, anything
|
|
you explicitly mounted).
|
|
|
|
## Project layout
|
|
|
|
```
|
|
bin/ai-sandbox the application: one script, ~1.3k lines
|
|
install.sh symlink into ~/.local/bin + optional alias
|
|
config/ example config, allowlist and env file
|
|
docs/ networking.md, threat-model.md, recipes.md
|
|
tests/run.sh runner; tests/unit/* (no containers) tests/e2e/* (real ones)
|
|
tests/spike_network.sh re-measure podman's behaviour on a new host
|
|
Makefile make test | e2e | lint | check | doctor | build
|
|
```
|
|
|
|
## Development
|
|
|
|
```bash
|
|
make test # unit: args, config, argv construction, env handling, rule generation
|
|
make e2e # + real containers: build cache, egress blocking, capability drop, lifecycle
|
|
make lint # shellcheck in a container
|
|
make style # bash -n
|
|
```
|
|
|
|
The Containerfile, the container entrypoint and the squid config are all generated
|
|
from `bin/ai-sandbox`, so the script remains the single source of truth
|
|
(`--print-containerfile`, `--print-entrypoint`).
|