Experimental/ai-sandbox
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: Experimental/ai-sandbox:ed2eb462d6ffa7346a141dedfe65795acc60403c
Branches Tags
Experimental/ai-sandbox:main
...
compare: Experimental/ai-sandbox:main
Branches Tags
Experimental/ai-sandbox:main
11 Commits
ed2eb462d6 ... main
Author SHA1 Message Date
ducoterra 144e322391 Add --publish: the one ingress control 
Everything else in the file gates egress, which left an everyday request unanswered: the
agent starts a dev server inside the sandbox and a human wants to look at it. This puts a
listener on the host that forwards into the container, so a browser, curl or an API
tester reaches the server the sandboxed process runs.

`--publish [ADDR:]HOST:CONTAINER`, config `publish` and AISB_PUBLISH, repeatable, with the
cost named in the flag: it is ingress, so whatever that server serves is reachable from
the address it binds. The default address is 127.0.0.1 because a bare podman -p binds
every interface (measured) -- a port committed in a repo config must not quietly be a port
on the LAN. publish_addr changes the default machine-wide; a spec with a non-loopback
address warns on every run. The grammar is smaller than podman's on purpose: one port per
entry, no ranges, no /udp, no `*` -- this list is reviewed line by line, and typing
0.0.0.0 is the whole warning. An address means three parts, so `127.0.0.1:8000` is refused
rather than guessed. No -p alias: that flag belongs to the harness (claude -p prints).

Fail closed, as elsewhere: --net none cannot forward a port and podman discards the
mapping while starting the container anyway, so "it ran" would mean "nothing is
listening"; a host port already in use is named before the container starts, not after a
build; pick_port() will not hand the gate a port this run publishes. The container is
labelled with the ports it was given, and doctor reports them.

Inside, publish_rules() accepts tcp on the published ports in the input chain and never
the output one. Measured (docs/networking.md): pasta delivers the connection over
loopback, so that rule counts 0 packets on the reference host; it is there for a host or a
bridge network that delivers on the interface, and so that `nft list table inet aisb`
answers "what can reach this container". The ports are validated again inside the
entrypoint, because a ruleset is not somewhere to splice an unvalidated string. And the
measurement that makes a configured preview look broken: a server bound to 127.0.0.1
inside answers inside and refuses the published port outside, so dev servers must bind
0.0.0.0 -- both skills say so in the change that writes the port.

Tests. Unit: the grammar and every port list that looks right and is not, config/env/flag
precedence and dedup, the canonical spec in --print-policy, the `# ingress:` header, the
flags reaching the invocation before the image, egress enforcement byte-identical with a
port open, which chain the rule lands in and that a non-numeric port never reaches nft as
code. e2e (tests/e2e/13_publish.sh): one real container with enforce=nft where the host
gets its 200 and the same run still refuses a domain outside the allowlist at 403, the
server runs as uid 1000 with no capabilities so it cannot widen its own port, and the host
listener is gone when the container exits.

LLM-Generated: true
 2026-10-30 22:53:57 -04:00
ducoterra ac2f310d10 Say what an image build is doing, and keep saying it 
A build is the slow part of a first run and the thing --install triggers: apt, a Node
tarball, an npm install. Every byte of podman's transcript used to go to a log file, so
minutes of silence looked exactly like a wedge -- and a wedge is what people interrupt,
leaving a half-built image.

One line per build step, plus a notice every AISB_BUILD_QUIET_SECS (15) that a step has
shown the terminal nothing, whether it is producing output the view drops or no output at
all. AISB_BUILD_OUTPUT=progress|full|quiet picks how much the terminal sees; the complete
transcript stays in $AISB_STATE_DIR/build-*.log in all three, because the progress view is
a filter and a failed build has to be diagnosable from the raw apt output. The first line
names the extra packages going in, which is what answers "why is this run building when
the last one did not".

podman build writes its transcript to *stdout* -- the same stdout the
$(ensure_image ...) call reads back as the image ref -- so the transcript is routed onto
stderr and the ref stays one clean line. A failed build now names the package apt could
not find instead of pointing at a file to go read, and reports the time it took.
Resolving a digest by pulling the base says so too: that is a download of the base image.

tests/unit/11_build.sh stubs podman for both heartbeat paths and checks the ref, the
failure text and that a canary credential appears nowhere.

LLM-Generated: true
 2026-10-30 22:53:57 -04:00
ducoterra 90d98dcc9e Keep the sandbox's own chatter off the stream the command owns 
stdout inside the container belongs to the command that ran. Two readers take it as
data: ai-sandbox-autophase's preflight reads its answer from that stream, and
progress.mjs reads the child pi's JSON event stream from it. Five notices in the
pi-seeding block went there instead of stderr, and the steady state (host file
unchanged, so cur = prev) prints one on *every* pi-bridged run -- the driver refused a
working sandbox on every project. They all go to stderr now, like log() and fail().

The preflight takes the aisb-ok marker from wherever it appears instead of insisting it
come first: a banner from an older image or from the user's own launcher is warned about
and quoted back, because that stream is parsed as data in every unit. It still refuses on
a non-zero exit, on silence, and on a launcher that runs no command -- the failures that
would repeat in every unit -- and quotes the foreign stdout instead of a bare exit code.

The same block, second contract: the thinking level is policy, so a home the sandbox
creates gets it on the *first* seed, not one run later via the merge.

tests/unit/06_entrypoint.sh runs the seeding block with the two streams split and asserts
stdout empty in every state (cold, steady, edited inside, unparseable, level changed), and
greps the entrypoint for any [aisb] line without >&2. tests/unit/10_autophase.sh adds the
noisy and the yapping launcher fixtures.

LLM-Generated: true
 2026-10-30 22:53:57 -04:00
ducoterra 8b70967907 Stop a trailing comment in an allow file killing every run 
cfg_read_allowlist() ended on `[[ -n "$line" ]] && CFG_allow+=(...)`, so a file whose
last line is a comment or blank returned 1 from the loop. load_config() calls it twice
in a non-final position, so under `set -e` every run died with exit 1 and no output at
all -- --dry-run and --print-policy included, with nothing to explain it.

Reading a file now always succeeds. An allowlist with nothing in it stays
validate_opts()'s call to refuse, with the message that says why.

LLM-Generated: true
 2026-10-30 22:50:04 -04:00
ducoterra 9347bca78f Lint the skill scripts with everything else 
make lint and make style now walk .agents/skills/*/scripts/*.sh, so a skill helper
is held to the same shellcheck pass as the sandbox, the tests and the installer.

LLM-Generated: true
 2026-10-30 22:49:52 -04:00
ducoterra a91744f390 stop the driver hanging on a terminal and on a launcher 
ai-sandbox-autophase could sit forever, printing nothing, in two places
that are both before the first unit -- which is why both looked like a
script that "does nothing".

The rc probe ran `timeout 15 bash -ic` in front of the user's terminal.
An interactive bash calls tcsetpgrp() to take that terminal, is in its
background, and is stopped for trying; a stopped process does not take
SIGTERM, so the timeout waited in front of it and the driver waited on
the command substitution behind it. It now runs in its own session
(setsid), with stdin closed, under `timeout -k`, and reads the
definition from a temp file rather than a pipe, because a grandchild an
rc file leaves behind holds a pipe's EOF open long after the shell it
came from has been reaped.

A launcher is called *with* the run, and a function ending at
`--env NAME` -- the shape everyone writes for typing by hand, including
every example this repo printed -- discards the flags and the
in-container command. The container then starts the image's default
harness with no prompt and, with --no-interactive gone, the terminal
attached: an agent waiting for keystrokes behind a pipe nobody reads.
`ap_forwards_args` checks the captured definition, --status and the
preflight name it with the one-line fix, and every child now starts with
stdin on /dev/null so a launcher cannot put the sandbox back on the
keyboard. Aliases and whole command lines need no forwarding: the shell
appends the words to those.

Tests: a launcher fixture that drops its arguments, one that leaves a
process behind, and shape assertions on ap_probe_rc.

LLM-Generated: true
 2026-10-29 21:07:39 -04:00
ducoterra 209f167e43 add doc-sandbox: answer from the installed program, not memory 
The explainer for questions about a flag, a config key, a refusal, a
mechanism or a security boundary. ai-sandbox generates its own
Containerfile, entrypoint, squid config and netfilter rules from one
script, so the text of the program is the documentation of the version
the user will run: the skill sets a strict ground-truth order -- the
installed script and --help, then the repo docs, then the tests as the
executable spec -- and its own fact sheet last, overridden by all of
them.

scripts/locate.sh resolves that order on an arbitrary machine: the
installed script and its version, the source tree behind it (via
$AISB_REPO, the script's own parent, $PWD, the git root), the docs with
their line counts, and the config/state/cache locations.

references/source-map.md maps a question topic to the function, doc and
test that hold the answer, plus the vocabulary (split_target,
classify_allows, gate vs --no-gate vs --proxy, POL[]/P_*/OPT_*) and the
FAQ. The skill names which safe surface answers what: the --dry-run,
--print-entrypoint, --print-containerfile and --print-policy previews
build nothing and reach no registry, while inspect, doctor --deep,
build, update and gate up do change the machine and are not for an
explanation. It records the two honest exceptions rather than
overclaiming -- --print-policy ensures the home volume, and --no-gate
hits DNS.

Read-only by construction: no build, no container, no network call, and
it reads pi_config_bridge() rather than a models.json holding a live
key. Its worst failure mode is an invented flag, so the answer format
requires the real spelling, the source it came from, what the thing does
not do, and a plain "no" plus the closest real thing when the answer is
no.

LLM-Generated: true
 2026-10-29 08:37:51 -04:00
ducoterra 34f94e38be add right-sandbox: write a project's own ai-sandbox policy 
Turns a project directory into that project's policy, so a plain
`ai-sandbox` in the directory is the correct, reviewable invocation. It
writes the two files load_config() actually reads -- .ai-sandbox/allow
and .ai-sandbox/config -- and nothing else: no sandbox.yaml, no local
override, no invented schema, because a file nobody reads is a policy
nobody has.

scripts/survey.sh gathers the evidence read-only and prints no values:
variable NAMES, remote and registry HOSTS, service PORTS, with URL
credentials stripped before anything is printed -- the same rule
--dry-run, --print-policy and the session log hold themselves to. The
agent decides from that evidence, then verifies with --dry-run and
--print-policy rather than assuming an allowlist entry means a working
toolchain.

references/settings-map.md maps every accepted key and flag to its
default, plus per-ecosystem allowlists and the survey-to-policy cheat
sheet. references/port-and-loopback.md is the honest answer on ports:
outbound to a service on the machine via the mapped loopback address
100.64.0.1 (needs an allow entry and a client that does not hand the
request to the egress proxy), and why a port the sandbox listens on is
not reachable from the host in --net allow -- with probes, because that
half is a reading of gen_rules(), not a measurement.

Policy files only. Never builds, never starts a container, and never
writes a real credential: it writes .ai-sandbox/env.example and tells
the user what to put in the gitignored 0600 file.

LLM-Generated: true
 2026-10-29 08:37:51 -04:00
ducoterra b273bfef09 name the sandbox's working directory, and drive phases through it 
Two features, one shared reason: what the sandbox calls things decides
what a run tells you afterwards.

The cwd is now mounted at the path it has on the host and that is the
working directory, so pwd, an error the agent quotes, a commit message and
the window title a harness derives from the cwd all name this project
instead of reading the same as every sandbox in the session. The mount is
unchanged: same directory, same rw, same :z, still the only writable one.

Only the destination changed, and a destination is where a bind mount can
do damage -- it hides whatever the image already had there. So
mirror_block_of() refuses the paths the image or the sandbox owns (/usr,
/etc, /var, /opt, /bin, /sbin, /lib*, /boot, /proc, /sys, /dev, /root,
/run, /tmp, /home, /home/sandbox*) and / itself, while /home/<user>/...
mirrors: the one standard hierarchy where work lives, and the one whose
per-user tree never meets the container's /home/sandbox volume or the
read-only .agents share inside it.

/tmp is what set the shape, and it was measured rather than assumed: with
--tmpfs /tmp and -v /tmp/x:/tmp/x in one run podman puts the tmpfs on top,
so pwd prints the right path, writes succeed, and the project is not there.
A failure that quiet earns a refusal plus a line of output instead:
workdir_note in --print-policy, a comment in --dry-run, one log line on a
real run. --workdir DIR still chooses the mount point; onto a refused path
it is honoured (it was asked for) and warned about. /project, the one
directory baked into the image, is every fallback, so a workdir always
exists -- --no-cwd included.

ai-sandbox-autophase runs a .agents/phases/ pipeline with the split the
sandbox forces: every executor agent and the .agents/validate.sh gate
inside, the commits and the pushes outside. The gate is in the writable
mount, so running it on the host is running agent-editable bash on the
host; the git remote is deliberately unreachable in there, because an agent
that can push can ship what you are about to review. It runs your launcher,
not a copy of it -- the definition is read out of your rc files and never
called, which is what lets --status and --dry-run promise no token fetch,
no build, no container. Host git runs with an empty hooks path and no ext::
transport, and delivery is refused if a sandboxed unit rewrote .git/config
or planted a hook. Needs --no-interactive to go with --no-tty: an
unattended run must not leave stdin open on your keyboard, and the two are
now separate choices on both sides.

Tests and tooling, most of it forced by what the two changes exposed:
mirror_block_of is extracted from the script and asked about /usr and
/home/sandbox, which a test cannot create (the 06_entrypoint.sh pattern),
and both real branches are driven from a scratch dir under $HOME, since
$TMPDIR is always the fallback -- which is why the whole suite had only
ever seen /project. e2e/12 holds a real container to the same promise,
including that /project, /home/sandbox and a home-volume marker survive a
mirrored run. The "nothing was started" checks now compare the set of
container ids before and after instead of counting every sandbox on the
machine, so a session you have open is not a test failure; the pty cases
fall back to python3 where util-linux's script is not installed.
shellcheck covers both programs, the completion file and the fixtures, so
the completion builds are word-by-word now, and a colon in the project path
is refused with the reason instead of podman's "invalid option type".

558 unit assertions, 719 with e2e; lint and bash -n clean.

LLM-Generated: true
 2026-10-28 10:06:31 -04:00
ducoterra a08a69f143 make xhigh the sandbox's thinking, and stop settings.json freezing 
Two changes, one bug found through the other.

Thinking is policy now: defaultThinkingLevel is xhigh unless you say otherwise
(thinking / --thinking / AISB_THINKING, with pi's levels plus 'host' to decline
and 'off' meaning pi's own no-reasoning level). It is written into the bridged
settings and re-applied by the entrypoint on every run, so a run cannot set its
own reasoning down for the runs after it. Unknown levels are refused with the
list to pick from. Nothing is written for codex, claude or a command after --,
and --no-pi-config declines it along with the bridge, because pi's own defaults
are what that flag asks for.

The reason xhigh had never reached a sandbox is worse than a missing default:
settings.json was managed as one whole file, and pi rewrites its own settings
(a theme change, "seen the changelog"), so the first such write matched nothing
and froze every host preference out permanently -- provider, model, thinking
budgets and level included, a real session left holding just
{"lastChangelogVersion": "1.4.0"}. It is merged per key instead, against a copy
of what we last wrote: a key edited in the sandbox stays edited, a key never
touched follows the host again, an untouched key the host dropped is dropped, a
key pi invented survives, and a file with no base copy takes the host values --
which repairs the volumes already frozen. Unparseable JSON is re-seeded with a
warning since pi cannot use it either; a file the container cannot read is
reported with the fix, because nothing inside the sandbox can repair one.

One trap worth remembering: the seeding script is passed to bash -c in single
quotes, so a single quote inside it closes the argument early and the rest runs
in the outer shell -- valid bash, wrong program, clean bash -n and clean
shellcheck. tests/unit/06_entrypoint.sh extracts the seeding script and asserts
it is quote-free, and extracts the merge program to test its semantics without
a container; tests/e2e/12_sandbox.sh does the same in a real run, including
forcing the level back after the sandbox sets it to off.

LLM-Generated: true
 2026-10-27 21:48:33 -04:00
ducoterra 6a70dd2b2b add bash completion, flags mirrored from parse_args 
Typing `ai-sandbox --` gave nothing, and a policy surface this wide
(egress modes, mounts, the agents share, the pi bridge, per-subcommand
flags) is not something to retype from memory. Nothing in the script or
install.sh emitted completions, so this adds one as a standalone file
rather than growing the script.

It mirrors parse_args() plus the per-subcommand parsers, and follows the
same rules the script does: a non-flag word that is not a subcommand ends
sandbox parsing, so after it the completer offers nothing rather than
flags that would silently become harness arguments; build, update, doctor
and clean only offer the flags they actually accept, since each dies on
an unknown one; inspect takes the full policy set, so it gets the full
list. Values complete from local state only -- --net and --harness from
their fixed modes, --env from exported host variables, --mount and
--workdir from directories, --allow from the two documented allow files.
Nothing here resolves a name or touches the network. Aliases that point
at the script (alias pi=ai-sandbox) register too, which is why the source
line belongs after the alias in ~/.bashrc.

Verified by driving COMP_WORDS for each case above and diffing the flag
list against parse_args(); bash -n is clean. Note the Makefile lint list
does not yet include contrib/, so shellcheck has not seen this file.

LLM-Generated: true
 2026-10-28 00:50:56 +00:00
34 changed files with 5788 additions and 103 deletions
Expand all files Collapse all files
.agents/skills/doc-sandbox/SKILL.md
+129
View File
@@ -0,0 +1,129 @@
---
name: doc-sandbox
description: Answers questions about ai-sandbox and ai-sandbox-autophase — what a flag or config key does, why a run refused to start (exit 77/78), how the egress allowlist, the squid gate and the in-container nftables ruleset work together, how the ~/.agents share or the pi config bridge or the base-image digest pinning behaves, what the image contains, how to run a given harness or language, what the sandbox does and does not protect against, and how the phase driver splits sandbox work from host git. Use when the user asks what something in ai-sandbox means, is about to, or just did, run ai-sandbox and wants an explanation, hits a sandbox error, or asks about the tool's design, security model or limitations. Answers from the installed program's own text plus the repo docs and tests, never from memory; runs only side-effect-free preview commands and never a build, container, or network call.
---
# doc-sandbox
ai-sandbox is one bash script that generates its own Containerfile, container entrypoint, squid
config and netfilter rules. That is what makes it answerable: the text of the program *is* the
documentation of the installed version, and three of its flags print the artifacts it generates.
Read those; do not recall.
## Ground truth, in this order
1. **The installed program** — `ai-sandbox --help`, and its source: `readlink -f "$(command -v
ai-sandbox)"`. If they disagree, the installed program is what the user will run.
2. **The repository**, if present: `README.md` (what and why), `docs/networking.md` (the
*measurements* behind the enforcement model), `docs/threat-model.md` (what is **not** protected),
`docs/recipes.md` (invocations per harness/ecosystem/pipeline), `AGENTS.md` (design rules).
3. **The tests** — the executable spec. `tests/unit/*` needs no containers and states exact shapes;
`tests/e2e/*` proves behaviour in a real container (e.g. `11_net.sh` for "can the agent lift the
ruleset"). When behaviour is subtle, the test answers better than prose.
4. **This skill's fact sheet** — fast, but if it conflicts with the above, the above wins and this
file is a bug to fix.
```bash
bash scripts/locate.sh # where each of those lives on this machine, and the version
```
## Working a question
1. Classify it: a **flag/key**, a **failure**, a **mechanism** ("how is egress enforced"), a
**security boundary** ("can it read my ssh keys"), or a **pipeline** question. Then read
`references/source-map.md` — it maps each to the function, doc and test that hold the answer.
2. Read that source. Grep, don't skim: `grep -n 'print-policy' bin/ai-sandbox`.
3. Check the version you read (`ai-sandbox version`) and say which one you read when it matters —
behaviour that changed recently includes the working-directory mirroring, the `settings.json`
per-key merge, the `xhigh` thinking policy and the base digest pinning.
4. If the question is about *this machine's* behaviour, prefer a preview command over an opinion
(§safe commands) and quote its output.
5. Answer in the format below. If the source does not contain it, say **it does not exist** and
name the closest real thing — a plausible invented flag is the worst failure mode available here.
## Answer format
* **The answer, one or two sentences**, in the user's terms.
* **The command** to run, copy-pasteable, with real flag spellings.
* **Where it comes from**: `bin/ai-sandbox` (`build_argv()`), or `docs/networking.md`, or
`tests/e2e/11_net.sh`. So they can check you.
* **What it does *not* do**, whenever the question touched security — e.g. "the allowlist limits
*where*, not *what*: an allowlisted remote plus a token can ship your code."
* A pointer to `docs/threat-model.md` when the honest answer is "this is out of scope" (kernel
escapes, anything allowlisted, anything you mounted).
## Safe to run while answering
```
ai-sandbox --help ai-sandbox version
ai-sandbox --dry-run the effective invocation, env values redacted
ai-sandbox --print-entrypoint the enforcement script baked into the image
ai-sandbox --print-containerfile the generated Containerfile
ai-sandbox --print-policy effective policy as key=value -- with one caveat, below
```
These build nothing, start no container, write no session state and reach no registry — that is a
deliberate property of the tool (preview modes are what make it scriptable), so use them freely.
They read config files, so they show the *effective* policy where you are. Two honest exceptions:
`--print-policy` also ensures the home-cache volume exists, so it fails without podman (a deviation
from the repo's own "preview must not write state" rule, worth reporting rather than repeating);
and with `--no-gate`, resolving domain entries does hit DNS.
Not for an explanation, unless the user asked about the machine and agrees:
* `inspect` — prints the same policy but resolves the base digest **now**, so it may contact the
registry.
* `doctor [--deep]` — host checks; `--deep` runs a real container and probes egress.
* `build`, `update`, `gate up`, `clean`, and any real run — these change the machine.
* `cat`ing `~/.pi/agent/models.json` to answer a question about the bridge: read the *code*
(`pi_config_bridge()`), not the file with a live key in it. Never echo a credential, an
`auth.json`, or anything from a `--env-file`.
## Facts you may state without reading (still: state the version)
| topic | fact |
|---|---|
| precedence | built-in default < `~/.config/ai-sandbox/{config,allow}` < `./.ai-sandbox/{config,allow}` < `AISB_*` < CLI. `allow/env/mount/install/label/publish` accumulate; other keys replace |
| arg parsing | stops at the first positional or unknown flag; the rest goes **to the harness**. `--` replaces the command. `--strict-args` makes an unknown flag an error |
| default net | `allow`, and an empty allowlist in `allow` mode is a hard error, never a silent free-for-all |
| enforcement | two stacked controls: an in-container `inet aisb` nft table with `policy drop` on output, installed by the entrypoint as root before it `setpriv`s to uid 1000 with no capabilities, plus a squid gate on the host loopback that filters by domain at CONNECT |
| refusals | exit **77** ruleset could not be installed; **78** policy file missing in the container. Both are the sandbox working; `--allow-unenforced` is the named escape hatch |
| gate ports | squid relays only **80, 443, 8443**; `--allow domain:8080` is not what it looks like. IP/CIDR entries are direct netfilter rules instead (default ports 80,443) |
| host services | `100.64.0.1` (config: `gate_addr`) inside the container *is* the host's loopback; only the gate's port is reachable through it unless you `--allow 100.64.0.1:<port>` |
| inbound | `--publish [ADDR:]HOST:CONTAINER` (config `publish`, `AISB_PUBLISH`): forwards a host port into the container. Bound to **127.0.0.1** unless the spec names an address (podman's bare `-p` binds every interface, measured), one port per entry, an address needs three parts, `--net none` refuses instead of starting with nothing listening, a busy host port is named before the build. The server must listen on `0.0.0.0` **inside**: a loopback-only server answers inside and refuses the published port (measured). Ingress only — egress enforcement is untouched in the same run |
| filesystem | cwd mounted rw **at its host path** (falls back to `/project` with a message when the host path would bury `/tmp`, `/usr`, `/etc`, `/var`, `/opt`, `/run`, the container home, or `/`); nothing else is mounted except the `.agents` share and the read-only policy mount at `/run/aisb` |
| the container user | uid/gid 1000, the image's own `sandbox` account, `$HOME` = its home (`--userns=keep-id`, so files the agent writes are owned by *you* on the host). Skills land at `~/.agents` on both sides and they are different directories |
| skills | host `~/.agents` → the container's `~/.agents` (`agents_dest`) **read-only**, no flag makes it writable; `--no-agents` to withhold it, `--agents-dir` for another directory. Its contents are readable by whatever runs in there |
| env | nothing inherited but `TERM LANG LC_ALL LC_CTYPE TZ` (plus `HOME`, `AISB`, `AISB_SESSION`); values travel in a `0600` file, never argv; `--env NAME` where NAME is unset **fails the run** |
| pi bridge | host `~/.pi/agent/{models,settings}.json` rewritten, not copied: `apiKey` → `"$AI_API_TOKEN"`, `!command` values dropped, `auth.json` never touched; `settings.json` merged per key; `defaultThinkingLevel` is policy (`xhigh`) re-applied every run unless `--thinking host` |
| image | one base (`ubuntu:24.04`) pinned to a **registry digest** that is part of the image tag; Node pinned + sha256-verified; `pi` version-pinned; `--install PKG` → a different tag; a build with no digest is labelled `UNPINNED` and `doctor` warns |
| limits | `cpus 4`, `memory 4g`, `pids 512`, `tmpfs 1g`; the container home is a named volume (`aisb-home`) that outlives the image, `--ephemeral-home` swaps it for a tmpfs |
| state | logs/config/cache under `~/.local/state/ai-sandbox`, `~/.config/ai-sandbox`, `~/.cache/ai-sandbox`; `~/.local/state/ai-sandbox/log` records every run **without secret values** |
| autophase | the agent *and* `.agents/validate.sh` run in the sandbox; the commits and every push happen on the host, and the git remote is deliberately unreachable from inside. Host git runs with an empty hooks path and refuses to deliver if a sandboxed unit changed `.git/config` or `.git/hooks/` |
## When the answer is "no"
Say it plainly, then give the closest real thing:
* "publish a port on my network so my phone can see it" → not as a committed setting.
`--publish 0.0.0.0:18080:8000` exists and warns on every run, because that is the sandbox becoming
a server on the LAN; `publish = 8000` (loopback) is the setting a repo file should carry. The old
answer — `AISB_EXTRA_ARGS=-p …` with `--net full` — buys nothing that `--publish` does not, and it
costs the egress policy.
* "make the sandbox able to write my skills" → no, by design: a skill is an instruction you follow
later on the host. Author it in the project and move it yourself.
* "let the pipeline push from inside the sandbox" → no: an agent that can push can ship what you are
about to review. `ai-sandbox-autophase` delivers on the host.
* "inherit my environment, it's annoying" → no; that ambient environment is exactly the exfil target.
Name what you need with `--env`.
* "silently fall back to no network policy when the ruleset fails" → no: exit 77.
If the user wants the missing thing, that is a change to `bin/ai-sandbox` — say which function
would carry it (`gen_rules()`, `build_argv()`, `parse_args()`), that it needs an entry in
`tests/e2e/`, and that per the repo's own rules it must fail closed and print no values.
## References
* `references/source-map.md` — question topic → function / doc / test, plus the FAQ and the
vocabulary (`split_target`, `classify_allows`, gate vs `--no-gate` vs `--proxy`, `POL[]`/`P_*`/`OPT_*`).
* `scripts/locate.sh` — find the installed script, the version, the repo docs, the state dirs.
.agents/skills/doc-sandbox/references/source-map.md
+126
View File
@@ -0,0 +1,126 @@
# Source map
Question topic → the function, doc and test that hold the answer. Line numbers move; **names do
not** — grep for these in the installed script (`readlink -f "$(command -v ai-sandbox)"`).
## Flags, keys, defaults
| question | read |
|---|---|
| "what does `--x` do / does it exist" | `usage()` (grouped by NETWORK / PORTS (INGRESS) / FILESYSTEM / ENVIRONMENT / IMAGE / PI CONFIG / LIMITS / RUN / ESCAPE HATCHES / SUBCOMMANDS), `parse_args()`, and `KNOWN_OPTS` + `suggest()` for the accepted set |
| "which config keys are legal" | `cfg_read_kv()` — the `case` lists them, the accumulate list is `cfg_accum_keys` (`allow env mount install label publish`). Anything else warns `unknown key` and is ignored |
| "what wins, my flag or the file" | `resolve_opts()` — every line is `${OPT_x:-${AISB_X:-$(cfgv x default)}}`: CLI, then env, then config, then default. `load_config()` shows the file order |
| "`AISB_*` variables" | the comment block above `usage()` and `--help`'s ENVIRONMENT section |
| "why did arg parsing eat my harness flag" | `parse_args()` stops at the first positional/unknown; `--` sets `COMMAND`; `--strict-args`; `default_command()`/`harness_argv()` decide what actually runs |
| subcommand behaviour | `main()` dispatch, then `cmd_build` / `cmd_update` / `cmd_doctor` / `cmd_gate` / `cmd_ps` / `cmd_clean` / `cmd_inspect` |
## Networking and enforcement
| question | read | prove with |
|---|---|---|
| how `--net allow` holds at all | `entrypoint_script()` → `gen_rules()`, plus `build_argv()`'s `--cap-add=NET_ADMIN` + the `setpriv` line | `ai-sandbox --print-entrypoint`; `docs/networking.md` |
| can the sandboxed process lift the policy | the capabilities the entrypoint drops before exec (`--inh-caps=-all --bounding-set=-all --no-new-privs`) | `tests/e2e/11_net.sh` (flush attempt, `unshare -n`, raw socket, DNS to 8.8.8.8) |
| why an entry did nothing | `split_target()` then `classify_allows()`: domains → the gate (CONNECT-time, subdomain match); IP/CIDR/ipv6 → direct netfilter rules, ports default 80,443 | `ai-sandbox --print-policy` → `gate_domains=`, `direct=`, `direct6=` |
| which ports the gate relays | `write_squid_conf()`: `Safe_ports` 80 443 8443, `SSL_ports` 443 8443, `http_access deny all` after the allowlist | `ai-sandbox gate logs` |
| gate lifecycle / ports / reuse | `pick_port()`, `gate_up()`, `teardown()`, `cmd_gate()`; `AISB_GATE_KEEP=1`; `gate_port`/`gate_addr` | `ai-sandbox gate status` |
| `--no-gate` vs `--proxy` vs the gate | `resolve_proxy_endpoints()` — `--proxy` uses your proxy (default port 3128); `--no-gate` resolves domains once on the host, opens DNS and warns; default is the built-in gate | `--dry-run` (the `HTTP(S)_PROXY` args) |
| host-loopback / `100.64.0.1` | `build_argv()`'s `--network=pasta:--map-host-loopback,${gate_addr}` and the `NO_PROXY` defaults | `docs/networking.md`'s measurement table |
| "can I see the server the sandbox is running" / what `publish` means | `resolve_publish()` (the grammar, the loopback default `publish_addr`, the refusals), `build_argv()`'s `--publish` + `aisb.publish` label, `write_policy_files()`'s `AISB_PUBLISH_PORTS`, and `publish_rules()` in the entrypoint. `pick_port()` skips published host ports | `--dry-run`'s `# ingress:` line; `docs/networking.md` "Publishing a port"; `tests/e2e/13_publish.sh` (host reaches it, egress still refused, loopback-only server inside does not answer, busy port refuses the run) |
| "does publishing weaken the sandbox?" | It is ingress only: the output chain, the gate and the allowlist are untouched, and the mapping is decided on the host before the container starts — the command has no `CAP_NET_ADMIN` and cannot add or widen a port | `tests/unit/06_entrypoint.sh` (the rule lands in `input`, never `output`), `tests/e2e/13_publish.sh`, threat model #14 |
| "is my machine able to do this" | `cmd_doctor()`, `tests/spike_network.sh` | `ai-sandbox doctor --deep` |
## Filesystem, mounts, the working directory
| question | read |
|---|---|
| why the project is at `/project` | `resolve_workdir()` and `mirror_block_of()` — the deny list is what the image or the sandbox owns, and `/tmp` is a measured case (the tmpfs covers a bind of the same path). `workdir_note=` in `--print-policy` gives the reason |
| what is mounted, rw or ro | `build_argv()`: the cwd, `/run/aisb:ro` (session policy), the home volume or tmpfs, the `.agents` share, then user `--mount`s |
| `--mount` grammar and defaults | `mount_dest()` — `SRC[:DEST][:ro\|rw]`, default `DEST=/mnt/<basename>`; the option must be `ro` or `rw`; the source must exist |
| skills sharing | `agents_src_of()`, `agents_dest_taken()`, the `.agents` block in `build_argv()`. Read-only, no override; a hand-written `--mount` on the same destination wins and the share is skipped with a log line |
| file ownership written by the agent | `--userns keep-id:uid=…,gid=…` in `build_argv()` (files land owned by *you*), checked by `doctor` |
| SELinux labels | `set_selinux()`, `POL[selinux]`, `AISB_SELINUX` |
## Environment and secrets
| question | read |
|---|---|
| what is inherited | `AISB_IMPLICIT_ENV` (`TERM LANG LC_ALL LC_CTYPE TZ`) and the explicit `HOME`/`AISB`/`AISB_SESSION` in `build_argv()` |
| how a value reaches the container | `build_env_file()` — a `0600` file, `NAME`, `NAME=VALUE` and `@file` forms; unset `--env NAME` **dies** on a real run and reports `env_missing=` in preview |
| can a secret leak through output | `print_argv()` (redacts the env file), `print_policy()` (names only), `log_session()` (net, counts, rc, command). `tests/unit/04_env.sh`, `tests/e2e/12_sandbox.sh` |
| what lands on disk per session | `session_init()` / `teardown()`: `$AISB_STATE_DIR/sessions/<id>` (`chmod 700`), removed unless `--keep` |
## The pi bridge and thinking policy
| question | read |
|---|---|
| what is copied vs rewritten | `pi_config_bridge()` — `apiKey` → `"$VAR"`, `!command` values dropped and reported, host paths dropped and reported, `auth.json` and `models-store.json` never touched |
| why `settings.json` behaves oddly | the per-key merge in the entrypoint's seeding block, and `.aisb-settings.json.wrote` — a key you edit in the sandbox stays yours; keys you never touched follow the host |
| when the run is refused for a credential | `pi_config_bridge()` + `cmd_doctor()`: unset **or empty** token variable → refused. `--pi-token-env NAME`, `--no-pi-token-env`, `--no-pi-config` |
| "the model endpoint is not in the allowlist" warning | `pi_egress_hint()`, `pi_provider_hosts_of()` |
| thinking level | `resolve_opts()` (validation of `off…max\|host`) and the seeding block: `AISB_PI_THINKING` is written over the merged settings every run; pi-only; `--no-pi-config` declines the policy too |
## Image, pinning, supply chain
| question | read |
|---|---|
| what is in the image | `containerfile()` (git curl jq build tools python3 Node pi), `gate_containerfile()` (squid from the same base) |
| why the tag is what it is | `image_hash()` (the policy, including the base digest, Node and pi versions) and `image_ref()`; `--install` changes it, so variants coexist |
| digest pinning and freshness | `base_pin_digest()`, `base_digest_online()`, `base_age_days()`, `base_ref_of()`, `cmd_update()`; `base_ttl_days`, `base_auto_update`, `AISB_BASE_PULL_FALLBACK`, `AISB_SKOPEO`. Preview modes resolve **cache-only** — that is why `--dry-run` never touches the network |
| "what version of squid filtered that" | the `aisb.*` image labels and `/etc/aisb/manifest.txt` inside the image (`image_hash()`/`containerfile()` write them) |
| rollback | `cmd_update()` keeps the previous image and prints the `podman rmi` for it |
| stale home | `ensure_home_volume()`, `cmd_clean()`'s `--volumes[=GLOB]` |
## The pipeline driver (`bin/ai-sandbox-autophase`)
| question | read |
|---|---|
| where does my launcher come from | `ap_resolve_launcher()`, `ap_probe_rc()` (sources an rc file in a throwaway `bash -ic` and returns the *definition*; it never calls it), `ap_rc_list()`, `AISB_AP_RC_FILES` |
| a launcher that ignores what it was asked to run | `ap_forwards_args()` / `ap_warn_forwarding()`: a **function** must end in `"$@"`, because every child is `NAME --no-tty --no-interactive -- <command>`. Aliases and whole command lines need nothing (`ap_build_run()`). Named in `--status`, refused by `ap_preflight()` |
| "it just sits there and does nothing" | two causes, both at the very start of a run: the launcher dropped the command so the container is running an interactive harness with no prompt (`ap_forwards_args()`), or the rc probe is waiting on the terminal (`ap_probe_rc()`: `setsid`, `</dev/null`, `timeout -k 5 15`, transcript into a file, because an interactive bash takes the terminal, gets stopped, and a stopped process ignores SIGTERM). Every child starts with stdin on `/dev/null` |
| how the child command is built | `ap_build_runner()` / `ap_build_run()` — `printf %q` per argument; `--sandbox-args` defaults `--no-tty --no-interactive`; the prompt is one argument that may contain quotes, newlines and `$` |
| what it shadows from the phase library | `ap_install_overrides()`: it first requires the library to provide `find_root next_unit phase_entries phase_next_unit unit_phase is_phase_end count_units sweep_finished_phases execute_unit unit_report notify_task commit_phase push_phase` (plus `first_prompt`/`fix_prompt`, kept as `ap_lib_*`), then shadows `pi`, `first_prompt`, `fix_prompt`, `run_validation`, `push_phase` — and nothing else. Order matters: the launcher resolves *before* `pi()` exists, or a launcher named `pi` would resolve to the shim |
| why a push was refused | `ap_git_fingerprint()`, `ap_git_guard()`, `ap_git_env()`, `ap_push_pending()`; `--trust-git-changes` is the (loud) override |
| redaction | `ap_redact()` — literal `--env NAME=value` and credentials inside remote URLs |
| what runs where | `ap_preflight()`, `ap_gate_preview()` (`--validate-in sandbox\|host`), `ap_run_pipeline()`, `ap_status()` |
| the contract without the skill installed | `tests/unit/10_autophase.sh` and `tests/fixtures/phase-lib/` |
## Vocabulary
* **gate** — the per-session squid container, `--network=host`, bound to the host loopback on a free
port ≥ `gate_port`, run as the `proxy` user with `--cap-drop=ALL`, destroyed with the session.
`gate up` is the explicit long-lived exception.
* **ruleset** — the `inet aisb` nft table installed inside the container's namespace: output
`policy drop` with room for loopback, established/related, ICMP, the direct IP entries, the gate's
address:port, and DNS only in `--no-gate` mode. Input is `policy drop` as well, accepting loopback,
established/related, a few icmpv6 types and — since `publish` exists — the tcp ports the host
forwards in (`aisb-publish`). Measured on the reference host pasta delivers those over loopback, so
that rule carries 0 packets there and does the work on a host that delivers on the interface.
* **`POL[...]` / `P_*` / `OPT_*` / `CFG_*`** — resolved policy / resolved lists / raw CLI flags /
config-file values. Reading raw env vars elsewhere is the bug pattern; a helper called as
`X="$(helper)"` runs in a subshell, so any `POL[..]` it sets is discarded.
* **fail closed** — no allowlist in `allow` mode, missing policy file (78), uninstallable ruleset
(77), unset `--env NAME`, unset-or-empty bridged token: all refusals, never degradations.
* **escape hatch** — `--allow-unenforced`, `--insecure-keep-root`, `--net full`, `--mount`,
`extra_args`, `AISB_EXTRA_ARGS`. Named to be uncomfortable; nothing uses them implicitly.
## FAQ (verify the version before quoting)
| question | answer |
|---|---|
| `--print-policy` says "could not create the home cache volume"? | There is no usable podman (missing, or not rootless, or no `/etc/subuid` range) — `--print-policy` ensures the `aisb-home` volume exists, unlike `--dry-run`, which is the fully side-effect-free preview. Run `ai-sandbox --dry-run` to see the policy and `ai-sandbox doctor` for the host. |
| Why exit 77? | The ruleset could not be installed, so the sandbox refused to run unfiltered. `ai-sandbox doctor`; check `podman info \| grep -i rootless` and `/etc/subuid`. Debug with `--allow-unenforced` only to see the error, not to ship it. |
| Why "no egress allowlist configured"? | `allow` mode with nothing in it is a hard error. `--allow`, `.ai-sandbox/allow`, or `--net none`/`--net full` if you mean it. |
| Can it read `~/.ssh`? | No — that path is not in the mount namespace unless you mounted it. Threat model #1 is what you *do* mount and allow. |
| Can it rewrite my skills? | No. `~/.agents` is read-only and there is no flag; but its contents are *readable*, so treat the shared tree as published to the agent. |
| Why is my project at `/project`? | Mirroring refused because the host path would bury a path the image or sandbox owns; `workdir_note=` says which, `--workdir` chooses. |
| `npm ci` works but a package fails | That ecosystem needs another host (python: `files.pythonhosted.org`; rust: `static.crates.io`/`index.crates.io`; go: `sum.golang.org`). Or the package fetched from an allowlisted host — allowing a domain is not a security review. |
| How do I let it push? | You don't, from inside. `--allow github.com` + `--env GH_TOKEN` or `--mount ~/.ssh:ro` does it, and the threat model explains the price. For pipelines, run `ai-sandbox-autophase`, which pushes on the host. |
| How do I see what the gate allowed? | `ai-sandbox gate logs` (and `gate status`, `gate down`). Don't put secrets in URLs: the gate sees what flows through it. |
| A stale container home after a rebuild? | It is the `aisb-home` volume. `ai-sandbox clean --volumes[=GLOB]`, or `--ephemeral-home` per run. |
| How do I see the dev server the sandbox runs? | `--publish 8000` (config `publish = 8000`): host `127.0.0.1:8000` → container `8000`, warned if you name a non-loopback address. The server inside must bind `0.0.0.0` — a `127.0.0.1`-only server answers inside and refuses the published port (measured). `--net none` refuses to publish, because podman drops the mapping there and starts anyway. |
| Is a published port a security hole? | It is ingress you asked for: whatever that server serves is reachable from the address it is bound to, for as long as the run lasts. Loopback by default (podman's bare `-p` binds every interface), no ranges, the agent cannot open or widen one, egress enforcement is untouched. Threat model #14. |
| How does the sandbox reach a database on my machine? | That is egress, not `publish`: `allow = 100.64.0.1:<port>` (and a `NO_PROXY` entry when it is HTTP on an unusual port). `publish` points the other way. |
| How do I refresh the base? | `ai-sandbox update --check` then `ai-sandbox update` (or `build --refresh-base`). TTL re-resolves a digest, it never downloads on a preview. |
| Do `--dry-run`/`--print-*` touch the network? | No — that determinism is deliberate. `inspect` is the one that asks the machine now. |
| Is the container root? | It starts root with `CAP_NET_ADMIN` just long enough to install the ruleset, then `setpriv` to uid 1000 with no capabilities and `no-new-privileges`. `--insecure-keep-root` is the opposite. |
| What is NOT protected? | `docs/threat-model.md`, and answer with it: allowlisted destinations, kernel/runtime escapes, anything you mounted, the cached home volume, DNS in `--no-gate` mode, the fact that a host-side push pushes what the agent wrote. |
.agents/skills/doc-sandbox/scripts/locate.sh
Executable
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env bash
# locate.sh -- where the ground truth for an ai-sandbox question lives on THIS machine.
#
# Read-only, prints no secrets, starts nothing. Answers questions about ai-sandbox from the
# program's own text and its docs, never from memory -- so first find them.
#
# bash locate.sh
set -uo pipefail
section() { printf '\n== %s\n' "$1"; }
note() { printf ' %s\n' "$1"; }
found() { printf ' [x] %s\n' "$1"; }
warn() { printf ' [!] %s\n' "$1"; }
BIN="$(command -v ai-sandbox 2>/dev/null || true)"
[[ -n "$BIN" ]] || BIN="$(command -v ai-sandbox-autophase 2>/dev/null || true)"
section "the program (the authority on flags and behaviour)"
if [[ -n "$BIN" ]]; then
real="$(readlink -f "$BIN" 2>/dev/null || printf '%s' "$BIN")"
found "$real"
note "version: $("$BIN" version 2>/dev/null || printf 'unknown -- older than the version subcommand')"
note "flags for this exact build: $BIN --help"
else
warn "ai-sandbox is not on PATH -- answer from the repository instead, and say so"
fi
# A candidate is the script itself, or a repo holding it (bin/ai-sandbox with README.md beside).
SCRIPT=""
[[ -n "$BIN" ]] && SCRIPT="$(readlink -f "$BIN" 2>/dev/null || printf '%s' "$BIN")"
candidates=()
[[ -n "${AISB_REPO:-}" ]] && candidates+=("$AISB_REPO")
[[ -n "${SCRIPT:-}" ]] && candidates+=("$(dirname "$(dirname "$SCRIPT")")")
candidates+=("$PWD" "$(git rev-parse --show-toplevel 2>/dev/null || true)")
REPO=""
for c in ${candidates[@]+"${candidates[@]}"}; do
[[ -n "$c" && -d "$c" ]] || continue
if [[ -f "$c/bin/ai-sandbox" && -f "$c/docs/threat-model.md" ]]; then REPO="$c"; break; fi
done
section "the source tree (the authority on why)"
if [[ -n "$REPO" ]]; then
found "$REPO"
for f in README.md AGENTS.md docs/networking.md docs/threat-model.md docs/recipes.md bin/ai-sandbox bin/ai-sandbox-autophase; do
[[ -f "$REPO/$f" ]] && note "$(printf '%-32s %s lines' "$f" "$(wc -l < "$REPO/$f")")"
done
[[ -d "$REPO/tests" ]] && note "tests/ -- the executable spec: unit/ needs no containers, e2e/ runs real ones"
[[ -z "$BIN" ]] && note "not installed, but this tree runs: $REPO/bin/ai-sandbox --help"
note "if the installed script and this tree differ, say which one you read"
else
note "no repository found (tried \$AISB_REPO, the installed script's parent, \$PWD, the git root)"
note "the installed script alone answers most questions: it is one file, and it generates"
note "the Containerfile, the entrypoint and the squid config from itself"
fi
section "read the machine, or read the script"
note "safe, deterministic, no build / container / state / registry:"
note " ai-sandbox --help every flag, this version"
note " ai-sandbox version"
note " ai-sandbox --dry-run the exact podman invocation, secret values redacted"
note " ai-sandbox --print-policy effective policy as key=value -- but it also ensures the"
note " home volume exists, so it needs podman on the host"
note " ai-sandbox --print-entrypoint the enforcement script baked into the image"
note " ai-sandbox --print-containerfile the generated Containerfile"
note "asks the machine now (may reach the registry, starts nothing): ai-sandbox inspect"
note "touches the host: doctor [--deep], build, update, gate up, clean, and any real run"
section "state a run leaves behind"
note "config: ${AISB_CONFIG_DIR:-$HOME/.config/ai-sandbox}/{config,allow}"
note "state: ${AISB_STATE_DIR:-$HOME/.local/state/ai-sandbox}/log (every run, no secret values)"
note "cache: ${AISB_CACHE_DIR:-$HOME/.cache/ai-sandbox}/base/ (the recorded base digests)"
note "repo: ./.ai-sandbox/{config,allow}"
.agents/skills/right-sandbox/SKILL.md
+243
View File
@@ -0,0 +1,243 @@
---
name: right-sandbox
description: Analyzes a project directory and writes that project's ai-sandbox policy — .ai-sandbox/allow plus .ai-sandbox/config (egress allowlist, environment variable names, bind mounts, published ports so a local web server can be run and previewed from the host, working directory, resource limits, harness, extra image packages, access to services on the host) — so that a plain `ai-sandbox` run in the directory is the correct, reviewable invocation. Use when the user asks to set up, configure, write, fix, review or tighten ai-sandbox or the sandbox for a project, wants a .ai-sandbox config committed with the code, wants to run a dev server or expose a port from inside the sandbox, or asks why a sandboxed agent cannot reach a package registry, a database, a localhost port or its model gateway. Writes policy files only — never builds, never starts a container, never puts a credential value in a file.
---
# right-sandbox
Produce the two files ai-sandbox reads from a project, and nothing else:
| file | contents | committed |
|---|---|---|
| `.ai-sandbox/allow` | one egress target per line, `#` comments ok | yes — the policy gets reviewed with the code |
| `.ai-sandbox/config` | `key = value`: net, harness, limits, mounts, env names, image | yes |
| `.ai-sandbox/env` | optional `NAME=VALUE`, mode `600` | **never** — gitignore it |
`load_config()` reads `./.ai-sandbox/{config,allow}` (after `~/.config/ai-sandbox/{config,allow}`)
and nothing else, so do not invent `sandbox.yaml`, `local`, `env.json` or a `phases/` directory in
there — a file nobody reads is a policy nobody has.
Precedence for every key: built-in default < user config < repo config < `AISB_*` env < CLI flag.
The repeatable keys (`allow`, `env`, `mount`, `install`, `label`) **accumulate** across all five
sources rather than replacing, so a repo config adds to the user's, it does not silence it.
## What makes a config right
1. **Names, never values.** Both committed files end up in the repository, in `--print-policy` and
in a reviewer's diff. Credentials live in the environment or in `.ai-sandbox/env` (`chmod 600`,
gitignored); the config says `env = AI_API_TOKEN`. A literal `NAME=VALUE` is only for things
that are not secrets, e.g. `env = DATABASE_URL=postgresql://dev@100.64.0.1:5432/app`.
2. **Every key and flag must be real.** An unknown config key is warned about and ignored — a typo
is a silently missing policy, which is worse than an error. The authority is the installed
program: `ai-sandbox --help`, plus `cfg_read_kv()`'s key list in the script.
`references/settings-map.md` holds the list as of writing — check it, do not trust it.
Known trap: `project_dest` is accepted by the parser and read by nothing; the key is `workdir`.
3. **The git remote stays out.** No `github.com`, `gitlab.com`, `bitbucket.org` or ssh host in
`allow` unless the user asks after hearing the cost: an allowlisted remote plus a token or a
mounted `~/.ssh` lets a prompt-injected agent ship code nobody reviewed (threat model #1).
`git status`, `diff`, `log`, `blame` all work without it, and `ai-sandbox-autophase` commits and
pushes on the host precisely so that this line can stay out. If a dependency genuinely has to be
fetched over a git host, say which dependency in a comment above the line and prefer a
read-only, short-lived token.
4. **Escape hatches never go in a committed config**: `net = full`, `extra_args`,
`--allow-unenforced`, `--insecure-keep-root`. They are per-run decisions named to be
uncomfortable, and a repo file outlives the run that justified it. A run refusing to start
(exit 77) is the policy working — debug the host, do not route around it.
5. **Least mount.** The cwd is already mounted read-write at its host path, and `~/.agents` is
shared read-only with no flag to change it. Never propose a writable skills mount: a sandbox
that can write a `SKILL.md` has written an instruction the user follows later on the host.
Every extra `mount` line needs a one-line reason in the file.
6. **Verify before reporting success** (§5). "It should work" is not an acceptable result for a
security policy; `--print-policy` and `--dry-run` answer without building or starting anything.
## 1. Survey the project
```bash
bash scripts/survey.sh [project-dir] # default: $PWD; resolve scripts/ against this skill's directory
```
It is read-only and prints no secret values — variable *names*, remote *hosts*, registry *hosts*
and service *ports*. Read its output, then read the files it names: the lockfiles, the compose
file, `.env.example`, the dev-server config, CI, and the Makefile or task runner. Prefer evidence
over inference: a `pyproject.toml` with an `[[tool.uv.index]]` is a private index, an empty
`node_modules` tells you nothing.
Also read what already exists — `.ai-sandbox/config`, `.ai-sandbox/allow`, the user's
`~/.config/ai-sandbox/{config,allow}` — and extend it rather than replacing a decision the user
made deliberately.
## 2. Decide each setting from evidence
| need | evidence | setting |
|---|---|---|
| package installs | lockfiles (`pnpm-lock.yaml`, `uv.lock`, `Cargo.lock`, `go.sum`, `Gemfile.lock`, `pom.xml`…) | `allow` entries for the registries that ecosystem really contacts — see the table in `references/settings-map.md` |
| model gateway | `~/.pi/agent/models.json` `baseUrl` host, or the base URL the user's harness is configured with | `allow = <gateway host>`; without it a bridged pi starts and cannot answer |
| a database / queue / cache | `docker-compose*.yml` published ports, `.env` URLs, `DATABASE_URL` | `allow = 100.64.0.1:<published port>` + an env var that points the app at `100.64.0.1` — see `references/port-and-loopback.md` |
| a server the project runs, to be seen from the host (dev server, storybook, API docs, a preview the agent should be able to start *and* drive) | `scripts.dev` / `start` in `package.json`, `vite.config.*`, `manage.py runserver`, `uvicorn …`, a `Dockerfile` `EXPOSE`, the port in the project's own README | `publish = <port>` (loopback by default) — ingress, not an `allow` line. `references/port-and-loopback.md` has the bind-`0.0.0.0` requirement, which is what makes the difference between configured and working |
| a non-HTTP backend | pinned host:port (a LAN inference box, an internal API) | `allow = <ip or cidr>:<port>` (IP/CIDR become netfilter rules; domains go through the gate) |
| credential the harness needs | `.env.example`, CI `secrets.*`, `--env NAME` in the user's launcher | `env = NAME` (name only), and list what the user must export |
| reference material / sibling repo | imports or docs that read `../something` | `mount = ../something:/something:ro` |
| push over ssh | `~/.ssh` and an `ssh://` remote | do **not** add by default; rule 3 |
| a tool the run needs that the image lacks | build logs, `apt-get` in the project's Dockerfile, `rg`, `jq`, a DB client, `ffmpeg` | `install = <apt package>` (each distinct set gets its own cached image tag). Decide it **here**: inside, the harness is uid 1000 with no root and no package manager, so `apt-get` is not something it can do later |
| heavy build / flaky cap | repo size, test suite runtime, the user's machine | `cpus`, `memory`, `pids`, `tmpfs_size` |
| harness that is not pi | `.claude/`, `codex` config, the user's habit | `harness = pi \| codex \| claude \| none \| "<your command>"` |
| paths must look like the host's | a harness that titles by cwd; logs the user pastes | leave `workdir = auto`; set it only when the host path buries an image path |
| project-specific skills/prompts | a project `.agents/` | nothing — it arrives with the project mount |
| untrusted-code review | the user says so | `net = none`, `agents = 0`, `home_cache = 0`, `read-only` on the CLI, not in the file |
Two decisions the tool already makes that you should not repeat in the file: the thinking level is
`xhigh` policy for a bridged pi, and the pi config bridge is on with the credential by environment
reference. Only write `thinking` or `pi_token_env` when the project genuinely differs, with a
comment saying why.
## 3. Write the files
`.ai-sandbox/allow` — one target per line, grouped and commented, because this file is the diff a
human reviews:
```
# model gateway -- without it the sandboxed harness cannot answer
gateway.example.com
# dependencies
registry.npmjs.org
nodejs.org # node-gyp / prebuild downloads
# services on this machine, reached through the mapped loopback address (§ports)
100.64.0.1:5432 # postgres, `docker compose up db`
```
`.ai-sandbox/config` — only the keys this project needs; a comment per non-obvious one:
```ini
# ai-sandbox policy for <project>. Committed on purpose: review it like code.
# Values belong in .ai-sandbox/env (0600, gitignored) or the environment, never here.
net = allow
harness = pi
# the toolchain needs a bit more than the default 4 cpu / 4g
cpus = 8
memory = 8g
# design docs live beside the repo
mount = ../specs:/specs:ro
# names only; ai-sandbox passes them by name through a 0600 file
env = AI_API_TOKEN
env = @./.ai-sandbox/env
# apt packages the test suite needs that the image lacks
install = postgresql-client
# ingress: the dev server this project runs, previewable on the host at
# http://127.0.0.1:8000 (loopback only; the server binds 0.0.0.0 inside)
publish = 8000
```
`env = @path` is the config-file form of `--env-file` (`--env-file F` becomes the entry `@F`), so
it works today and fails closed if the file is missing — note in the file that a newcomer must
`cp .ai-sandbox/env.example .ai-sandbox/env && chmod 600` it. If you create `env.example`, put
names with dummy values in it and add `.ai-sandbox/env` to `.gitignore` (never overwrite an
existing ignore file's rules — append).
Never write `.ai-sandbox/env` yourself with real values: write the example, then tell the user what
to put in it.
## 4. Egress in one paragraph (get the shape right)
* A **domain** entry is matched by the gate at CONNECT time, including subdomains. Its `:port` does
not add a netfilter rule and the gate only relays **80, 443, 8443** (`Safe_ports`/`SSL_ports` in
the generated squid config), so `--allow host:8080` will not do what it looks like.
* An **IP, CIDR or `[ipv6]`** entry becomes a direct netfilter rule, default ports 80,443 unless you
write `:port` — this is how non-HTTP traffic (postgres, redis, a LAN model box) is allowed.
* `--no-gate` skips the proxy: domains are resolved once, on the host, at start and **DNS is
opened**; weaker for CDNs and rotating IPs. `--proxy host:port` uses a proxy you already run
(default port 3128). Neither belongs in a committed config without a comment justifying it.
* An empty allowlist in `allow` mode is a hard error. Never "fix" that by widening net.
* `publish` is the one **ingress** key and it is not an allow entry: `allow = 100.64.0.1:5432` lets
the sandbox reach your machine, `publish = 5432:5432` lets your machine reach the sandbox. Writing
the wrong one of these is how a service ends up open on the wrong side.
## 5. Verify (no builds, no containers, no network)
```bash
cd <project>
ai-sandbox --dry-run # the exact podman invocation + the # egress / # env header lines;
# works with no podman and leaves no session directory behind
ai-sandbox --print-policy # net=, allow=, gate_domains=, direct=, env_names=, env_missing=,
# mounts=, workdir=, agents=, thinking= -- as key=value lines
```
Start with `--dry-run`: it is the one preview path with no side effects at all. `--print-policy`
gives the friendlier key=value form but also ensures the home-cache volume exists, so it needs
podman present — treat a failure there as "this machine has no podman", not as a policy problem.
Check, in this order, and fix the files instead of explaining the output:
1. no `warning: --env NAME requested but not set` for a name nobody intended (unset names are fine
to ship — say so in the report — but each must be a deliberate line, not a typo).
2. every allow entry appears in `domains=[…]` or `direct=[…]` (or `gate_domains=`/`direct=` under
`--print-policy`) — an entry in neither is a parse problem.
3. `workdir=` is the host path, not `/project`, unless the fallback was intended (`workdir_note=`
says why).
4. `mounts=` and `agents=` are what you wrote and nothing more.
5. the dry run shows `--network pasta:--map-host-loopback,100.64.0.1`, `--cap-drop=ALL`,
`--security-opt no-new-privileges`, `--userns keep-id:…` and no secret values.
6. if you wrote `publish`: the `# ingress: publish=[…]` line shows `127.0.0.1` on every entry unless
the user asked for an address, and the project's dev command binds `0.0.0.0` — a published port
pointing at a `127.0.0.1`-only server is configured, silent and useless.
7. `--print-policy | grep -i` for each credential name from §1: the *name* may appear, a *value*
must not.
Then, with the user watching (this builds the image the first time and starts a container):
```bash
ai-sandbox --shell # egress, mounts, env present
ai-sandbox --net none -- make test # the offline half of the toolchain
ai-sandbox doctor # host readiness, incl. the ports it would publish
```
Probe the two or three endpoints the project actually needs from inside (`curl -sS -m 5 -o
/dev/null -w '%{http_code}\n' https://registry.npmjs.org/`) rather than assuming an allowlist entry
means a working toolchain. If you wrote `publish`, probe the other direction too, from the host,
while a sandbox is still running (`references/port-and-loopback.md` §2 has the two-shell probe): a
published port nobody answered during the review is not a verified setting.
## 6. Report
Finish with exactly these five points, so the user can act without reading the files:
1. **Files written** — paths, plus the `.gitignore` change if any.
2. **`cd <dir> && ai-sandbox`** is now the invocation (and the alias note if they use one).
3. **What they must export** first (`env =` names that are unset), and where values may go.
4. **What is deliberately not allowed, and what is open** — the git remote first, then anything you
declined to open because the honest answer was "not without weakening the sandbox". If you wrote
`publish`, name each port, its host address and what listens there: an open port belongs in the
report even though it was requested.
5. **What you verified and how** — the two preview commands and any live probe, with results.
## Ports, localhost, "the app must be reachable"
Read `references/port-and-loopback.md` before promising anything about ports. Two directions, two
settings, and the wrong one is how a service ends up open on the wrong side:
* **the sandbox → a service on your machine**: `allow = 100.64.0.1:<port>` (the mapped loopback
address), plus a client that does not send that request to the egress proxy.
* **your machine → the sandbox**: `publish = <port>`, bound to `127.0.0.1` unless the spec names an
address. It is ingress: egress enforcement is untouched by it, and the agent inside cannot open or
widen a port of its own. The server has to bind `0.0.0.0` *inside* — that is the difference between
a configured preview and a working one, and it is measured, not folklore.
Verify on the machine rather than repeating any of this from memory: `--dry-run` shows
`# ingress: publish=[…]`, and `tests/e2e/13_publish.sh` is what the claims rest on.
## References
* `references/settings-map.md` — every accepted config key and flag, defaults, per-ecosystem
allowlists, the survey→policy cheat sheet.
* `references/port-and-loopback.md` — inbound vs outbound, `100.64.0.1`, the gate's port limits,
probes, the three workable dev-server patterns.
* `scripts/survey.sh` — the read-only evidence gatherer (prints no values).
* The tool's own docs, if this repo is present: `README.md`, `docs/networking.md` (measurements),
`docs/threat-model.md` (what is *not* protected), `docs/recipes.md` (invocations).
.agents/skills/right-sandbox/references/port-and-loopback.md
+210
View File
@@ -0,0 +1,210 @@
# Ports and loopback
Two separate problems wear the same name. Get the direction right before touching a config.
| | what it is | how | direction |
|---|---|---|---|
| **outbound** | the sandboxed process reaches a service running on your machine (dev postgres, redis, a mock API, a local model server) | `allow = 100.64.0.1:<port>` — an ordinary allow entry | egress |
| **inbound** | your browser on the host reaches a server the sandbox started | `publish = <port>` / `--publish` | ingress |
They are not the same setting and one does not imply the other. `allow = 100.64.0.1:5432` does not
open a port; `publish = 5432:5432` does not allow egress. Getting these backwards is how a database
ends up reachable from the wrong side.
Container-to-container is a third case, already measured on the reference host: two containers on a
shared user-defined network **cannot reach each other**, so do not design a sidecar proxy, a
service-mesh sandbox or a "sandbox plus its own database container" topology. The database runs on
the host (compose) and is reached as in §1.
## 1. Outbound: a service on your machine
With `--net allow` the container is started as
`--network pasta:--map-host-loopback,<gate_addr>` (`gate_addr`, default `100.64.0.1`). Inside the
container, **`100.64.0.1:PORT` lands on the host's `127.0.0.1:PORT`**. `127.0.0.1` inside the
container is the container itself and almost never what you meant.
Two layers have to agree, and this is where every "it still does not work" comes from:
1. **The ruleset.** Only the gate's port is routable through that address by default. Add:
```
allow = 100.64.0.1:5432 # postgres published by docker compose on the host
```
(IP entries become direct netfilter rules — the right shape for a non-HTTP protocol.)
2. **The proxy variables.** When a gate is in use, the container gets `HTTP(S)_PROXY`/`ALL_PROXY`
and `NO_PROXY=localhost,127.0.0.1,::1`. A client that honours them sends the request to the gate,
which allows the destination if it is in the allowlist **and the port is one of 80, 443, 8443**.
So:
| the service | what the config needs |
|---|---|
| postgres, redis, amqp, ssh, a raw socket — anything that ignores proxy variables | `allow = 100.64.0.1:<port>` |
| HTTP on 80, 443 or 8443 | `allow = 100.64.0.1:<port>` (the gate relays it) |
| HTTP on any other port (3000, 8000, 5173, 9200…) | `allow = 100.64.0.1:<port>` **and** a proxy bypass for that address, and measure which of the two `NO_PROXY` values wins |
The bypass, and the honest way to confirm it:
```ini
# .ai-sandbox/config -- the dev API on the host is not reachable through the egress gate
env = NO_PROXY=100.64.0.1,localhost,127.0.0.1,::1
env = no_proxy=100.64.0.1,localhost,127.0.0.1,::1
```
ai-sandbox sets `NO_PROXY` with `--env` *before* the env file, so the interaction between that and
an env-file value is podman's flag handling, not ai-sandbox's. Do not reason about it: run
`ai-sandbox --dry-run`, look at the order of the flags, and probe for real:
```bash
ai-sandbox --allow 100.64.0.1:3000 --shell
curl -sS -m 3 -o /dev/null -w '%{http_code}\n' http://100.64.0.1:3000/health
env | grep -i -E 'proxy|no_proxy'
```
Then point the app at the mapped address without putting a secret in a committed file — a URL with
no password in it is not a secret:
```ini
env = DATABASE_URL=postgresql://dev@100.64.0.1:5432/app_dev
```
If `100.64.0.1` collides with a real network the project must reach, change `gate_addr` (and
remember the gate itself is reached at that address too).
## 2. Inbound: publishing a port the sandbox listens on
`--publish` (config key `publish`, repeatable) forwards a host port into the container, so a client
on the host — a browser, curl, an API tester — reaches a server the sandboxed process runs. It is
the only ingress setting in the tool, and the ruleset names the ports it opened
(`nft list table inet aisb` → the `aisb-publish` rule).
Shapes, and what they bind. **The default address is `127.0.0.1`**, because a bare podman `-p` binds
every interface (measured) and a port in a committed config must not quietly be a port on the LAN:
| spec | host side | container side |
|---|---|---|
| `8000` | `127.0.0.1:8000` | `8000` |
| `18080:8000` | `127.0.0.1:18080` | `8000` |
| `0.0.0.0:18080:8000` | every interface — **warns on every run** | `8000` |
| `[::1]:18080:8000` | ipv6 loopback | `8000` |
Rules the parser enforces, and the reasons to repeat them to the user:
* **One port per entry.** No ranges, no `/udp`: this list is reviewed line by line, and a range is a
line nobody reads.
* **An address means three parts.** `127.0.0.1:8000` is refused rather than guessed at.
* **`--net none` refuses to publish.** Measured: podman discards port mappings there with one line on
its own stderr and starts the container anyway, so "it ran" would mean "nothing is listening".
* **A busy host port is named before the container starts**, not after a build. Suggest
`publish = <other>:<container>` rather than telling the user to go find `ss` output.
* **No `-p` flag.** `-p` passes through to the harness (`claude -p` prints), which is deliberate.
* **No `*`.** podman's shorthand for "every interface" is refused; write `0.0.0.0`. Typing the address
is the warning.
### The one thing that makes a published port look broken
**The server has to bind `0.0.0.0` inside the container.** Measured: a server bound to
`127.0.0.1` inside answers `200` from inside and refuses the published port from the host, because
pasta delivers the connection to the container's own address, not to its loopback. So the config is
not enough — the project's dev command has to cooperate:
| stack | what the preview command needs |
|---|---|
| vite | `npm run dev -- --host 0.0.0.0` |
| Next / webpack dev servers | `--hostname 0.0.0.0` |
| Django | `runserver 0.0.0.0:8000` |
| Flask | `flask --app … run --host 0.0.0.0 -p 8000` |
| uvicorn / FastAPI | `uvicorn --host 0.0.0.0 --port 8000 …` |
| a bare static server | `python3 -m http.server 8000 --bind 0.0.0.0` |
A dev-server config file that pins `host: 127.0.0.1` (`vite.config.*`, `webpack-dev-server`,
`gunicorn -b 127.0.0.1:8000`) needs that changed, or the preview command overrides it on the command
line. Say which, or the user will conclude the sandbox is broken.
### What publishing does not do
* **It does not weaken egress.** The same run answers the host and still refuses every domain
outside the allowlist; the entrypoint still drops all capabilities before the command runs.
`tests/e2e/13_publish.sh` asserts both halves in one container, so this is not a claim to re-derive.
* **It does not let the agent open more ports.** The mapping is decided on the host before the
container starts; inside, the command has no `CAP_NET_ADMIN` and cannot add or widen one.
* **It does not survive the run.** The listener is pasta's, removed with the container. If a port is
still bound afterwards, something was started with `--keep` — `ai-sandbox ps`, `ai-sandbox clean`.
### What to write, and what not to
Loopback publishing is a legitimate project setting: it says "this project runs a server on this
port, and previewing it is part of working on it".
```ini
# .ai-sandbox/config
publish = 8000 # the app: http://127.0.0.1:8000
publish = 18080:3000 # the storybook, on a host port that is free
```
Comment each one with what runs there — a port without a name is a port a later reader cannot
remove. Do **not** commit:
* a **non-loopback** address (`0.0.0.0:…`, `192.168.x.x:…`). It warns on every run because it means
what it says: anything that can route to the machine can reach whatever the sandbox serves. If the
user genuinely needs a phone on their WiFi to reach the preview, that is a per-run flag, said out
loud, not a line in a file that outlives the run.
* `publish_addr = 0.0.0.0` in a repo config. It changes the default for every bare `publish` line in
that directory, which is the same mistake wearing a different hat. It belongs in
`~/.config/ai-sandbox/config` if anywhere.
* a port for a service the project does not actually run, "in case".
### Verify it, in this order
```bash
ai-sandbox --dry-run # '# ingress: publish=[127.0.0.1:8000:8000]' and
# '--publish 127.0.0.1:8000:8000' in the invocation
ai-sandbox --print-policy | grep '^publish=' # canonical addr:host:container, loopback unless you
# wrote an address
```
Then the only check that answers the real question, with the user watching (a real container; the
port exists only while it runs):
```bash
ai-sandbox --publish 8000 --shell
# inside: python3 -m http.server 8000 --bind 0.0.0.0 >/tmp/s.log 2>&1 &
# inside: curl -sS -o /dev/null -m2 -w 'inside=%{http_code}\n' http://127.0.0.1:8000/
# host: curl -sS -o /dev/null -m3 -w 'host=%{http_code}\n' http://127.0.0.1:8000/
```
`host=200` is the result. `host=000` with `inside=200` is the bind address, not the sandbox — go back
to the table above. A refusal naming the port means something else holds it. `--print-entrypoint |
grep -A6 'chain input'` shows the accept rule shape; on the reference host pasta forwards over
loopback, so that rule's packet counter stays at 0 while the port serves — the rule is the audit
trail and the portability guarantee, not what carries the traffic there.
### When the honest answer is still "don't publish"
1. **Run the server on the host.** The project mount *is* their disk: the agent edits, the host's
`npm run dev` / `runserver` picks it up, the browser hits a normal host port. Nothing is published
and nothing is opened. Still the right answer when the agent does not need to drive the server —
and that is most "preview the app" requests.
2. **Publish it**, when the server itself has to be the sandboxed process: the agent seeds a
database it starts, runs a migration then serves, needs to watch the server's own logs, or the
test *is* hitting the endpoint. Loopback, named in `.ai-sandbox/config`, with the bind address
fixed in the same change.
3. **Split the loop**: the agent builds and unit-tests in the sandbox, writes a static export or a
`make preview` target, and the human runs that. Keeps the review step where it belongs.
If the ask is "publish it on the LAN so I can check it on my phone", that is a per-run
`--publish 0.0.0.0:18080:8000` with the warning read out, never the committed config — and if the
user wants it permanently, say plainly that the sandbox is then a server on their network.
## 3. Notes that keep coming up
* The gate binds the **host's loopback** only, on a free port at or above `gate_port` (18081);
`ai-sandbox gate status` and `gate logs` show a long-lived one. `pick_port()` skips ports this run
publishes, so a `publish = 18081` does not lose the run.
* `NO_PROXY` covering `localhost` is correct and must stay — tooling that refuses to proxy the
container's own loopback (dev databases, auth servers, vite HMR) breaks otherwise.
* `--no-gate` opens DNS inside the container; the mapped loopback address still needs its `allow`
line.
* `--net none` has no interface at all: neither direction works, which makes it the right mode for
`make test` on a toolchain that is already installed — and the reason it refuses `--publish`.
* `ai-sandbox doctor` reports the resolved `publish` lines and warns about a non-loopback bind or a
host port already in use. It never binds anything itself.
.agents/skills/right-sandbox/references/settings-map.md
+143
View File
@@ -0,0 +1,143 @@
# Settings map
Check this against the installed program before writing a config: `ai-sandbox --help`, and
`cfg_read_kv()` / `resolve_opts()` in the script. Where they disagree, the script wins and this
file is the bug.
## Precedence
```
built-in default < ~/.config/ai-sandbox/{config,allow} < ./.ai-sandbox/{config,allow}
< AISB_* environment < command-line flag
```
`allow`, `env`, `mount`, `install`, `label`, `publish` **accumulate** from every source (deduped);
every other key replaces. So a repo file cannot revoke a user-wide `allow` line — say so if the user
expects a repo config to tighten their own machine. It also cannot *un*publish a port the user's
own config opens.
## The container, for orientation
Anything named "container" below is inside the sandbox: uid/gid **1000**, the image's own `sandbox`
account, its home the value of `HOME` in there (`--userns=keep-id` makes files the agent writes
owned by *you* on the host). So `~/.agents` exists on both sides of the wall and they are different
directories — the host one is bind-mounted read-only onto the container one at `agents_dest`.
`/project` is the image's own directory and the fallback working directory, not the normal one.
## Config keys (`key = value`, `#` comments)
| key | default | notes |
|---|---|---|
| `net` | `allow` | `none` \| `allow` \| `full`. `allow` requires a non-empty allowlist. |
| `harness` | `pi` | `pi` \| `codex` \| `claude` \| `none` \| a command line, e.g. `pi --model turbo` |
| `workdir` | `auto` | `auto` mirrors the host path; or an absolute path. **`project_dest` is parsed and never read — use `workdir`.** |
| `cpus` | `4` | |
| `memory` | `4g` | `512m`/`4g` shape |
| `pids` | `512` | |
| `tmpfs_size` | `1g` | sizes `/tmp` and, with `home_cache = 0`, the container home |
| `base_image` | `docker.io/library/ubuntu:24.04` | may itself be `name@sha256:…` |
| `node_version` | `22.23.4` | checksum-verified against nodejs.org |
| `pi_version` | `1.4.0` | npm package version baked into the image |
| `base_ttl_days` | `7` | `0` re-resolve every build, `-1` never |
| `base_auto_update` | `1` | `0` keeps the recorded digest offline |
| `selinux` | `:z` when `getenforce` exists | `:z`, `:Z` or empty |
| `home_cache` | `1` | `0` = throwaway tmpfs home (also `--ephemeral-home`) |
| `gate_addr` | `100.64.0.1` | the address that *means* the host's loopback inside the container |
| `gate_port` | `18081` | preferred first port; next free is used |
| `publish_addr` | `127.0.0.1` | the host address a bare `publish = 8000` binds. Machine-wide setting: put it in the user config, never in a repo file, and do not widen it for the user's convenience |
| `extra_args` | empty | appended verbatim to `podman run`, word-split. Escape hatch: do not commit it |
| `pi_config` | `1` | bridge `~/.pi/agent/{models,settings}.json` with the credential as `$VAR` |
| `pi_token_env` | `AI_API_TOKEN` | the variable the bridged config reads |
| `pi_token_auto` | `1` | pass that variable automatically when set |
| `pi_agent_dir` | `~/.pi/agent` | where to read the host config from |
| `thinking` | `xhigh` | `off minimal low medium high xhigh max host`; pi only, re-applied every run |
| `agents` | `1` | share `~/.agents` read-only when the directory exists |
| `agents_dir` | `~/.agents` | which directory; a project's own `.agents/` needs nothing |
| `agents_dest` | the container's `~/.agents` | keep it under the container `$HOME` — that is the only place a harness looks for user-level skills |
| `allow` | — | repeatable; same grammar as `--allow` |
| `env` | — | repeatable; `NAME`, `NAME=VALUE` (no secret in a committed file), or `@path` = an env file |
| `mount` | — | repeatable; `SRC[:DEST][:ro\|rw]`, default `DEST=/mnt/<basename>` |
| `install` | — | repeatable; apt packages, each distinct set is its own image tag |
| `label` | — | repeatable; `K=V` on the container |
| `publish` | — | repeatable; **ingress**: `[ADDR:]HOST:CONTAINER` forwarded into the container. One port per entry, no ranges, no `/udp`; an address needs three parts. `8000`, `18080:3000`, `0.0.0.0:18080:3000` (LAN-visible: warned, do not commit) |
Anything else warns (`unknown key`) and is ignored.
## Flags
```
egress --net none|allow|full --allow HOST[:PORT] --allow-file F --proxy HOST:PORT --no-gate
ingress --publish [ADDR:]HOST:CONTAINER (no `-p`: that flag belongs to the harness)
files --mount SRC[:DEST][:ro] --no-cwd --workdir DIR|auto --read-only
--home-cache --ephemeral-home --agents | --no-agents --agents-dir PATH
env --env NAME --env NAME=VALUE --env-file F (F should be 0600)
image --base-image REF --install PKG --no-build
harness --harness NAME --shell --thinking LEVEL --pi-token-env NAME --no-pi-token-env
--no-pi-config
limits --cpus N --memory SIZE --pids N --tmpfs-size SIZE
run --name NAME --label K=V --keep --interactive | --no-interactive --no-tty
preview --dry-run|-n --print-policy --print-entrypoint --print-containerfile
hatches --allow-unenforced --insecure-keep-root --strict-args
```
Parsing stops at the first positional or unknown argument, which becomes an argument **to the
harness** — that is what makes `alias pi=ai-sandbox` behave like the harness. `--` replaces the
command entirely. `--strict-args` makes an unknown flag an error instead (use it in scripts).
Subcommands: `build [--rebuild] [--gate] [--refresh-base]`, `update [--check]`, `doctor [--deep]`,
`gate up|down|status|logs`, `ps`, `clean [--all] [--volumes[=GLOB]]`, `inspect`, `version`.
Exit status: the command's own code; **77** the egress ruleset could not be installed; **78** the
policy file was missing in the container. Both refusals are the sandbox working.
## Allow-entry grammar
| entry | effect |
|---|---|
| `example.com` | gate matches that name and anything below it, at CONNECT time |
| `example.com:8443` | same; the port does **not** create a netfilter rule, and the gate only relays 80/443/8443 |
| `10.0.0.5:5432` | direct netfilter rule for that host and port — the way to allow non-HTTP |
| `10.20.0.0/16` | direct rule for the subnet, default ports 80,443 unless `:port` |
| `[fd00::5]:8443` | direct ipv6 rule |
| `localhost` | resolves on the host → a direct rule for `127.0.0.1`, i.e. the *container's* loopback. For your machine's services use `100.64.0.1` |
## Per-ecosystem starting points
Add only what the project's own lockfiles and build files prove it contacts. Everything left out
fails loudly, which is the feature.
| stack | entries |
|---|---|
| npm | `registry.npmjs.org` (+ `nodejs.org` and, for prebuilds, `release-assets.githubusercontent.com` — read the git-host warning before adding anything from `github.com`) |
| python | `pypi.org`, `files.pythonhosted.org` (the file CDN is separate; both are needed). A private index: whatever host `--index-url`, `PIP_INDEX_URL`, `[[tool.uv.index]]` or `tool.poetry.source` names |
| rust | `crates.io`, `static.crates.io`, `index.crates.io` (+ `static.rust-lang.org` if it installs toolchains) |
| go | `proxy.golang.org`, `sum.golang.org` (+ the host of a private module path, `allow = <host>` and `GOPRIVATE` in the env) |
| ruby | `rubygems.org`, `index.rubygems.org` |
| php | `repo.packagist.org`, `api.github.com` (composer metadata; check before allowing) |
| jvm | `repo1.maven.org`, `repo.maven.apache.org`, `plugins.gradle.org`, `services.gradle.org`, `dl.google.com` |
| dotnet | `api.nuget.org` |
| terraform | `registry.terraform.io`, `releases.hashicorp.com` |
| playwright / puppeteer | their download hosts (`cdn.playwright.dev`, `storage.googleapis.com`) — or commit the browsers and go `--net none` |
| container builds inside the sandbox | `registry-1.docker.io`, `auth.docker.io`, `production.cloudflare.docker.com` |
## Gotchas worth a comment in the file
* `--env NAME=VALUE` shows up in `ps`; the name-only and `--env-file` forms do not.
* An unset variable named with `--env` **fails the run** (in preview it is reported as
`env_missing=` instead) — that fail-closed behaviour is why preview output is worth reading.
* The image's home is a named volume (`aisb-home`) that outlives the image: after a rebuild the
home may be stale. `ai-sandbox clean --volumes` reclaims unused `aisb-*` volumes.
* A project under `/tmp`, `/usr`, `/etc`, `/var`, `/opt`, `/run`, `/home` itself or the container's
own home cannot mirror its host path and lands in `/project`, with a line saying so. Do not "fix"
that with `--workdir /tmp/...`; the tmpfs covers it.
* The host path may not contain `:` at all — podman cannot parse such a mount spec, and
ai-sandbox refuses rather than passing the error along.
* `publish` is the only **ingress** setting there is, and it is not an allowlist entry. `allow =
100.64.0.1:5432` lets the sandbox reach a service on your machine; `publish = 5432:5432` lets your
machine reach a port inside the container. Confusing the two is how a database ends up exposed.
Read `references/port-and-loopback.md` before writing either.
* Nothing is inherited but `TERM LANG LC_ALL LC_CTYPE TZ` (plus `HOME`, set to the container home,
`AISB=1`, `AISB_SESSION`). A tool that needs `PATH`, a dotfile or anything else from your real home
needs an explicit grant.
* `.env` files inside the project are readable by the agent — that is the point of the mount, and
it is why the *sandbox* token should be a narrow one.
.agents/skills/right-sandbox/scripts/survey.sh
Executable
+263
View File
@@ -0,0 +1,263 @@
#!/usr/bin/env bash
# survey.sh -- gather the evidence a .ai-sandbox policy needs from a project directory.
#
# Read-only, and it prints no secret values: variable NAMES, registry HOSTS, service PORTS. A URL
# is reduced to its host and any credentials inside one are stripped before anything is printed --
# the rule ai-sandbox holds itself to in --dry-run, --print-policy and the session log.
#
# bash survey.sh [project-dir] # default: $PWD
#
# It reports what it found and proposes nothing on its own authority: the agent decides, writes the
# two files, and verifies them with --print-policy / --dry-run.
set -uo pipefail
DIR="${1:-$PWD}"
[[ -d "$DIR" ]] || { printf 'not a directory: %s\n' "$DIR" >&2; exit 2; }
cd "$DIR" || exit 2
section() { printf '\n== %s\n' "$1"; }
note() { printf ' %s\n' "$1"; }
found() { printf ' [x] %s\n' "$1"; }
warn() { printf ' [!] %s\n' "$1"; }
list() { while IFS= read -r l; do [[ -n "$l" ]] && printf ' %s\n' "$l"; done; }
# Text in, sorted unique hosts out, with URL credentials gone. Never emits a path or a value.
hosts_of() {
sed -E -e 's#(://)[^/@[:space:]"'"'"']*@#\1#g' -e 's#^[a-z][a-z0-9+.-]*://##' -e 's#^//##' -e 's#^[A-Za-z0-9._-]+@##' \
| grep -oE '^[A-Za-z0-9._*+-]+(:[0-9]{1,5})?' | sort -u || true
}
# Existing files by basename glob, shallow, skipping build and dependency noise.
files_matching() {
local g
for g in "$@"; do
find . -maxdepth 3 \( -name node_modules -o -name .git -o -name .venv -o -name venv \
-o -name target -o -name dist -o -name build -o -name __pycache__ -o -name vendor \) -prune \
-o -name "$g" -print 2>/dev/null
done
}
printf 'project survey -- %s\n' "$DIR"
note "reads only, writes nothing, prints no secret values"
# --------------------------------------------------------------------------- the tool
section "ai-sandbox on this machine"
if command -v ai-sandbox >/dev/null 2>&1; then
found "$(ai-sandbox version 2>/dev/null || command -v ai-sandbox)"
note "script: $(readlink -f "$(command -v ai-sandbox)")"
note "policy: $PWD/.ai-sandbox/{config,allow} user: ${AISB_CONFIG_DIR:-$HOME/.config/ai-sandbox}/{config,allow}"
note "the flag and key list for THIS version: ai-sandbox --help"
else
warn "ai-sandbox is not on PATH -- install it before writing a policy nobody can run"
fi
# --------------------------------------------------------------------------- repository
section "repository"
remote_hosts=""
if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
note "branch $(git branch --show-current 2>/dev/null || echo '?'), $(git status --porcelain 2>/dev/null | wc -l) dirty path(s)"
remote_hosts="$(git remote -v 2>/dev/null | awk '{print $2}' | hosts_of | cut -d: -f1 | sort -u)"
if [[ -n "$remote_hosts" ]]; then
printf ' remotes (hosts only):%s\n' "$(printf '%s' "$remote_hosts" | tr '\n' ' ')"
note "allowlisting a remote host hands the sandbox the ability to ship what it wrote"
fi
else
note "not a git repository"
fi
# --------------------------------------------------------------------------- stacks
section "toolchains (manifest -> the endpoints that ecosystem needs)"
declare -A STACK=(
[package-lock.json]="npm | registry.npmjs.org"
[pnpm-lock.yaml]="pnpm | registry.npmjs.org"
[yarn.lock]="yarn | registry.npmjs.org"
[pyproject.toml]="python | pypi.org files.pythonhosted.org"
[requirements.txt]="python | pypi.org files.pythonhosted.org"
[poetry.lock]="python/poetry | pypi.org files.pythonhosted.org"
[uv.lock]="python/uv | pypi.org files.pythonhosted.org"
[Pipfile.lock]="python/pipenv | pypi.org files.pythonhosted.org"
[Cargo.lock]="rust | crates.io static.crates.io index.crates.io"
[go.sum]="go | proxy.golang.org sum.golang.org"
[Gemfile.lock]="ruby | rubygems.org index.rubygems.org"
[composer.json]="php | repo.packagist.org"
[pom.xml]="maven | repo1.maven.org"
[build.gradle]="gradle | repo.maven.apache.org plugins.gradle.org services.gradle.org"
[build.gradle.kts]="gradle | repo.maven.apache.org plugins.gradle.org services.gradle.org"
[mix.exs]="elixir | hex.pm repo.hex.pm"
[deno.json]="deno | deno.land registry.npmjs.org"
[*.csproj]="dotnet | api.nuget.org"
[*.tf]="terraform | registry.terraform.io releases.hashicorp.com"
)
stacks=0
for f in "${!STACK[@]}"; do
hits="$(files_matching "$f")"
[[ -n "$hits" ]] || continue
found "$(printf '%-26s %s' "${STACK[$f]}" "$(printf '%s\n' "$hits" | head -2 | tr '\n' ' ')")"
stacks=$((stacks + 1))
done
(( stacks )) || note "no recognised manifest within depth 3 -- script tree, docs tree, or an unusual layout"
note "also read: $(files_matching Makefile Taskfile.yml Dockerfile 'Dockerfile.*' Justfile package.json pyproject.toml | head -6 | tr '\n' ' ')"
section "custom registries named in the project's own files"
idx="$(grep -rIhoE --include='*.toml' --include='*.txt' --include='*.cfg' --include='*.json' \
--include='*.yaml' --include='*.yml' --include='*.npmrc' --include='*.pip.conf' --include='Makefile' \
'(--index-url|--extra-index-url|--add-registry|index-url|GOPROXY|npmRegistryServer)[[:space:]]*[=:]?[[:space:]]*"?[^[:space:]#",]+' \
. 2>/dev/null \
| sed -E 's/^(-{1,2})?[A-Za-z-]*(index-url|add-registry|GOPROXY|npmRegistryServer)[[:space:]]*[=:]?[[:space:]]*"?//' \
| hosts_of)"
if [[ -n "$idx" ]]; then list <<< "$idx"; note "these decide whether the build works at all"; else note "none found"; fi
# --------------------------------------------------------------------------- services and ports
section "services and ports"
compose="$(files_matching 'docker-compose*.yml' 'docker-compose*.yaml' compose.yml compose.yaml)"
if [[ -n "$compose" ]]; then
for c in $compose; do
note "${c#./}:"
svc="$(grep -oE '^ [a-zA-Z0-9_.-]+:[[:space:]]*$' "$c" | tr -d ' :' | tr '\n' ' ')"
[[ -n "$svc" ]] && printf ' services: %s\n' "$svc"
ports="$(grep -oE '[0-9]{1,5}:[0-9]{1,5}' "$c" | sort -u | tr '\n' ' ')"
[[ -n "$ports" ]] && printf ' published: %s -> one "allow = 100.64.0.1:<port>" per port you grant\n' "$ports"
done
else
note "no compose file"
fi
devports="$(grep -rIhoE --include='vite.config.*' --include='next.config.*' --include='webpack*.js' \
--include='package.json' --include='*.env' --include='*.env.*' --include='settings.py' \
--include='Procfile' --include='manage.py' \
'(--[Pp][Oo][Rr][Tt][[:space:]=]+[0-9]{4,5})|([Pp][Oo][Rr][Tt][[:space:]]*[=:][[:space:]]*"?[0-9]{4,5})' . 2>/dev/null \
| grep -oE '[0-9]{4,5}$' | sort -u | tr '\n' ' ')"
[[ -n "$devports" ]] && note "ports named in dev-server configs: $devports"
exposed="$(files_matching Dockerfile 'Dockerfile.*' | xargs -r grep -hoiE '^EXPOSE[[:space:]]+[0-9]+' 2>/dev/null \
| grep -oE '[0-9]+' | sort -u | tr '\n' ' ')"
[[ -n "$exposed" ]] && note "EXPOSEd in the Dockerfile: $exposed"
# The address matters as much as the port: a dev server pinned to loopback answers nothing on a
# published port, so quote the project's own setting back when it says 127.0.0.1.
binds="$(grep -rIhoE --include='vite.config.*' --include='next.config.*' --include='webpack*.js' \
--include='gunicorn*' --include='Procfile' --include='settings.py' --include='*.toml' \
'(host|HOST|BIND|bind|--host|-b)[^0-9A-Za-z]{0,4}(0\.0\.0\.0|127\.0\.0\.1|\[::1\]|localhost)' . 2>/dev/null \
| grep -oE '0\.0\.0\.0|127\.0\.0\.1|\[::1\]|localhost' | sort | tr '\n' ' ')"
[[ -n "$binds" ]] && note "addresses the project's own config binds to: $binds"
note "outbound (sandbox -> a service on this machine): allow = 100.64.0.1:<port>."
note "inbound (host -> a server the sandbox runs): publish = <port> -- loopback unless an address"
note " is written in the spec. The two are not interchangeable: publish is ingress, allow is egress."
note " a published port only reaches a server listening on 0.0.0.0 inside (measured), so a config that"
note " binds loopback above needs that said in the same change. references/port-and-loopback.md"
# --------------------------------------------------------------------------- environment names
section "environment variable NAMES the project references (no values)"
names="$(
files_matching .env .env.* '*.env.example' '*.env.sample' '*.env.template' env.example \
| xargs -r grep -hoE '^[[:space:]]*(export[[:space:]]+)?[A-Za-z_][A-Za-z0-9_]*[[:space:]]*=' 2>/dev/null
[[ -n "$compose" ]] && printf '%s\n' $compose | xargs -r grep -hoE '^[[:space:]]+-?[[:space:]]*[A-Z][A-Z0-9_]{2,}[:=]' 2>/dev/null
grep -rIhoE 'secrets\.[A-Z][A-Z0-9_]+' .github 2>/dev/null
files_matching Dockerfile 'Dockerfile.*' | xargs -r grep -hoE '^ENV[[:space:]]+[A-Za-z_][A-Za-z0-9_]*' 2>/dev/null | awk '{print $2}'
)"
names="$(printf '%s\n' "$names" | sed -E 's/^[[:space:]]*(export[[:space:]]+)?//; s/^secrets\.//; s/[[:space:]]*[=:]$//' \
| sort -u | grep -vE '^(PATH|HOME|USER|SHELL|PWD|LANG|LC_ALL|LC_CTYPE|TZ|TERM|HOSTNAME|SHLVL|ENV|_)$')"
if [[ -n "$names" ]]; then
printf ' %s\n' "$(printf '%s\n' "$names" | tr '\n' ' ')"
note "grant what a run needs as 'env = NAME' -- names only"
secretn="$(printf '%s\n' "$names" | grep -iE 'KEY|TOKEN|SECRET|PASSW|CREDENTIAL|AUTH' | tr '\n' ' ')"
[[ -n "$secretn" ]] && warn "credential-shaped (values must never enter a committed file): $secretn"
else
note "none in .env*/compose/Dockerfile/CI -- check the README and the user's launcher"
fi
# --------------------------------------------------------------------------- the model endpoint
section "model endpoint on this host (from the pi agent config)"
pi_dir="${AISB_PI_AGENT_DIR:-$HOME/.pi/agent}"
if [[ -r "$pi_dir/models.json" ]]; then
if command -v jq >/dev/null 2>&1; then
eps="$(jq -r '.. | .baseUrl? // empty' "$pi_dir/models.json" 2>/dev/null | hosts_of)"
else
eps="$(grep -oE '"baseUrl"[[:space:]]*:[[:space:]]*"[^"]+"' "$pi_dir/models.json" 2>/dev/null | sed -E 's/.*"baseUrl"[[:space:]]*:[[:space:]]*"//' | hosts_of)"
fi
if [[ -n "$eps" ]]; then
list <<< "$eps"
note "the bridge turns apiKey into \$AI_API_TOKEN and copies nothing -- but this host must be allowed"
note "no credential value was read or printed"
else
note "models.json is readable, no baseUrl in it"
fi
else
note "no readable $pi_dir/models.json -- nothing to bridge"
fi
# --------------------------------------------------------------------------- existing policy
section "existing policy"
KV_KEYS="net harness workdir cpus memory pids tmpfs_size base_image node_version pi_version \
base_ttl_days base_auto_update selinux home_cache gate_addr gate_port publish publish_addr extra_args \
pi_config pi_token_env pi_token_auto pi_agent_dir thinking agents agents_dir agents_dest \
allow env mount install label"
for f in .ai-sandbox/config .ai-sandbox/allow; do
[[ -r "$f" ]] && found "$f ($(wc -l < "$f") lines)" || note "$f: absent"
done
if [[ -r .ai-sandbox/config ]]; then
while IFS= read -r line; do
line="${line%%#*}"
line="$(printf '%s' "$line" | sed -E 's/^[[:space:]]+//; s/[[:space:]]+$//')"
[[ -n "$line" && "$line" == *=* ]] || continue
k="$(printf '%s' "${line%%=*}" | sed -E 's/[[:space:]]+$//')"
v="$(printf '%s' "${line#*=}" | sed -E 's/^[[:space:]]+//; s/[[:space:]]+$//; s/^"//; s/"$//')"
[[ " $KV_KEYS " == *" $k "* ]] || warn "unknown key (ai-sandbox ignores it): $k"
[[ "$k" != project_dest ]] || warn "project_dest is parsed and read by nothing -- use workdir"
if [[ "$k" == env && "$v" == *=* && "${v%%=*}" =~ (KEY|TOKEN|SECRET|PASSW|CREDENTIAL|AUTH) ]]; then
warn "env = ${v%%=*}=... is a literal value for a credential-shaped name in a committed file"
fi
if [[ "$k" == net && "$v" == full ]]; then warn "net = full in a committed config: that is an escape hatch, not a policy"; fi
if [[ "$k" == extra_args ]]; then warn "extra_args in a committed config: the podman escape hatch -- it belongs on one command line"; fi
if [[ "$k" == publish ]]; then
case "$v" in
*/*) warn "publish = $v has a protocol suffix: one port per entry, no /udp" ;;
*-*) warn "publish = $v is a range: ai-sandbox takes one port per entry, so a reviewed line is never a block of ports" ;;
\[*) note "publish = $v -- an ipv6 bind in brackets, then host:container" ;;
*:*:*) note "publish = $v -- addr:host:container; anything not loopback is an open door and warns on every run" ;;
*:*) note "publish = $v -- host port -> container port, bound to 127.0.0.1 (the default)" ;;
*[!0-9]*) warn "publish = $v is not a port, host:container or addr:host:container -- ai-sandbox refuses it" ;;
*) note "publish = $v -- 127.0.0.1:$v -> container $v; the server must bind 0.0.0.0 inside" ;;
esac
fi
if [[ "$k" == publish_addr ]]; then
case "$v" in
127.*|localhost|::1|\[::1\]) note "publish_addr = $v -- loopback, the safe default" ;;
*) warn "publish_addr = $v in a repo config widens every bare publish line in this directory -- the user's config, if anywhere" ;;
esac
fi
done < .ai-sandbox/config
fi
if [[ -r .ai-sandbox/allow ]]; then
while IFS= read -r line; do
line="$(printf '%s' "${line%%#*}" | sed -E 's/^[[:space:]]+//; s/[[:space:]]+$//')"
[[ -n "$line" ]] || continue
body="$line"
host="$(printf '%s' "$line" | sed -E 's/^\[([^]]*)\].*/\1/; s/^([^:]*):.*/\1/')"
if printf '%s' "$body" | grep -qE '^[0-9]{1,3}(\.[0-9]{1,3}){3}(/[0-9]{1,2})?(:[0-9]{1,5})?$'; then
note "allow $body -- ipv4/cidr: a direct netfilter rule (ports 80,443 unless :port)"
elif printf '%s' "$line" | grep -qE '^\[[^]]*\](:[0-9]{1,5})?$' \
|| printf '%s' "$host" | grep -qE '^([0-9a-fA-F]{0,4}:){2,7}[0-9a-fA-F]{0,4}$'; then
note "allow $body -- ipv6: a direct rule; write it as [addr]:port"
elif printf '%s' "$host" | grep -qE '^[A-Za-z0-9-]+(\.[A-Za-z0-9-]+)+$'; then
note "allow $body -- domain: the gate filters it at CONNECT (only 80,443,8443 relayed)"
else
warn "allow $body -- matches no grammar (domain, ip, cidr, [ipv6]:port): check the spelling"
fi
if [[ -n "$remote_hosts" ]] && printf '%s\n' "$remote_hosts" | grep -qxF -- "$host"; then
warn "allow $body is a git remote of this repository -- the sandbox could push what it wrote"
fi
done < .ai-sandbox/allow
fi
if [[ -e .ai-sandbox/env ]]; then
mode="$(stat -c '%a' .ai-sandbox/env 2>/dev/null || echo '?')"
[[ "$mode" == 600 || "$mode" == 400 ]] && found ".ai-sandbox/env present, mode $mode" || warn ".ai-sandbox/env is mode $mode -- chmod 600"
git check-ignore -q .ai-sandbox/env 2>/dev/null || warn ".ai-sandbox/env is not gitignored"
else
note "no .ai-sandbox/env -- write env.example with names and dummy values; the user writes the real one"
fi
section "next"
note "1. decide net / allow / env / mount / install / publish / limits from the above (SKILL.md step 2)"
note "2. write .ai-sandbox/allow and .ai-sandbox/config, commented, names only"
note "3. verify with ai-sandbox --print-policy and --dry-run (no build, no container, no network)"
note "4. then one live probe per endpoint the project really needs -- and from the host too if you"
note " wrote publish: that port exists only while a sandbox runs, and only 0.0.0.0 inside answers it"
AGENTS.md
+148 -9
View File
@@ -16,13 +16,31 @@ read-only with no escape hatch (`agents`/`--agents`/`--agents-dir`, and `--no-ag
a sandbox that could write your skills could write instructions you follow later on the
host. See `agents_src_of()` and the tests in `tests/unit/09_agents.sh` / `tests/e2e/12_sandbox.sh`.
There is exactly one **ingress** control: `--publish` / `publish` forwards a host port into the
container so a server the sandbox runs can be previewed from the host. It binds `127.0.0.1` unless the
spec names an address (measured: podman's bare `-p` binds every interface), takes one port per entry,
refuses to start under `--net none` rather than run with nothing listening, and the in-container
ruleset names the ports it opened. It changes nothing about egress — the same run that answers a
browser still refuses every domain outside the allowlist (`tests/e2e/13_publish.sh`).
There is a second program, `bin/ai-sandbox-autophase`: the driver for a
`.agents/phases/` pipeline. It runs every executor agent *and the `.agents/validate.sh`
gate* through the sandbox, and does the commits and pushes on the host — the git remote
is not in the allowlist and must never be, because an agent that can push can ship what
you are about to review. It owns no policy: the allowlist, the token and every other
choice come from the user's own launcher command (a function/alias found in their rc
files, or `--pi-cmd`). It is documented in the README and `docs/recipes.md`.
## Non-negotiables
1. **Single script.** The Containerfile, the container entrypoint, the squid
config, the nftables rules and the bridged pi config are all generated from
`bin/ai-sandbox` (`containerfile()`, `entrypoint_script()`, `write_squid_conf()`,
`gen_rules()` inside the entrypoint, `pi_config_bridge()`). Keep it that way; do
not split policy into files that can drift from the script.
not split policy into files that can drift from the script. (The rule is about
*policy*: `bin/ai-sandbox-autophase` is a second program, and it generates none of
those artifacts and holds none of those choices — it delegates all of them to the
user's launcher.)
2. **Fail closed.** Any path where the sandbox cannot guarantee its policy must
refuse to start rather than degrade: no allowlist in `allow` mode, missing
policy file in the entrypoint (exit 78), ruleset that cannot be installed
@@ -53,14 +71,19 @@ host. See `agents_src_of()` and the tests in `tests/unit/09_agents.sh` / `tests/
## Layout
```
bin/ai-sandbox the application
install.sh symlink + optional alias
config/*.example documented examples (never loaded directly)
docs/networking.md the measurements behind the enforcement model
docs/threat-model.md what is and is not protected -- keep it honest
docs/recipes.md per-harness and per-language setups
tests/run.sh runner: unit (no containers) | --e2e (real containers)
tests/spike_network.sh re-measure the host
bin/ai-sandbox the application
bin/ai-sandbox-autophase pipeline driver: agent + gate in the sandbox, git on the host
install.sh symlink both + optional alias
contrib/ai-sandbox.bash bash completion (flags mirrored from parse_args; keep in sync)
config/*.example documented examples (never loaded directly)
docs/networking.md the measurements behind the enforcement model
docs/threat-model.md what is and is not protected -- keep it honest
docs/recipes.md per-harness, per-language and pipeline setups
tests/run.sh runner: unit (no containers) | --e2e (real containers)
tests/fixtures/phase-lib contract fixture for the driver: the phased-execution library's
function names and call shapes, so 10_autophase.sh runs with no
skill installed and no containers
tests/spike_network.sh re-measure the host
```
## Commands
@@ -105,6 +128,122 @@ networking or the image, `make e2e` green too.
* `entrypoint_script()` — the enforcement half. It is rendered into the image and
runs as root there. A bug here is a security bug. `tests/unit/06_entrypoint.sh`
checks the generated rule shape; `tests/e2e/11_net.sh` checks it in a container.
Its pi-seeding block is passed to `bash -c` inside single quotes, so a single quote
anywhere in there silently closes the argument and the rest runs in the *outer* shell
— valid bash, wrong program, clean `bash -n` and clean shellcheck. Nothing in it may
use `'`; jq programs there are written unquoted (`.defaultThinkingLevel=\$lvl`), and
`tests/unit/06_entrypoint.sh` fails the build if that ever changes. (It bit once while
editing a comment: one apostrophe in a word like `pi's` is enough.)
**Stdout inside the container belongs to the command that ran.** Everything the sandbox
says about itself goes to stderr — `log()` and `fail()` do, and so must every new line,
because the driver's preflight reads its answer from that stream and progress.mjs reads the
child pi event stream from it. The pi-seeding notices were on stdout once, and the steady
state (host file unchanged, so `cur = prev`) prints one on *every* pi-bridged run: it made
`ai-sandbox-autophase` refuse a working sandbox on every project. `tests/unit/06_entrypoint.sh`
now runs the seeding block with the two streams split and asserts stdout empty in each state
(cold, steady, edited-in-sandbox, unparseable, level changed), and greps the entrypoint for
any `[aisb]` line without `>&2`. Same block, second contract: the thinking level is policy,
so it is written on the *first* seed too, not only by the merge on the second run.
* `split_target()` / `classify_allows()` — the mapping from what the user typed to
what the firewall and the proxy enforce.
* `resolve_publish()` / `publish_rules()` — the one ingress path, whose two ends have to agree: the
canonical `addr:host:container` spec built on the host, and the container-side accept rule rendered
from `AISB_PUBLISH_PORTS`. Three things are load-bearing. (1) The bind default is loopback — widen
it and a preview in a committed config becomes a listener on the LAN, which is the mistake the flag
exists to avoid. (2) The rule belongs in the `input` chain and never the `output` chain;
`tests/unit/06_entrypoint.sh` asserts which chain, because a publish rule in `output` is an egress
hole wearing an ingress flag. (3) The port list is validated again *inside* the entrypoint before it
reaches nft, and nft is fussy about order (`… counter accept comment "x"`, verdict first): get it
wrong and the ruleset refuses to install, the sandbox exits 77 with the feature dead, and every
`--dry-run` still looks perfect. Only `make e2e` catches that class, so a change here is an e2e
change. Measured, and in `docs/networking.md` for a reason: pasta delivers the connection over
loopback (the `aisb-publish` counter stays at 0 while the port serves), and a server bound to
`127.0.0.1` inside answers nothing on the published port.
* `build_argv()` — every isolation flag lives here.
* `build_in()` / `build_progress()` — a build is minutes of apt, a Node tarball and an
npm install, so it has to announce itself and keep announcing: silence is what a hang
looks like from the terminal, and an interrupted build is a half-built image. Note that
`podman build` writes its transcript to **stdout** — the same stdout that the
`$(ensure_image ...)` call reads back as the image ref — so the transcript is routed
onto stderr. Print it
to stdout and the build log comes back spliced into the ref. Whatever the terminal shows
(`AISB_BUILD_OUTPUT=progress|full|quiet`), the complete transcript belongs in
`$AISB_STATE_DIR/build-*.log`, because the progress view is a filter and a failed build
has to be diagnosable from the raw apt output. Tests: `tests/unit/11_build.sh` (a stub
podman, both heartbeat paths, the ref stays one clean line, a canary value appears
nowhere).
* `resolve_workdir()` / `mirror_block_of()` — what the project is called inside the
container. Mirroring the host path is the convenience; the deny list is the safety, because
a bind hides whatever the image has at that destination and podman puts `--tmpfs /tmp` on
*top* of a bind of the same path. Extend the list with paths the image or the sandbox
owns, never with paths a project happens to live under, and keep `/project` reachable:
it is the fallback for every refusal and for `--no-cwd`, and the image always creates it.
A fallback must be announced (`workdir_note` in `--print-policy`, a comment in
`--dry-run`, one log line on a real run). Tests: `tests/unit/03_argv.sh` drives both
branches, asks the extracted function about the paths a test cannot create (`/usr`,
`/home/sandbox`), and `tests/e2e/12_sandbox.sh` proves it in a container.
* `bin/ai-sandbox-autophase` — four things are load-bearing, each a quiet way
to break the pipeline:
- **It shadows five functions of the sourced pipeline library** — `pi` (a child is a
container), `first_prompt`/`fix_prompt` (the agent is told what it cannot do in
there), `run_validation` (the gate runs where the agent runs) and `push_phase`
(every push, including the library's own, goes through `ap_push_pending`, so the
fingerprint guard, the credential redaction and the error contract cover all of
them) — and nothing else: unit order, retries, provider backoff, report recovery,
the scoped commit and notifications stay the library's. The `declare -F` gate
refuses to start if the library stops providing any of them, so a renamed function
fails loudly instead of quietly running around a guard. Shadow order matters: the
launcher is resolved *before* `pi()` is defined, or a launcher named `pi` resolves
to the shim that wraps it (and a PATH program is captured as an absolute path for
the same reason).
- **Nothing runs on the host against a repository the agent just edited.** `ap_git_env`
exports an empty `core.hooksPath`, `GIT_PAGER=cat`, `GIT_CONFIG_NOSYSTEM` and
`protocol.ext.allow=never`; `ap_git_fingerprint` hashes `.git/config` plus every
file in `.git/hooks/` before the first unit and `ap_git_guard` re-checks it before
every delivery, refusing the run unless `--trust-git-changes`. Git executes those
paths, so a `pre-commit` planted inside the sandbox is otherwise code running as
the user who invoked the pipeline.
- **The rc probe reads a definition and never calls it.** `ap_probe_rc` sources one
file in a throwaway `bash -ic` (interactive, because `~/.bashrc` guards on `$-`) with
`exit` neutered, and returns the text. `--status`/`--dry-run` must therefore never
fetch a token or start a container. The captured body is embedded in each child's
launcher script with `printf %q` per argument — the prompt is one argument that can
contain quotes, newlines and `$` at once, so nothing may build that command line by
concatenating raw text. The body is *never printed*: `ap_launcher_preview` names a
function rather than echoing it, because its body is the `pass` call.
- **The probe must not be able to touch, or wait on, the terminal.** It is
`setsid` + `</dev/null` + `timeout -k 5 15` with the transcript in a **temp file**,
and all three are load-bearing: an interactive bash started from a terminal calls
`tcsetpgrp()`, is in that terminal's background, and is **stopped** for trying — and a
stopped process does not take SIGTERM, so a plain `timeout` waited in front of it
forever and the driver waited behind it before printing anything. A pipe is the other
half: `$(...)` waits for EOF, which any grandchild the rc files leave behind (an
agent, a helper daemon) holds open after `timeout` has reaped the shell it was
watching. Only SIGKILL reaps a stopped process, hence `-k`. Read the definition into a
file and none of that can hold the run. Tests: the shape assertions plus the
backgrounding fixture in `tests/unit/10_autophase.sh`.
- **A function launcher is called *with* the run, so it must forward its arguments.**
Every child is `NAME --no-tty --no-interactive -- <in-container command>` with stdin
on `/dev/null`; a function ending at `--env NAME` — the shape everyone writes for
typing by hand — silently discards all of it, and the container then runs the image's
default harness with no prompt, which on an attached terminal is an agent waiting for
keystrokes behind a pipe nobody reads. `ap_forwards_args` checks the captured
definition (aliases and command lines need it not: the shell appends), `--status` and
the preflight name it, and every child starts `</dev/null` so the launcher cannot put
the terminal back. Keep the `"$@"` in every launcher this repo prints — `install.sh`,
`README.md`, `docs/recipes.md`.
- **The preflight reads the marker, not the first line.** `ap_preflight` asks for one
sandboxed shell and takes the line starting `aisb-ok` wherever it appears, because a banner
from an older image or from a launcher of the user's own is not a broken sandbox — it is
warned about and quoted back, since the same stream is parsed as data in every unit. It
still refuses on a non-zero exit, on silence, and on a launcher that runs no command: those
are the failures that would repeat in every unit. Tests: the noisy and yapping launcher
fixtures in `tests/unit/10_autophase.sh`.
- **The push is the delivery contract.** `ap_push_pending` runs at start, at every
phase boundary and at end of run; start is a warning, the boundary and the end are
fatal, because exiting 0 with the work unpushed is the silent miss this script
exists to prevent. Nothing printed may carry a value: `ap_redact` covers literal
`--env NAME=value` and credentials inside remote URLs, which is exactly what a git
error quotes back. Tests: `tests/unit/10_autophase.sh` (canary token, canary
credential in the remote URL, the composed command checked against the real
`ai-sandbox --dry-run`, and one run against the real library).
Makefile
+8 -3
View File
@@ -2,12 +2,14 @@
# machine plus podman; nothing needs to be installed system-wide.
SHELL := /bin/bash
BIN := bin/ai-sandbox
BINS := bin/ai-sandbox bin/ai-sandbox-autophase
SHELLCHECK_IMAGE := docker.io/koalaman/shellcheck:stable
.PHONY: help test e2e check lint style doctor build clean-demo all
help: ## show this help
@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk -F':.*?## ' '{printf " %-12s %s\n", $$1, $$2}'
test: ## unit tests only (no containers, no network)
./tests/run.sh
@@ -19,12 +21,15 @@ check: lint test ## lint + unit tests (what CI would run)
lint: ## shellcheck every shell file, in a container (nothing installed locally)
@podman run --rm -v "$(CURDIR):/src:ro,z" -w /src $(SHELLCHECK_IMAGE) \
--severity=warning --shell=bash --exclude=SC1091 \
bin/ai-sandbox install.sh tests/run.sh tests/lib.sh tests/spike_network.sh \
$$(ls tests/unit/*.sh tests/e2e/*.sh) \
$(BINS) install.sh contrib/ai-sandbox.bash \
tests/run.sh tests/lib.sh tests/spike_network.sh \
$$(ls tests/unit/*.sh tests/e2e/*.sh tests/fixtures/*.sh tests/fixtures/phase-lib/*.sh) \
$$(ls .agents/skills/*/scripts/*.sh 2>/dev/null) \
&& echo "shellcheck: clean"
style: ## bash-style syntax check of every shell file (fast, no deps)
@for f in $(BIN) install.sh tests/*.sh tests/unit/*.sh tests/e2e/*.sh; do \
@for f in $(BINS) install.sh contrib/*.bash tests/*.sh tests/unit/*.sh tests/e2e/*.sh \
tests/fixtures/*.sh tests/fixtures/phase-lib/*.sh .agents/skills/*/scripts/*.sh; do \
[ -f "$$f" ] || continue; bash -n "$$f" || exit 1; \
done; echo "bash -n: clean"
PLAN.md
+187
View File
@@ -127,6 +127,181 @@ the same destination wins because podman refuses duplicates. A missing or unread
directory is a silent skip like the pi bridge's; an explicit `--agents-dir` that resolves
to nothing is a hard error.
### 11 — Thinking level as policy, and a settings merge that does not freeze
`defaultThinkingLevel` is now `xhigh` by default (`thinking`, `--thinking`,
`AISB_THINKING`; pi's own default is `medium`), written into the bridged settings and
re-applied by the entrypoint on every run, so a run cannot lower the reasoning of the runs
after it. `--thinking host` hands the key back to `settings.json`, `off` is pi's own
"no reasoning" level, and an unknown value is refused with the list to pick from. Nothing
is written for codex, claude or a command after `--`, and `--no-pi-config` declines the
policy along with the bridge.
Getting there found a worse bug. `settings.json` was managed as one managed file, and pi
rewrites its own settings (a theme change, "seen the changelog"), so the first such write
matched nothing and froze *every* host preference out of the sandbox for good -- provider, model,
thinking budgets and the level included, with a real session left holding
`{"lastChangelogVersion": "1.4.0"}`. It is merged per key instead, against a copy of what
we last wrote (`.aisb-settings.json.wrote`): sandbox edits win per key, untouched keys
follow the host again, untouched keys the host dropped are dropped, keys pi invented
survive, and a file that is missing a base copy takes the host values -- which is what
repairs the volumes already frozen. Unparseable JSON is re-seeded with a warning (pi
cannot use it either); a file the container cannot *read* is reported with the fix, because
nothing inside the sandbox can repair it.
One trap worth remembering: the seeding script is passed to `bash -c` in single quotes, so
a single quote anywhere inside it closes the argument early and the remainder becomes code
in the outer shell -- valid bash, wrong program, invisible to `bash -n` and shellcheck.
`tests/unit/06_entrypoint.sh` now extracts the seeding script and asserts it is quote-free,
and extracts the merge program to test its semantics without a container.
### 12 — The pipeline driver: agent in, git out
`bin/ai-sandbox-autophase` runs a `.agents/phases/` pipeline (the shape the
phased-execution skill drives) with the split that the sandbox forces: executor agents
*and* `.agents/validate.sh` inside, commits and pushes outside. Two reasons, both about
the writable mount. `validate.sh` lives in it, so running the gate on the host is
running agent-editable bash on the host (`--validate-in host` for a project whose
toolchain is only on the machine, warned every run). And the remote is deliberately not
allowlisted -- an agent that can push can ship what you are about to review -- so
somebody else has to deliver, and that somebody is this script, between unit runs: after
each phase commit, again before the next phase starts, again at the end, and a re-push of
whatever an earlier run could not deliver.
The launcher is the user's own command, because the allowlist and the token are theirs.
A script cannot see an interactive shell's functions, so `sandbox-pi` (default,
`--pi-cmd`/`AISB_AP_PI_CMD`) is looked up in `~/.bashrc.d/*`, `~/.bashrc`,
`~/.bash_aliases`, `~/.profile`: the rc file is sourced in a throwaway `bash -ic`
with `exit` neutered and the *definition* returned -- never called, which is what lets
`--status`/`--dry-run` promise no token fetch, no image build, no container. That probe
is a `setsid`, `</dev/null`, `timeout -k` read into a **file**: an interactive bash
started from a terminal takes that terminal, is stopped for trying, and a stopped process
does not take the SIGTERM meant for it, so a piped probe could wedge the whole script in
silence -- and any grandchild the rc files leave behind holds a pipe's EOF open long
after the shell it came from is gone. A function launcher is called *with* the run
(`NAME --no-tty --no-interactive -- pi <args> <prompt>`, stdin on `/dev/null`), so it has
to forward its arguments; one that ends at `--env NAME` drops the command, and is named
by `--status` and refused by the preflight rather than running the image default with no
prompt. Each child then re-sources and runs it, so `pass` runs per unit rather than once
a night.
Integration is by shadowing five functions of the sourced library (`pi`,
`first_prompt`, `fix_prompt`, `run_validation`, `push_phase`) behind a `declare -F`
contract check, so the library keeps scheduling, retries, provider backoff, report
recovery and the scoped commit, and a library that changes shape is refused rather than
half-wired.
What it found on the way, in order of how much it mattered:
* **A repository is what git runs, not just what it records.** Committing on the host
after an agent had the tree writable means `.git/config` and `.git/hooks/` are now
input to a host process. Host git therefore runs with an empty `core.hooksPath`, no
pager, no system config and `protocol.ext.allow=never`, and delivery is *refused* if a
unit changed either (`--trust-git-changes` to override). `tests/unit/10_autophase.sh`
plants a `pre-commit` from inside the fake sandbox and asserts it never runs, in either
mode.
* **Shadowing `pi` and resolving a launcher called `pi` are the same name.** Resolve
first, install the shims after; a PATH program is captured as an absolute path, or the
function wins and the child recurses.
* **A preview can leak a body.** The captured launcher definition contains the `pass`
call, and the child command is that text plus arguments -- so previews name the
launcher instead of printing it, and `ap_redact` covers `--env NAME=value` and
credentials inside remote URLs, which is exactly what a failing `git push` quotes back.
* **A report is a gate too.** Running the library's `push_phase` around the guard, rather
than letting it push on its own, is the difference between "the harness pushes" and
"every push is checked".
### 13 — Naming the directory the agent works in
Until now `pwd` was `/project` in every sandbox on the machine, which is two problems
wearing one costume: paths an agent prints (`/project/src/api.py:41`) have to be translated
back to the host by whoever reads them, and a harness that titles its window after the cwd
makes every tab of a session read the same. So the cwd is mounted at the path it has on the
host, and `--workdir` now defaults to `auto`, which means precisely that.
Nothing about the *mount* changed -- same directory, same `rw`, same `:z`, still the only
writable one. What changed is the destination, and the destination is where a bind mount
can do damage: it hides whatever the image already had at that path. Hence
`mirror_block_of()`, whose list is not "places people keep code" but "paths the image or
this sandbox owns": `/usr`, `/etc`, `/var`, `/opt`, `/bin`, `/sbin`, `/lib*`, `/boot`,
`/proc`, `/sys`, `/dev`, `/root`, `/run`, `/tmp`, `/home` and `/home/sandbox*`. `/home`
needs its own rule because it is the one standard hierarchy where work lives:
`/home/<user>/…` mirrors, while `/home` and the container's own home do not -- that is
where the home volume and the read-only `.agents` share are mounted.
`/tmp` decided the shape of the function, and it was measured rather than assumed
(podman's behaviour *is* the question): with `--tmpfs /tmp` and `-v /tmp/x:/tmp/x` in one
run, podman puts the tmpfs on top. `pwd` prints the right path, writes succeed, and the
project is not there. A failure that quiet, in the one place the sandbox promises not to
fail quietly, earns a refusal plus a line of output: `workdir_note` in `--print-policy`, a
comment line in `--dry-run`, one log line on a real run. `/project` -- the one directory
baked into the image -- is what everything falls back to, so a run always has a working
directory that exists, `--no-cwd` included.
The tests could not create most of the deny list, so `tests/unit/03_argv.sh` extracts
`mirror_block_of` from the script and asks it about `/usr`, `/home/sandbox` and `/`
directly (the pattern `06_entrypoint.sh` already uses for the entrypoint), and drives both
real branches through the CLI with a scratch directory under `$HOME` -- `lib.sh:
mirrorable_dir`, because `$TMPDIR` is always the fallback branch, which is why the whole
suite had only ever seen `/project`. `tests/e2e/12_sandbox.sh` then holds a real container
up to the same promise: `pwd`, writes landing on the host, and `/project`, `/home/sandbox`
and a home-volume marker all still present through a mirrored run.
Two things the naming turned up on the way:
* The first version took the first path component with `${top%%/*}`, which on any path with
a second slash yields the empty string -- so every deny rule matched nothing, everything
mirrored, and `bash -n` and shellcheck were both clean. The fix is the table of paths in
*both* directions in `03_argv.sh`; one assertion on the happy path would not have seen it.
* A cwd containing `:` cannot be handed to podman in a `-v` spec at all (`invalid option
type "/project"`), and mirroring doubles the exposure by making that string the
destination too. It is refused now, with the reason, instead of podman's. (`--mount
type=bind,src=…,dst=…` parses it happily; switching the mount form for one character is not
worth the relabel-option churn that comes with it.)
### 14 — Publishing a port: the one control that runs the other way
Everything up to here gates egress, which left an everyday request unanswered: the agent starts a
dev server *inside* the sandbox and a human wants to look at it. The documented answer used to be
"run the server on the host", which is good advice until the agent has to drive the server — seed the
database it starts, migrate then serve, hit its own endpoints, read its logs. So: `--publish` /
`publish`, ingress, with the cost named in the flag rather than discovered later.
Two measurements decided the shape. A bare podman `-p 8000:8000` binds **every interface** on the
host, so a port committed in a repo config would also be a port opened on the LAN — hence the default
bind address is `127.0.0.1`, an address in the spec is written as `addr:host:container` and warns,
and `publish_addr` exists as a machine-wide override nobody should put in a project file. And the
`input` chain, which reads as `policy drop` with no room for a forward, turns out not to block it:
`nft -a list chain inet aisb input` while a published port serves traffic shows the
`aisb-publish` rule at **0 packets**, because pasta forwards into the namespace over loopback and
`iifname "lo" accept` carries it. The explicit rule is therefore not what makes publishing work here;
it is what makes it work on a host that delivers on the interface, and what makes
`nft list table inet aisb` answer "what can reach this container" instead of leaving that question to
a flag on the host.
A third measurement became the documentation, because it is what makes a configured preview look
broken: a server bound to `127.0.0.1` *inside* answers 200 from inside and refuses the published port
from the host. pasta delivers to the container's own address, so the project's dev command has to bind
`0.0.0.0` — `--host 0.0.0.0`, `runserver 0.0.0.0:8000` — and the skill is told to say so in the same
change that writes the port.
The grammar is deliberately smaller than podman's: `port`, `host:container`, `addr:host:container`,
one port per entry, no ranges and no `/udp`, because this list is reviewed line by line and a range is
a line nobody reads. An address requires all three parts, so `127.0.0.1:8000` is refused with the
grammar in the message instead of silently meaning whichever of the two things podman would have
picked. No `-p` alias: `-p` belongs to the harness (`claude -p` prints), and an unknown flag already
passes through. Refusals follow the usual rule — `--net none` cannot forward a port and podman
discards the mapping while starting the container anyway, so "it ran" would mean "nothing is
listening"; and a host port already in use is named before the container starts, not after a build.
`pick_port()` skips ports this run publishes, or the gate would bind the port the sandbox asked for
first and lose the run.
The unit tests check the shape (`--publish 127.0.0.1:8000:8000`, the `# ingress:` header, which chain
the rule lands in, a non-numeric port never reaching nft as code) and the e2e file checks the only
question that matters: a real container with `enforce=nft`, a host `curl` that gets 200, **the same
run still refusing a non-allowlisted domain at 403**, the server running as uid 1000 with no
capabilities so it cannot widen its own port, and the host listener gone once the container exits.
That test paid for itself immediately: the first version of the rule put `comment` before `accept`,
nft refused it, and the sandbox exited 77 — correctly refusing to run unenforced, and silently
killing the feature, with `--dry-run` looking perfect.
## Follow-ups (not yet done)
* **`--net unix` mode**: `--network=none` plus a unix-socket proxy and an in-container
@@ -150,8 +325,20 @@ to nothing is a hard error.
* **Scoped sandbox tokens**: if the gateway can mint a short-lived, per-sandbox key,
`--pi-token-env` should ask for one rather than forwarding the long-lived host
credential. The bridge deliberately does not guess a minting endpoint.
* **`--thinking` for the other harnesses**: codex and claude have their own reasoning
knobs; a policy that spans harnesses needs to know each one's config shape rather than
assume pi's.
* **Bridging extensions/skills on request**: `--pi-mount-ext` or similar for the
extension paths named in `enabledExtensionPaths`, which are host paths and therefore
currently dropped; needs the same explicit-grant shape as `--mount`.
* **Published ports in the driver**: `.ai-sandbox/config` already reaches every child unit, so a
pipeline can publish — but two units running in sequence want the same port, and a long run holding
one is a decision the driver should make out loud (`--publish` per unit, or a range handed out from
a pool) rather than inherit.
* **`publish = :8000`** (podman's "pick a free host port"): useful for parallel sandboxes, and it
needs the chosen port reported back to the terminal, since nothing in the config says what to open
in the browser.
* **UDP publishing**: deliberately left out of v1. It drags in the protocol suffix the parser
refuses, and the sandboxed use cases (dev servers, APIs, hot-reload) are all TCP.
* **`models.json` review surface**: `inspect` could print the exact diff between host
config and bridged config, so a change is reviewable before a run rather than after.
README.md
+237 -15
View File
@@ -16,9 +16,9 @@ $ pi --allow aipi.reeseapps.com --env AI_API_TOKEN
pi
```
`pi` there is an alias for `ai-sandbox`. Inside, the agent sees `/project` (your
cwd, read-write), nothing else on your disk, and the internet only where you
pointed it.
`pi` there is an alias for `ai-sandbox`. Inside, the agent sees your cwd at the path it
has on the host — `~/Projects/billing-api`, not a generic `/project` — read-write, and
nothing else on your disk, with the internet only where you pointed it.
## Install
@@ -85,6 +85,37 @@ If you bridge your pi config (below), ai-sandbox tells you when the model endpoi
your own `models.json` points at is not in the allowlist -- pi would otherwise start
and sit there unable to answer, which reads like a broken sandbox.
### Opening a port: ingress, on purpose
Everything above is egress. To look at a server the sandbox is running -- a browser on
your machine hitting the dev server the agent started -- publish a port:
```bash
ai-sandbox --publish 8000 # host 127.0.0.1:8000 -> container 8000
ai-sandbox --publish 18080:8000 # a different host port
ai-sandbox --publish 0.0.0.0:18080:8000 # every interface: warned about, on purpose
```
| | |
|---|---|
| Default address | `127.0.0.1`. A bare port never lands on your network; `publish_addr` changes the default, an address in the spec overrides it. |
| Shape | `port`, `host:container`, or `addr:host:container`. One port per entry: no ranges, no `/udp` -- this list is reviewed line by line. An address means three parts, so `127.0.0.1:8000` is refused rather than guessed, and podman's `*` shorthand is refused too: writing `0.0.0.0` makes the next reader see what is open. |
| Cost | It is **ingress**: whatever that server serves, or accepts, is reachable from that address. Egress is untouched: the same run that answers your browser still refuses a domain outside the allowlist (`tests/e2e/13_publish.sh`). |
| In the container | The ruleset accepts tcp on the published ports and says so (`nft list table inet aisb`), and the entrypoint logs the ports it opened. |
| Needs a network | `--net none` + `--publish` is a refusal, not a warning: measured, podman discards port mappings and starts the container anyway. |
| The server must bind `0.0.0.0` | Measured: pasta delivers the connection to the container's own address, so `--bind 127.0.0.1` inside answers on the inside and answers nothing on the outside. `npm run dev -- --host 0.0.0.0`, `uvicorn --host 0.0.0.0`. |
```ini
# .ai-sandbox/config -- the dev server this project runs, previewed from the host
publish = 8000 # http://127.0.0.1:8000
publish = 18080:3000 # vite, on a host port that is free
```
The reverse -- a sandboxed process reaching a database, queue or mock API you run on
your own machine -- is egress and needs no new flag: `allow = 100.64.0.1:<port>`, where
`100.64.0.1` is the address that stands for the host's loopback. See
[`docs/networking.md`](docs/networking.md).
## Environment
Nothing is inherited except `TERM LANG LC_ALL LC_CTYPE TZ`.
@@ -101,14 +132,35 @@ Asking for a variable that is not set fails the run closed.
## Filesystem
Your cwd is mounted at `/project` and is the working directory. That is the only
*writable* mount, and nothing else on the host is visible except the one exception
below. Everything else is opt-in:
Your cwd is mounted at **the path it has on the host**, and that is the working
directory. So `pwd`, an error the agent quotes back, a commit message and the window title
your harness derives from the cwd all name *this* project instead of reading the same as
every sandbox in the session. The mount itself is unchanged by the naming: same directory,
same read-write, nothing else visible except the one exception below.
A bind mount hides whatever was already at that path in the image, so a host path that
would land on something the image or the sandbox owns (`/usr`, `/etc`, `/var`, `/opt`,
`/tmp`, `/run`, `/home/sandbox`, `/`) falls back to `/project` — and says so, because a
working directory you did not choose is worth a line:
```
[aisb] ai-sandbox: cannot mirror /tmp/scratch/billing-api: the sandbox owns /tmp*, so the
project is mounted at /project instead (--workdir DIR to choose)
```
`/tmp` is the one case worth naming: the sandbox mounts a tmpfs there, and podman puts
that tmpfs *over* a bind of the same path, so a project under `/tmp` would come out as an
empty directory that happens to be writable and to have the right name. Measured, not
guessed — see [docs/networking.md](docs/networking.md).
`--workdir DIR` picks the mount point yourself (`auto`, the default, is the mirroring
behaviour above). The table below is everything else, all opt-in:
```bash
--mount ~/.ssh:ro # if the agent must be able to push
--mount ../docs:/docs:ro # read-only reference material
--no-cwd # a scratch sandbox with nothing mounted
--workdir /src # mount and cd there instead of mirroring the host path
--no-cwd # a scratch sandbox with nothing mounted (lands in /project)
--read-only # read-only root filesystem
--ephemeral-home # do not persist /home/sandbox (default: persist in a volume)
```
@@ -154,14 +206,41 @@ purpose is to be unable to use it:
| `apiKey: "sk-..."` or `apiKey: "!pass show ..."` | `apiKey: "$AI_API_TOKEN"` -- pi resolves it itself, no value is written to disk |
| a header like `X-Trace: "!date +%s"` | dropped, and reported: a host command is not run in the sandbox |
| `baseUrl: "https://aipi.reeseapps.com/v1"` | unchanged, along with every model, limit and thinking level |
| settings: provider, model, theme, thinking budgets, telemetry | unchanged |
| settings: provider, model, theme, thinking budgets, telemetry | unchanged, merged per key |
| a settings value pointing into `~` | dropped, and reported: that path is not mounted |
| `auth.json`, `models-store.json`, sessions, extensions | never touched |
The rewritten pair is seeded into `/home/sandbox/.pi/agent` at container start, owned by
your uid, mode `600`. It is a managed file: a change on the host is picked up on the
next run, an edit made *inside* the sandbox is left alone, and deleting it inside
re-seeds from the host.
your uid, mode `600`. The two files are managed differently, because they are different
kinds of thing:
| file | rule | why |
|---|---|---|
| `models.json` | replaced when missing or unchanged, left alone once you edit it in the sandbox | it is a catalog; a partial merge of a provider list is worse than either choice |
| `settings.json` | **merged per key** | pi rewrites its own settings — a theme change, "seen the changelog" — and managing the file whole meant that one write froze *every* host preference out of the sandbox permanently |
So in `settings.json`: a key you change inside the sandbox stays yours, a key you never
touched follows the host again, a key the host drops disappears unless you had changed
it, and a key pi added (`lastChangelogVersion`) survives.
### Thinking is xhigh, and it is policy
`defaultThinkingLevel` is the one setting `ai-sandbox` insists on, because pi's own
default is `medium` and a sandbox run is usually the kind of work you want reasoned
about hard:
```bash
pi --allow aipi.reeseapps.com # xhigh, whatever your host settings.json says
pi --thinking max ... # this run only
pi --thinking off ... # no reasoning, e.g. a mechanical refactor at scale
pi --thinking host ... # stop forcing it; your settings.json decides
```
`--thinking` is a policy, not a preference: it is written over the merged settings on
every run, so a run cannot set its own thinking level down for the runs after it. It is
`pi`-only — codex, claude and anything you put after `--` read their own config, and
ai-sandbox leaves that alone. `--no-pi-config` declines the policy too, since "pi starts
with its own defaults" is what that flag is for.
The token itself follows the normal rule -- it is the one variable the bridged config
names, so it is passed by name (never by value, and never into `ps`):
@@ -185,6 +264,34 @@ distinct policy and tagged by content hash: git, curl, jq, build tools, python3,
PKG` adds packages and produces a different tag, so variants coexist, and
`--base-image` swaps the base.
### A build is not a hang
A run whose policy names an image that does not exist yet — a first run, a new
`--install`, a refreshed base, a new release of ai-sandbox itself — builds that image
before it starts anything: apt, a Node tarball, an `npm install`. Tens of seconds to a
few minutes, said out loud as it happens:
```console
ai-sandbox: building the sandbox image with 2 extra package(s): rustc cmake -- one build per policy change, reused by every run after it
ai-sandbox: build: localhost/ai-sandbox:ubuntu-1a2b3c4d5e6f (transcript: ~/.local/state/ai-sandbox/build-sandbox-1a2b3c4d5e6f.log)
ai-sandbox: build: [4/21] RUN apt-get update && apt-get install -y --no-install-recomme
ai-sandbox: build: still on [4/21] RUN apt-get update && apt-get install -y --no-in (52s, 61s total)
ai-sandbox: built localhost/ai-sandbox:ubuntu-1a2b3c4d5e6f in 84s
```
One line per build step, plus a notice every `AISB_BUILD_QUIET_SECS` (15) that a step has
shown nothing — a step downloading a toolchain and a step wedged look identical from the
terminal, and only the clock tells them apart. The names go in the first line because
that is what answers "why is *this* run building when the last one did not".
Whatever the terminal shows, the **complete** transcript of every build is written to
that file under `AISB_STATE_DIR`, and the last 25 lines of it are printed when a build
fails — with the package names called out, since a name apt has never heard of is what
`--install` usually gets wrong and it drowns in the package lists.
`AISB_BUILD_OUTPUT=full` streams podman's raw transcript instead; `=quiet` shows only the
announcements. None of it goes to stdout: `podman build` writes its progress there, and
this script's stdout is the image ref the run is then started with.
### The base is pinned by digest
A tag is a moving pointer: `FROM ubuntu:24.04` means "whatever Canonical points at
@@ -226,7 +333,8 @@ ai-sandbox ps | clean [--all] [--volumes[=GLOB]] | inspect | gate up|status|logs
```
State lives under `~/.local/state/ai-sandbox` (session dirs, `log` of every run
with no secret values, build logs). Config in `~/.config/ai-sandbox`.
with no secret values, and `build-*.log` — the full transcript of each image build,
which is what to read when a build fails). Config in `~/.config/ai-sandbox`.
`/home/sandbox` is a named volume (`aisb-home`) so npm and tool caches survive
between runs. It outlives the image, so after changing the image you may be looking
@@ -253,14 +361,128 @@ the design: [docs/networking.md](docs/networking.md).
does not protect you from (allowlisted destinations, container escapes, anything
you explicitly mounted).
## An unattended pipeline: `ai-sandbox-autophase`
A phase pipeline (`.agents/phases/todo/…`) is the harshest thing to ask of a sandbox:
hours of unattended work by an agent that installs dependencies, runs the tests and
wants to commit and push. It gets its own driver, and the split inside that driver is
the design:
| in the sandbox | on the host |
|---|---|
| the executor agent of every task, and the phase's final pass | the phase commit |
| `.agents/validate.sh`, the pass/fail gate | the push — and the re-push of anything an earlier run left behind |
The git remote stays unreachable from inside on purpose: an agent that could push could
ship what you are about to review. `git status`, `diff`, `log` and `blame` work in there
(the repository is the mount, git is in the image), and the agent is told in its own
prompt that `push`/`fetch`/`pull` cannot work, so it does not spend attempts discovering
that. Delivery is the driver's job: it pushes when a phase commits, again before the next
phase starts, and again at the end of the run, so nothing sits unpushed between runs. A
push failure stops the run and prints the command that finishes it — the commit is local
and safe, and the next run takes it out.
The gate runs in the sandbox for the same reason the agent does: `.agents/validate.sh`
sits in the writable mount, so running it on the host is running agent-editable bash on
the host. `--validate-in host` exists for a project whose toolchain only exists on the
machine, and says so on every run.
```
[2026-10-28 02:11:03] preflight: one sandboxed shell, to prove the launcher, the image and the policy
aisb-ok aisb=1 git=2.43.0 user=sandbox cwd=/home/you/Projects/foo
[2026-10-28 02:11:04] checking for commits a previous run left unpushed
(up to date with origin/main)
[2026-10-28 02:11:04] ━━ 01_one/01_task.md — attempt 1/3 ━━
(every line prefixed with the time, as the pipeline prints them)
(committed: 9f1c2ab phase: 01_one)
(pushed: 9f1c2ab phase: 01_one)
[2026-10-28 02:41:52] ✓ pipeline done — 2 unit(s) delivered this run: 01_one/01_task.md 01_one/00_phase.md
```
### It runs *your* command, not a copy of it
The driver does not know your allowlist and does not try to. It runs your launcher — by
default the `sandbox-pi` shell function you keep in `~/.bashrc.d`:
```bash
sandbox-pi () {
export AI_API_TOKEN=$(pass aipi/dev-token)
ai-sandbox --allow aipi.reeseapps.com --allow github.com \
--allow pi.dev --allow release-assets.githubusercontent.com \
--env AI_API_TOKEN "$@"
}
```
That last `"$@"` is the whole contract. A function launcher is called *with* the run: the
flags, then `--`, then the command to run inside, and one that ends at `--env AI_API_TOKEN`
drops all of it. It then starts the image's default harness with no prompt — and a `pi`
with no prompt on the other side of a launcher that attached your terminal sits waiting
for keystrokes while its output goes into a pipe nobody is watching. The driver names a
launcher like that in `--status` and refuses to start the pipeline; aliases and whole
command lines do not need it, because the shell appends the words to those by itself.
A script cannot see an interactive shell's functions, so the driver looks the name up in
`~/.bashrc.d/*`, `~/.bashrc`, `~/.bash_aliases` and `~/.profile` (or `AISB_AP_RC_FILES`)
and takes the *definition* — probing never calls it, so a preview cannot fetch a token.
Reading that definition is done in a throwaway session of its own, with no stdin and a
bounded read: an interactive bash started from your terminal takes your terminal, gets
stopped for it, and a stopped process ignores the timeout sent to it, so the probe used to
be able to wedge the whole script before it printed a single line.
Each child then runs exactly what your shell would have, `pass` included, once per unit
so an overnight run is not holding yesterday's token. An alias or a plain program works
too; `--pi-cmd` also accepts a whole command line.
```bash
cd your/project
ai-sandbox-autophase --status # launcher, pending units, git plan. Runs nothing.
ai-sandbox-autophase --dry-run # the exact child and gate commands
ai-sandbox-autophase # every pending task, in pipeline order
```
Every child is started as `<your command> --no-tty --no-interactive -- pi <args> <prompt>`,
with its stdin on `/dev/null`: your flags first, then the two that keep an unattended run
off your terminal (no TTY to corrupt the JSON stream, no stdin for your keystrokes to fall
into), then `--`, after which ai-sandbox reads nothing but the command. Those two are
flags, and a launcher is free to lose them, so the `/dev/null` is what actually guarantees
the second one.
| flag | |
|---|---|
| `--pi-cmd NAME` | the launcher: function, alias, program or whole command line |
| `--rc-files A:B` | which rc files to look in |
| `--sandbox-args ARGS` | flags added to every run (default `--no-tty --no-interactive`) |
| `--validate-in sandbox\|host` | where the gate runs |
| `--no-push` | commit locally, deliver nothing |
| `--no-preflight` | skip the throwaway container that proves the chain works before spending tokens |
| `--trust-git-changes` | deliver even though the sandbox edited `.git/config` or `.git/hooks/` |
| `--unit`, `--phase`, `--max-units` | scope a run |
It does *all* the pushing — including the pipeline library's own — so that one function
answers for every commit that leaves the machine. And because a repository is not
something git merely records but something it **runs**, host-side git is started with an
empty hooks path, no pager, no system config and no `ext::` transport, and the run
refuses to deliver if a sandboxed unit rewrote `.git/config` or planted a hook. A
`pre-commit` written inside the sandbox is the sandbox executing code on the host, and
the phase commit is not worth finding that out the hard way.
`MAX_FIX_ATTEMPTS`, `PHASE_MODEL`, `PHASE_THINKING`, `PI_TRUST`, `FRESH_FIX`, `QUIET`,
`PHASE_NOTIFY` and `PHASE_COMMIT` behave as the phased-execution skill documents them —
the driver reuses that pipeline and replaces only the parts that assume the agent is a
local process.
## Project layout
```
bin/ai-sandbox the application: one script, ~1.3k lines
install.sh symlink into ~/.local/bin + optional alias
bin/ai-sandbox the application: one script, ~2.2k lines
bin/ai-sandbox-autophase the pipeline driver: agent in the sandbox, git on the host
install.sh symlink both into ~/.local/bin + optional alias
contrib/ai-sandbox.bash bash completion for both
config/ example config, allowlist and env file
docs/ networking.md, threat-model.md, recipes.md
tests/run.sh runner; tests/unit/* (no containers) tests/e2e/* (real ones)
tests/fixtures/phase-lib contract fixture: the pipeline library's shape, without skill
or containers, for the driver's tests
tests/spike_network.sh re-measure podman's behaviour on a new host
Makefile make test | e2e | lint | check | doctor | build
```
@@ -268,7 +490,7 @@ Makefile make test | e2e | lint | check | doctor | build
## Development
```bash
make test # unit: args, config, argv construction, env handling, rule generation
make test # unit: args, config, argv construction, env handling, rules, pipeline driver
make e2e # + real containers: build cache, egress blocking, capability drop, lifecycle
make lint # shellcheck in a container
make style # bash -n
bin/ai-sandbox
+696 -41
View File
File diff suppressed because it is too large Load Diff
bin/ai-sandbox-autophase
Executable
+991
View File
@@ -0,0 +1,991 @@
#!/usr/bin/env bash
# ai-sandbox-autophase -- run the .agents/phases/ pipeline with the agent inside
# ai-sandbox and the git history outside it.
#
# Who runs where is the whole design:
#
# * the executor agents run INSIDE the sandbox, so the code they write, the tests
# they run and the dependencies they download cannot touch the host;
# * .agents/validate.sh runs INSIDE it too, because that file sits in the writable
# mount -- a gate executed on the host is the sandboxed agent executing arbitrary
# bash on the host the moment it edits its own jailer;
# * the commit and the push run on the HOST, between unit runs, by this script.
# The egress allowlist does not contain the git remote and adding it would let an
# agent push what you are about to review. `git status`, `git diff`, `git log` and
# `git blame` all work in there -- the repository is in the mount and git is in the
# image -- only the remote is out of reach. So the pipeline cannot expect the agent
# to deliver anything to the remote, and this script does that delivery instead:
# one atomic commit per completed phase (the library's, scoped to the phase's own
# work), pushed as soon as it exists, re-pushed between phase runs if a previous
# run left something behind.
#
# The sandbox launcher is YOUR command, not a copy of it. Default: the `sandbox-pi`
# shell function people keep in ~/.bashrc.d / ~/.bashrc, which knows the allowlist and
# how to fetch the token. A non-interactive script never loads an interactive rc, so
# the command is looked up in those rc files (function, alias or executable) and every
# child then runs exactly what your own shell would have run -- the `pass` call
# included, once per unit so an overnight run is not holding a day-old token.
#
# Usage: ai-sandbox-autophase [options] (see --help)
set -uo pipefail
VERSION="1.0.0"
PROG="ai-sandbox-autophase"
# --------------------------------------------------------------------------
# options: defaults -> environment -> command line
# --------------------------------------------------------------------------
AISB_AP_PI_CMD="${AISB_AP_PI_CMD-sandbox-pi}" # function, alias or program
AISB_AP_RC_FILES="${AISB_AP_RC_FILES:-}" # colon list; default: your rc files
AISB_AP_LIB="${AISB_AP_LIB:-}" # phased-execution lib.sh
AISB_AP_SKILL_DIR="${AISB_AP_SKILL_DIR:-}" # ... or its skill directory
AISB_AP_VALIDATE="${AISB_AP_VALIDATE:-sandbox}" # sandbox | host
AISB_AP_PUSH="${AISB_AP_PUSH:-1}" # push between phase runs
AISB_AP_PREFLIGHT="${AISB_AP_PREFLIGHT:-1}" # prove the chain before spending tokens
# No colon on purpose: an explicitly empty AISB_AP_SANDBOX_ARGS means "no extra flags",
# which is how a launcher that is not ai-sandbox (or a test) opts out of them.
AISB_AP_SANDBOX_ARGS="${AISB_AP_SANDBOX_ARGS---no-tty --no-interactive}"
AISB_AP_NOTES="${AISB_AP_NOTES:-1}" # append the sandbox rules to the prompts
AISB_AP_TRUST_GIT="${AISB_AP_TRUST_GIT:-0}" # allow a run that rewrote .git config/hooks
AP_MODE="pipeline" # pipeline | unit | phase | status | dryrun
AP_UNIT=""
AP_PHASE=""
AP_MAX_UNITS=0
ap_ts() { date '+%Y-%m-%d %H:%M:%S'; }
ap_log() { printf '[%s] %s\n' "$(ap_ts)" "$*"; }
ap_warn() { printf '[%s] ⚠ %s\n' "$(ap_ts)" "$*" >&2; }
ap_die() { printf '[%s] ✗ ERROR: %s\n' "$(ap_ts)" "$*" >&2; exit 1; }
ap_need() { [[ -n "${2:-}" ]] || ap_die "$1 needs a value (see --help)"; }
usage() {
cat <<EOF
$PROG $VERSION -- run the phased-execution pipeline through ai-sandbox.
USAGE
$PROG every pending task, in pipeline order
$PROG --unit 03_api/02_routes.md one task (or a phase's 00_phase.md)
$PROG --phase 03_api one phase to completion
$PROG --max-units 4 stop after N units (0 = no cap)
$PROG --status what would run: launcher, pending units, git plan
$PROG --dry-run the exact child and gate commands
--status and --dry-run read your rc files and nothing else: no container, no image
build, no token fetched, no network.
WHAT RUNS WHERE
in the sandbox the executor agent of every unit, and .agents/validate.sh
on the host the phase commit, and every push -- this script's and the pipeline
library's alike, routed through here so that nothing is delivered
without the checks under GIT
The agent's repository is readable in there (status/diff/log/blame) and it is told,
in its own prompt, that the remote is unreachable. An agent that could push could
ship what you are about to review, so the remote stays out of the allowlist and this
script delivers the commits: after each phase, and again before the next phase and
at the end of the run, so nothing sits unpushed between runs.
LAUNCHER
--pi-cmd NAME Command that starts a sandboxed pi. Default: \$AISB_AP_PI_CMD or
'sandbox-pi'. Resolved in this order: an exported function, a
function or alias in your rc files, then PATH. A value with
spaces is taken verbatim: --pi-cmd 'ai-sandbox --allow api.example'
A FUNCTION must forward its arguments, because this is what a
run appends to it:
NAME --no-tty --no-interactive -- pi <pi args> <prompt>
so the definition has to end in ... --env AI_API_TOKEN "\$@"
One that drops them runs the image's default harness with no
prompt instead, which says nothing and waits forever. A launcher
like that is named by --status and refused by the preflight.
--rc-files LIST Colon-separated rc files to look in.
Default: ~/.bashrc.d/*, ~/.bashrc, ~/.bash_aliases, ~/.profile
--sandbox-args ARGS Flags added to the launcher for every run.
Default: --no-tty --no-interactive -- nothing of an unattended
run is attached to this terminal: no TTY to corrupt the JSON
stream, and your keystrokes cannot reach the agent.
--lib PATH phased-execution lib.sh. Default: \$AISB_AP_LIB, then
\$AISB_AP_SKILL_DIR/scripts/lib.sh, then
~/.agents/skills/phased-execution/scripts/lib.sh,
.agents/skills/..., ~/.pi/agent/skills/...
GATE
--validate-in WHERE sandbox (default) or host. The gate file lives in the writable
mount, so choosing host runs agent-editable bash outside the
sandbox, and says so loudly.
--no-preflight Skip the throwaway container that proves launcher, image,
entrypoint and egress policy work. On by default, so a broken
sandbox fails in seconds instead of after an hour of attempts.
GIT
--no-push Leave commits local. PHASE_PUSH=0 in the environment means the
same thing here and is honoured: nothing goes out either way.
--trust-git-changes Deliver even though the sandbox edited .git/config or
.git/hooks/. Default: refuse: a repository RUNS those, so a
hook written inside the sandbox is code running on the host.
Host git runs with an empty hooks path, no pager, no system
config and no ext:: transport with or without this.
A failed push stops the run and prints the hand-fix command. The commit itself is
local and safe, and the between-phase push of the next run takes it out once the
remote answers.
Prompts: --no-notes drops the block that tells the agent it is inside a sandbox and
that the remote is unreachable (on by default -- without it an agent will spend
attempts trying to push).
Everything else is the pipeline's own environment: MAX_FIX_ATTEMPTS, PHASE_MODEL,
PHASE_THINKING, PI_TRUST, FRESH_FIX, QUIET, PHASE_NOTIFY, PHASE_COMMIT,
PHASE_STAGE_DENY. See the phased-execution skill.
EXIT STATUS
0 everything pending was delivered (or there was nothing to do)
1 a unit failed after all attempts, a commit/push did not complete, or delivery was
refused because the sandbox changed .git/config or .git/hooks/
2 bad usage
130/143 interrupted; the unit stays in todo/ and its session is resumed next run
EOF
}
while (($#)); do
case "$1" in
--pi-cmd) ap_need "$1" "${2:-}"; AISB_AP_PI_CMD="$2"; shift 2 ;;
--pi-cmd=*) AISB_AP_PI_CMD="${1#*=}"; shift ;;
--rc-files) ap_need "$1" "${2:-}"; AISB_AP_RC_FILES="$2"; shift 2 ;;
--rc-files=*) AISB_AP_RC_FILES="${1#*=}"; shift ;;
--lib) ap_need "$1" "${2:-}"; AISB_AP_LIB="$2"; shift 2 ;;
--lib=*) AISB_AP_LIB="${1#*=}"; shift ;;
--skill-dir) ap_need "$1" "${2:-}"; AISB_AP_SKILL_DIR="$2"; shift 2 ;;
--skill-dir=*) AISB_AP_SKILL_DIR="${1#*=}"; shift ;;
--sandbox-args) ap_need "$1" "${2:-}"; AISB_AP_SANDBOX_ARGS="$2"; shift 2 ;;
--sandbox-args=*) AISB_AP_SANDBOX_ARGS="${1#*=}"; shift ;;
--validate-in) ap_need "$1" "${2:-}"; AISB_AP_VALIDATE="$2"; shift 2 ;;
--validate-in=*) AISB_AP_VALIDATE="${1#*=}"; shift ;;
--no-push) AISB_AP_PUSH=0; shift ;;
--trust-git-changes) AISB_AP_TRUST_GIT=1; shift ;;
--no-preflight) AISB_AP_PREFLIGHT=0; shift ;;
--no-notes) AISB_AP_NOTES=0; shift ;;
--unit) ap_need "$1" "${2:-}"; AP_MODE=unit; AP_UNIT="$2"; shift 2 ;;
--unit=*) AP_MODE=unit; AP_UNIT="${1#*=}"; shift ;;
--phase) ap_need "$1" "${2:-}"; AP_MODE=phase; AP_PHASE="$2"; shift 2 ;;
--phase=*) AP_MODE=phase; AP_PHASE="${1#*=}"; shift ;;
--max-units) ap_need "$1" "${2:-}"; AP_MAX_UNITS="$2"; shift 2 ;;
--max-units=*) AP_MAX_UNITS="${1#*=}"; shift ;;
--status) AP_MODE=status; shift ;;
--dry-run|-n) AP_MODE=dryrun; shift ;;
-h|--help) usage; exit 0 ;;
-V|--version) printf '%s %s\n' "$PROG" "$VERSION"; exit 0 ;;
*) printf '%s: unknown option: %s (see --help)\n' "$PROG" "$1" >&2; exit 2 ;;
esac
done
case "$AISB_AP_VALIDATE" in
sandbox) ;;
host)
# Said once, up front, whatever mode this is: .agents/validate.sh is inside the
# mount the agent can write, so this choice runs agent-editable bash outside the
# sandbox. It is a legitimate choice (a project whose toolchain only exists on the
# host) and it is never a quiet one.
ap_warn "the validation gate will run on the HOST: .agents/validate.sh is writable by the sandboxed agent, so this runs agent-editable bash outside the sandbox"
;;
*) ap_die "--validate-in must be 'sandbox' or 'host' (got '$AISB_AP_VALIDATE')" ;;
esac
[[ "$AP_MAX_UNITS" =~ ^[0-9]+$ ]] || ap_die "--max-units must be a number"
# --------------------------------------------------------------------------
# the launcher: run the command the user's own shell would run
# --------------------------------------------------------------------------
# rc files to look in, in the order a login shell reads them. Only existence is
# checked here; ap_probe_rc is what sources them, and only to read a definition.
AP_RC_LIST=()
ap_rc_list() {
local -a out=()
local f
if [[ -n "$AISB_AP_RC_FILES" ]]; then
local IFS=':'
for f in $AISB_AP_RC_FILES; do [[ -n "$f" ]] && out+=("$f"); done
else
for f in "$HOME"/.bashrc.d/* "$HOME/.bashrc" "$HOME/.bash_aliases" "$HOME/.profile"; do
[[ -f "$f" && -r "$f" ]] && out+=("$f")
done
fi
AP_RC_LIST=("${out[@]+"${out[@]}"}")
}
# Source ONE rc file in a throwaway interactive bash and print what it defines for
# NAME -- the definition text only, never a call: probing must not fetch a token or
# start a container. Interactive because a ~/.bashrc that opens with `case $- in *i*)`
# defines nothing for a non-interactive shell. `exit` is neutered first, because plenty
# of rc files exit when they dislike the conditions and must not take the probe with them.
#
# Output, in this order, so it parses without ambiguity:
# _AP_PATH=<PATH as the rc files left it>
# _AP_FN | _AP_ALIAS (absent when NAME is defined nowhere in that file)
# <the definition>
#
# Three things here exist because of one bug: an interactive bash started from a terminal
# takes the terminal. It calls tcsetpgrp(), it is in the background of that terminal, so
# the kernel stops it -- and a stopped process does not take SIGTERM, so `timeout` waited
# in front of it forever while the driver waited behind it with nothing on screen. Run
# from a script (no terminal) it never happened, which is how it survived. Hence:
#
# * setsid, so the probe has no controlling terminal to fight over at all; it is still
# interactive, which is the part the rc guards look at;
# * stdin from /dev/null, so nothing it sources can read from the run's keyboard;
# * the transcript into a FILE rather than a pipe: a command substitution waits for the
# pipe's EOF, which is held open by any grandchild the rc files leave behind (an
# agent, a daemon, an editor helper) even after timeout has reaped the shell it was
# watching. A file has no such promise -- timeout returns, we read what is there.
# * `timeout -k`, because SIGKILL is the one signal a stopped process cannot sit on.
#
# setsid is util-linux and is present wherever rootless podman is; without it the probe
# still runs, just without that protection.
AP_SETSID=()
command -v setsid > /dev/null 2>&1 && AP_SETSID=(setsid)
ap_probe_rc() { # <name> <file>
local name="$1" file="$2" tmp rc=0
tmp="$(mktemp "${TMPDIR:-/tmp}/aisb-ap-probe.XXXXXX")" || return 1
AISB_AP_P_NAME="$name" AISB_AP_P_FILE="$file" \
timeout -k 5 15 ${AP_SETSID[@]+"${AP_SETSID[@]}"} bash -ic '
exit() { return "${1:-0}"; }
. "$AISB_AP_P_FILE" </dev/null >/dev/null 2>&1
shopt -s expand_aliases 2>/dev/null || true
printf "_AP_PATH=%s\n" "$PATH"
if declare -F -- "$AISB_AP_P_NAME" >/dev/null 2>&1; then
printf "_AP_FN\n"; declare -f -- "$AISB_AP_P_NAME"
elif alias -- "$AISB_AP_P_NAME" >/dev/null 2>&1; then
printf "_AP_ALIAS\n"; alias -- "$AISB_AP_P_NAME"
fi
' > "$tmp" 2>/dev/null < /dev/null || rc=$?
# A probe that had to be killed is worth naming: "the launcher is not defined in your rc
# files" would be the wrong thing to leave you with when the real story is that this one
# file blocked for 15s, and the file is the one thing you can act on.
if (( rc == 124 || rc == 137 )); then
ap_warn "$file took longer than 15s to read (something in it waits) -- skipped"
fi
cat "$tmp" 2>/dev/null
rm -f "$tmp"
}
AP_KIND="" # function | alias | program | arguments
AP_LAUNCHER="" # resolved name, for messages
AP_SOURCE="" # where it was found
AP_DEF="" # captured definition text, when there is one
AP_ARG_FORWARD=1 # 0 when a captured function cannot pass the command line on
AP_CHILD_PATH="" # PATH as the rc files left it
AP_CHILD=() # the launcher itself; AP_RUN below is what a child is started with
# Two word lists live in globals: AP_WORDS is the sandbox flags of every run, AP_CMD_WORDS
# the words of a --pi-cmd that named a whole command line. Do not reuse one for the other.
AP_WORDS=()
AP_CMD_WORDS=()
ap_words() { # <string> -> AP_WORDS
AP_WORDS=()
read -r -a AP_WORDS <<< "$1"
}
# A captured function/alias becomes a small launcher script: neuter `exit`, source the
# rc files (the command may lean on other functions or aliases from them), define what
# we captured, then call it. Every argument is re-quoted with printf %q, because the
# prompt is one argument that can contain quotes, newlines and dollar signs at once.
ap_build_runner() {
local -a body=()
local f
[[ -n "$AP_CHILD_PATH" ]] && body+=("PATH=$(printf '%q' "$AP_CHILD_PATH")" "export PATH")
body+=('exit() { return "${1:-0}"; }')
body+=('shopt -s expand_aliases 2>/dev/null || true')
for f in ${AP_RC_LIST[@]+"${AP_RC_LIST[@]}"}; do
body+=(". $(printf '%q' "$f") </dev/null >/dev/null 2>&1 || true")
done
# An rc file that turned on errexit must not kill the launcher halfway through the
# run: the child's own exit code is the only status that matters here.
body+=('set +e')
body+=("$AP_DEF")
body+=('_ap_run_named() {')
body+=(' local n="$1"; shift')
body+=(' local -a q=(); local a')
body+=(' for a in "$@"; do q+=("$(printf "%q" "$a")"); done')
body+=(' eval "$n ${q[*]}"')
body+=('}')
body+=("_ap_run_named $AP_LAUNCHER \"\$@\"")
AP_CHILD=(bash -c "$(printf '%s\n' "${body[@]}")" "$PROG-launcher")
}
# A captured FUNCTION has to hand its arguments on: every run appends its own sandbox
# flags, then --, then the in-container command, and a definition that never mentions
# "$@" throws all of that away. It fails silently, which is what makes it worth checking
# for: the container then starts the image's default harness with no prompt, and with a
# terminal on the other side of the launcher that is an interactive agent waiting for
# input nobody will type, behind a pipe that swallows its output. An alias needs no
# check -- the shell appends the words to an alias by itself -- and neither does a
# program or a whole command line, for the same reason.
#
# The test is on the definition text, never on a call: a probe must not fetch a token.
# Anything matching $@ / $* / ${1 / $1 counts as forwarding; a false pass is a
# preflight failure with a diagnosis, a false alarm is only ever a warning.
ap_forwards_args() { # <definition text> -> 0 when it can pass the command line on
printf '%s' "$1" | grep -qE '\$[@*]|[$][{@#0-9]|\$[1-9]'
}
ap_check_forwarding() {
AP_ARG_FORWARD=1
[[ "$AP_KIND" == function ]] || return 0
ap_forwards_args "$AP_DEF" || AP_ARG_FORWARD=0
}
# Said once per run, wherever the launcher was reported. Never prints the definition:
# the body of a launcher is the thing that fetches the token.
ap_warn_forwarding() {
(( AP_ARG_FORWARD )) && return 0
ap_warn "$AP_LAUNCHER does not forward its arguments, so the run flags and the in-container command this script appends to it are dropped -- it runs the image's default harness with no prompt instead, which waits for a terminal nothing sends. End the function with \"\$@\" (see --help)."
}
ap_resolve_launcher() {
local name="$AISB_AP_PI_CMD" f out kind path def
AP_CMD_WORDS=()
[[ -n "$name" ]] || ap_die "no sandbox launcher: \$AISB_AP_PI_CMD and --pi-cmd are both empty"
# An explicit command line wins: the user said exactly what to run.
if [[ "$name" == *[[:space:]]* ]]; then
read -r -a AP_CMD_WORDS <<< "$name"
((${#AP_CMD_WORDS[@]})) || ap_die "--pi-cmd has no command in it"
AP_KIND=arguments
AP_LAUNCHER="${AP_CMD_WORDS[0]}"
AP_SOURCE="--pi-cmd"
AP_CHILD=("${AP_CMD_WORDS[@]}")
return 0
fi
[[ "$name" =~ ^[A-Za-z_][A-Za-z0-9_-]*$ ]] \
|| ap_die "launcher '$name' is not a command name; pass a whole command line instead (see --pi-cmd)"
# 1. a function the caller exported
if declare -F -- "$name" >/dev/null 2>&1; then
AP_KIND=function; AP_LAUNCHER="$name"; AP_SOURCE="the calling shell"
AP_DEF="$(declare -f -- "$name")"
ap_check_forwarding
ap_build_runner
return 0
fi
# 2. a function or an alias inside the rc files
ap_rc_list
for f in ${AP_RC_LIST[@]+"${AP_RC_LIST[@]}"}; do
out="$(ap_probe_rc "$name" "$f")"
path="$(printf '%s\n' "$out" | sed -n '1s/^_AP_PATH=//p')"
kind="$(printf '%s\n' "$out" | sed -n '2p')"
if [[ "$kind" == "_AP_FN" || "$kind" == "_AP_ALIAS" ]]; then
def="$(printf '%s\n' "$out" | tail -n +3)"
if [[ "$kind" == "_AP_FN" ]]; then AP_KIND="function"; else AP_KIND="alias"; fi
AP_LAUNCHER="$name"; AP_SOURCE="$f"; AP_DEF="$def"; AP_CHILD_PATH="$path"
ap_check_forwarding
ap_build_runner
return 0
fi
done
# 3. a program on PATH -- recorded as the absolute path, because a bare word would be
# resolved as a shell name first, and `pi` is one of the names this script shadows.
if command -v -- "$name" >/dev/null 2>&1; then
AP_KIND=program; AP_LAUNCHER="$(command -v -- "$name")"; AP_SOURCE="$AP_LAUNCHER"
AP_CHILD=("$AP_LAUNCHER")
return 0
fi
ap_die "cannot resolve the sandbox launcher '$name':
not an exported function,
not a function or alias in: ${AP_RC_LIST[*]:-<no rc files found>},
not a program on PATH.
Point --rc-files at the file that defines it, or name the command outright:
--pi-cmd 'ai-sandbox --allow api.example.com --env API_KEY'"
}
# The full argv prefix of a child run: the launcher, this run's own sandbox flags, then
# -- so that everything after it is the in-container command verbatim. ai-sandbox stops
# reading its own options at --, which is what lets a launcher function with a fixed
# policy carry an arbitrary harness command line.
AP_RUN=()
ap_build_run() {
AP_WORDS=()
read -r -a AP_WORDS <<< "$AISB_AP_SANDBOX_ARGS"
AP_RUN=("${AP_CHILD[@]}" ${AP_WORDS[@]+"${AP_WORDS[@]}"})
}
# Anything printed that could carry a value is redacted first: a --pi-cmd that sets a
# literal value, and a remote URL with a credential in it, which is what a git error
# quotes back verbatim. Values never appear in output, only their names.
ap_redact() {
sed -E -e 's#([A-Za-z][A-Za-z0-9+.-]*://[^/:@ ]+:)[^@[:space:]]+@#\1***@#g' \
-e 's/(--[a-z-]*env[= ]+[A-Za-z_][A-Za-z0-9_]*)=[^[:space:]]+/\1=<redacted>/g'
}
# The launcher part of a preview. A captured function or alias is NOT printed: its body
# is the thing that fetches the token, and a preview must not echo it -- naming it is
# the useful and the safe thing to print. A command line is printed with any literal
# value redacted.
ap_launcher_preview() {
case "$AP_KIND" in
function|alias) printf '<%s>' "$AP_LAUNCHER" ;;
program) printf '%s' "$AP_LAUNCHER" ;;
arguments) printf '%s' "$(printf '%s ' ${AP_CMD_WORDS[@]+"${AP_CMD_WORDS[@]}"} | ap_redact)" ;;
*) printf '<launcher>' ;;
esac
}
ap_child_preview() {
local a out=""
out="$(ap_launcher_preview) "
for a in ${AP_WORDS[@]+"${AP_WORDS[@]}"}; do out+="$(printf '%q' "$a") "; done
printf '%s-- pi <pi args> <prompt>' "$out"
}
# --------------------------------------------------------------------------
# the pipeline library, and the four things that change when the agent is remote
# --------------------------------------------------------------------------
ap_find_lib() {
local -a c=()
[[ -n "$AISB_AP_LIB" ]] && c+=("$AISB_AP_LIB")
[[ -n "$AISB_AP_SKILL_DIR" ]] && c+=("$AISB_AP_SKILL_DIR/scripts/lib.sh")
c+=("$HOME/.agents/skills/phased-execution/scripts/lib.sh"
".agents/skills/phased-execution/scripts/lib.sh"
"$HOME/.pi/agent/skills/phased-execution/scripts/lib.sh")
local f
for f in "${c[@]}"; do
[[ -f "$f" && -r "$f" ]] && { printf '%s\n' "$f"; return 0; }
done
return 1
}
AP_LIB="$(ap_find_lib)" || ap_die "the phased-execution library (lib.sh) was not found.
Looked in: \$AISB_AP_LIB, \$AISB_AP_SKILL_DIR/scripts/lib.sh,
~/.agents/skills/phased-execution/scripts/lib.sh, .agents/skills/..., ~/.pi/agent/skills/...
Install the phased-execution skill, or point --lib at its scripts/lib.sh."
# shellcheck source=/dev/null
source "$AP_LIB"
# The library owns the pipeline. This script shadows exactly four of its functions --
# the ones that assume the agent is a local process -- and nothing else, so unit order,
# validation, retries, provider-outage backoff, report recovery, the scoped phase
# commit and the notifications all stay the library's.
#
# pi() the library calls `pi` once per child attempt; this turns that
# call into a container. The child contract is unchanged: stdout is
# still the JSON stream progress.mjs reads, and the exit code is
# still pi's own, because ai-sandbox propagates the in-container
# command's status and its own chatter goes to stderr.
# first_prompt() the agent also has to know what it cannot do in there.
# fix_prompt()
# run_validation() the gate runs where the agent runs.
# This happens after the launcher is resolved, and that order is not cosmetic: one of
# the names installed here is `pi`, and a launcher called `pi` has to resolve to the
# user's command rather than to the shim that wraps it.
ap_install_overrides() {
# Every push goes through ap_push_pending, including the library's own: it commits at
# the phase's commit point and then calls push_phase, so shadowing that function is
# what puts the fingerprint guard, the credential redaction and the error contract in
# front of all of it. Requiring it is deliberate -- if the library ever stops calling
# it, this script must find out here rather than quietly pushing around the guard.
for _ap_fn in find_root next_unit phase_entries phase_next_unit unit_phase is_phase_end \
count_units sweep_finished_phases execute_unit unit_report notify_task \
commit_phase push_phase; do
declare -F -- "$_ap_fn" >/dev/null 2>&1 \
|| ap_die "$AP_LIB does not provide $_ap_fn(): it is not the phased-execution library, or it has changed shape. Refusing to run a half-wired pipeline."
done
for _ap_fn in first_prompt fix_prompt; do
declare -F -- "$_ap_fn" >/dev/null 2>&1 \
|| ap_die "$AP_LIB does not provide $_ap_fn(): refusing to run a pipeline whose prompts it cannot build."
# the library's version, kept under its own name so the wrappers below can extend it
eval "ap_lib_$_ap_fn ()$(declare -f -- "$_ap_fn" | tail -n +2)"
done
unset _ap_fn
AP_SANDBOX_NOTES='
## Sandbox rules for this run (ai-sandbox)
You are running inside a rootless podman container. This directory is the only
writable mount and your home is a sandbox home, not the host one.
* git works locally: git status, git diff, git log and git blame are available and the
repository is the real one. The git REMOTE is unreachable from here, and that is
policy rather than a fault to fix: git push, git fetch, git pull, git ls-remote and
git clone <url> cannot work, so never attempt them and never report them as a
failure of the task. The harness commits and pushes on the host when the phase
completes; a task is delivered by leaving the work in the tree and green.
* Egress is limited to an allowlist you cannot change. A connection that is refused or
times out to anything outside it is the policy working: say so in your report rather
than routing around it.
* No root, no package manager, no system services. Use the toolchain in the image
(node, python3, make, gcc, git, jq) and the project own vendored dependencies.
* .agents/validate.sh, .agents/phases/ and .agents/reports/ belong to the harness.
Never edit the gate to make it pass.
'
# Every child starts with its stdin closed. `--no-interactive` in the sandbox flags is
# what normally keeps podman from attaching the terminal, but a launcher is the user's
# code and can drop that flag -- and ai-sandbox attaches stdin when it has a terminal,
# which is how a run ends up waiting for keystrokes behind a pipe that shows nothing.
# /dev/null is the only honest stdin for an unattended run.
pi() { "${AP_RUN[@]}" -- pi "$@" </dev/null; }
first_prompt() {
local text
text="$(ap_lib_first_prompt "$@")" || return 1
printf '%s\n' "$text"
[[ "$AISB_AP_NOTES" == 1 ]] && printf '%s\n' "$AP_SANDBOX_NOTES"
return 0
}
fix_prompt() {
local text
text="$(ap_lib_fix_prompt "$@")" || return 1
printf '%s\n' "$text"
[[ "$AISB_AP_NOTES" == 1 ]] && printf '%s\n' "$AP_SANDBOX_NOTES"
return 0
}
run_validation() { # <logfile>
if [[ "$AISB_AP_VALIDATE" == host ]]; then
bash .agents/validate.sh >"$1" 2>&1 </dev/null
else
"${AP_RUN[@]}" -- bash .agents/validate.sh >"$1" 2>&1 </dev/null
fi
}
push_phase() { ap_push_pending "phase ${1:-?}"; } # <- guarded, redacted, honest
}
# --------------------------------------------------------------------------
# git, on the host, in between the runs
# --------------------------------------------------------------------------
# The agent has the repository: git status/diff/log are how it sees what it changed. But
# the COMMIT happens out here, and git is not a passive reader of a repository -- it runs
# what the repository tells it to. So two things:
#
# 1. host-side git starts with the knobs that execute things turned off: no system
# config, no pager, no prompt, no ext:: transport, and an empty hooks directory
# instead of .git/hooks;
# 2. the two paths that carry them (.git/config and .git/hooks/) are fingerprinted
# before the first unit and re-checked before every delivery. If a sandboxed run
# edited either, the run stops: a pre-commit hook written in there is the sandbox
# executing code on the host, and refusing to commit is cheap next to that.
# --trust-git-changes is the way through when the edit was yours.
AP_EMPTY_HOOKS=""
GIT_FINGERPRINT0=""
GIT_FINGERPRINT_MSG=""
ap_git_env() {
AP_EMPTY_HOOKS="$(mktemp -d "${TMPDIR:-/tmp}/aisb-ap-hooks.XXXXXX")" \
|| ap_die "could not create the empty hooks directory that keeps .git/hooks out of this run"
export GIT_CONFIG_NOSYSTEM=1 GIT_PAGER=cat GIT_TERMINAL_PROMPT=0
export GIT_CONFIG_COUNT=2
export GIT_CONFIG_KEY_0=core.hooksPath GIT_CONFIG_VALUE_0="$AP_EMPTY_HOOKS"
export GIT_CONFIG_KEY_1=protocol.ext.allow GIT_CONFIG_VALUE_1=never
}
ap_git_fingerprint() { # -> hash of the repo's local config and hook files, or nothing
local gd
gd="$(git rev-parse --git-dir 2>/dev/null)" || return 0
[[ -n "$gd" ]] || return 0
{ git config --local --list 2>/dev/null
find "$gd/hooks" -type f -printf '%p %s\n' 2>/dev/null | sort
find "$gd/hooks" -type f -exec sha1sum {} + 2>/dev/null | sort
} | sha1sum | cut -d' ' -f1
}
# 0 when nothing dangerous moved; 1 with the reason when it did.
ap_git_guard() {
[[ -n "$GIT_FINGERPRINT0" ]] || return 0
[[ "$AISB_AP_TRUST_GIT" == 1 ]] && return 0
local now
now="$(ap_git_fingerprint)"
[[ -z "$now" || "$now" == "$GIT_FINGERPRINT0" ]] && return 0
GIT_FINGERPRINT_MSG="$(git rev-parse --git-dir 2>/dev/null)"
return 1
}
ap_git_guard_fail() {
printf '✗ ERROR: refusing to deliver (commit/push) -- the sandbox changed this repository'\''s own git configuration.\n' >&2
printf ' A repository runs .git/config and .git/hooks/, it does not merely record them: a hook\n' >&2
printf ' written inside the sandbox would execute on the host the moment this script committed.\n' >&2
printf ' look at it by hand: git config --local --list; ls -l "%s/hooks"\n' "${GIT_FINGERPRINT_MSG:-.git}" >&2
printf ' if the change was yours: re-run with --trust-git-changes\n' >&2
}
ap_push_fail() { # <git output> <command> <fatal>
local out="$1" cmd="$2" fatal="$3"
printf '✗ ERROR: %s\n' "the push FAILED. The commit is local and safe; the remote does not have it." >&2
printf '%s\n' "$out" | ap_redact | sed 's/^/ /' >&2
printf ' finish by hand: %s\n' "$cmd" >&2
printf ' nothing is lost: the next run pushes it again once the remote answers.\n' >&2
return "$fatal"
}
# ap_push_pending <context> [fatal] -- idempotent: pushes what the remote is missing.
ap_push_pending() {
local context="$1" fatal="${2:-1}"
if ! ap_git_guard; then
ap_git_guard_fail
return "$fatal"
fi
if [[ "$AISB_AP_PUSH" != 1 ]]; then
echo " (push disabled by --no-push: commits stay local)"
return 0
fi
if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
echo " (not a git repository -- nothing to push)"
return 0
fi
local remote branch ahead out
remote="$(git remote 2>/dev/null | head -n1)"
if [[ -z "$remote" ]]; then
echo " (no git remote configured -- commits stay local)"
return 0
fi
branch="$(git symbolic-ref --short -q HEAD || true)"
if [[ -z "$branch" ]]; then
printf '✗ ERROR: %s -- detached HEAD, nothing to push.\n' "$context" >&2
printf ' the commits are local; check out a branch and push them.\n' >&2
return "$fatal"
fi
if git rev-parse --abbrev-ref --disambiguate '@{u}' >/dev/null 2>&1; then
ahead="$(git rev-list --count '@{u}..HEAD' 2>/dev/null || echo 0)"
if [[ "$ahead" == "0" ]]; then
echo " (up to date with $remote/$branch)"
return 0
fi
ap_log "pushing $ahead unpushed commit(s) on $branch ($context)"
if ! out="$(git push 2>&1)"; then
ap_push_fail "$out" "git push" "$fatal"
return $?
fi
else
ap_log "pushing $branch to $remote (no upstream set yet)"
if ! out="$(git push -u "$remote" "$branch" 2>&1)"; then
ap_push_fail "$out" "git push -u $remote $branch" "$fatal"
return $?
fi
fi
echo " (pushed: $(git log -1 --oneline))"
return 0
}
# What the host intends to do with git. Printed by --status and --dry-run, neither of
# which contacts the remote: 'unpushed' is computed from local refs only.
ap_git_plan() {
if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
printf 'git: not a repository -- the pipeline leaves its work in the tree\n'
return 0
fi
local branch remote ahead="n/a"
branch="$(git symbolic-ref --short -q HEAD || echo '(detached)')"
remote="$(git remote 2>/dev/null | head -n1)"
if [[ -n "$remote" ]] && git rev-parse --abbrev-ref --disambiguate '@{u}' >/dev/null 2>&1; then
ahead="$(git rev-list --count '@{u}..HEAD' 2>/dev/null || echo '?')"
fi
# The remote's NAME is resolved by name, never printed as a URL: one with a
# credential in it would land in the log and the session output.
printf 'git: branch=%s remote=%s unpushed=%s\n' "$branch" \
"$(if [[ -n "$remote" ]]; then git remote get-url "$remote" 2>/dev/null | ap_redact; else printf 'none'; fi)" "$ahead"
printf 'git: commit and push happen on the host, between phase runs; the agent cannot reach the remote and is not given it\n'
}
# --------------------------------------------------------------------------
# preflight: one throwaway container, before any agent tokens are spent
# --------------------------------------------------------------------------
ap_preflight() {
local out got="" line errf rc=0
ap_log "preflight: one sandboxed shell, to prove the launcher, the image and the policy"
errf="$(mktemp)"
# </dev/null for the same reason as pi() below: this call has the terminal on its stdin
# and the answer in a pipe, so anything that reads stdin inside the container would sit
# there forever with nothing on screen to explain it.
out="$("${AP_RUN[@]}" -- bash -c 'printf "aisb-ok aisb=%s git=%s user=%s cwd=%s\n" "${AISB:-none}" "$(git --version 2>/dev/null | cut -d" " -f3 || echo none)" "$(id -un)" "$PWD"' 2>"$errf" </dev/null)" || rc=$?
# The answer is the marker line, wherever it turned up. stdout inside the container belongs
# to the command the driver asked for and the sandbox's own notices belong on stderr; but an
# image built before that was settled, or a launcher that echoes, can still put a line in
# front. Name that loudly -- the same stream is read as data by progress.mjs in every unit
# -- but do not refuse a sandbox that answered for it. The refusal is for a non-zero exit,
# for silence, and for a launcher that runs nothing.
while IFS= read -r line; do
[[ "$line" == aisb-ok* ]] && { got="$line"; break; }
done <<< "$out"
if (( rc != 0 )) || [[ -z "$got" ]]; then
printf '%s: preflight failed (exit %s): the launcher does not give a working sandbox.\n' "$PROG" "$rc" >&2
printf ' command: %s\n' "$(ap_child_preview)" >&2
tail -n 12 "$errf" | sed 's/^/ | /' >&2
if (( AP_ARG_FORWARD == 0 )); then
printf ' %s is a function that ignores its arguments, so the flags above and everything after -- were dropped\n' "$AP_LAUNCHER" >&2
printf ' and the container ran the image default harness with no prompt instead of the shell asked for. Forward them:\n' >&2
printf ' %s() { ai-sandbox --allow ... --env NAME "$@"; }\n' "$AP_LAUNCHER" >&2
elif [[ -z "${out//[[:space:]]/}" ]]; then
printf ' the sandbox started and answered nothing at all: a launcher that runs no command, or one\n' >&2
printf ' that prints its answer somewhere other than stdout, fails exactly like this. It has to run\n' >&2
printf ' follows the -- it was given (a function needs "$@" at the end; see --help).\n' >&2
else
printf ' stdout carried no aisb-ok answer at all; this is what came back instead:\n' >&2
printf '%s\n' "$out" | awk -v max=6 '{ if (++n <= max) printf " > %s\n", $0 }' >&2
fi
rm -f "$errf"
ap_die "Fix the launcher or the sandbox before running the pipeline: every unit would fail this way. (--no-preflight to skip the check.)"
fi
rm -f "$errf"
printf ' %s\n' "$got"
if [[ "$out" != "$got" ]]; then
ap_warn "the sandbox put lines of its own on stdout before answering. That stream is data: the agent"
ap_warn "stream is parsed from it, and so is this check. Notices belong on stderr -- rebuild the"
ap_warn "image if they read [aisb], or move any echo in your launcher there. Continuing: it answered."
printf '%s\n' "$out" | awk -v max=4 '!/^aisb-ok/ { if (++n <= max) printf " > %s\n", $0 }' >&2
fi
if [[ "$got" == *"git=none"* ]]; then
ap_warn "no git inside the sandbox -- the agent cannot read its own diff. Add it to the image (--install git) or expect blind fixer runs."
fi
return 0
}
# --------------------------------------------------------------------------
# modes
# --------------------------------------------------------------------------
ap_gate_preview() {
local a out=""
out="$(ap_launcher_preview) "
for a in ${AP_WORDS[@]+"${AP_WORDS[@]}"}; do out+="$(printf '%q' "$a") "; done
printf '%s-- bash .agents/validate.sh' "$out"
}
ap_report_config() {
# The launcher line is printed through the same redaction as everything else: a
# --pi-cmd may carry a literal --env value, and nothing that prints repeats a value.
printf 'launcher : %s\n' "$(printf '%s' "$AISB_AP_PI_CMD" | ap_redact)"
printf ' = %s, from %s\n' "$AP_KIND" "${AP_SOURCE:-nowhere}"
if (( AP_ARG_FORWARD == 0 )); then
printf ' ! %s drops the arguments appended to it; the child command cannot reach the\n' "$AP_LAUNCHER"
printf ' sandbox, so this run would fail the preflight. End the function with "$@".\n'
fi
printf 'child : %s\n' "$(ap_child_preview)"
if [[ "$AISB_AP_VALIDATE" == host ]]; then
printf 'gate : .agents/validate.sh, on the host (see --validate-in)\n'
else
printf 'gate : .agents/validate.sh, in the sandbox\n'
fi
printf 'pipeline : %s\n' "$AP_LIB"
printf 'attempts : MAX_FIX_ATTEMPTS=%s, PI_TRUST=%s, FRESH_FIX=%s, QUIET=%s, PHASE_COMMIT=%s\n' \
"${MAX_FIX_ATTEMPTS:-3}" "${PI_TRUST:-0}" "${FRESH_FIX:-0}" "${QUIET:-0}" "${PHASE_COMMIT:-1}"
}
# The pending units of one phase, in the library's execution order: the task files
# sorted, then 00_phase.md as the final pass. next_unit() reports the first pending unit
# rather than consuming it, so a listing cannot be built from it -- it has to walk the
# same shape the scheduler does. --status compares the total against the library's own
# count_units(), which is what notices if that shape ever changes.
ap_phase_units() { # <phase>
local p="$1" f t
if [[ -d "$PHASE_TODO/$p" ]]; then
for f in "$PHASE_TODO/$p"/[0-9]*.md; do # globs, so a name cannot be mis-split
[[ -f "$f" ]] || continue
t="${f##*/}"; [[ "$t" == "00_phase.md" ]] && continue
printf '%s/%s\n' "$p" "$t"
done
[[ -f "$PHASE_TODO/$p/00_phase.md" ]] && printf '%s/00_phase.md\n' "$p"
elif [[ -f "$PHASE_TODO/$p" ]]; then
printf '%s\n' "$p"
fi
return 0
}
ap_list_units() {
local p u n=0
for p in $(phase_entries); do
while IFS= read -r u; do
[[ -n "$u" ]] || continue
printf ' %s\n' "$u"
n=$((n + 1))
(( n >= 10000 )) && break # a listing is never worth an unbounded loop
done < <(ap_phase_units "$p")
done
[[ "$n" == 0 ]] && printf ' (none -- the pipeline is complete)\n'
local libn
libn="$(count_units 2>/dev/null || echo '?')"
if [[ "$libn" != "?" && "$libn" != "$n" ]]; then
ap_warn "the library counts $libn pending unit(s) but this listing found $n -- its directory shape has changed; trust the library"
fi
return 0
}
ap_preview() {
ap_report_config
echo
echo "one container per unit attempt; the prompt travels as one argument:"
printf ' %s\n' "$(ap_child_preview)"
echo
echo "the gate:"
printf ' %s\n' "$(ap_gate_preview)"
echo
echo "pending units:"
ap_list_units
echo
ap_git_plan
echo
echo "Nothing was started: --dry-run builds no image, starts no container, fetches no"
echo "token and contacts no remote. The launcher was resolved by reading rc files only."
}
ensure_git_env() {
# Two switches mean the same thing here, so they are kept in step in both directions:
# PHASE_PUSH=0 in the environment means "do not deliver", and --no-push means the
# library must not push behind our back either.
[[ "${PHASE_PUSH:-1}" == 1 ]] || AISB_AP_PUSH=0
[[ "$AISB_AP_PUSH" == 1 ]] || export PHASE_PUSH=0
}
ap_run_pipeline() {
local unit phase last_phase="" last_unit="" n=0 rc=0
local -a delivered=()
sweep_finished_phases
ensure_git_env
command -v node >/dev/null 2>&1 || ap_die "node is required on the host to render task progress"
[[ "$AISB_AP_PREFLIGHT" == 1 ]] && ap_preflight
# Anything an earlier run failed to push goes out before new work. A failure here is
# a warning: the pipeline can still make progress locally and every later push takes
# the same commits up.
ap_log "checking for commits a previous run left unpushed"
ap_push_pending "start of run" 0 || ap_warn "some commits are still local -- continuing, and each phase push retries them"
while unit="$(next_unit)"; do
[[ -n "$unit" ]] || break
# next_unit() reports the first pending unit; the progress that removes it is
# execute_unit(). A unit that comes back twice has not been delivered, and looping
# on it would run the pipeline in place forever -- stop and say so.
if [[ -n "$unit" && "$unit" == "$last_unit" ]]; then
echo "✗ ERROR: $unit was reported pending again after completing -- nothing moved it to $PHASE_DONE/. Stopping rather than repeating it forever." >&2
return 1
fi
last_unit="$unit"
phase="$(unit_phase "$unit")"
if [[ -n "$last_phase" && "$phase" != "$last_phase" ]]; then
# In between phase runs: deliver the phase that just committed.
if ! ap_push_pending "between $last_phase and $phase"; then
echo "✗ ERROR: pipeline stopped -- the phase commit is not on the remote (see above)" >&2
return 1
fi
fi
rc=0
execute_unit "$unit" || rc=$?
if (( rc != 0 )); then
# execute_unit printed the detail: task failure, or commit/push failure with the
# hand-fix command.
echo "✗ ERROR: pipeline stopped -- $unit did not complete (see the error output above)" >&2
echo " re-run this script to continue where it stopped" >&2
return 1
fi
delivered+=("$unit")
last_phase="$phase"
n=$((n + 1))
if is_phase_end "$unit"; then notify_task "$unit" phase; else notify_task "$unit" task; fi
if (( AP_MAX_UNITS > 0 && n >= AP_MAX_UNITS )); then
ap_log "--max-units $AP_MAX_UNITS reached; $(count_units) unit(s) still pending"
break
fi
done
# Delivery is the point of the run, so a remote that will not answer makes the run a
# failure: exiting 0 with the work committed but unpushed is the silent miss this
# script exists to prevent. The commits are safe either way and the next run retries.
if ! ap_push_pending "end of run"; then
echo "✗ ERROR: the run delivered the work but could not push it (see above)" >&2
return 1
fi
echo
ap_log "✓ pipeline done -- ${#delivered[@]} unit(s) delivered this run: ${delivered[*]:-none}"
echo " remaining in $PHASE_TODO/: $(count_units) task(s)"
return 0
}
# One unit, or one phase to completion: the library's own entry points, with the
# between-run delivery around them.
ap_run_selected() {
local unit phase="$AP_PHASE" rc=0 n=0
sweep_finished_phases
ensure_git_env
command -v node >/dev/null 2>&1 || ap_die "node is required on the host to render task progress"
[[ "$AISB_AP_PREFLIGHT" == 1 ]] && ap_preflight
if [[ "$AP_MODE" == unit ]]; then
unit="$AP_UNIT"
[[ "$unit" == *.md ]] || unit="$unit.md"
[[ -f "$PHASE_TODO/$unit" ]] || ap_die "$unit not found in $PHASE_TODO/"
execute_unit "$unit" || rc=$?
if (( rc == 0 )); then
if is_phase_end "$unit"; then notify_task "$unit" phase; else notify_task "$unit" task; fi
fi
ap_push_pending "after $unit" || return 1
return "$rc"
fi
[[ -d "$PHASE_TODO/$phase" || -f "$PHASE_TODO/$phase.md" ]] \
|| ap_die "phase $phase not found in $PHASE_TODO/"
while unit="$(phase_next_unit "$phase")"; do
[[ -n "$unit" ]] || break
rc=0
execute_unit "$unit" || rc=$?
if (( rc != 0 )); then
echo "✗ ERROR: phase $phase did not complete (see the error output above)" >&2
return 1
fi
n=$((n + 1))
if is_phase_end "$unit"; then notify_task "$unit" phase; else notify_task "$unit" task; fi
done
ap_push_pending "after phase $phase" || return 1
ap_log "✓ phase $phase complete ($n unit(s))"
}
main() {
ap_resolve_launcher
ap_build_run
ap_install_overrides # after the resolve: see the note on ap_install_overrides
# Previews report it in their config block; a real run has no config block, so it hears
# it here instead -- before the first container, not after an hour of nothing.
[[ "$AP_MODE" == status || "$AP_MODE" == dryrun ]] || ap_warn_forwarding
# Preview modes stop here: no image build, no container, no state written, no
# network -- and no sweep of finished phase directories either, which writes.
local root
root="$(find_root)" || root=""
[[ -n "$root" ]] || ap_die "no .agents/phases/todo found in this or parent directories (run it inside a phased project)"
cd "$root" || ap_die "cannot enter $root"
if [[ "$AP_MODE" != status && "$AP_MODE" != dryrun ]]; then
ap_git_env # see the note above
GIT_FINGERPRINT0="$(ap_git_fingerprint)"
fi
case "$AP_MODE" in
status) ap_status ;;
dryrun) ap_preview ;;
unit|phase) ap_run_selected ;;
*) ap_run_pipeline ;;
esac
}
ap_status() {
ap_report_config
echo
echo "pending units:"
ap_list_units
echo
ap_git_plan
}
ap_interrupt() { # <code> <signal>
echo
printf '[%s] ✗ ERROR: interrupted (%s) -- the unit stays in the todo/ directory; re-run to continue where it stopped\n' "$(ap_ts)" "$2" >&2
exit "$1"
}
trap 'ap_interrupt 130 SIGINT' INT
trap 'ap_interrupt 143 SIGTERM' TERM
# the empty hooks directory exists only to keep .git/hooks out of our own git calls
trap '[ -z "$AP_EMPTY_HOOKS" ] || rm -rf "$AP_EMPTY_HOOKS"' EXIT
main "$@"
config/config.example
+32 -1
View File
@@ -19,12 +19,37 @@ net = allow
# used if it is busy.
# gate_port = 18081
# --- published ports (ingress) ----------------------------------------------
# Everything else here is egress. 'publish' opens a listener on the host that
# forwards into the container, so a browser on the host can look at a dev server the
# sandbox is running. Repeatable; shapes are 'port', 'host:container' and
# 'addr:host:container'. One port per entry: no ranges, no /udp.
#
# The cost is ingress: whatever that server serves, or accepts, is reachable from the
# address it is bound to. Egress is untouched either way.
# publish = 8000 # 127.0.0.1:8000 -> container 8000
# publish = 18080:3000 # 127.0.0.1:18080 -> container 3000
# publish = 0.0.0.0:18080:3000 # every interface -- warned about, do not commit it
#
# A server inside the container has to listen on 0.0.0.0 to be reachable this way:
# pasta delivers the connection to the container's own address, not to its loopback.
#
# The address a bare 'publish = 8000' binds on the host. Machine-wide, not per-project:
# leave it loopback in a repo config. Set it in this file only, and only if you know
# what is routed to you.
# publish_addr = 127.0.0.1
# --- harness ---------------------------------------------------------------
# Default in-container command: pi | codex | claude | none | "<your command>"
# harness = pi
# --- mounts and layout -----------------------------------------------------
# workdir = /project
# Where the current directory is mounted inside the container, which is also the
# working directory. 'auto' (the default) uses the path it has on the host, so the
# agent's pwd and the paths it prints are the ones you act on afterwards. It falls
# back to /project when the host path would cover something the image or the sandbox
# owns (/usr, /etc, /var, /opt, /tmp, /run, /home/sandbox) and announces the fallback.
# workdir = auto
# --- limits ----------------------------------------------------------------
# cpus = 4
@@ -62,6 +87,12 @@ net = allow
# pi_token_env = AI_API_TOKEN
# pi_token_auto = 1
#
# The thinking level a sandboxed pi reasons at: off, minimal, low, medium, high, xhigh
# or max. pi's own default is medium; ai-sandbox states xhigh and re-applies it on every
# run, so one run cannot lower the thinking of the next. 'host' stops forcing it and lets
# settings.json decide. pi only -- other harnesses read their own config.
# thinking = xhigh
#
# Where to read the host config from (default ~/.pi/agent).
# pi_agent_dir = ~/.pi/agent
contrib/ai-sandbox.bash
Executable
+122
View File
@@ -0,0 +1,122 @@
# bash completion for ai-sandbox.
#
# Install (either):
# mkdir -p ~/.local/share/bash-completion/completions
# cp contrib/ai-sandbox.bash ~/.local/share/bash-completion/completions/ai-sandbox
# (it also registers ai-sandbox-autophase, which shares this file)
# or source it from ~/.bashrc -- put the `source` line *after* the alias, since
# bash needs `complete` registered under the alias name (`alias pi=ai-sandbox`).
#
# The flag list mirrors parse_args() in bin/ai-sandbox -- keep them in sync.
# compgen prints one candidate per line; reading them instead of dropping them through an
# unquoted $(...) keeps a candidate that contains a space (a file name, a phase directory)
# in one piece. Process substitution, not a pipe: COMPREPLY has to survive the call.
_aisb_fill() { local w; while IFS= read -r w; do COMPREPLY+=("$w"); done; }
_ai_sandbox_completions() {
local cur prev word first="" sub="" i
if type _get_comp_words_by_ref >/dev/null 2>&1; then
_get_comp_words_by_ref -n : cur prev
else
cur="${COMP_WORDS[COMP_CWORD]}"; prev="${COMP_WORDS[COMP_CWORD-1]:-}"
fi
local subs="build update doctor gate ps clean inspect version help"
# flags that consume the next word, so their values are not mistaken for
# positional (harness) arguments.
local value_flags=" --net --allow --allow-file --proxy --publish --mount --workdir --agents-dir
--env --env-file --base-image --install --harness --pi-token-env --cpus --memory
--pids --tmpfs-size --name --label "
# Walk the words before the cursor to find the subcommand, skipping option values.
local skip=0
for ((i = 1; i < COMP_CWORD; i++)); do
word="${COMP_WORDS[i]}"
if (( skip )); then skip=0; continue; fi
if [[ "$word" == --* && "$value_flags" == *" $word "* ]]; then skip=1; continue; fi
[[ "$word" == -* ]] && continue
[[ -z "$first" ]] && first="$word"
done
[[ " $subs " == *" $first "* ]] && sub="$first"
case "$prev" in # values of the flag just typed
--net) COMPREPLY=(); _aisb_fill < <(compgen -W "none allow full" -- "$cur"); return ;;
--harness) COMPREPLY=(); _aisb_fill < <(compgen -W "pi codex claude none" -- "$cur"); return ;;
--env) COMPREPLY=(); _aisb_fill < <(compgen -A export -- "$cur"); return ;;
--allow) # hosts already in the allow files; no DNS, no network
local hosts
hosts="$(sed 's/#.*//' "$HOME/.config/ai-sandbox/allow" ./.ai-sandbox/allow 2>/dev/null | tr ',[:space:]' '\n\n')"
COMPREPLY=(); _aisb_fill < <(compgen -W "$hosts" -- "$cur")
(( ${#COMPREPLY[@]} )) || COMPREPLY=( "$cur" ) # don't fall back to filenames
return ;;
--proxy|--publish) COMPREPLY=( "$cur" ); return ;; # values are ours to type, not to guess
--allow-file|--env-file) COMPREPLY=(); _aisb_fill < <(compgen -f -- "$cur"); return ;;
--workdir) COMPREPLY=(); _aisb_fill < <(compgen -W "auto" -- "$cur") # 'auto' mirrors the host path
(( ${#COMPREPLY[@]} )) || _aisb_fill < <(compgen -d -- "$cur"); return ;;
--mount|--agents-dir) COMPREPLY=(); _aisb_fill < <(compgen -d -- "$cur"); return ;;
esac
# A non-flag word before the cursor that is not a subcommand means the rest goes
# to the harness: ai-sandbox stops parsing there, so offer nothing, not sandbox flags.
[[ -n "$first" && -z "$sub" ]] && return 1
[[ -z "$first" && "$cur" != -* ]] && { COMPREPLY=(); _aisb_fill < <(compgen -W "$subs" -- "$cur"); return; }
# subcommand-specific flags (each subcommand dies on anything else);
# `inspect` takes the full policy flag set, so it falls through to the list below.
case "$sub" in
build) COMPREPLY=(); _aisb_fill < <(compgen -W "--rebuild -f --refresh-base -B --gate" -- "$cur"); return ;;
update) COMPREPLY=(); _aisb_fill < <(compgen -W "--check --no-gate" -- "$cur"); return ;;
doctor) COMPREPLY=(); _aisb_fill < <(compgen -W "--deep -d" -- "$cur"); return ;;
clean) COMPREPLY=(); _aisb_fill < <(compgen -W "--all -a --volumes -V" -- "$cur"); return ;;
gate) COMPREPLY=(); _aisb_fill < <(compgen -W "up down status logs" -- "$cur"); return ;;
ps|version) return 1 ;;
esac
COMPREPLY=()
_aisb_fill < <(compgen -W \
"--net --allow --allow-file --proxy --no-gate --publish --mount --no-cwd --workdir --home-cache --ephemeral-home --read-only --agents --no-agents --agents-dir --env --env-file --base-image --install --harness --pi-token-env --no-pi-token-env --no-pi-config --cpus --memory --pids --tmpfs-size --shell --interactive --no-interactive --no-tty --keep --name --label --no-build --dry-run --print-policy --print-entrypoint --print-containerfile --allow-unenforced --insecure-keep-root --strict-args -h --help -V" \
-- "$cur")
}
# ai-sandbox-autophase: the pipeline driver. Its own flag list, mirroring its parser.
_ai_sandbox_autophase_completions() {
local cur
if type _get_comp_words_by_ref >/dev/null 2>&1; then
_get_comp_words_by_ref -n : cur
else
cur="${COMP_WORDS[COMP_CWORD]}"
fi
case "$cur" in
--validate-in=*) COMPREPLY=(); _aisb_fill < <(compgen -W "sandbox host" -- "${cur#*=}"); return ;;
esac
COMPREPLY=()
_aisb_fill < <(compgen -W \
"--pi-cmd --rc-files --lib --skill-dir --sandbox-args --validate-in --no-push --no-preflight --no-notes --trust-git-changes --unit --phase --max-units --status --dry-run -n -h --help -V" \
-- "$cur")
# --unit and --phase name pending work: offer the phase directories first, then the
# ordinary file completion the caller would otherwise have got.
local todo="${PWD}/.agents/phases/todo"
case "${COMP_WORDS[COMP_CWORD-1]}" in
--unit|--phase)
if [[ -d "$todo" ]]; then
local d
for d in "$todo"/"$cur"*; do
[[ -e "$d" ]] && COMPREPLY+=("${d#"$todo"/}") # named as the driver wants them
done
_aisb_fill < <(compgen -f -- "$cur")
fi ;;
esac
}
complete -o default -F _ai_sandbox_completions ai-sandbox 2>/dev/null || true
complete -o default -F _ai_sandbox_autophase_completions ai-sandbox-autophase 2>/dev/null || true
if [[ $- == *i* ]]; then # also complete the harness alias
for _aisb in $(compgen -A alias 2>/dev/null); do
if alias "$_aisb" 2>/dev/null | grep -q 'ai-sandbox'; then
complete -o default -F _ai_sandbox_completions "$_aisb" 2>/dev/null || true
fi
done
unset _aisb
fi
docs/networking.md
+41
View File
@@ -50,6 +50,43 @@ domains you allowlisted, resolved by the gate itself. CDNs, SNI and CONNECT-time
filtering all work because squid terminates the CONNECT request and resolves the
name itself.
## Publishing a port (ingress)
`--publish` is the one control that runs the other way: podman/pasta puts a listener on the
host and forwards what arrives into the container, so a client on the host can reach a server
the sandboxed process listens on. Nothing about the egress model changes -- the same run that
answers your browser still refuses every domain outside the allowlist
(`tests/e2e/13_publish.sh` asserts exactly that, in one container).
Binding and parsing decisions, and why:
| decision | reason |
|---|---|
| default bind address `127.0.0.1` | measured: a bare `-p 8000:8000` binds `*:8000`, i.e. every interface. A port in a committed config must not quietly be a port on the LAN. `publish_addr` or an explicit `addr:` changes it, and a non-loopback address warns. |
| one port per entry, no ranges, no `/udp` | this list is a policy a human reviews; a range is a line nobody reads. |
| `*` ("every interface") is refused; `0.0.0.0` is accepted | podman's shorthand hides what it opens. Making someone type the address is the whole warning. |
| an address requires three parts | `0.0.0.0:8000` means two different things in podman's grammar. Refusing it with a message beats guessing. |
| `--net none` + `--publish` refuses | measured: podman prints one line about discarding the mapping and starts the container regardless. "It ran" with nothing listening is the worst outcome here. |
| a busy host port refuses the run | measured before the container starts, because podman's own error arrives after any build. |
| the ruleset accepts the published ports on the `input` hook | see below: it changes nothing on the reference host, and it is the audit trail on every host. |
Where the packet actually arrives was the interesting measurement. The `input` chain is
`policy drop`, so a port forward looks doomed from the reading of it -- and it is not:
| Probe | Result | Consequence |
|---|---|---|
| `--net allow --publish H:8000`, server on `0.0.0.0`, `curl` from the host | **200**, with `enforce=nft` in the same run | the existing ruleset does not block a published port |
| `nft -a list chain inet aisb input` while that port serves traffic | `aisb-publish` counter: **0 packets** | pasta forwards the connection into the namespace over **loopback**, so `iifname "lo" accept` carries it |
| server bound to `127.0.0.1` **inside**, port published | host: connection refused; inside: 200 | pasta delivers to the container's own address, not to its loopback → dev servers must bind `0.0.0.0` |
| `ss -ltnp` on the host while published | `LISTEN 127.0.0.1:H ... pasta.avx2` | the listener is pasta's, on loopback, and it is gone when the container exits |
| container's own view of its interface | `wlp4s0` with the **host's** LAN address | pasta mirrors the host address into the namespace, which is why a *remote* host cannot reach the published port: only pasta's own socket on the host accepts, and it accepts on the address we named |
So `publish_rules()` in the entrypoint is deliberately not what makes publishing work here.
It is there because (a) a different host, or a bridge network, can deliver the packet on the
container's own interface, where `policy drop` would leave a port that looks configured and
answers nothing, and (b) `nft list table inet aisb` should answer "what can reach this
container" instead of leaving that question to a flag on the host.
## Measurements
| Probe | Result | Consequence |
@@ -60,6 +97,8 @@ name itself.
| `--network=host` in a rootless container | works, has internet | makes the host-loopback gate possible |
| `--network=pasta` (default), egress | works | pasta is a plain internet tap, no filtering |
| container → host `127.0.0.1` via `--map-host-loopback,100.64.0.1` | **works** | the gate address |
| host → container via `-p 127.0.0.1:H:C`, `--net allow`, ruleset installed | **works**; the `input` hook sees 0 packets for it | ingress arrives on `lo` — see "Publishing a port" above |
| `-p H:C` with no address | binds `*:H` on the host | the default bind address in `ai-sandbox` is `127.0.0.1`, not podman's |
| `--map-host-loopback,127.0.0.1` | rejected by passt (`Invalid address to remap to host`) | the magic address must not itself be loopback |
| `--network=pasta:--map-gw` | works (the gateway address already maps to the host by default) | `--map-host-loopback` chosen for explicitness |
| squid as root with `--cap-drop=ALL` | `setgid: (1) Operation not permitted`, gate dies | run the gate as `proxy` from the start |
@@ -72,6 +111,8 @@ name itself.
| `--pids-limit 64 --memory 256m --cpus 0.5` | `pids.max=64`, `memory.max=268435456` visible inside | limits are real (note: `/proc/cpuinfo` still lists all cores; use `nproc`/cgroup limits, not cpuinfo) |
| `-v $dir:/project` with SELinux enforcing, `--userns=keep-id:uid=1000,gid=1000` | write **denied** | bind mounts need `:z` on SELinux hosts |
| same, with `:z` | write succeeds, file owned by the host uid (1000:1000) | project files keep sane ownership |
| `--tmpfs /tmp` **and** `-v /tmp/x:/tmp/x` in one run | the tmpfs wins: `mount` shows `tmpfs on /tmp/x`, `pwd` prints `/tmp/x`, writes succeed | a project mounted at its own path under `/tmp` would be an empty, writable, correctly-named directory that is not the project — so `mirror_block_of()` refuses `/tmp` rather than degrading quietly |
| `-v $dir:$dir` (a per-user path) beside the `/tmp` tmpfs, the `/home/sandbox` volume and the `.agents` share | `pwd` is the host path, writes land on the host owned by the host uid, `/project` + the home volume + the share all still there | mirroring buries nothing outside the denied set; asserted per release in `tests/e2e/12_sandbox.sh` |
## Why not the other designs
docs/recipes.md
+182 -3
View File
@@ -50,12 +50,37 @@ pi --allow aipi.reeseapps.com --pi-token-env GW_TOKEN
# pass it yourself, which is what you want in shared scripts
pi --allow aipi.reeseapps.com --env AI_API_TOKEN --no-pi-token-env
# no bridge: pi starts with its own defaults inside the volume
# no bridge: pi starts with its own defaults inside the volume (and no thinking policy)
pi --no-pi-config --allow aipi.reeseapps.com
```
Edits made inside the sandbox are kept; changes to the host config are picked up on
the next run; deleting `~/.pi/agent/models.json` inside the sandbox re-seeds it.
`models.json` is kept in step with the host unless you edit it in the sandbox, where your
edit then wins; deleting it inside the sandbox re-seeds it. `settings.json` is merged per
key, so a preference you changed in the sandbox stays changed and every one you did not
touch follows the host again.
### How hard it thinks
`defaultThinkingLevel` is policy, not preference: `xhigh` unless you say otherwise, and
re-applied every run rather than remembered from whatever the last sandbox set.
```bash
# the everyday case: xhigh
pi --allow aipi.reeseapps.com
# one-off: the top of pi's range, or no reasoning at all for a mechanical job
pi --thinking max --allow aipi.reeseapps.com
pi --thinking off --net allow --allow registry.npmjs.org -- npm ci
# let settings.json decide, the way it did before this was policy
pi --thinking host --allow aipi.reeseapps.com
# per repo, chosen by .ai-sandbox/config
# thinking = high
```
Nothing is written for `codex`, `claude` or a command after `--`: those read their own
configuration and ai-sandbox does not touch it.
## Your skills and standing instructions
@@ -134,6 +159,53 @@ pi --allow registry-1.docker.io --allow auth.docker.io --allow production.cloudf
pi --allow aipi.reeseapps.com --allow 10.20.0.5:5432
```
## Ports: two directions, two answers
**Outbound — a service on your machine** (dev postgres, redis, a mock API, a local model
server). That is egress, so it is an allow entry, not a flag. `100.64.0.1` is the address that
stands for the host's loopback inside the container:
```ini
# .ai-sandbox/allow
100.64.0.1:5432 # postgres from `docker compose up db`, on the host
# .ai-sandbox/config -- point the app at the mapped address (no secret is a URL like this)
env = DATABASE_URL=postgresql://dev@100.64.0.1:5432/app_dev
# HTTP on anything but 80/443/8443 also has to skip the proxy, or the gate is asked for it:
env = NO_PROXY=100.64.0.1,localhost,127.0.0.1,::1
env = no_proxy=100.64.0.1,localhost,127.0.0.1,::1
```
**Inbound — a server the sandbox runs, to be seen from the host.** Publish it. Bound to
loopback by default, so `http://127.0.0.1:8000` in your browser works and nothing on your
network can reach it:
```bash
# the one-off preview: the agent runs the server, you look at it
ai-sandbox --allow aipi.reeseapps.com --publish 8000 -- npm run dev -- --host 0.0.0.0
# a host port that is free, a different container port
ai-sandbox --publish 18080:3000 -- npm run dev -- --host 0.0.0.0
# from a project's own config, so `ai-sandbox` alone is the invocation
# .ai-sandbox/config: publish = 8000
# publish = 18080:3000
```
Four things worth knowing before it looks broken (all measured, all in
[`networking.md`](networking.md)):
* the server must listen on `0.0.0.0` **inside** — `--host 0.0.0.0`, `uvicorn --host
0.0.0.0`, `runserver 0.0.0.0:8000`. A `127.0.0.1`-only server answers inside and refuses the
published port.
* egress is unaffected. A published port is ingress: the run keeps its allowlist, its
unprivileged uid and its ruleset, and the ruleset names the ports it opened.
* `--net none` refuses to publish rather than start with nothing listening; a host port that
is already taken is named before the container starts, not after the build.
* a non-loopback bind (`publish = 0.0.0.0:18080:3000`) means exactly what it says: anything that
can route to this machine can reach that server. It warns, and it does not belong in a
committed config — that file outlives the run that justified it.
## Git
```bash
@@ -211,3 +283,110 @@ registry.npmjs.org
Then `pi` alone in that directory is the whole invocation — and because both files
are in the repository, the sandbox policy gets reviewed with the code.
## An unattended pipeline: phases in the sandbox, pushes on the host
The nightly case: `.agents/phases/todo/` full of task files, nobody watching, an
agent that must not be able to reach the git remote or touch the host.
```bash
# ~/.bashrc.d/ai-sandbox.bash -- the launcher, the way you already write it, plus the
# "$@" that lets a run hand it the command to sandbox
sandbox-pi () {
export AI_API_TOKEN=$(pass aipi/dev-token)
ai-sandbox --allow aipi.reeseapps.com --allow github.com \
--allow pi.dev --allow release-assets.githubusercontent.com \
--env AI_API_TOKEN "$@"
}
```
A function written for typing by hand usually stops at `--env AI_API_TOKEN`, and that is
the one thing the driver cannot work with silently: everything after the function name is
the run, so dropping it starts the image's default harness with no prompt. With a terminal
behind it that is an interactive agent waiting for input, its output going into a pipe the
driver is reading — a run that appears to do nothing at all. `--status` names a launcher
that takes no arguments and the preflight refuses it; the fix is the `"$@"` above.
```bash
cd ~/Projects/foo
ai-sandbox-autophase --status # what it resolved, what is pending, the git plan
ai-sandbox-autophase --dry-run # the exact child and gate commands, nothing run
ai-sandbox-autophase # every pending task, in pipeline order
```
What it does with that function is the interesting part. A script cannot see an
interactive shell's functions, so it reads the rc files, takes the *definition*, and
starts each child as:
```
sandbox-pi --no-tty --no-interactive -- pi --no-approve --session-dir .agents/phase-sessions --name <unit> --mode json <prompt>
```
`pass` runs again per unit, so a run that starts at 23:00 is not holding a token
fetched at 09:00. Reading the definition is all the probe does: `--status` and
`--dry-run` never call the launcher, fetch a token, build an image or start a
container. It reads it in a session of its own with no stdin — an interactive bash
started from a terminal tries to take that terminal, gets stopped for trying, and a
stopped process does not take the timeout meant for it. Every child is started with its
stdin on `/dev/null` for the same reason in the other direction: nothing inside the
sandbox can be waiting for your keyboard.
| you want | flag |
|---|---|
| a different launcher | `--pi-cmd sandbox-claude`, `--pi-cmd pi`, or a whole command line |
| the function somewhere else | `--rc-files ~/.bashrc.d/zz-ai.bash` |
| fewer flags of its own | `--sandbox-args ''` (the defaults are `--no-tty --no-interactive`) |
| the gate on the host, because the toolchain lives there | `--validate-in host` (warns: the gate file is in the writable mount) |
| one task, one phase, a short run | `--unit 03_api/02_routes.md`, `--phase 03_api`, `--max-units 4` |
| commits that stay local while you watch | `--no-push` |
| no warm-up container (it costs a build the first time) | `--no-preflight` |
Where the work goes when it finishes: the executor agents never commit; the driver
commits each completed phase on the host and pushes it, checks again before starting
the next phase, and pushes whatever is left at the end of the run. A failed push
stops the run and prints the command that finishes it — the commit is never lost, and
the next run's push sweeps it up:
```
[2026-10-28 02:14:07] pushing 1 unpushed commit(s) on main (between 03_api and 04_docs)
(pushed: 9f1c2ab phase: 03_api)
```
Inside the sandbox the agent has the repository but not the remote, and it is told so
in its own prompt (`--no-notes` to drop that block) — `git status`, `git diff`, `git
log` work; `git push`, `fetch`, `pull` and `clone <url>` are not attempted, so no
fixer round is spent rediscovering that. That is also why the gate runs in the sandbox:
`.agents/validate.sh` is inside the writable mount, and a jailer the prisoner can edit
is not a jailer.
One consequence of committing out here is worth naming: a repository is not only what
git records, it is what git **runs** — `.git/config`, and `.git/hooks/` especially. So
the driver's git calls run with an empty hooks path, no pager, no system config and no
`ext::` transport, and the run refuses to deliver if a sandboxed unit changed either:
```
✗ ERROR: refusing to deliver (commit/push) -- the sandbox changed this repository's own git configuration.
look at it by hand: git config --local --list; ls -l .git/hooks
if the change was yours: re-run with --trust-git-changes
```
The commits and the work stay exactly where they are; nothing is pushed and no hook
runs either way.
A project whose tests need something the image lacks gets it from the launcher:
```bash
sandbox-pi () {
export AI_API_TOKEN=$(pass aipi/dev-token)
ai-sandbox --allow aipi.reeseapps.com --allow pypi.org --allow files.pythonhosted.org \
--install python3-pip --install ripgrep --env AI_API_TOKEN "$@"
}
```
A new `--install` set is a new image tag, so the first run after changing it builds: that
one run says which packages are going in and reports every step until the image exists
(`AISB_BUILD_OUTPUT=quiet` if you would rather it did not talk). Every later run starts
without a word about the image, because there is nothing left to do.
`doctor`, `inspect`, `clean` and the rest of ai-sandbox work unchanged underneath it —
`--status` prints the policy the launcher will apply, without applying it.
docs/threat-model.md
+33 -4
View File
@@ -34,6 +34,8 @@ registry you don't control, or someone with access to your host user account.
| Writing to the host's package manager, `/etc`, global npm/pip | read-only-ish image, `HOME` inside the container, optional `--read-only` | `tests/e2e/12_sandbox.sh` |
| Credential persistence you did not ask for | `--ephemeral-home`; the default home volume is named and inspectable | `tests/e2e/12_sandbox.sh` |
| Silent wide-open starts | an empty allowlist in `allow` mode is a hard error | `tests/unit/01_args.sh` |
| An agent shipping work you have not looked at (`git push`) | the git remote is not in the allowlist, so nothing in the container can reach it; `ai-sandbox-autophase` commits and pushes on the host, outside the sandbox, between unit runs | `tests/unit/10_autophase.sh` |
| The gate becoming the escape (`validate.sh` is inside the writable mount) | the pass/fail gate runs in the sandbox as the same unprivileged user, not on the host | `tests/unit/10_autophase.sh` |
## What does **not** stop it
@@ -88,10 +90,37 @@ Be honest about these, and design your work accordingly.
uses is your real one. So the bridged config is a convenience, not a narrower
credential: prefer a per-sandbox token from your gateway if it can mint one, and
note that a bridged run whose token variable is unset or empty is refused rather
than started. Sandboxed edits to `~/.pi/agent` persist in the home volume and are
kept over later host changes (see the README), which is convenience with a
consequence: a hostile run could rewrite the sandboxed pi config for later runs of
that volume. `--ephemeral-home` removes that.
than started. Sandboxed edits to `~/.pi/agent` persist in the home volume and win
over later host changes (see the README), which is convenience with a consequence: a
hostile run could rewrite the sandboxed pi config for later runs of that volume —
point `baseUrl` at a host it controls, swap a model, change the provider. Merging
`settings.json` per key limits this (a run can only pin the keys it actually writes,
and everything else keeps following your host config), and `defaultThinkingLevel` is
re-forced every run so a run cannot quietly lower the reasoning of the next. It is
not eliminated: `--ephemeral-home` removes it, and `inspect` shows what is bridged.
13. **A push on the host still pushes what the agent wrote.** `ai-sandbox-autophase`
keeps the remote away from the sandbox, but somebody has to deliver the work, and
that somebody is the harness running as you. It commits only the phase's own
changes and only after `.agents/validate.sh` passes — so read that gate as part of
your trust base. It lives in the writable mount, which is exactly why the gate runs
*in the sandbox* by default; `--validate-in host` hands that judgement to you and
warns on every run.
14. **A published port is a door you opened.** `--publish` (and `publish` in a config
file) puts a listener on the host that forwards into the container, so a client
outside can reach a server the agent is running. That is the feature — preview a dev
server, poke a local API — and it is also the whole risk: whatever that server serves,
and whatever it accepts, is reachable from the address it is bound to, by things other
than you. The default binds `127.0.0.1`, because a bare podman `-p` binds every
interface (measured); a spec with a non-loopback address warns on every run and should
not be committed. Egress stays enforced either way: a published port is ingress, not a
loosened allowlist, and the same run that answers your browser still refuses a domain
outside it (`tests/e2e/13_publish.sh`). The agent inside cannot widen its own port or
open a new one — the mapping is decided on the host before the container starts and the
command runs with no capabilities. What it *can* do is serve whatever it likes on a port
you asked for. Publish the port of the server you meant to preview, and treat that port
as published content for as long as the run lasts.
## Residual risk rating
install.sh
+28 -9
View File
@@ -3,12 +3,15 @@
#
# ./install.sh install to ~/.local/bin, print the alias line
# ./install.sh --alias pi also append the alias to ~/.bashrc
# ./install.sh --prefix DIR install somewhere else
# ./install.sh --prefix DIR install somewhere else ($AISB_PREFIX does the same)
# ./install.sh --uninstall remove what this script installed
#
# Both programs are installed: ai-sandbox itself, and ai-sandbox-autophase, which runs
# the .agents/phases/ pipeline with the agent in the sandbox and the git push on the host.
set -Eeuo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PREFIX="${HOME}/.local/bin"
PREFIX="${AISB_PREFIX:-${HOME}/.local/bin}"
ALIAS_NAME=""
UNINSTALL=0
@@ -19,16 +22,19 @@ while (($#)); do
--alias) ALIAS_NAME="${2:?--alias needs a name, e.g. --alias pi}"; shift 2 ;;
--alias=*) ALIAS_NAME="${1#*=}"; shift ;;
--uninstall) UNINSTALL=1; shift ;;
-h|--help) sed -n '2,10p' "$0"; exit 0 ;;
-h|--help) sed -n '2,12p' "$0"; exit 0 ;;
*) echo "unknown option: $1" >&2; exit 2 ;;
esac
done
target="$PREFIX/ai-sandbox"
targets=("$PREFIX/ai-sandbox" "$PREFIX/ai-sandbox-autophase")
target="${targets[0]}"
rcfile="${HOME}/.bashrc"
if (( UNINSTALL )); then
[[ -L "$target" || -f "$target" ]] && { rm -f "$target"; echo "removed $target"; } || echo "nothing at $target"
for t in "${targets[@]}"; do
[[ -L "$t" || -f "$t" ]] && { rm -f "$t"; echo "removed $t"; } || echo "nothing at $t"
done
if [[ -n "$ALIAS_NAME" ]]; then
tmp="$(mktemp)"
grep -v "^alias ${ALIAS_NAME}=" "$rcfile" > "$tmp" 2>/dev/null || true
@@ -37,12 +43,14 @@ if (( UNINSTALL )); then
exit 0
fi
[[ -x "$HERE/bin/ai-sandbox" ]] || { echo "bin/ai-sandbox is missing or not executable" >&2; exit 1; }
command -v podman > /dev/null || echo "warning: podman is not installed; ai-sandbox needs it"
mkdir -p "$PREFIX"
ln -sfn "$HERE/bin/ai-sandbox" "$target"
echo "installed $target -> $HERE/bin/ai-sandbox"
for src in ai-sandbox ai-sandbox-autophase; do
[[ -x "$HERE/bin/$src" ]] || { echo "bin/$src is missing or not executable" >&2; exit 1; }
ln -sfn "$HERE/bin/$src" "$PREFIX/$src"
echo "installed $PREFIX/$src -> $HERE/bin/$src"
done
if [[ -n "$ALIAS_NAME" ]]; then
line="alias ${ALIAS_NAME}='${target}'"
@@ -56,9 +64,20 @@ else
echo
echo "To use it as a harness wrapper, add this to $rcfile:"
echo " alias pi='$target'"
echo "then: source $rcfile && cd your/project && pi --allow aipi.reeseapps.com --env AI_API_TOKEN"
echo "then: source $rcfile && cd your/project && pi --allow api.example.com --env API_KEY"
fi
echo
echo "next: $target doctor # sanity-check this machine"
echo " $target build # build the sandbox image now (otherwise built on first run)"
echo
echo "To run a phased project inside the sandbox, wrap this in a function in ~/.bashrc.d"
echo "(that is what ai-sandbox-autophase looks for, since a script cannot see a shell"
echo "function it did not define):"
echo " sandbox-pi () {"
echo " export AI_API_TOKEN=\$(pass aipi/token)"
echo ' ai-sandbox --allow api.example.com --env AI_API_TOKEN "$@"'
echo " }"
echo ' (the "$@" is not decoration: a run appends its flags, then --, then the command'
echo " to sandbox, and a function that drops them runs the image default with no prompt)"
echo "then: cd your/project && ai-sandbox-autophase --status"
tests/e2e/11_net.sh
+3 -1
View File
@@ -22,7 +22,9 @@ trap 'kill '"$SRV"' 2>/dev/null' EXIT
sleep 1
t_name "allow mode: enforcement in one run"
sb $GATE --env AISB_PROBE_HOSTSVC="$HOSTPORT" -- bash /project/egress_probe.sh
# the probe is a file in the project, reached at whatever path the container mounts it at
PROJ_DEST="$([[ -n "$(mirror_block_of "$PROJ")" ]] && echo /project || echo "$PROJ")"
sb $GATE --env AISB_PROBE_HOSTSVC="$HOSTPORT" -- bash "$PROJ_DEST/egress_probe.sh"
assert_contains "ok_allowed_http=200" "allowlisted http works (through the gate)"
assert_contains "ok_allowed_https=200" "allowlisted https works (CONNECT)"
assert_contains "ok_gate_denied_domain=403" "a non-allowlisted domain is refused by the gate"
tests/e2e/12_sandbox.sh
+88 -5
View File
@@ -22,17 +22,51 @@ sb $DARK --harness pi -- bash -lc "command -v pi"
assert_rc 0 "the default pi binary exists in the image"
t_name "the project directory is mounted, writable, and owned by the host user"
sb $DARK -- bash -c 'pwd; touch /project/from-container.txt; echo wrote'
assert_contains "/project" "working directory is /project"
# Where it is mounted depends on the host path; see the mirroring block below, which is
# the other branch of the same decision.
DEST="$([[ -n "$(mirror_block_of "$WORK_DIR")" ]] && echo /project || echo "$WORK_DIR")"
sb $DARK -- bash -c "pwd; touch '$DEST/from-container.txt'; echo wrote"
assert_contains "$DEST" "working directory is the project mount"
[[ -f "$WORK_DIR/from-container.txt" ]] && _ok "writes land on the host" || _bad "writes land on the host"
if [[ "$(stat -c %u "$WORK_DIR/from-container.txt")" == "$(id -u)" ]]; then
_ok "the file is owned by the host uid, not by root"
else
_bad "the file is owned by the host uid" "owner=$(stat -c %u "$WORK_DIR/from-container.txt") expected=$(id -u)"
fi
sb $DARK -- bash -c 'cat /project/host-file.txt'
sb $DARK -- bash -c "cat '$DEST/host-file.txt'"
assert_contains "hello from the host" "existing project files are readable"
t_name "the container names the directory what the host calls it"
# The reason for mirroring: pwd, an error message and a harness' window title name *this*
# project instead of every sandbox alike. Only a real container answers the question --
# podman decides what a bind hides, and it is the one thing about the arrangement that can
# go quietly wrong.
if [[ "$DEST" == "/project" ]]; then
assert_contains "cannot mirror" "the fallback is announced rather than silent"
fi
mirrorable_dir || true
if [[ -z "$USER_MIRROR_DIR" ]]; then
_ok "no path on this host may be mirrored ($HOME is inside a sandbox?) -- skipped"
else
MD="$USER_MIRROR_DIR"
echo "hello from the host" > "$MD/host-file.txt"
cd "$MD" || exit 1
sb $DARK -- bash -c 'pwd; cat host-file.txt; touch from-container.txt; echo wrote; ls -d /project; test -w /home/sandbox && echo home_writable && echo marked > /home/sandbox/mirror-marker'
assert_contains "$MD" "pwd is the host path, not /project"
assert_contains "hello from the host" "the mirrored mount is the project, not an empty directory"
assert_contains "wrote" "and it is writable"
[[ -f "$MD/from-container.txt" ]] \
&& _ok "writes land on the host through the mirrored path" || _bad "writes land on the host"
assert_absent "cannot mirror" "nothing is announced when the path mirrors"
assert_contains "/project" "the image's own /project is still there"
assert_contains "home_writable" "the home volume is still mounted at /home/sandbox"
# The home volume is the state the next run expects, so check it survived the mirror.
sb $DARK -- bash -c 'cat /home/sandbox/mirror-marker 2>&1; ls'
assert_contains "marked" "and the marker written through a mirrored run is still there"
assert_contains "host-file.txt" "a mirrored run sees its own project, not the previous one"
cd "$WORK_DIR" || exit 1
fi
t_name "nothing else on the host is visible"
echo secret > "$WORK/not-shared.txt"
sb $DARK -- bash -c "cat '$WORK/not-shared.txt' 2>&1 || echo NOT_VISIBLE"
@@ -145,12 +179,12 @@ assert_absent "demo-skill" "not even as a directory name"
# visible at once, which is what a harness that reads both (pi) expects.
mkdir -p "$WORK_DIR/.agents/skills/project-skill"
printf 'name: project-skill\ninstructions: project only\n' > "$WORK_DIR/.agents/skills/project-skill/SKILL.md"
sb $DARK -- bash -c 'cat /project/.agents/skills/project-skill/SKILL.md; ls "$HOME/.agents/skills" | tr "\n" " "'
sb $DARK -- bash -c "cat '$DEST/.agents/skills/project-skill/SKILL.md'; ls \"\$HOME/.agents/skills\" | tr '\n' ' '"
assert_contains "project only" "the project's own .agents comes in with the project mount"
assert_contains "demo-skill" "and the user-level one is still alongside it"
t_name "read-only root filesystem"
sb $DARK --read-only -- bash -c 'touch /usr/bin/nope 2>/dev/null && echo "BAD: wrote to rootfs" || echo rootfs_locked=ok; touch /project/rw-still-works && echo project_still_writable'
sb $DARK --read-only -- bash -c "touch /usr/bin/nope 2>/dev/null && echo 'BAD: wrote to rootfs' || echo rootfs_locked=ok; touch '$DEST/rw-still-works' && echo project_still_writable"
assert_contains "rootfs_locked=ok" "--read-only blocks writes outside mounts"
assert_contains "project_still_writable" "the project mount stays writable"
@@ -245,6 +279,55 @@ sb $DARK --harness pi -- bash -c 'rm -f "$HOME/.pi/agent/models.json"; echo rese
sb $DARK --harness pi -- bash -lc 'grep -q 524288 "$HOME/.pi/agent/models.json" && echo reseeded || echo not_reseeded'
assert_contains "reseeded" "deleting it inside the sandbox takes the host version again"
t_name "settings.json follows the host per key, and the thinking level is policy"
# The host file says medium and no ai-sandbox policy is in it: anything that arrives as
# xhigh can only have been written by the policy.
printf '{ "defaultProvider": "gateway", "defaultModel": "turbo", "theme": "dark/dark",\n "defaultThinkingLevel": "medium", "thinkingBudgets": { "xhigh": 65536 } }\n' > "$AGENT/settings.json"
sb $DARK --harness pi -- bash -lc 'jq -c . "$HOME/.pi/agent/settings.json"'
assert_contains '"defaultThinkingLevel":"xhigh"' "xhigh is enforced over the host file's medium"
assert_contains '"thinkingBudgets":{"xhigh":65536}' "and the rest of the preferences come along"
# What pi does to its own settings: rewrites the file, adding a key of its own. That used
# to freeze every host preference out of the sandbox for good.
sb $DARK --harness pi -- bash -c '
f="$HOME/.pi/agent/settings.json"
jq ".theme=\"light/light\" | .defaultThinkingLevel=\"off\" | .lastChangelogVersion=\"9.9.9\"" "$f" > "$f.n" && mv "$f.n" "$f"
echo edited' >/dev/null
# the host then moves the model and the budgets, drops the provider entirely, and says
# medium itself -- so a later --thinking host run has something to hand over
printf '{ "defaultModel": "deepthink", "theme": "sepia/sepia",\n "defaultThinkingLevel": "medium", "thinkingBudgets": { "xhigh": 131072 } }\n' > "$AGENT/settings.json"
sb $DARK --harness pi -- bash -lc 'jq -c . "$HOME/.pi/agent/settings.json"'
MERGED="$OUT"
printf '%s' "$MERGED" | grep -qF '"theme":"light/light"' && _ok "the theme edited in the sandbox survives" \
|| _bad "the theme edited in the sandbox survives" "$MERGED"
printf '%s' "$MERGED" | grep -qF '"lastChangelogVersion":"9.9.9"' && _ok "a key only pi wrote survives" \
|| _bad "a key only pi wrote survives" "$MERGED"
printf '%s' "$MERGED" | grep -qF '"defaultModel":"deepthink"' && _ok "an untouched model follows the host again" \
|| _bad "an untouched model follows the host again" "$MERGED"
printf '%s' "$MERGED" | grep -qF '"xhigh":131072' && _ok "and so do the thinking budgets" \
|| _bad "and so do the thinking budgets" "$MERGED"
printf '%s' "$MERGED" | grep -qF 'defaultProvider' && _bad "an untouched key the host dropped is dropped" \
|| _ok "an untouched key the host dropped is dropped"
printf '%s' "$MERGED" | grep -qF '"defaultThinkingLevel":"xhigh"' && _ok "thinking is forced back to xhigh after the sandbox set off" \
|| _bad "thinking is forced back to xhigh after the sandbox set off" "$MERGED"
# (the jq programs below avoid `{a, b}` object construction: bash brace-expands an
# unquoted word containing a comma before jq ever sees it)
sb $DARK --harness pi --thinking host -- bash -lc 'jq -r .defaultThinkingLevel "$HOME/.pi/agent/settings.json"'
assert_eq "medium" "$(printf '%s' "$OUT" | tail -1)" "with --thinking host the host's own medium is handed over"
sb $DARK --harness pi -- bash -c 'printf "{oops" > "$HOME/.pi/agent/settings.json"' >/dev/null
sb $DARK --harness pi -- bash -lc 'jq -c . "$HOME/.pi/agent/settings.json"'
assert_contains "deepthink" "a settings.json that is not JSON is re-seeded rather than left broken"
assert_contains '"defaultThinkingLevel":"xhigh"' "with the policy still applied"
sb $DARK --harness pi -- bash -lc 'jq -r .defaultThinkingLevel "$HOME/.pi/agent/settings.json"'
assert_eq "xhigh" "$(printf '%s' "$OUT" | tail -1)" "and it stays xhigh, not the medium that came back"
sb $DARK --harness pi -- bash -c 'chmod 000 "$HOME/.pi/agent/settings.json"' >/dev/null
sb $DARK --harness pi -- bash -lc 'echo ran'
assert_contains "cannot read settings.json" "a file this container cannot read is reported, not blamed on JSON"
assert_contains "--ephemeral-home" "and the report names a way out"
unset AI_API_TOKEN
sb_timeout 90 $DARK --harness pi
assert_rc 1 "a run without the credential is refused before anything starts"
tests/e2e/13_publish.sh
+115
View File
@@ -0,0 +1,115 @@
#!/usr/bin/env bash
# tests/e2e/13_publish.sh -- ingress: a port the sandbox listens on, reached from the host.
#
# Publish is the one place where something arrives *from* the host, so it is tested with a
# real container and a real client. --dry-run can print --publish all day and the port can
# still answer nothing, and the ruleset can reject the very rule that carries it (it did,
# once: nft wants the verdict before the comment, and the sandbox correctly refused to start
# at exit 77 -- correctly, but nobody sees that as "the feature works").
#
# Two of the assertions below are the measurements quoted by docs/networking.md:
# * with the full enforcement ruleset installed, a published port answers on the host;
# * a server bound to 127.0.0.1 *inside* does not, because pasta delivers the connection
# to the container's own address, not to its loopback.
# If either changes on a new host, the docs are wrong and this file is what says so.
source "$TESTS/lib.sh"
PROJ="$WORK/proj"; mkdir -p "$PROJ"; cd "$PROJ" || exit 1
export AISB_CONFIG_DIR="$WORK/cfg-empty"; mkdir -p "$AISB_CONFIG_DIR"
if ! curl -m8 -sS -o /dev/null http://example.com 2>/dev/null; then
echo " (skipped: no outbound internet, so the enforced run this needs cannot start)"
summary; exit 0
fi
free_port() { python3 -c 'import socket;s=socket.socket();s.bind(("127.0.0.1",0));print(s.getsockname()[1]);s.close()'; }
listening() { ss -H -ltn 2>/dev/null | grep -c "[:.]${1}\$"; }
# Start a sandbox in the background running an HTTP server, and hold the container open
# until the host writes $PROJ/.stop -- the port only exists while the container does, so the
# probe and the run have to overlap. $AISB_TEST_BIND says what the server binds to inside.
hold_server() { # hold_server <out-file> <host-port> <in-container bind> [extra flags...]
local out="$1" hp="$2" bind="$3"; shift 3
rm -f "$PROJ/.stop"
"$AISB" --net allow --allow example.com --env "AISB_TEST_BIND=$bind" "$@" --publish "$hp:8000" -- \
bash -lc '
python3 -m http.server 8000 --bind "$AISB_TEST_BIND" >/tmp/srv.log 2>&1 &
for i in $(seq 40); do curl -sf -m1 -o /dev/null http://127.0.0.1:8000/ && break; sleep 0.25; done
echo "inside=$(curl -s -o /dev/null -w %{http_code} -m2 http://127.0.0.1:8000/)"
echo "egress_denied=$(curl -s -o /dev/null -w %{http_code} -m8 http://example.org 2>/dev/null)"
echo "egress_allowed=$(curl -s -o /dev/null -w %{http_code} -m15 http://example.com 2>/dev/null)"
echo "uid=$(id -u) capeff=$(grep CapEff /proc/self/status | cut -f2)"
i=0; while [ $i -lt 160 ] && [ ! -f .stop ]; do sleep 0.25; i=$((i+1)); done
echo "srvlog=$(tail -1 /tmp/srv.log | cut -c1-60)"' > "$out" 2>&1 &
}
# Probe from the host, which is the whole point: nothing inside this container can answer
# the question "can a browser on the host reach it".
host_probe() { # host_probe <port> [tries] -> the last http code seen
local hp="$1" n="${2:-60}" i code=000
for ((i = 0; i < n; i++)); do
code="$(curl -s -m2 -o /dev/null -w '%{http_code}' "http://127.0.0.1:$hp/" 2>/dev/null || echo 000)"
[[ "$code" == 200 ]] && break
sleep 0.25
done
printf '%s' "$code"
}
finish() { touch "$PROJ/.stop"; wait "$1" 2>/dev/null; sleep 1; }
t_name "a published port answers on the host, with enforcement installed"
HP="$(free_port)"
hold_server "$WORK/a.out" "$HP" 0.0.0.0
CODE="$(host_probe "$HP")"
finish $!
A="$(cat "$WORK/a.out")"
assert_eq "200" "$CODE" "the host reaches the server the sandbox is running"
printf '%s' "$A" | grep -q 'enforce=nft' && _ok "the ruleset installed -- the publish rule is valid nft" \
|| _bad "the ruleset installed" "$(printf '%s' "$A" | grep -A3 FATAL | head -4)"
printf '%s' "$A" | grep -q 'ingress: input accepts tcp 8000' && _ok "the entrypoint says which ports it opened" \
|| _bad "the entrypoint reports the ingress"
printf '%s' "$A" | grep -q 'srvlog=.*"GET / HTTP/1.1" 200' && _ok "the request landed in the server's own log" \
|| _bad "the request landed in the server log" "$(printf '%s' "$A" | grep srvlog)"
# The promise the feature has to keep: it is ingress *only*. The same run that served the
# host still refused a non-allowlisted domain and still ran unprivileged.
printf '%s' "$A" | grep -q 'egress_denied=403' && _ok "egress outside the allowlist is still refused by the gate" \
|| _bad "egress still refused" "$(printf '%s' "$A" | grep egress_denied)"
printf '%s' "$A" | grep -q 'egress_allowed=200' && _ok "and the allowlist still works in the same run" \
|| _bad "allowlisted egress" "$(printf '%s' "$A" | grep egress_allowed)"
printf '%s' "$A" | grep -q 'uid=1000' && _ok "the server runs as the sandbox user" || _bad "uid"
printf '%s' "$A" | grep -q 'capeff=0000000000000000' && _ok "with no capabilities -- it cannot widen its own port" \
|| _bad "capeff" "$(printf '%s' "$A" | grep capeff)"
assert_eq "0" "$(listening "$HP")" "the host listener is gone when the container exits (no door left watching nothing)"
t_name "a server bound to loopback inside is not reachable from the host"
# Measured, and the doc says it: pasta delivers a published connection to the container's own
# address, so `npm run dev` with its default 127.0.0.1 answers inside and answers nothing
# outside. A preview that is silently not reachable is worse than one that says bind 0.0.0.0.
HP2="$(free_port)"
hold_server "$WORK/b.out" "$HP2" 127.0.0.1
CODE2="$(host_probe "$HP2" 12)"
finish $!
B="$(cat "$WORK/b.out")"
printf '%s' "$B" | grep -q 'inside=200' && _ok "the server does listen -- inside its own loopback" || _bad "listens inside"
[[ "$CODE2" != 200 ]] && _ok "and the published port does not reach it (bind 0.0.0.0 inside)" \
|| _bad "a loopback-only server was reachable from the host" "code=$CODE2 -- update docs/networking.md"
t_name "--net none refuses to publish instead of starting with nothing listening"
snapshot_containers
sb_timeout 30 --net none --publish "$(free_port)" -- true
assert_rc 1 "--net none + --publish is an error"
assert_contains "cannot forward a port" "and it says why: podman drops the mapping and runs anyway"
assert_no_new_containers "and no container was started to find that out"
t_name "a host port already in use is named before the container starts"
HP3="$(free_port)"
python3 -m http.server "$HP3" --bind 127.0.0.1 --directory "$WORK" >/dev/null 2>&1 &
SRV=$!
sleep 1
snapshot_containers
sb_timeout 30 --net allow --allow example.com --publish "$HP3:8000" -- true
assert_rc 1 "a busy host port refuses the run"
assert_contains "already in use" "and names the port"
assert_no_new_containers "without starting a container first"
kill "$SRV" 2>/dev/null
summary
tests/fixtures/phase-lib/lib.sh
Vendored
+161
View File
@@ -0,0 +1,161 @@
#!/usr/bin/env bash
# tests/fixtures/phase-lib/lib.sh -- a stand-in for the phased-execution skill's lib.sh.
#
# It is a CONTRACT fixture, not a reimplementation: the function names, the child call
# shape (`pi "${PI_ARGS[@]}" --session-dir ... --mode json "$prompt"` piped to a progress
# renderer), the gate call (`run_validation <logfile>`) and the host-side commit + push
# at a phase's commit point are what bin/ai-sandbox-autophase is written against, and
# what breaks if that library changes shape. Keeping it here means the unit tests run
# with no skill installed and no containers; tests/unit/10_autophase.sh also runs the
# real library once, when it is installed.
#
# Sourced, never executed.
PHASE_TODO=".agents/phases/todo"
PHASE_DONE=".agents/phases/complete"
PHASE_REPORTS=".agents/reports"
PHASE_SESSIONS=".agents/phase-sessions"
MAX_FIX_ATTEMPTS="${MAX_FIX_ATTEMPTS:-3}"
FIXTURE_LIB_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ts() { date '+%Y-%m-%d %H:%M:%S'; }
warn() { echo "[$(ts)] ⚠ $*" >&2; }
die() { echo "[$(ts)] ✗ ERROR: $*" >&2; exit 1; }
find_root() {
local d; d="$(pwd)"
while :; do
[[ -d "$d/.agents/phases/todo" ]] && { printf '%s\n' "$d"; return 0; }
[[ "$d" == "/" ]] && return 1
d="$(dirname "$d")"
done
}
# A phase entry is anything in todo/ whose name starts with a digit: a phase directory, or
# a single .md file. Globs rather than ls|grep, so a name cannot be mis-split.
phase_entries() {
local e
{
( cd "$PHASE_TODO" 2>/dev/null || exit 0
for e in [0-9]*; do [[ -e "$e" ]] && printf '%s\n' "$e"; done )
} | sort
}
phase_next_unit() {
local p="$1" f t
if [[ -d "$PHASE_TODO/$p" ]]; then
for f in "$PHASE_TODO/$p"/[0-9]*.md; do
[[ -f "$f" ]] || continue
t="${f##*/}"; [[ "$t" == "00_phase.md" ]] && continue
printf '%s/%s\n' "$p" "$t"; return 0
done
[[ -f "$PHASE_TODO/$p/00_phase.md" ]] && { printf '%s/00_phase.md\n' "$p"; return 0; }
elif [[ -f "$PHASE_TODO/$p" ]]; then
printf '%s\n' "$p"
fi
return 0
}
next_unit() {
local p u
for p in $(phase_entries); do
u="$(phase_next_unit "$p")"
[[ -n "$u" ]] && { printf '%s\n' "$u"; return 0; }
done
return 0
}
count_units() {
local p n=0 f
for p in $(phase_entries); do
if [[ -d "$PHASE_TODO/$p" ]]; then
for f in "$PHASE_TODO/$p"/[0-9]*.md; do [[ -f "$f" ]] && n=$((n + 1)); done
elif [[ -f "$PHASE_TODO/$p" ]]; then
n=$((n + 1))
fi
done
printf '%s\n' "$n"
}
unit_phase() { local u="$1"; [[ "$u" == */* ]] && printf '%s\n' "${u%%/*}" || printf '%s\n' "${u%.md}"; }
unit_base() { local u="${1%.md}"; printf '%s\n' "${u//\//__}"; } # 01_api/02_x.md -> 01_api__02_x
is_phase_end() { local u="$1"; if [[ "$u" == */* ]]; then [[ "${u##*/}" == "00_phase.md" ]]; else return 0; fi; }
unit_report_dir() {
local u="$1" p
if [[ "$u" == */* ]]; then p="${u%%/*}"; else p="."; fi
printf '%s/%s\n' "$PHASE_REPORTS" "$p"
}
unit_report() { # <unit> <attempt> <ext>
printf '%s/%s.a%s.%s\n' "$(unit_report_dir "$1")" "$(unit_base "$1")" "$2" "$3"
}
move_unit() {
local u="$1"
if [[ "$u" == */* ]]; then
mkdir -p "$PHASE_DONE/${u%%/*}"
mv -f "$PHASE_TODO/$u" "$PHASE_DONE/${u%%/*}/${u##*/}"
rmdir "$PHASE_TODO/${u%%/*}" 2>/dev/null || true
else
mkdir -p "$PHASE_DONE"
mv -f "$PHASE_TODO/$u" "$PHASE_DONE/$u"
fi
}
sweep_finished_phases() { return 0; }
build_pi_args() { PI_ARGS=(--no-approve); }
first_prompt() { printf 'Execute the unit %s. Reply with a report.\n' "$1"; }
fix_prompt() { printf 'Fix the failures: %s\n' "${1:-}"; }
run_validation() { bash .agents/validate.sh >"$1" 2>&1; }
# The host-side commit + push, at the phase's commit point: same shape as the real
# library's commit_phase/push_phase (commit, then push when PHASE_PUSH=1).
commit_phase() {
local phase="$1"
git rev-parse --is-inside-work-tree >/dev/null 2>&1 || return 0
git add -A >/dev/null 2>&1 || return 1
if ! git diff --cached --quiet; then
git commit --no-gpg-sign -m "phase: $phase" >/dev/null 2>&1 || return 1
fi
# the real library hands the commit to the remote through push_phase, so this fixture
# does too -- which is also the function the driver shadows to keep every push (and
# every refusal to push) in one place.
push_phase "$phase" || return 1
return 0
}
# Mirrors the library's own entry point; the driver replaces it.
push_phase() {
git remote >/dev/null 2>&1 || return 0
git push >/dev/null 2>&1 || return 1
return 0
}
notify_task() { return 0; }
execute_unit() {
local unit="$1" errors="" rc=0
mkdir -p "$PHASE_DONE" "$PHASE_REPORTS" "$PHASE_SESSIONS" "$(unit_report_dir "$unit")"
echo "[$(ts)] ━━ $unit — attempt 1/$MAX_FIX_ATTEMPTS ━━" # the real library prints this; tests read it
build_pi_args
local prompt out errf vrc
prompt="$(first_prompt "$unit")"
out="$(unit_report "$unit" 1 md)"
errf="$(unit_report "$unit" 1 err)"
pi "${PI_ARGS[@]}" --session-dir "$PHASE_SESSIONS" --name "$(unit_base "$unit")" \
--mode json "$prompt" 2>"$errf" | bash "$FIXTURE_LIB_DIR/progress.sh" "$out"
rc="${PIPESTATUS[0]}"
(( rc == 0 )) || errors+="child exited $rc"$'\n'
run_validation "$(unit_report "$unit" 1 validate)"; vrc=$?
(( vrc == 0 )) || errors+=".agents/validate.sh failed"$'\n'
[[ -n "$errors" ]] && { printf '%s' "$errors" >&2; return 1; }
move_unit "$unit"
if is_phase_end "$unit"; then
commit_phase "$unit" || return 1
fi
return 0
}
tests/fixtures/phase-lib/progress.sh
Vendored
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# tests/fixtures/phase-lib/progress.sh -- the fixture's stand-in for progress.mjs.
#
# Same job and same contract: read the child's JSON stream on stdin, write the final
# assistant text to $1, exit non-zero when the stream produced no report.
set -uo pipefail
out="$1"; shift
text=""
while IFS= read -r line; do
case "$line" in
*'"type": "message_end"'* | *'"type":"message_end"'*)
t="$(printf '%s' "$line" | sed -n 's/.*"text": *"\([^"]*\)".*/\1/p')"
[[ -n "$t" ]] && text="$t"
;;
esac
done
if [[ -z "$text" ]]; then
printf '(no final assistant message)\n' > "$out"
exit 1
fi
printf '%s\n' "$text" > "$out"
tests/lib.sh
+78
View File
@@ -31,6 +31,84 @@ sb_timeout() {
return 0
}
# Extract one function from the script itself, the way 06_entrypoint.sh extracts the
# entrypoint: the test then asks the real code a question instead of restating its answer.
aisb_fn() { # aisb_fn <function-name> -> that function's source, on stdout
awk -v fn="$1" '$0 ~ "^" fn "\\(\\) \\{" {f=1} f {print} f && /^\}$/ {exit}' "$AISB"
}
# Where the sandbox would mount a host path. Empty means it mirrors the path; anything
# else names the container path the bind would bury. Extracted rather than reimplemented:
# the answer depends on the container's own filesystem (/usr, /tmp, /home/sandbox), and
# most of that hierarchy cannot be created by a test.
mirror_block_of() { # mirror_block_of <absolute host path> -> blocking path or empty
local src; src="$(aisb_fn mirror_block_of)"
[[ -n "$src" ]] || { echo "mirror_block_of not found in $AISB" >&2; return 2; }
( set -uo pipefail; eval "$src"; mirror_block_of "$1" )
}
# A directory whose host path the sandbox is allowed to mirror, in USER_MIRROR_DIR (empty
# when there is none). The scratch dir ($WORK) is under $TMPDIR, and /tmp is one of the
# paths the sandbox owns, so a run from there always takes the fallback branch -- the
# mirroring branch needs a per-user tree, and $HOME is the one place an unprivileged test
# can create it. Nothing is written into the real ~/.config, ~/.local or ~/.cache.
#
# Assigns a global instead of printing: the cleanup trap below belongs to the test file,
# so this must not be called as $(mirrorable_dir) -- a command substitution would set the
# trap in a subshell that exits, and takes the directory with it.
USER_MIRROR_DIR=""
mirrorable_dir() {
local base="${HOME%/}" qbase
USER_MIRROR_DIR=""
[[ -n "$base" && -d "$base" && -w "$base" ]] || return 1
[[ -z "$(mirror_block_of "$base")" ]] || return 1
USER_MIRROR_DIR="$(mktemp -d "$base/.ai-sandbox-test-mirror.XXXXXX" 2>/dev/null)" || return 1
[[ -n "$USER_MIRROR_DIR" ]] || return 1
# Removed when the test file's subshell exits, including on a failed assert. The base is
# quoted in now rather than read from $HOME later, so the trap cannot widen.
printf -v qbase '%q' "$base"
# shellcheck disable=SC2064 # immediate expansion of the captured base is the point
trap "rm -rf -- $qbase/.ai-sandbox-test-mirror.* >/dev/null 2>&1 || true" EXIT
return 0
}
# Run a command with stdin, stdout and stderr on a pty: -t and -i are decided from the
# real terminal, so a test about them cannot use the runner's pipes. util-linux's 'script'
# is the usual tool; python3's pty module does the same job where script is not installed
# (measured: all three descriptors are ttys and the exit code propagates). Sets OUT and RC
# like sb(). Args are joined for 'script', so pass simple words, not quoted sentences.
pty_available() { command -v script > /dev/null 2>&1 || command -v python3 > /dev/null 2>&1; }
pty_run() {
OUT="" RC=0
if command -v script > /dev/null 2>&1; then
OUT="$(script -qec "$*" /dev/null 2>&1 </dev/null)" && RC=0 || RC=$?
elif command -v python3 > /dev/null 2>&1; then
OUT="$(python3 -c '
import os, pty, sys
st = pty.spawn(sys.argv[1:])
try: sys.exit(os.waitstatus_to_exitcode(st))
except AttributeError: sys.exit(st >> 8)' "$@" 2>&1 </dev/null)" && RC=0 || RC=$?
else
OUT="no pty tool available"
RC=127
fi
return 0
}
# "nothing was started", as a hermetic question. Counting every sandbox container on the
# machine fails when the developer has a session open, or when an interrupted run left one
# behind -- neither of which is this test starting a container. Take the set of ids before,
# and ask whether it changed.
snapshot_containers() { AISB_CONTAINER_SNAPSHOT="$(podman ps -aq --filter label=ai-sandbox=1 2>/dev/null | sort)"; }
assert_no_new_containers() { # assert_no_new_containers <label> (after snapshot_containers)
local now extra
now="$(podman ps -aq --filter label=ai-sandbox=1 2>/dev/null | sort)"
if [[ "$now" == "${AISB_CONTAINER_SNAPSHOT:-}" ]]; then _ok "$1"; return 0; fi
extra="$(comm -13 <(printf '%s\n' "${AISB_CONTAINER_SNAPSHOT:-}") \
<(printf '%s\n' "$now") | tr '\n' ' ')"
_bad "$1" "container ids appeared: ${extra:-none (one disappeared instead)}"
}
# sb_env VAR=val ... -- args (env prefix, without leaking into the test shell)
sb_env() {
local -a envs=()
tests/run.sh
+3 -2
View File
@@ -30,7 +30,8 @@ command -v podman >/dev/null || { echo "podman is required to run the tests" >&2
# Syntax-check every shell file in the project before running anything.
SYNTAX_BAD=0
for f in "$AISB" "$HERE"/*.sh "$HERE"/unit/*.sh "$HERE"/e2e/*.sh "$ROOT"/install.sh; do
for f in "$AISB" "$HERE"/*.sh "$HERE"/unit/*.sh "$HERE"/e2e/*.sh "$HERE"/fixtures/*.sh \
"$HERE"/fixtures/phase-lib/*.sh "$ROOT"/install.sh "$ROOT"/contrib/*.bash; do
[[ -f "$f" ]] || continue
if ! syn="$(bash -n "$f" 2>&1)"; then
echo "SYNTAX: $f" >&2; printf '%s\n' "$syn" | sed 's/^/ /' >&2; SYNTAX_BAD=1
@@ -79,7 +80,7 @@ for f in "${FILES[@]}"; do
# test should depend on or accidentally share. Files that exercise the .agents
# bridge point AISB_AGENTS_DIR at a fixture of their own.
export AISB_AGENTS_DIR="${AISB_AGENTS_DIR:-$W/absent-agents-dir}"
unset AI_API_TOKEN AISB_PI_TOKEN_ENV AISB_PI_CONFIG AISB_PI_TOKEN_AUTO
unset AI_API_TOKEN AISB_PI_TOKEN_ENV AISB_PI_CONFIG AISB_PI_TOKEN_AUTO AISB_THINKING
# NOTE: HOME is deliberately not overridden -- podman storage lives there.
unset AISB_ALLOW AISB_ENV AISB_MOUNT AISB_INSTALL AISB_NET AISB_EXTRA_ARGS AISB_SESSION_OVERRIDE
unset AISB_AGENTS AISB_AGENTS_DEST
tests/unit/01_args.sh
+37
View File
@@ -33,6 +33,43 @@ sb --allow; assert_rc 1 "--allow with no value"
sb --allow-file /nope/missing; assert_rc 1 "--allow-file must exist"
sb --mount /nope/missing --dry-run; assert_rc 1 "--mount path must exist"
t_name "publish takes a port, a pair, or an address and a pair"
sb --dry-run $A --publish 8000; assert_rc 0 "a bare port parses"
sb --dry-run $A --publish 18080:8000; assert_rc 0 "host:container parses"
sb --dry-run $A --publish 0.0.0.0:8000:8000; assert_rc 0 "addr:host:container parses"
sb --dry-run $A --publish '[::1]:8000:8000'; assert_rc 0 "bracketed ipv6 parses"
sb --dry-run $A --publish=8000; assert_rc 0 "--publish= form parses"
sb --strict-args --publish; assert_rc 1 "--publish is a known flag that needs a value"
sb --strict-args --pub 8000 --dry-run $A; assert_contains "did you mean --publish" "a near miss suggests --publish"
# Every one of these is a port list that *looks* right and is not. Silently reading one as
# something else is how a config ends up publishing a port nobody wrote.
t_name "publish specs that are refused"
for bad in 0 99999 abc 8000/udp 8000:8000:8000:8000 '::1:8000:8000' '8000:' ':8000' '8000::9000' '0.0.0.0:8000' '*:8000:8000' '0.0.0.0:8000:8000:8000'; do
sb_timeout 15 --dry-run $A --publish "$bad"
assert_rc 1 "refuses '$bad'"
done
# Three fields where the first is not an address: podman would get 'bind on 8000' and fail with an
# error that names none of our words, so the address has to look like an address here.
sb_timeout 15 --dry-run $A --publish 8000:8000:8000
assert_rc 1 "refuses a port written where an address belongs"
assert_contains "is not an address" "and says the three shapes that are valid"
sb_timeout 15 --dry-run $A --publish all:8000:8000
assert_rc 1 "refuses a word for an address"
sb --dry-run $A --publish box.internal:8000:8000
assert_contains "--publish box.internal:8000:8000" "a dotted host is taken as an address (and warned: not loopback)"
assert_contains "anything that can route to this machine" " ... by the same warning an explicit IP gets"
sb_timeout 15 --dry-run $A --publish 18080:8000 --publish 18080:9000
assert_rc 1 "two container ports on one host port is refused"
assert_contains "published twice" "and says which"
sb_timeout 15 --dry-run --net none --publish 8000
assert_rc 1 "--net none refuses to publish"
assert_contains "cannot forward a port" "because podman would drop the mapping and run anyway"
# -p is the harness's flag (claude -p prints), so it must stay a pass-through.
sb --dry-run $A -p 8000
assert_contains "pi -p 8000" "-p belongs to the harness, not to the sandbox"
t_name "fail-closed network policy"
sb --dry-run; assert_rc 1 "allow mode with no allowlist refuses to start"
assert_contains "Refusing to start wide open" "explains that it refuses to be wide open"
tests/unit/02_config.sh
+60
View File
@@ -76,3 +76,63 @@ echo 'bogus_key = 1' >> "$CFG/config"
sb --print-policy --net none
assert_contains "unknown key: bogus_key" "unknown key warns"
assert_rc 0 "unknown key is not fatal"
t_name "an allow file may end on a comment or a blank line"
# Regression: cfg_read_allowlist() ended on its `[[ -n "$line" ]]`, so a file whose last
# line yields no target returned 1. load_config() calls it in a non-final position, so under
# `set -e` every run died with exit 1 and *zero* bytes of output -- --dry-run and
# --print-policy included. The test above ends its file on "host # trailing comment", which
# yields a target and so never exercised this.
for last in '# a trailing comment' '' ' '; do
printf '# user-wide allowlist\napi.openai.com\n%s\n' "$last" > "$CFG/allow"
printf 'pypi.org\n%s\n' "$last" > "$REPO/.ai-sandbox/allow"
sb --print-policy --net allow
assert_rc 0 "user allow file ending on [$last] is not fatal"
assert_contains "api.openai.com" "user target survives a file ending on [$last]"
assert_contains "pypi.org" "repo target survives a file ending on [$last]"
done
t_name "publish accumulates like the other list keys"
cat > "$CFG/config" <<'EOF'
publish = 8000
EOF
cat > "$REPO/.ai-sandbox/config" <<'EOF'
publish = 18080:8000
EOF
sb --print-policy --net allow --allow example.com
assert_contains "publish=127.0.0.1:8000:8000" "user config publishes"
assert_contains "127.0.0.1:18080:8000" "repo config adds to it rather than replacing it"
sb_env AISB_PUBLISH=9000,9100:80 --print-policy --net allow --allow example.com
assert_contains "127.0.0.1:9000:9000" "AISB_PUBLISH is a comma list of the same shapes"
assert_contains "127.0.0.1:9100:80" " ... including host:container"
sb --print-policy --net allow --allow example.com --publish 9200
assert_contains "127.0.0.1:9200:9200" "a flag adds to both files"
sb --print-policy --net allow --allow example.com --publish 8000
N="$(printf '%s' "$OUT" | grep -o '127.0.0.1:8000:8000' | wc -l | tr -d ' ')"
assert_eq "1" "$N" "the same port from two sources collapses to one"
t_name "publish_addr is the default bind address, and only that"
cat > "$CFG/config" <<'EOF'
publish_addr = 127.0.0.2
EOF
rm -f "$REPO/.ai-sandbox/config"
sb --print-policy --net allow --allow example.com --publish 8000
assert_contains "publish=127.0.0.2:8000:8000" "config file sets the address a bare port binds"
sb_env AISB_PUBLISH_ADDR=127.0.0.3 --print-policy --net allow --allow example.com --publish 8000
assert_contains "127.0.0.3:8000:8000" "AISB_PUBLISH_ADDR beats the config file"
sb_env AISB_PUBLISH_ADDR=localhost --print-policy --net allow --allow example.com --publish 8000
assert_contains "publish=localhost:8000:8000" "a name is accepted where an address goes"
printf '%s' "$OUT" | grep -qF 'route to this machine' \
&& _bad "localhost warned about as if it were the LAN" || _ok "localhost counts as loopback: no LAN warning"
sb --print-policy --net allow --allow example.com --publish 0.0.0.0:8000:8000
assert_contains "publish=0.0.0.0:8000:8000" "an address written in the spec is the address used"
printf '%s' "$OUT" | grep '^publish=' | grep -qF '127.0.0.2' \
&& _bad "publish_addr overwrote an explicit address" || _ok "publish_addr only fills in a bare port"
# The fix must not trade one silence for another: with no target anywhere, the run still
# refuses, and says why.
printf '# nothing but a header here\n' > "$CFG/allow"
printf '# nothing but a header here\n' > "$REPO/.ai-sandbox/allow"
sb_timeout 20 --print-policy --net allow
assert_rc 1 "a header-only allow file still refuses to start"
assert_contains "no egress allowlist configured" "and refuses out loud"
tests/unit/03_argv.sh
+195 -5
View File
@@ -5,6 +5,9 @@ source "$TESTS/lib.sh"
PROJ="$WORK/proj"; mkdir -p "$PROJ"; cd "$PROJ" || exit 1
export AISB_CONFIG_DIR="$WORK/cfg-empty"; mkdir -p "$AISB_CONFIG_DIR"
A="--allow example.com"
# Every run in this file is a preview. What that is worth is asserted at the end, against
# the set of containers that existed before it started.
snapshot_containers
t_name "isolation flags are always on"
sb --dry-run $A
@@ -36,12 +39,15 @@ sb --dry-run $A --no-gate
assert_absent "HTTP_PROXY=" "--no-gate sets no proxy"
t_name "filesystem"
# Where the cwd lands depends on the host path, so the expected destination is asked of
# the same function the script uses instead of being hardcoded here.
PROJ_DEST="$([[ -n "$(mirror_block_of "$PROJ")" ]] && echo /project || echo "$PROJ")"
sb --dry-run $A
assert_contains "$PROJ:/project" "cwd is mounted at /project"
assert_contains "--workdir /project" "cwd is the working directory"
assert_contains "$PROJ:$PROJ_DEST" "cwd is mounted at its own path or the fallback"
assert_contains "--workdir $PROJ_DEST" "cwd is the working directory"
assert_contains "/run/aisb:ro" "policy dir is mounted read-only"
sb --dry-run $A --no-cwd
assert_absent "$PROJ:/project" "--no-cwd mounts nothing"
assert_absent "$PROJ" "--no-cwd mounts nothing"
sb --dry-run $A --workdir /src
assert_contains "$PROJ:/src" "--workdir changes the mount point"
mkdir -p "$WORK/extra"
@@ -58,6 +64,119 @@ sb --dry-run $A --ephemeral-home
assert_contains "/home/sandbox:rw,nosuid,nodev" "--ephemeral-home uses a tmpfs home"
assert_absent "type=volume,source=aisb-home" "--ephemeral-home drops the volume"
t_name "the container names the directory what the host calls it"
# Mirroring the host path is what makes pwd, an error message and a harness' window title
# name *this* project. It is a decision about the host path, so both branches need a real
# directory: $TMPDIR is a path the sandbox owns, a per-user tree is not.
mirrorable_dir || true
MIRROR_DIR="$USER_MIRROR_DIR"
if [[ -z "$MIRROR_DIR" ]]; then
_ok "no writable host path here may be mirrored ($HOME is itself inside a sandbox?) -- skipped"
else
cd "$MIRROR_DIR" || exit 1
sb --dry-run $A
assert_contains "$MIRROR_DIR:$MIRROR_DIR" "the cwd is mounted at the path it has on the host"
assert_contains "--workdir $MIRROR_DIR" "and worked in at that path"
assert_absent "# workdir:" "nothing to announce when the path mirrors"
sb --dry-run $A --print-policy
assert_eq "" "$(printf '%s' "$OUT" | grep '^workdir_note=' || true)" "and no note in the policy"
sb --dry-run $A --workdir /src
assert_contains "$MIRROR_DIR:/src" "--workdir still picks the mount point over mirroring"
assert_contains "--workdir /src" "--workdir is what reaches podman"
sb --dry-run $A --workdir auto
assert_contains "--workdir $MIRROR_DIR" "'auto' is the default, spelled out"
cd "$PROJ" || exit 1
fi
t_name "a host path the sandbox owns is not mirrored, and says so"
# /tmp is the guaranteed case: the sandbox mounts a tmpfs there, and podman puts that
# tmpfs on top of a bind of the same path -- so a mirrored /tmp project would come out as
# an empty tmpfs. Measured, see docs/networking.md.
cd /tmp || exit 1
sb --dry-run $A
assert_contains "--workdir /project" "/tmp falls back to the image's own directory"
assert_contains "-v /tmp:/project" "and mounts the cwd there instead"
assert_contains "# workdir: cannot mirror /tmp" "--dry-run explains the fallback"
assert_contains "the sandbox owns /tmp*" "naming what it would have buried"
assert_contains "--workdir DIR to choose" "and the way to decide otherwise"
sb_env AI_API_TOKEN=canary-not-a-real-token --dry-run $A --env AI_API_TOKEN
assert_absent "canary-not-a-real-token" "the note carries no values, only the decision"
sb --dry-run $A --print-policy
assert_contains "workdir=/project" "the policy says which directory was chosen"
assert_contains "workdir_note=" "and why"
assert_contains "cwd=/tmp" "while still printing the host path"
cd "$PROJ" || exit 1
t_name "which host paths may be mirrored is a decision about the container's filesystem"
# The refusals are about what lives at a path *inside* the image, and a test cannot create
# /usr or /home/sandbox. So ask the function itself, extracted from the script, about the
# paths it was written for -- the same trick 06_entrypoint.sh uses for the entrypoint.
[[ -n "$(aisb_fn mirror_block_of)" ]] \
&& _ok "the mirror decision is one function in the script" \
|| _bad "the mirror decision is one function in the script"
# Blocked: everything the base image owns, plus this sandbox's own mounts and home.
for p in / /usr/src/proj /var/lib/pg/data /opt/stack/app /etc/proj /tmp/build/proj \
/run/user/1000/proj /home /home/sandbox /home/sandbox/work /bin/proj /boot/efi \
/dev/shm/x /lib/x86_64-linux-gnu /lib32/x /lib64/ld /proc/1/cwd /root/proj \
/sbin/init /sys/firmware; do
b="$(mirror_block_of "$p")"
if [[ -n "$b" ]]; then _ok "$p is not mirrored (would bury $b)"
else _bad "$p is not mirrored" "it was mirrored"; fi
done
assert_eq "/tmp" "$(mirror_block_of /tmp/a/b)" "the note names the top-level path, not the project"
assert_eq "/home/sandbox" "$(mirror_block_of /home/sandbox/work)" "the container home is named as itself"
assert_eq "/" "$(mirror_block_of /)" "the whole root is refused"
# Free: per-user trees under /home, and the hierarchy the image leaves empty.
for p in /home/you/Projects/foo /home/dev /home/sandboxer/proj /mnt/data/proj /srv/www \
/media/usb/proj /code/x /work/proj /data/proj /home/sandboxish/x; do
b="$(mirror_block_of "$p")"
if [[ -z "$b" ]]; then _ok "$p mirrors to itself"
else _bad "$p mirrors to itself" "refused because of $b"; fi
done
t_name "a project path podman cannot express is refused with a reason"
# Measured: podman answers `-v /a:b:/project` with `invalid option type "/project"`, which
# is a strange thing to read when all you typed was `pi`. Mirroring puts the same string in
# the destination too, so it is refused before podman is involved.
COLON="$WORK/a:b"; mkdir -p "$COLON"; cd "$COLON" || exit 1
sb --dry-run $A
assert_rc 1 "a ':' in the project path is refused"
assert_contains "podman cannot parse" "and says whose parser"
assert_contains "--no-cwd" "and names the way out"
sb --dry-run $A --no-cwd
assert_rc 0 "--no-cwd needs nothing mounted, so the path is fine"
cd "$PROJ" || exit 1
t_name "--workdir takes a container path or 'auto', nothing else"
sb --dry-run $A --workdir relative/dir
assert_rc 1 "a relative --workdir is refused"
assert_contains "must be an absolute path or 'auto'" "and says what it wanted"
sb --dry-run $A --workdir /
assert_rc 1 "--workdir / is refused"
assert_contains "over the whole container" "with the reason"
sb --dry-run $A --workdir /src/
assert_contains "$PROJ:/src" "a trailing slash is not part of the destination"
# An explicit path onto something the sandbox owns is honoured -- it was asked for -- but
# the hiding rule does not stop applying because a human chose the path, so it is warned.
sb --dry-run $A --workdir /tmp/proj
assert_rc 0 "--workdir onto a path the sandbox owns is still honoured"
assert_contains "$PROJ:/tmp/proj" "podman is told what was asked for"
assert_contains "warning: --workdir /tmp/proj" "and the covering is announced"
assert_contains "/tmp is a tmpfs" "with the reason"
sb --dry-run $A --workdir /opt/proj
assert_contains "the sandbox owns /opt*" "the same for a directory the image owns"
sb --dry-run $A --workdir /srv/app
assert_absent "warning: --workdir" "a path nothing owns is chosen in silence"
sb_env AISB_PROJECT_DEST=/srv/app --dry-run $A
assert_contains "$PROJ:/srv/app" "AISB_PROJECT_DEST still chooses it"
mkdir -p "$WORK/cfg-wd"
printf 'workdir = /srv/cfg\n' > "$WORK/cfg-wd/config"
# No subshell here: OUT and the counters have to survive it.
AISB_CONFIG_DIR_SAVED="$AISB_CONFIG_DIR"; export AISB_CONFIG_DIR="$WORK/cfg-wd"
sb --dry-run $A
assert_contains "$PROJ:/srv/cfg" "the config file chooses it too"
export AISB_CONFIG_DIR="$AISB_CONFIG_DIR_SAVED"
t_name "labels, limits, extra args"
sb --dry-run $A --label team=x
assert_contains "--label team=x" "--label"
@@ -78,9 +197,80 @@ pos_cmd="$(printf '%s' "$line" | tr ' ' '\n' | grep -n -m1 '^hello$' | cut -d: -
if [[ -n "$pos_img" && -n "$pos_lbl" && $pos_lbl -lt $pos_img ]]; then _ok "extra flags precede the image"; else _bad "extra flags precede the image" "img=$pos_img label=$pos_lbl"; fi
if [[ -n "$pos_cmd" && -n "$pos_img" && $pos_img -lt $pos_cmd ]]; then _ok "the command follows the image"; else _bad "the command follows the image" "img=$pos_img cmd=$pos_cmd"; fi
t_name "published ports reach the invocation, bound to loopback"
sb --dry-run $A --publish 8000
assert_contains "--publish 127.0.0.1:8000:8000" "a bare port is bound to loopback on both ends"
assert_contains "# ingress: publish=[127.0.0.1:8000:8000" "the dry run says what it will let in"
assert_contains "--label aisb.publish=8000" "the container is labelled with the port it publishes"
sb --dry-run $A --publish 18080:8000
assert_contains "--publish 127.0.0.1:18080:8000" "host and container ports are separate"
sb --dry-run $A --publish 8000 --publish 18080:9000
assert_count 2 "--publish " "two entries, two flags"
sb --dry-run $A --publish 0.0.0.0:18080:8000
assert_contains "--publish 0.0.0.0:18080:8000" "an explicit address is passed through"
assert_contains "anything that can route to this machine" "and said out loud as an open door"
sb --dry-run $A --net full --publish 8000
assert_contains "--publish 127.0.0.1:8000:8000" "--net full can publish too"
# The promise the flag has to keep: it is ingress only. Everything that gates egress must be
# byte-identical with a port open, or "preview the app" would be a way to turn the policy off.
t_name "publishing does not touch egress enforcement"
sb --dry-run $A --publish 8000
assert_contains "--network pasta:--map-host-loopback,100.64.0.1" "the gate address is still the only door"
assert_contains "--cap-add=NET_ADMIN" "the entrypoint still gets NET_ADMIN to install the ruleset"
assert_contains "--cap-drop=ALL" "and still drops everything else"
assert_contains "HTTP_PROXY=http://100.64.0.1:18081" "the proxy env is still set"
assert_contains "--security-opt no-new-privileges" "no-new-privileges survives"
# An ipv6 bind keeps its brackets: podman splits the spec on ':' too, and an address full of
# them has to arrive as one field or the port lands inside the address.
sb --dry-run $A --publish '[::1]:8000:8000'
assert_contains '\[::1\]:8000:8000' "the ipv6 bind keeps its brackets in the invocation"
# --publish must precede the image like every other flag, or podman reads it as the command.
sb --dry-run $A --publish 8000 -- echo hello
line="$(printf '%s' "$OUT" | tail -1)"
pos_img="$(printf '%s' "$line" | tr ' ' '\n' | grep -n -m1 'localhost/ai-sandbox' | cut -d: -f1)"
pos_pub="$(printf '%s' "$line" | tr ' ' '\n' | grep -n -m1 -e '--publish' | cut -d: -f1)"
if [[ -n "$pos_img" && -n "$pos_pub" && $pos_pub -lt $pos_img ]]; then _ok "--publish precedes the image"; else _bad "--publish precedes the image" "img=$pos_img publish=$pos_pub"; fi
t_name "dry run is side-effect free"
sb --dry-run $A
RUNNING="$(podman ps --format '{{.Names}}' | grep -c 'aisb' || true)"
# Ask about the container *this* invocation would have created, not about every sandbox on
# the machine: another session that happens to be running is not this test starting one.
# The session id is unique per run, so its label is the hermetic question.
SID="$(printf '%s' "$OUT" | tr ' ' '\n' | sed -n 's/^aisb\.session=//p' | head -1)"
[[ -n "$SID" ]] && _ok "the dry run names its session" || _bad "the dry run names its session"
CREATED="$(podman ps -a --filter "label=aisb.session=$SID" --format '{{.Names}}' | wc -l | tr -d ' ')"
DIRS="$(ls -A "$AISB_STATE_DIR/sessions" 2>/dev/null | wc -l | tr -d ' ')"
assert_eq "$RUNNING" 0 "dry run started no container"
assert_eq "$CREATED" 0 "dry run created no container, under any name"
assert_eq "$DIRS" 0 "dry run left no session directory behind"
assert_no_new_containers "and no preview in this file created one either"
t_name "attachment: stdin is a choice, and an unattended run does not want it"
# -t and -i are decided from the real stdin/stdout, and the test runner's are pipes. The
# pipe case is checked either way; the attached case needs a pty, which lib.sh builds from
# 'script' or python3 and reports as skipped when the host has neither.
inv_line() { printf '%s' "$OUT" | tr -d '\r' | tail -1; }
has_opt() { inv_line | grep -qE "(^| )$1( |$)"; }
sb_pty() { pty_run "$AISB" --dry-run --net none "$@"; }
sb --dry-run --net none
if has_opt "-i" || has_opt "-t"; then _bad "pipes get neither" "$(inv_line)"; else _ok "pipes get neither: nothing is attached to a terminal that is not there"; fi
sb --dry-run --net none --interactive
if has_opt "-i"; then _ok "--interactive still forces it on with pipes"; else _bad "--interactive still forces it on with pipes" "$(inv_line)"; fi
if ! pty_available; then
_ok "neither 'script' nor python3 here -- the attached cases are skipped"
else
sb_pty
assert_rc 0 "on a pty"
if has_opt "-i" && has_opt "-t"; then _ok "a terminal run is attached, which is what an interactive shell expects"; else _bad "a terminal run is attached" "$(inv_line)"; fi
sb_pty --no-interactive
assert_rc 0 "--no-interactive"
if has_opt "-i"; then _bad "--no-interactive drops -i" "$(inv_line)"; else _ok "--no-interactive drops -i: the command cannot read this terminal"; fi
if has_opt "-t"; then _ok "--no-interactive leaves the TTY alone (it is a separate choice)"; else _bad "--no-interactive leaves the TTY alone" "$(inv_line)"; fi
sb_pty --no-tty --no-interactive
if has_opt "-t" || has_opt "-i"; then _bad "--no-tty --no-interactive detaches both" "$(inv_line)"; else _ok "--no-tty --no-interactive detaches both"; fi
sb_pty --strict-args --no-interactive
assert_rc 0 "--no-interactive is a real flag, not a pass-through to the harness"
if has_opt "-i"; then _bad "--no-interactive is parsed, not forwarded" "$(inv_line)"; else _ok "--no-interactive is parsed, not forwarded"; fi
fi
tests/unit/05_policy.sh
+15
View File
@@ -41,6 +41,21 @@ assert_rc 1 "non-numeric port rejected"
sb --print-policy --net allow --allow api.openai.com:8443
assert_contains "gate_domains=api.openai.com" "domain+port still routed through the gate"
t_name "publish is its own line, because it is a different direction"
sb --print-policy --net allow --allow example.com
assert_contains "publish=" "published ports are reported even when there are none"
assert_eq "publish=" "$(printf '%s' "$OUT" | grep '^publish=')" " ... and the line is empty, not 'none'"
sb --print-policy --net allow --allow example.com --publish 8000 --publish 18080:9000
assert_contains "publish=127.0.0.1:8000:8000 127.0.0.1:18080:9000" "canonical specs: address, host port, container port"
sb --print-policy --net allow --allow example.com --publish 0.0.0.0:8000
assert_rc 1 "a two-part spec with an address is not guessed at"
# An allow entry and a published port are answers to different questions; a config that
# reads like one must not quietly become the other.
sb --print-policy --net allow --allow 100.64.0.1:5432 --publish 8000
assert_contains "direct=100.64.0.1 5432" "the allow entry stayed an egress rule"
assert_contains "publish=127.0.0.1:8000:8000" "and the published port stayed ingress"
t_name "mixed list"
sb --print-policy --net allow --allow api.openai.com --allow 10.0.0.5:5432 --allow files.pythonhosted.org
assert_contains "api.openai.com" "domain 1"
tests/unit/06_entrypoint.sh
+174 -2
View File
@@ -41,9 +41,21 @@ printf '::1 8080\n' > "$POL/direct6.list"
RULES="$AISB_STATE_DIR/rules.f"
# Render the ruleset by extracting the generator and feeding it the policy above.
# Render the ruleset the way the entrypoint would. publish_rules() is a function of its own
# in there, so it comes along: extracting only gen_rules() would ask a half-program to run.
render_rules() { # render_rules <AISB_PUBLISH_PORTS value> -> the ruleset on stdout
local g="$WORK/render.$$.sh"
{ awk '/^publish_rules\(\) \{/{f=1} f{print} f&&/^\}$/{exit}' "$EP"
awk '/^gen_rules\(\) \{/{f=1} f{print} f&&/^\}$/{exit}' "$EP"
printf 'POLICY_DIR=%q\nAISB_PROXY_IPS="100.64.0.1"\nAISB_PROXY_PORTS="18081"\nAISB_ALLOW_ICMP=1\nAISB_ALLOW_DNS=0\nAISB_PUBLISH_PORTS=%q\n' "$POL" "$1"
printf 'gen_rules\n'; } > "$g"
bash "$g" 2>"$WORK/render.err"
}
GEN="$WORK/gen.sh"
awk '/^gen_rules\(\) \{/{f=1} f{print} f&&/^\}$/{exit}' "$EP" > "$GEN"
{ printf 'POLICY_DIR=%q\nAISB_PROXY_IPS="100.64.0.1"\nAISB_PROXY_PORTS="18081"\nAISB_ALLOW_ICMP=1\nAISB_ALLOW_DNS=0\n' "$POL"
awk '/^publish_rules\(\) \{/{f=1} f{print} f&&/^\}$/{exit}' "$EP" > "$GEN"
awk '/^gen_rules\(\) \{/{f=1} f{print} f&&/^\}$/{exit}' "$EP" >> "$GEN"
{ printf 'POLICY_DIR=%q\nAISB_PROXY_IPS="100.64.0.1"\nAISB_PROXY_PORTS="18081"\nAISB_ALLOW_ICMP=1\nAISB_ALLOW_DNS=0\nAISB_PUBLISH_PORTS=""\n' "$POL"
printf 'gen_rules\n'; } >> "$GEN"
bash "$GEN" > "$RULES" 2>"$WORK/gen.err" && _ok "rules render" || _bad "rules render" "$(tail -3 "$WORK/gen.err")"
@@ -58,9 +70,169 @@ grep -q 'ip daddr 100.64.0.1 tcp dport 18081' "$RULES" && _ok "gate endpoi
grep -q 'aisb-est' "$RULES" && _ok "established traffic accepted" || _bad "established rule"
grep -c 'accept comment "aisb-direct"' "$RULES" | grep -qx 3 && _ok "exactly 3 direct rules" || _bad "direct rule count: $(grep -c 'aisb-direct' "$RULES")"
t_name "the published ports appear in the input chain, and nowhere else"
# Nothing published means no ingress rule at all: the ruleset must not open a port that was
# never asked for, and an empty set must not render an empty port list (nft refuses those).
PUBRULES="$(render_rules "8000 18080 8000")"
# Is a line matching <regex> present inside the named chain? An ingress rule in the output
# chain would be an egress hole, which is the one thing this feature must never become.
in_chain() { # in_chain <chain> <regex>
awk -v c="chain $1 {" -v re="$2" 'index($0, c){inside=1; next} /^ }/{inside=0}
inside && $0 ~ re {found=1} END{exit !found}' <<< "$PUBRULES"
}
printf '%s' "$PUBRULES" | grep -qF 'comment "aisb-publish"' && _ok "a rule names the published ports" || _bad "aisb-publish rule" "$(printf '%s' "$PUBRULES" | sed -n '/chain input/,/}/p')"
printf '%s' "$PUBRULES" | grep -qF 'tcp dport { 8000,18080 }' && _ok "both ports, de-duplicated" || _bad "port set" "$(printf '%s' "$PUBRULES" | grep aisb-publish)"
in_chain input 'aisb-publish' && _ok "the publish rule is in the input chain" || _bad "the publish rule is in the input chain"
in_chain output 'aisb-publish' && _bad "the publish rule is also an egress hole" || _ok "publishing opens no egress path"
in_chain input 'policy drop' && _ok "input still defaults to drop with a port open" || _bad "input policy drop survived"
printf '%s' "$PUBRULES" | grep -qF 'aisb-default-drop' && _ok "the egress default-drop is untouched" || _bad "default-drop rule"
# The policy file is host-written and mounted read-only, but a ruleset is not somewhere to
# splice an unvalidated string: anything that is not a port is dropped on the floor.
BADRULES="$(render_rules '8000 nft delete table inet aisb accept')"
printf '%s' "$BADRULES" | grep -qF 'delete table' && _bad "a port value reached the ruleset as code" || _ok "a non-numeric port never reaches the ruleset"
printf '%s' "$BADRULES" | grep -qF 'tcp dport { 8000 }' && _ok " ... and the valid port in it still lands" || _bad "valid port kept" "$(printf '%s' "$BADRULES" | grep aisb-publish)"
render_rules '' | grep -qF 'aisb-publish' && _bad "a rule appeared with nothing published" || _ok "no publish rule when nothing is published"
t_name "no DNS hole when the gate is in front"
grep -E 'th dport 53' "$RULES" >/dev/null && _bad "DNS is closed in gate mode" || _ok "DNS is closed in gate mode"
t_name "the seeding script survives being passed in single quotes"
# Everything between `bash -c '` and `' aisb-seed` is one single-quoted argument of the
# outer script. One quote inside it closes that argument early and the rest of the file
# silently becomes code in the *outer* shell -- valid bash, wrong program, so neither
# `bash -n` nor shellcheck notices. Hence this test.
INNER="$WORK/seeding-inner.sh"
awk "/setpriv .*bash -c '/{f=1;next} /^ *' aisb-seed/{f=0} f" "$EP" > "$INNER"
[[ -s "$INNER" ]] && _ok "the seeding script was extracted" || _bad "the seeding script was extracted"
if grep -q "'" "$INNER"; then
_bad "it contains no single quote" "found: $(grep -n "'" "$INNER" | head -2 | tr '\n' ' ')"
else
_ok "it contains no single quote"
fi
bash -n "$INNER" && _ok "and it is valid bash on its own" || _bad "and it is valid bash on its own"
grep -q '\$lvl' "$INNER" && _ok "the thinking level is enforced inside it" || _bad "thinking level enforced"
t_name "settings.json is merged per key, not managed whole"
# Run the merge exactly as the entrypoint does, with the program extracted from the image
# script: T = what the sandbox has, B = what we last wrote, N = what the host says now.
MJQ="$WORK/merge.jq"
awk '/^SETTINGS_MERGE_JQ=/{f=1;next} f && /^.$/{exit} f' "$EP" > "$MJQ"
[[ "$(wc -l < "$MJQ")" -gt 3 ]] && _ok "the merge program was extracted" || _bad "the merge program was extracted"
M_IN="$WORK/m"; mkdir -p "$M_IN"
cat > "$M_IN/new.json" <<'EOF'
{ "defaultProvider": "gateway", "defaultModel": "turbo", "theme": "dark/dark",
"thinkingBudgets": { "xhigh": 65536 } }
EOF
cat > "$M_IN/wrote.json" <<'EOF'
{ "defaultProvider": "gateway", "defaultModel": "turbo", "theme": "dark/dark",
"thinkingBudgets": { "xhigh": 65536 } }
EOF
cat > "$M_IN/sandbox.json" <<'EOF'
{ "defaultProvider": "gateway", "defaultModel": "turbo", "theme": "light/light",
"thinkingBudgets": { "xhigh": 65536 }, "lastChangelogVersion": "1.4.0", "mine": 7 }
EOF
merge_out() { jq -c -n --slurpfile T "$1" --slurpfile B "$2" --slurpfile N "$3" -f "$MJQ"; }
M="$(merge_out "$M_IN/sandbox.json" "$M_IN/wrote.json" "$M_IN/new.json")"
[[ "$(printf '%s' "$M" | jq -r .theme)" == "light/light" ]] \
&& _ok "a key edited in the sandbox stays edited" || _bad "sandbox edit kept: $M"
[[ "$(printf '%s' "$M" | jq -r .lastChangelogVersion)" == "1.4.0" ]] \
&& _ok "a key only pi wrote survives" || _bad "sandbox-only key: $M"
[[ "$(printf '%s' "$M" | jq -r .mine)" == "7" ]] \
&& _ok "and so does one the sandbox invented" || _bad "invented key: $M"
# host moves two keys, drops a third, keeps one the sandbox changed
cat > "$M_IN/new2.json" <<'EOF'
{ "defaultProvider": "gateway", "defaultModel": "deepthink",
"thinkingBudgets": { "xhigh": 131072 } }
EOF
M2="$(merge_out "$M_IN/sandbox.json" "$M_IN/wrote.json" "$M_IN/new2.json")"
[[ "$(printf '%s' "$M2" | jq -r .defaultModel)" == "deepthink" ]] \
&& _ok "an untouched key follows the host again" || _bad "host key arrives: $M2"
[[ "$(printf '%s' "$M2" | jq -r .thinkingBudgets.xhigh)" == "131072" ]] \
&& _ok "including a nested one (whole key, whole value)" || _bad "nested key: $M2"
[[ "$(printf '%s' "$M2" | jq -r 'has("theme")')" == "true" ]] \
&& _ok "a key the host dropped stays if the sandbox had changed it" || _bad "dropped-but-edited key: $M2"
# the same drop against a file the sandbox never touched: it goes away, as it did when
# the whole file was re-seeded -- otherwise settings only ever accumulate
cat > "$M_IN/clean.json" <<'EOF'
{ "defaultProvider": "gateway", "defaultModel": "turbo", "theme": "dark/dark",
"thinkingBudgets": { "xhigh": 65536 } }
EOF
M3="$(merge_out "$M_IN/clean.json" "$M_IN/wrote.json" "$M_IN/new2.json")"
[[ "$(printf '%s' "$M3" | jq -r 'has("theme")')" == "false" ]] \
&& _ok "an untouched key the host dropped is dropped" || _bad "untouched dropped key: $M3"
# no base copy (the first run after this shipped, or .wrote deleted): the host wins on
# every key it has, which is what repairs a settings file pi had frozen.
M4="$(merge_out "$M_IN/sandbox.json" /dev/null "$M_IN/new.json")"
[[ "$(printf '%s' "$M4" | jq -r '.thinkingBudgets.xhigh')" == "65536" ]] \
&& _ok "with no base, host preferences come back (the frozen-file repair)" || _bad "no-base merge: $M4"
[[ "$(printf '%s' "$M4" | jq -r .lastChangelogVersion)" == "1.4.0" ]] \
&& _ok "and pi's own keys are kept" || _bad "no-base, pi key: $M4"
printf '{oops' > "$M_IN/broken.json"
jq -n --slurpfile T "$M_IN/broken.json" --slurpfile B /dev/null --slurpfile N "$M_IN/new.json" \
-f "$MJQ" > /dev/null 2>&1 \
&& _bad "unparseable input fails loudly for the caller to handle" || _ok "unparseable input fails loudly for the caller to handle"
t_name "the sandbox keeps its own chatter off the stream the command owns"
# stdout inside the container is data. ai-sandbox-autophase reads its preflight answer from
# it and progress.mjs reads the child pi event stream from it, so every word the sandbox says
# about itself belongs on stderr. Five notices in this very block were once on stdout, and one
# of them fires on every pi-bridged run -- which made the driver refuse a working sandbox.
# Hence a test that runs the block and separates the two streams, not one that reads it.
if grep -nE '(echo|printf).*\[aisb\]' "$EP" | grep -v '>&2' | grep -q .; then
_bad "every [aisb] line in the entrypoint is redirected" \
"$(grep -nE '(echo|printf).*\[aisb\]' "$EP" | grep -v '>&2' | head -2 | tr '\n' ' ')"
else
_ok "every [aisb] line in the entrypoint is redirected"
fi
# The extracted block, run as the entrypoint runs it (bash -c with the policy directory, the
# level and the merge program as $1..$3). Only this snippet sees the substituted HOME, and it
# starts no container, so podman's own HOME is untouched.
SP="$WORK/seed"; mkdir -p "$SP/pi"
printf '{ "providers": { "gateway": { "models": [ { "id": "turbo" } ] } } }\n' > "$SP/pi/models.json"
printf '{ "theme": "dark/dark" }\n' > "$SP/pi/settings.json"
SEED_OUT="$WORK/seed.out"; SEED_ERR="$WORK/seed.err"
seed_run() { # <level>
HOME="$SP/home" bash -c "$(cat "$INNER")" aisb-seed "$SP" "${1-xhigh}" "$(cat "$MJQ")" \
> "$SEED_OUT" 2> "$SEED_ERR"
}
seed_says() { grep -qF -- "$1" "$SEED_ERR"; }
seed_stdout() { [[ -s "$SEED_OUT" ]] && printf '%s' "$(tr '\n' ';' < "$SEED_OUT")"; }
seed_level() { jq -r '.defaultThinkingLevel // "absent"' "$SP/home/.pi/agent/settings.json" 2>/dev/null; }
seed_run xhigh; RC=$?
assert_eq 0 "$RC" "seeding runs outside a container"
[[ -z "$(seed_stdout)" ]] && _ok "and it writes nothing on stdout (cold seed)" \
|| _bad "stdout belongs to the command (cold seed): $(seed_stdout)"
seed_says "seeded models.json" && _ok "the notice went to stderr" || _bad "the notice went to stderr"
seed_says "seeded settings.json" && _ok "including the settings one" || _bad "including the settings one"
# the level is policy, so a home we create must not run one unit at the default of pi itself
assert_eq "xhigh" "$(seed_level)" "the first seed already carries the thinking policy"
seed_run xhigh; RC=$?
assert_eq 0 "$RC" "and on the next run"
[[ -z "$(seed_stdout)" ]] && _ok "nothing on stdout once the files exist (the steady state)" \
|| _bad "steady-state stdout is not clean: $(seed_stdout)"
seed_says "refreshed models.json from the host config" \
&& _ok "the line that broke the driver is on stderr" || _bad "the line that broke the driver is on stderr"
sed -i 's/turbo/juggernaut/' "$SP/home/.pi/agent/models.json"
seed_run xhigh
[[ -z "$(seed_stdout)" ]] && _ok "an in-sandbox edit is reported without touching stdout" \
|| _bad "edited-file stdout: $(seed_stdout)"
seed_says "left models.json alone" && _ok "and the explanation still arrives" || _bad "and the explanation still arrives"
printf '{oops' > "$SP/home/.pi/agent/settings.json"
seed_run xhigh
[[ -z "$(seed_stdout)" ]] && _ok "even a repair is announced off stdout" || _bad "broken-json stdout: $(seed_stdout)"
seed_says "not usable JSON" && _ok "with the warning on stderr" || _bad "with the warning on stderr"
seed_run low
assert_eq "low" "$(seed_level)" "a changed level is applied over the merged file"
seed_says "thinking -> low" && _ok "and announced there too" || _bad "and announced there too"
[[ -z "$(seed_stdout)" ]] && _ok "with stdout still empty" || _bad "level-change stdout: $(seed_stdout)"
t_name "privilege drop is the last thing that happens"
tail -5 "$EP" | grep -q 'setpriv' && _ok "exec setpriv at the end" || _bad "setpriv last: $(tail -3 "$EP" | tr '\n' ' ')"
grep -q -- '--bounding-set=-all' "$EP" && _ok "bounding set cleared" || _bad "bounding set cleared"
tests/unit/08_pi_config.sh
+69 -3
View File
@@ -37,12 +37,15 @@ cat > "$AGENT/models.json" <<EOF
EOF
# sessionDir points at the *real* host home on purpose: that is the case the bridge
# has to drop, and HOME must stay the real home for podman's sake.
# defaultThinkingLevel is deliberately NOT xhigh here: the host has to disagree with the
# ai-sandbox policy, otherwise "the policy wrote xhigh" cannot be told apart from "the
# host said xhigh and we copied it".
cat > "$AGENT/settings.json" <<EOF
{
"defaultProvider": "gateway",
"defaultModel": "turbo",
"theme": "dark/dark",
"defaultThinkingLevel": "xhigh",
"defaultThinkingLevel": "medium",
"enableInstallTelemetry": false,
"thinkingBudgets": { "xhigh": 65536 },
"sessionDir": "$HOME/pi-sessions"
@@ -110,13 +113,74 @@ t_name "settings travel, host paths do not"
S="$(gen_file settings.json)"
if [[ -n "$S" ]]; then
for pair in '"defaultProvider": "gateway"' '"defaultModel": "turbo"' '"theme": "dark/dark"' \
'"defaultThinkingLevel": "xhigh"' '"xhigh": 65536' '"enableInstallTelemetry": false'; do
'"xhigh": 65536' '"enableInstallTelemetry": false'; do # the thinking level is policy, not a preference that travels
grep -qF "$pair" "$S" && _ok "kept $pair" || _bad "kept $pair"
done
grep -qF 'sessionDir' "$S" && _bad "a value pointing at the host home is dropped" || _ok "a value pointing at the host home is dropped"
fi
assert_contains "dropped settings sessionDir" "and it says so"
# `--print-policy --keep` renders the bridge into the session directory and stops there:
# a unit file never starts a container, and --keep keeps the rendered files to look at.
rendered_level() { # rendered_level <level-or-default>
clear_sessions
sb --keep --print-policy --net none ${1:+--thinking "$1"}
local f; f="$(gen_file settings.json)"
[[ -n "$f" ]] || { printf 'none'; return; }
jq -r '.defaultThinkingLevel // "absent"' "$f" 2>/dev/null || printf 'invalid'
}
t_name "the thinking level is policy, written over the settings that travel"
sb --keep --print-policy --net none
assert_contains "thinking=xhigh (enforced on every run)" "xhigh is the default, not pi's medium"
assert_contains "pi config: thinking -> xhigh" "and it is announced"
# the fixture says medium, so an xhigh here can only have come from the policy
assert_eq "xhigh" "$(rendered_level '')" "the policy beats the host file's medium"
# the level is a preference, so it belongs to settings.json and nowhere else
clear_sessions
sb --keep --print-policy --net none
if grep -q 'defaultThinkingLevel' "${M:-/nonexistent}" 2>/dev/null; then
_bad "models.json is untouched by it" "the key appeared in the model catalog"
else
_ok "models.json is untouched by it"
fi
S="$(gen_file settings.json)"
grep -qF '"defaultProvider": "gateway"' "${S:-/nonexistent}" \
&& _ok "the rest of the settings still travel" || _bad "the rest of the settings still travel"
grep -qF '"xhigh": 65536' "${S:-/nonexistent}" \
&& _ok "including the host's thinking budgets" || _bad "the host's thinking budgets"
grep -qF 'sessionDir' "${S:-/nonexistent}" \
&& _bad "and a host path is still dropped while the level is added" || _ok "and a host path is still dropped while the level is added"
t_name "every level, including 'host' and 'off', and nothing in between"
assert_eq "low" "$(rendered_level low)" "a level of your choosing"
assert_eq "max" "$(rendered_level max)" "and the top of pi's range"
assert_eq "off" "$(rendered_level off)" "off is pi's own level, not 'do not manage it'"
assert_eq "medium" "$(rendered_level host)" "host leaves the key to settings.json (medium there)"
rendered_level host >/dev/null
assert_contains "thinking=host (not applied: the config decides)" "and says so"
rendered_level bogus >/dev/null
assert_rc 1 "an unknown level is refused"
assert_contains "must be off, minimal, low, medium, high, xhigh, max or host" "with the list to pick from"
sb_env AISB_THINKING=medium --print-policy --net none
assert_contains "thinking=medium" "AISB_THINKING works"
sb_env AISB_THINKING=medium --thinking max --print-policy --net none
assert_contains "thinking=max" "the flag beats the environment"
t_name "a host with no settings.json still gets the policy"
mv "$AGENT/settings.json" "$WORK/settings-away.json"
assert_eq "xhigh" "$(rendered_level '')" "the level does not depend on the host having the file"
clear_sessions
sb --keep --print-policy --net none
assert_contains "pi_endpoints" "the model bridge still happened alongside it"
mv "$WORK/settings-away.json" "$AGENT/settings.json"
t_name "no bridge means no policy either, and it says so"
clear_sessions
sb --keep --no-pi-config --print-policy --net none
assert_contains "thinking=xhigh (not applied: pi config bridging is off)" "--no-pi-config declines the thinking policy too"
if [[ -n "$(gen_file settings.json)" ]]; then _bad "nothing is rendered for pi at all"; else _ok "nothing is rendered for pi at all"; fi
t_name "auth.json and the credential store are never bridged"
if find "$AISB_STATE_DIR/sessions" \( -name auth.json -o -name models-store.json \) | grep -q .; then
_bad "only models.json and settings.json are bridged"
@@ -130,6 +194,7 @@ else
fi
t_name "the token is passed by name, once, and announced"
sb --keep --print-policy --net none
assert_contains "passing \$AI_API_TOKEN from the host" "the auto-pass is announced"
assert_eq "env_names=AI_API_TOKEN" "$(printf '%s' "$OUT" | grep '^env_names=' || true)" "exactly one variable is passed"
assert_absent "$AI_API_TOKEN" "and only its name is reported"
@@ -142,12 +207,13 @@ assert_contains "will not reach the container" "preview warns instead of launchi
t_name "a bridged provider with no credential in reach is refused, not attempted"
unset AI_API_TOKEN
snapshot_containers
sb --net none --harness pi
assert_rc 1 "the run refuses to start"
assert_contains 'asks for $AI_API_TOKEN, but it is not going to reach the container' "it names the variable and why"
assert_contains "env = AI_API_TOKEN" "it shows the config-file way"
assert_contains "--no-pi-config" "and the way to opt out"
assert_eq "0" "$([[ -n "$(podman ps -q --filter label=ai-sandbox=1)" ]] && echo 1 || echo 0)" "and nothing was started"
assert_no_new_containers "and nothing was started (not even a container someone else runs)"
t_name "an empty token is treated as missing"
sb_env AI_API_TOKEN= --net none --harness pi # set, but empty
tests/unit/10_autophase.sh
+707
View File
@@ -0,0 +1,707 @@
#!/usr/bin/env bash
# tests/unit/10_autophase.sh -- bin/ai-sandbox-autophase: the pipeline driver.
#
# Nothing here starts a container. The two things the driver is built on are both
# stood in for, faithfully:
#
# * the LAUNCHER -- a sandbox-pi() function written into a fixture rc file, exactly
# like the one people keep in ~/.bashrc.d. It records the argv it was handed (that
# is what "the launcher, then --, then the harness command" has to mean) and then
# runs what it was told to run, so the pipeline is exercised for real. It also
# exports a canary token the way the real one calls `pass`, which is how these
# tests check that no printed surface repeats a value.
# * the PIPELINE LIBRARY -- tests/fixtures/phase-lib is a contract fixture: the same
# function names, the same `pi ... | progress` call, the same gate call, the same
# host-side commit+push at a phase's commit point. The last test runs the REAL
# library as well, when it is installed.
#
# The git remote is a local bare repository: a push really happens, and really fails on
# demand, with no network involved.
source "$TESTS/lib.sh"
APBIN="$(dirname "$AISB")/ai-sandbox-autophase"
LIB_FIX="$TESTS/fixtures/phase-lib/lib.sh"
CANARY="CANARY-TOKEN-9d41f6"
REAL_GIT="$(command -v git)"
mkdir -p "$WORK/bin"
t_name "the driver exists"
[[ -x "$APBIN" ]] && _ok "bin/ai-sandbox-autophase is executable" || _bad "bin/ai-sandbox-autophase is executable"
# --- the launcher, as it lives in an rc file ---------------------------------
mkdir -p "$WORK/rc"
cat > "$WORK/rc/00-nothing.bash" <<'EOF'
# defines nothing the driver asked for: the search must keep looking
unrelated_setup() { PATH="$PATH:/nowhere"; }
EOF
cat > "$WORK/rc/sandbox.bash" <<EOF
sandbox-pi() {
export AI_API_TOKEN='$CANARY'
local trace="\${AISB_AP_TRACE:-/dev/null}"
local -a argv0=("\$@") cmd=()
while (( \$# )); do
case "\$1" in
--allow|--env|--pi-token-env|--mount) shift 2 ;;
--no-tty|--no-interactive|--dry-run) shift ;;
--) shift; cmd=("\$@"); break ;;
*) shift ;;
esac
done
{
printf 'LAUNCH argc=%s cmd=%s\n' "\${#argv0[@]}" "\${cmd[0]:-none}"
local a
for a in "\${argv0[@]}"; do printf 'A:%s\n' "\$a"; done
# what this child was started with on stdin: the driver closes it, so an agent inside
# the sandbox cannot end up reading the terminal of an unattended run
printf 'STDIN:%s\n' "\$(readlink /proc/self/fd/0)"
} >> "\$trace"
(( \${#cmd[@]} )) || return 0
"\${cmd[@]}"
}
EOF
cat > "$WORK/rc/aliases.bash" <<'EOF'
alias sandbox-alias='sandbox-pi'
EOF
# The launcher exactly as it is often written for interactive use -- a fixed policy, and
# nothing after it. It works when *you* type its name, and drops the whole in-container
# command when something appends one, which is every run the driver makes. Kept as a
# fixture so the driver is checked to name it instead of hanging on it.
cat > "$WORK/rc/stubby.bash" <<'EOF'
stubby-pi() {
export AI_API_TOKEN='CANARY-STUBBY-4f2c81'
ai-sandbox --allow api.example.com --env AI_API_TOKEN
}
EOF
# An rc file that leaves a grandchild behind, the way one that starts an agent or a
# helper daemon does. The definition probe must not wait for it: reading a definition is
# not a conversation with the file's process tree.
cat > "$WORK/rc/daemon.bash" <<'EOF'
daemon-pi() {
sleep 10 &
ai-sandbox "$@"
}
EOF
cat > "$WORK/rc/broken.bash" <<'EOF'
broken-sandbox() {
local -a cmd=()
while (($#)); do case "$1" in --) shift; cmd=("$@"); break ;; *) shift ;; esac; done
echo "Error: podman: executable not found" >&2
return 125
}
EOF
# A launcher that puts a notice of its own on stdout before the command runs -- the shape the
# sandbox entrypoint had, where the pi-config notice came first on every bridged run. The
# sandbox does answer, so the run is workable: what the driver owes the user is a warning that
# names the pollution (the same stream is parsed as data every unit), not a refusal.
cat > "$WORK/rc/noisy.bash" <<'EOF'
noisy-pi() {
local -a cmd=()
while (($#)); do case "$1" in --) shift; cmd=("$@"); break ;; *) shift ;; esac; done
printf 'LAUNCH cmd=%s\n' "${cmd[0]:-none}" >> "${AISB_AP_TRACE:-/dev/null}"
printf '[aisb] pi config: refreshed models.json from the host config\n'
"${cmd[@]}"
}
EOF
# The same talking with no answer in it. That is a broken launcher, and it still has to stop
# the run -- with the foreign stdout quoted back rather than a bare exit code.
cat > "$WORK/rc/yap.bash" <<'EOF'
yap-pi() {
local -a cmd=()
while (($#)); do case "$1" in --) shift; cmd=("$@"); break ;; *) shift ;; esac; done
printf 'hello, I am a launcher that runs nothing\n'
}
EOF
# --- the harness and the git shim -------------------------------------------
cat > "$WORK/bin/pi" <<'EOF'
#!/usr/bin/env bash
# fake pi: does the work, writes a session file, emits the stream a real pi emits.
name=""; sessdir="."; prompt=""
while (($#)); do
case "$1" in
--session-dir) sessdir="$2"; shift 2 ;;
--name) name="$2"; shift 2 ;;
--session|--mode|--model|--thinking) shift 2 ;;
--no-session|--no-approve|--approve) shift ;;
--*) shift ;;
*) prompt="$1"; shift ;;
esac
done
[[ -n "$prompt" ]] || { echo "fake pi: no prompt arrived" >&2; exit 3; }
# the token has to reach the sandboxed agent -- recorded as a length, never a value
printf 'token_bytes=%s\n' "${#AI_API_TOKEN}" >> "${FAKE_PI_MARKER:-/dev/null}"
printf '%s\n---\n' "$prompt" >> "${FAKE_PI_PROMPTS:-/dev/null}"
printf 'work by %s\n' "${name:-unit}" >> work.txt
# an agent that tries to make the *host* run something when the harness commits
if [[ -n "${FAKE_PI_PLANT_HOOK:-}" ]]; then
mkdir -p .git/hooks
printf '#!/bin/sh\ntouch "%s"\n' "$FAKE_PI_PLANT_HOOK" > .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit
fi
mkdir -p "$sessdir"
{
printf '{"type": "session", "id": "fake-%s"}\n' "${name:-s}"
printf '{"type": "message_end", "message": {"role": "assistant", "content": [{"type": "text", "text": "report for %s"}]}}\n' "${name:-s}"
printf '{"type": "agent_end"}\n'
} | tee "${sessdir}/${name:-session}.jsonl"
EOF
cat > "$WORK/bin/git" <<'EOF'
#!/usr/bin/env bash
printf '%s\n' "$*" >> "${FAKE_GIT_TRACE:-/dev/null}"
if [[ "$1" == push && -n "${FAKE_GIT_FAIL:-}" ]]; then
echo "fatal: unable to access '${FAKE_GIT_FAIL}': Could not resolve host" >&2
exit 1
fi
exec "$REAL_GIT" "$@"
EOF
chmod +x "$WORK/bin/pi" "$WORK/bin/git"
export GIT_AUTHOR_NAME=test GIT_AUTHOR_EMAIL=t@example.invalid
export GIT_COMMITTER_NAME=test GIT_COMMITTER_EMAIL=t@example.invalid
# --- a phased project, with a local bare repo as its remote ------------------
# Every project gets its own bare remote: they all commit a branch called main, and a
# shared remote would turn the second project's push into a non-fast-forward that has
# nothing to do with what is being tested.
mkproj() { # <dir> [phases] -> the remote it pushed its init commit to
local p="$1" n="${2:-1}" REMOTE
# Named after the project, not counted: a counter incremented in a command
# substitution would stay in the subshell and hand every project the same remote,
# which is a non-fast-forward waiting to happen.
REMOTE="$WORK/remotes/$(basename "$p").git"
mkdir -p "$WORK/remotes"
"$REAL_GIT" init -q --bare "$REMOTE" >/dev/null 2>&1
mkdir -p "$p/.agents/phases/todo/01_one"
printf '# task one\n\nDo the thing.\n' > "$p/.agents/phases/todo/01_one/01_task.md"
printf '# final pass one\n\nWrap up phase 01_one.\n' > "$p/.agents/phases/todo/01_one/00_phase.md"
if [[ "$n" == 2 ]]; then
mkdir -p "$p/.agents/phases/todo/02_two"
printf '# task two\n\nDo the next thing.\n' > "$p/.agents/phases/todo/02_two/01_task.md"
printf '# final pass two\n\nWrap up phase 02_two.\n' > "$p/.agents/phases/todo/02_two/00_phase.md"
fi
printf '#!/usr/bin/env bash\nexit 0\n' > "$p/.agents/validate.sh"
printf '.agents/phase-sessions/\n*.log\n' > "$p/.gitignore"
( cd "$p" && "$REAL_GIT" init -q -b main . && "$REAL_GIT" remote add origin "$REMOTE" \
&& "$REAL_GIT" add -A && "$REAL_GIT" commit -q --no-gpg-sign -m 'init' \
&& "$REAL_GIT" push -q -u origin main ) >/dev/null 2>&1
printf '%s\n' "$REMOTE" # so a test can count what actually reached the remote
}
TRACE="$WORK/launcher.trace"
GTRACE="$WORK/git.trace"
MARKER="$WORK/token.marker"
PROMPTS="$WORK/prompts.log"
: > "$TRACE"; : > "$GTRACE"; : > "$MARKER"; : > "$PROMPTS"
printf 'the driver stdin, which no child is allowed to see\n' > "$WORK/stdin.marker"
DEF_RC="$WORK/rc/sandbox.bash:$WORK/rc/aliases.bash:$WORK/rc/00-nothing.bash"
# A prefix assignment in front of a shell function sticks around for the rest of the
# script, so ap_run reads every T_* into a local and then unsets it: each run starts
# from the defaults unless the test asked otherwise, and cannot inherit the last one.
ap_run() { # [args...]
local cmd="${T_CMD-sandbox-pi}" rc="${T_RC-$DEF_RC}" lib="${T_LIB-$LIB_FIX}"
local validate="${T_VALIDATE-sandbox}" sargs="${T_SANDBOX_ARGS---no-tty --no-interactive}"
local push="${T_PUSH-1}" pre="${T_PREFLIGHT-1}" notes="${T_NOTES-1}" gfail="${T_GITFAIL-}"
local ppush="${T_PHASE_PUSH-}" dryout="${T_DRYOUT-}" plant="${T_PLANT-}" stdin_src="${T_STDIN-$WORK/stdin.marker}"
unset T_CMD T_RC T_LIB T_VALIDATE T_SANDBOX_ARGS T_PUSH T_PREFLIGHT T_NOTES T_GITFAIL T_PHASE_PUSH T_DRYOUT T_PLANT T_STDIN
# stdin is a file with a known name, never the test runner's own: the launcher records
# what it was started with, and that is how "a child never inherits the terminal" is
# checked. A driver that forwards stdin would show up as this path in the trace.
OUT="$(env "PATH=$WORK/bin:$PATH" \
"AISB_AP_LIB=$lib" "AISB_AP_RC_FILES=$rc" "AISB_AP_PI_CMD=$cmd" \
"AISB_AP_VALIDATE=$validate" "AISB_AP_SANDBOX_ARGS=$sargs" \
"AISB_AP_PUSH=$push" "AISB_AP_PREFLIGHT=$pre" "AISB_AP_NOTES=$notes" \
"AISB_AP_TRACE=$TRACE" "FAKE_GIT_TRACE=$GTRACE" "FAKE_PI_MARKER=$MARKER" \
"FAKE_PI_PROMPTS=$PROMPTS" "REAL_GIT=$REAL_GIT" "PHASE_NOTIFY=0" \
${gfail:+"FAKE_GIT_FAIL=$gfail"} ${ppush:+"PHASE_PUSH=$ppush"} ${dryout:+"AISB_AP_DRYOUT=$dryout"} \
${plant:+"FAKE_PI_PLANT_HOOK=$plant"} \
"$APBIN" "$@" 2>&1 < "$stdin_src")" && RC=0 || RC=$?
return 0
}
gt() { grep -c "^$1" "$GTRACE" 2>/dev/null || true; } # count of git calls by verb
trace_has() { grep -qF -- "$1" "$TRACE"; }
PROJ="$WORK/proj"; mkproj "$PROJ" 1 >/dev/null
cd "$PROJ" || exit 1
t_name "the launcher is resolved from the rc files, and is never run to find it"
ap_run --status
assert_rc 0 "--status"
assert_contains "sandbox-pi" "names the configured command"
assert_contains "function, from $WORK/rc/sandbox.bash" "found as a function, in the file that defines it"
assert_absent "$CANARY" "the token never appears"
if [[ -s "$TRACE" ]]; then _bad "resolving does not run the launcher" "the launcher was called"; else _ok "resolving does not run the launcher"; fi
ap_run --dry-run
assert_absent "$CANARY" "--dry-run does not either"
t_name "an alias is a launcher too"
T_CMD=sandbox-alias ap_run --status
assert_rc 0 "--status with an alias"
assert_contains "alias, from $WORK/rc/aliases.bash" "found as an alias"
t_name "a program on PATH, and a whole command line"
T_CMD=pi ap_run --status
assert_rc 0 "a program on PATH"
assert_contains "program, from $WORK/bin/pi" "and where it came from"
T_CMD='ai-sandbox --allow api.example.com --env API_KEY' ap_run --status
assert_rc 0 "a command line"
assert_contains "arguments, from --pi-cmd" "used verbatim"
T_CMD='ai-sandbox --env API_KEY=SECRETLITERALVALUE --allow api.example.com' ap_run --status
assert_absent "SECRETLITERALVALUE" "a literal --env value in --pi-cmd is not echoed back"
assert_contains "--env API_KEY" "while the name still is"
t_name "an unresolvable launcher is refused with what was searched"
T_CMD=no-such-sandbox-cmd ap_run --status
assert_rc 1 "nothing by that name anywhere"
assert_contains "cannot resolve the sandbox launcher" "says what failed"
assert_contains "$WORK/rc/00-nothing.bash" "lists the rc files it searched"
T_CMD='' ap_run --status
assert_rc 1 "an explicitly empty launcher is an error, not a silent fallback"
assert_contains "no sandbox launcher" "and says so"
# Extract one function from the driver itself, so a test asks the real code a question
# instead of restating its answer (the way aisb_fn() does for bin/ai-sandbox).
ap_fn() { # <name>
awk -v fn="$1" '$0 ~ "^" fn "\\(\\) \\{" {f=1} f {print} f && /^\}$/ {exit}' "$APBIN"
}
t_name "a launcher that drops its arguments is named, and never hangs a run"
# The shape every interactive sandbox-pi takes: a fixed policy and no "$@". Appended to,
# it runs the image's default harness with no prompt, so an unattended run of the real
# thing sat in front of an agent waiting for keystrokes, behind a pipe that showed
# nothing. The driver has to say so, in the preview and before the first unit.
: > "$TRACE"
T_CMD=stubby-pi T_RC="$WORK/rc/stubby.bash" ap_run --status
assert_rc 0 "--status with a launcher that takes no arguments"
assert_contains "drops the arguments appended to it" "the preview says so"
assert_contains "End the function with" "and says how to fix it"
assert_absent "CANARY-STUBBY" "without repeating what the function exports"
assert_absent "export AI_API_TOKEN" "and without printing its body"
T_CMD=stubby-pi T_RC="$WORK/rc/stubby.bash" ap_run --max-units 1
assert_rc 1 "a run with it is refused, not left waiting"
assert_contains "preflight failed" "at the preflight, before any unit"
assert_contains "ignores its arguments" "with the reason named"
assert_contains 'ai-sandbox --allow ... --env NAME "$@"' "and the fix written out"
if grep -q 'cmd=pi' "$TRACE"; then _bad "no unit runs"; else _ok "no unit runs"; fi
# the fixture launcher in the tests forwards "\$@", and must not be nagged about it
ap_run --status
assert_absent "drops the arguments" "a launcher that forwards them is not flagged"
t_name "reading the rc files cannot be stopped, and cannot be held open"
# An interactive bash started from a terminal takes that terminal: it calls tcsetpgrp(),
# gets SIGTTOU because it is in the background, and stops -- and a stopped process does
# not take SIGTERM, so the probe waited on it forever with the driver waiting behind it.
# Everything a run does starts in this function, previews included, so the whole script
# looked wedged with nothing printed. The three defences are asserted here by shape;
# stopping a process is not something a test can arrange without a terminal of its own.
probe_src="$(ap_fn ap_probe_rc)"
for want in 'AP_SETSID' '< /dev/null' 'timeout -k 5 15' 'mktemp' '> "$tmp"'; do
if printf '%s' "$probe_src" | grep -qF -- "$want"; then
_ok "the probe is bounded, file-backed and off the terminal ($want)"
else
_bad "the probe is bounded, file-backed and off the terminal" "missing: $want"
fi
done
if grep -qF 'AP_SETSID=(setsid)' "$APBIN"; then
_ok "and its own session, so it has no controlling terminal to be stopped by"
else
_bad "and its own session, so it has no controlling terminal to be stopped by" "no AP_SETSID=(setsid) in $APBIN"
fi
# and the consequence a user would feel: an rc file that leaves a process behind must not
# be able to hold the read open. sleep 10 in the fixture is the stand-in for the agent or
# helper daemon a real rc file starts; if the probe reads a pipe instead of a file, this
# test does not fail fast -- it fails 10 seconds later, which is the whole point.
: > "$TRACE"
start=$SECONDS
T_CMD=daemon-pi T_RC="$WORK/rc/daemon.bash" ap_run --status
elapsed=$(( SECONDS - start ))
assert_rc 0 "--status with an rc file that backgrounds a process"
assert_contains "function, from $WORK/rc/daemon.bash" "resolved all the same"
(( elapsed < 8 )) && _ok "without waiting for what it left behind (${elapsed}s)" \
|| _bad "without waiting for what it left behind" "took ${elapsed}s"
t_name "a child is the launcher, then --, then the harness command"
: > "$TRACE"
PROJ_A="$WORK/pa"; mkproj "$PROJ_A" 1 >/dev/null
cd "$PROJ_A" || exit 1
ap_run --dry-run
assert_rc 0 "--dry-run"
assert_contains "-- pi <pi args> <prompt>" "the pi command rides after --"
assert_contains "--no-tty" "no TTY for the JSON stream to corrupt"
assert_contains "--no-interactive" "and your keystrokes cannot reach the agent"
: > "$TRACE"
T_SANDBOX_ARGS='' ap_run --dry-run
assert_absent "--no-tty" "--sandbox-args replaces the defaults rather than adding to them"
assert_rc 0 "and an explicitly empty value means none, not the default"
: > "$TRACE"
ap_run --max-units 1
assert_rc 0 "one unit"
trace_has "A:--no-interactive" "the launcher got the run flags"
if grep -q '^LAUNCH argc=[0-9]* cmd=pi$' "$TRACE"; then
_ok "pi is the in-container command"
else
_bad "pi is the in-container command" "$(grep '^LAUNCH' "$TRACE" | head -3)"
fi
# Every child starts with stdin on /dev/null, whatever this test's own stdin is: the
# flag that normally keeps podman off the terminal (--no-interactive) is a flag, and a
# launcher is free to drop it.
if grep -q '^STDIN:/dev/null$' "$TRACE"; then
_ok "the child's stdin is /dev/null, not the caller's"
else
_bad "the child's stdin is /dev/null, not the caller's" "$(grep '^STDIN' "$TRACE" | head -3)"
fi
if grep -qF "stdin.marker" "$TRACE"; then _bad "the run's own stdin never reaches a child"; else _ok "the run's own stdin never reaches a child"; fi
argc="$(grep '^LAUNCH argc=.* cmd=pi$' "$TRACE" | head -1 | sed 's/.*argc=\([0-9]*\).*/\1/')"
[[ -n "$argc" && "$argc" -ge 8 ]] && _ok "the whole harness command line reached the launcher ($argc arguments)" \
|| _bad "the whole harness command line reached the launcher" "argc=$argc"
if grep -A1 '^A:--$' "$TRACE" | grep -q '^A:pi$'; then
_ok "-- is the separator, and nothing after it is a sandbox flag"
else
_bad "-- is the separator, and nothing after it is a sandbox flag"
fi
t_name "the prompt arrives as one argument, with the sandbox rules in it"
if grep -qF 'Sandbox rules for this run' "$PROMPTS"; then
_ok "the agent is told it is inside a sandbox"
else
_bad "the agent is told it is inside a sandbox" "$(tail -5 "$PROMPTS")"
fi
grep -qF 'git push' "$PROMPTS" && _ok "it is told the remote is unreachable, by name" \
|| _bad "it is told the remote is unreachable, by name"
grep -qF '01_one/01_task.md' "$PROMPTS" && _ok "the unit it is running is in the prompt" \
|| _bad "the unit it is running is in the prompt"
: > "$PROMPTS"
T_NOTES=0 ap_run --unit 01_one/00_phase.md
assert_rc 0 "one unit, notes off"
if grep -qF 'Sandbox rules' "$PROMPTS"; then _bad "--no-notes drops the block"; else _ok "--no-notes drops the block"; fi
grep -qF '01_one/00_phase.md' "$PROMPTS" && _ok "the library's own prompt still arrives" \
|| _bad "the library's own prompt still arrives" "$(head -3 "$PROMPTS")"
t_name "the token reaches the agent"
if grep -q "token_bytes=${#CANARY}" "$MARKER"; then
_ok "exported by the launcher, present in the child"
else
_bad "exported by the launcher, present in the child" "$(tail -1 "$MARKER" 2>/dev/null)"
fi
t_name "the gate runs where the agent runs"
: > "$TRACE"
PROJ_B="$WORK/pb"; mkproj "$PROJ_B" 1 >/dev/null
cd "$PROJ_B" || exit 1
ap_run --unit 01_one/01_task.md
assert_rc 0 "one task"
if grep -q '^A:.agents/validate.sh$' "$TRACE"; then
_ok ".agents/validate.sh is handed to the launcher"
else
_bad ".agents/validate.sh is handed to the launcher" "$(tail -5 "$TRACE")"
fi
: > "$TRACE"
T_VALIDATE=host ap_run --unit 01_one/00_phase.md
assert_rc 0 "one unit, host gate"
if grep -q '^A:.agents/validate.sh$' "$TRACE"; then _bad "host mode does not consult the launcher"; else _ok "host mode does not consult the launcher"; fi
assert_contains "outside the sandbox" "and the choice is announced, because it is the risky one"
T_VALIDATE=host ap_run --status
assert_contains "on the host (see --validate-in)" "and --status says where the gate would run"
assert_contains "gate will run on the HOST" "and says so loudly: the gate file is agent-writable"
T_VALIDATE=sideways ap_run --status
assert_rc 1 "and a gate location that is neither is refused"
t_name "a completed phase is committed and pushed by the host"
PROJ1="$WORK/p1"; R1="$(mkproj "$PROJ1" 1)"
: > "$GTRACE"; cd "$PROJ1" || exit 1
ap_run
assert_rc 0 "the pipeline ran to completion"
if grep -q 'report for 01_one__00_phase' "$PROJ1/.agents/reports/01_one/01_one__00_phase.a1.md" 2>/dev/null; then
_ok "the final pass ran last, and its report is on disk"
else
_bad "the final pass ran last, and its report is on disk" "$(ls "$PROJ1/.agents/reports/01_one" 2>/dev/null)"
fi
if grep -q '^push' "$GTRACE"; then _ok "a push went out for the phase"; else _bad "a push went out for the phase"; fi
assert_contains "up to date with origin/main" "and the end-of-run check confirmed the remote has it"
if [[ -d "$PROJ1/.agents/phases/complete/01_one" && ! -d "$PROJ1/.agents/phases/todo/01_one" ]]; then
_ok "the phase moved to complete/"
else
_bad "the phase moved to complete/" "$(ls "$PROJ1/.agents/phases"/* 2>/dev/null)"
fi
if ( cd "$PROJ1" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then
_ok "the host made the phase commit"
else
_bad "the host made the phase commit" "$( cd "$PROJ1" && "$REAL_GIT" log --oneline | head -3 )"
fi
assert_eq 2 "$( git --git-dir="$R1" rev-list --count --all )" "the remote has init + the phase"
t_name "the driver owns delivery: push at the phase boundary, re-check, push again at the end"
PROJ2="$WORK/p2"; R2="$(mkproj "$PROJ2" 2)"
: > "$GTRACE"; cd "$PROJ2" || exit 1
ap_run
assert_rc 0 "two phases ran"
commits="$(gt commit)"; pushes="$(gt push)"
[[ "$commits" == 2 ]] && _ok "both phases committed on the host" || _bad "both phases committed on the host" "commits=$commits"
[[ "$pushes" == "$commits" ]] && _ok "every commit is delivered, not left for a later run ($pushes)" \
|| _bad "every commit is delivered, not left for a later run" "commits=$commits pushes=$pushes"
assert_contains "phase 01_one" "the push is attributed to the phase that made it"
assert_contains "up to date with origin/main" "and the next phase's start re-checks before running it"
assert_eq 3 "$( git --git-dir="$R2" rev-list --count --all )" "the remote has init + both phases"
# PHASE_PUSH is the library's switch; it means the same thing here, so it is honoured
# rather than overridden -- and honoured means nothing goes out, in either direction.
PROJ3="$WORK/p3"; R3="$(mkproj "$PROJ3" 2)"
: > "$GTRACE"; cd "$PROJ3" || exit 1
T_PHASE_PUSH=0 ap_run
assert_rc 0 "the pipeline still ran"
if grep -q '^push' "$GTRACE"; then _bad "PHASE_PUSH=0 pushes nothing"; else _ok "PHASE_PUSH=0 pushes nothing"; fi
assert_contains "push disabled" "and says so"
assert_eq 1 "$( git --git-dir="$R3" rev-list --count --all )" "the remote is untouched"
t_name "--no-push leaves the commits local, and says so"
PROJ4="$WORK/p4"; R4="$(mkproj "$PROJ4" 1)"
: > "$GTRACE"; cd "$PROJ4" || exit 1
ap_run --no-push
assert_rc 0 "the pipeline ran"
if grep -q '^push' "$GTRACE"; then _bad "nothing is pushed" "$(grep '^push' "$GTRACE")"; else _ok "nothing is pushed"; fi
assert_contains "push disabled" "and that is stated, not silent"
if [[ "$(gt commit)" == 1 ]] && ( cd "$PROJ4" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then
_ok "the work is still committed locally"
else
_bad "the work is still committed locally" "$( cd "$PROJ4" && "$REAL_GIT" log --oneline | head -3 )"
fi
assert_eq 1 "$( git --git-dir="$R4" rev-list --count --all )" "the remote is untouched"
t_name "a failed push stops the run, with the hand-fix command"
PROJ5="$WORK/p5"; mkproj "$PROJ5" 1 >/dev/null
: > "$GTRACE"; cd "$PROJ5" || exit 1
T_GITFAIL="https://user:pushcanary@remote.invalid/repo.git" ap_run
assert_rc 1 "the run stops on a failed push"
assert_contains "the push FAILED" "says what failed"
assert_contains "finish by hand" "and how to finish it"
assert_absent "pushcanary" "a credential in the remote URL is not repeated in the output"
: > "$GTRACE"
ap_run # the remote answers this time
assert_contains "unpushed commit" "the next run notices what is still local"
assert_contains "pushed:" "and takes it out"
# and a run that could not deliver never reports success
PROJ5B="$WORK/p5b"; mkproj "$PROJ5B" 1 >/dev/null; cd "$PROJ5B" || exit 1
T_GITFAIL="https://user:pushcanary@remote.invalid/repo.git" ap_run
assert_rc 1 "a run that could not deliver does not report success"
assert_contains "the push FAILED" "and says which part failed"
if ( cd "$PROJ5B" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then
_ok "its commit is still local and safe"
else
_bad "its commit is still local and safe"
fi
if ( cd "$PROJ5" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then
_ok "the phase stayed complete through it"
else
_bad "the phase stayed complete through it"
fi
t_name "nothing a preview prints carries a value"
PROJ6="$WORK/p6"; mkproj "$PROJ6" 1 >/dev/null
cd "$PROJ6" || exit 1
: > "$TRACE"
out_status="$(cd "$PROJ6" && env PATH="$WORK/bin:$PATH" AISB_AP_LIB="$LIB_FIX" \
AISB_AP_RC_FILES="$WORK/rc/sandbox.bash" AISB_AP_PI_CMD=sandbox-pi \
AISB_AP_TRACE="$TRACE" "$APBIN" --status 2>&1)"
out_dry="$(cd "$PROJ6" && env PATH="$WORK/bin:$PATH" AISB_AP_LIB="$LIB_FIX" \
AISB_AP_RC_FILES="$WORK/rc/sandbox.bash" AISB_AP_PI_CMD=sandbox-pi \
AISB_AP_TRACE="$TRACE" "$APBIN" --dry-run 2>&1)"
for v in "$out_status" "$out_dry"; do
if printf '%s' "$v" | grep -qF -- "$CANARY"; then
_bad "a preview does not echo the launcher's secret"
else
_ok "a preview does not echo the launcher's secret"
fi
if printf '%s' "$v" | grep -qF "export AI_API_TOKEN"; then
_bad "a preview does not print the launcher's body"
else
_ok "a preview does not print the launcher's body"
fi
done
if [[ -s "$TRACE" ]]; then _bad "a preview runs nothing"; else _ok "a preview runs nothing"; fi
assert_eq "$( git -C "$PROJ6" rev-list --count HEAD )" "$( git -C "$PROJ6" rev-list --count HEAD )" "a preview changes no git state"
if [[ -e "$PROJ6/.agents/reports" ]]; then _bad "a preview writes no reports"; else _ok "a preview writes no reports"; fi
if [[ -e "$PROJ6/.agents/phases/complete/01_one" ]]; then _bad "a preview does not move units"; else _ok "a preview does not move units"; fi
OUT="$out_status"
assert_contains "01_one/01_task.md" "but it does list what is pending"
assert_contains "01_one/00_phase.md" "task first, then the final pass"
OUT="$out_dry"
assert_contains "one container per unit" "and --dry-run says what it would do"
t_name "preflight refuses to start a pipeline the sandbox cannot run"
cd "$PROJ6" || exit 1
: > "$TRACE"
T_CMD=broken-sandbox T_RC="$WORK/rc/broken.bash" ap_run --max-units 1
assert_rc 1 "preflight stops the run"
assert_contains "preflight failed" "and says so"
assert_contains "executable not found" "with the launcher's own error, for diagnosis"
if grep -q 'cmd=pi' "$TRACE"; then _bad "no unit runs"; else _ok "no unit runs"; fi
T_PREFLIGHT=0 ap_run --max-units 1
assert_rc 0 "--no-preflight is available for the runs that cannot afford it"
t_name "a sandbox that talks on stdout is named, not refused"
cd "$PROJ6" || exit 1
: > "$TRACE"
T_CMD=noisy-pi T_RC="$WORK/rc/noisy.bash" ap_run --max-units 1
assert_rc 0 "a launcher that answers is a working sandbox, banner or not"
assert_contains "aisb-ok" "the answer is taken from wherever it is in the stream"
assert_contains "put lines of its own on stdout" "and the pollution itself is named"
assert_contains "> [aisb] pi config: refreshed" "with the offending line quoted back"
if grep -q 'cmd=pi' "$TRACE"; then _ok "the pipeline went on to run its unit"; else _bad "the pipeline went on to run its unit: $(tr '\n' ';' < "$TRACE")"; fi
t_name "a launcher that talks instead of answering is still refused"
cd "$PROJ6" || exit 1
: > "$TRACE"
T_CMD=yap-pi T_RC="$WORK/rc/yap.bash" ap_run --max-units 1
assert_rc 1 "no aisb-ok answer means no run"
assert_contains "preflight failed" "and it says so"
assert_contains "no aisb-ok answer at all" "naming what the check was looking for"
assert_contains "> hello, I am a launcher that runs nothing" "quoting the stdout that came back instead"
if grep -q 'cmd=pi' "$TRACE"; then _bad "no unit runs"; else _ok "no unit runs"; fi
t_name "bad usage is refused before anything happens"
cd "$PROJ6" || exit 1
ap_run --not-a-flag
assert_rc 2 "an unknown option"
ap_run --max-units lots
assert_rc 1 "--max-units wants a number"
cd / || exit 1
ap_run --status
assert_rc 1 "outside a phased project"
assert_contains "no .agents/phases/todo" "and it says what is missing"
cd "$PROJ6" || exit 1
t_name "a library that does not match the contract is refused"
printf 'find_root() { printf "%%s\\n" "$PWD"; }\nnext_unit() { :; }\n' > "$WORK/rc/half-lib.sh"
T_LIB="$WORK/rc/half-lib.sh" ap_run --status
assert_rc 1 "a library missing pieces is not run against"
assert_contains "does not provide" "and the missing function is named"
t_name "a hook planted from inside the sandbox never runs on the host"
# The commit is made out here, by us, after an agent had the tree writeable. A repository
# does not merely record .git/hooks and .git/config -- it runs them. So the delivery
# refuses when they changed, and even when it does not, our git never looks in .git/hooks.
HOOKPROOF="$WORK/hook-ran"
PROJ9="$WORK/p9"; R9="$(mkproj "$PROJ9" 1)"
cd "$PROJ9" || exit 1
: > "$GTRACE"
T_PLANT="$HOOKPROOF" ap_run
assert_rc 1 "a run that rewrote the repository's own git config does not deliver"
assert_contains "refusing to deliver" "and says why"
assert_contains "--trust-git-changes" "and names the way through"
if [[ -e "$HOOKPROOF" ]]; then _bad "the planted pre-commit hook did not run"; else _ok "the planted pre-commit hook did not run"; fi
if grep -q '^push' "$GTRACE"; then _bad "nothing was pushed"; else _ok "nothing was pushed"; fi
assert_eq 1 "$( git --git-dir="$R9" rev-list --count --all )" "the remote still has only the init commit"
ap_run --trust-git-changes
assert_rc 0 "the named escape hatch does deliver"
assert_contains "pushed:" "and says so"
if [[ -e "$HOOKPROOF" ]]; then _bad "and still the hook did not run"; else _ok "and still the hook did not run: our git uses an empty hooks path"; fi
assert_eq 2 "$( git --git-dir="$R9" rev-list --count --all )" "the phase commit is on the remote now"
# and it is refused even when this script is not the one that would push: the point is
# that the operator hears about it, not that a push is blocked
PROJ9B="$WORK/p9b"; mkproj "$PROJ9B" 1 >/dev/null; cd "$PROJ9B" || exit 1
T_PLANT="$HOOKPROOF" ap_run --no-push
assert_rc 1 "--no-push does not make it safe to ignore"
assert_contains "refusing to deliver" "and the same message explains it"
if [[ -e "$HOOKPROOF" ]]; then _bad "the hook did not run here either"; else _ok "the hook did not run here either"; fi
t_name "what the driver composes is a real ai-sandbox invocation"
# The fixture launcher stands in for ai-sandbox everywhere else, because this suite runs
# without containers. Here the launcher IS ai-sandbox, in --dry-run: if the flags land on
# the wrong side of --, or --no-interactive were forwarded to the harness, or the project
# were not the mount, ai-sandbox itself says so.
cat > "$WORK/rc/drylaunch.bash" <<EOF
sandbox-dry() {
$AISB --dry-run --net none --allow api.example.com "\$@" >> "\${AISB_AP_DRYOUT:-/dev/null}" 2>&1
}
EOF
PROJ8="$WORK/p8"; mkproj "$PROJ8" 1 >/dev/null
cd "$PROJ8" || exit 1
DRYOUT="$WORK/ai-sandbox.dryrun"; : > "$DRYOUT"
T_CMD=sandbox-dry T_RC="$WORK/rc/drylaunch.bash" T_PREFLIGHT=0 T_DRYOUT="$DRYOUT" \
ap_run --max-units 1
# the child produces no JSON (ai-sandbox printed an invocation instead), so the unit
# does not complete -- what is under test is the command that produced that output
if grep -q 'effective podman invocation' "$DRYOUT"; then
_ok "ai-sandbox accepted the composed command"
else
_bad "ai-sandbox accepted the composed command" "$(head -4 "$DRYOUT")"
fi
if grep -q 'egress: net=none' "$DRYOUT"; then
_ok "the launcher's own policy is still what is enforced"
else
_bad "the launcher's own policy is still what is enforced" "$(head -2 "$DRYOUT")"
fi
if grep -qF -- '--no-interactive' "$DRYOUT" || grep -qF -- '--no-tty' "$DRYOUT"; then
_bad "the run's sandbox flags are consumed by ai-sandbox, not passed on to pi"
else
_ok "the run's sandbox flags are consumed by ai-sandbox, not passed on to pi"
fi
if grep -qE -- ' pi (--no-approve|--session-dir)' "$DRYOUT"; then
_ok "pi is the in-container command, after --"
else
_bad "pi is the in-container command, after --" "$(tail -1 "$DRYOUT" | cut -c1-200)"
fi
if grep -qE -- '(^| )-i( |$)' "$DRYOUT" || grep -qE -- '(^| )-t( |$)' "$DRYOUT"; then
_bad "nothing is attached to the terminal: no -t in the JSON stream, no -i to steal keystrokes"
else
_ok "nothing is attached to the terminal: no -t in the JSON stream, no -i to steal keystrokes"
fi
# The project is mounted at its own host path where the sandbox can mirror it, and at
# /project where it cannot -- the driver composes neither, it just does not disturb it.
MOUNT_AT="$([[ -n "$(mirror_block_of "$PROJ8")" ]] && echo /project || echo "$PROJ8")"
if grep -qF -- "$PROJ8:$MOUNT_AT" "$DRYOUT"; then
_ok "the project directory is the mount the agent writes into"
else
_bad "the project directory is the mount the agent writes into"
fi
if grep -qF -- 'bash .agents/validate.sh' "$DRYOUT"; then
_ok "and so is the gate: validate.sh runs in the same sandbox, as the same user"
else
_bad "and so is the gate: validate.sh runs in the same sandbox, as the same user" "$(tail -2 "$DRYOUT" | cut -c1-200)"
fi
# ---------------------------------------------------------------------------
# The real library, real renderer, real commit -- with the same fake sandbox.
t_name "the real phased-execution library, end to end"
REAL_LIB="$HOME/.agents/skills/phased-execution/scripts/lib.sh"
if [[ ! -f "$REAL_LIB" ]]; then
_ok "the skill is not installed here -- the real-library run is skipped"
else
PROJ7="$WORK/p7"; mkproj "$PROJ7" 1 >/dev/null
: > "$GTRACE"; cd "$PROJ7" || exit 1
# shellcheck disable=SC2034 # assert_rc() in tests/lib.sh is what reads RC
OUT="$(env PATH="$WORK/bin:$PATH" AISB_AP_LIB="$REAL_LIB" \
AISB_AP_RC_FILES="$WORK/rc/sandbox.bash" AISB_AP_PI_CMD=sandbox-pi \
AISB_AP_VALIDATE=sandbox AISB_AP_TRACE="$TRACE" FAKE_GIT_TRACE="$GTRACE" \
FAKE_PI_MARKER="$MARKER" REAL_GIT="$REAL_GIT" PHASE_NOTIFY=0 QUIET=1 \
"$APBIN" 2>&1)" && RC=0 || RC=$?
assert_rc 0 "the real pipeline completed a phase inside the sandbox launcher"
if grep -rq 'report for 01_one__00_phase' "$PROJ7/.agents/reports/" 2>/dev/null; then
_ok "the real progress renderer read the child stream and wrote the report"
else
_bad "the real progress renderer read the child stream and wrote the report" "$(ls -R "$PROJ7/.agents/reports" 2>/dev/null | head)"
fi
if compgen -G "$PROJ7/.agents/reports/01_one/*.md" > /dev/null; then
_ok "the executor reports were written where the skill puts them"
else
_bad "the executor reports were written where the skill puts them" "$(ls -R "$PROJ7/.agents" 2>/dev/null | head -20)"
fi
if ( cd "$PROJ7" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then
_ok "the real commit_phase made the scoped commit"
else
_bad "the real commit_phase made the scoped commit" "$( cd "$PROJ7" && "$REAL_GIT" log --oneline | head -3 )"
fi
assert_contains "pushed:" "and the push went out"
if printf '%s' "$OUT" | grep -qF -- "$CANARY"; then _bad "no token in the real pipeline output"; else _ok "no token in the real pipeline output"; fi
if ( cd "$PROJ7" && "$REAL_GIT" ls-tree -r --name-only HEAD | grep -q '.agents/phases/complete/01_one/00_phase.md' ); then
_ok "the moved phase file is in the commit"
else
_bad "the moved phase file is in the commit" "$( cd "$PROJ7" && "$REAL_GIT" ls-tree -r --name-only HEAD | head -10 )"
fi
if grep -qF 'Sandbox rules for this run' "$PROMPTS"; then
_ok "and the real prompts carried the sandbox rules"
else
_bad "and the real prompts carried the sandbox rules"
fi
fi
cd "$WORK" || exit 1
tests/unit/11_build.sh
Executable
+166
View File
@@ -0,0 +1,166 @@
#!/usr/bin/env bash
# tests/unit/11_build.sh -- feedback while an image is being built.
#
# The bug this file exists for: a build is the slow part of a first run and the thing
# --install triggers, and every byte of podman's transcript used to go to a log file.
# Minutes of silence reads as a hang. So these tests ask the running script what a user
# would see, with a stub for podman rather than a container.
#
# The stub stands in for the two behaviours of the real thing that this file is about:
# 'image exists' answers no, so a build is needed, and 'build' writes its transcript to
# *stdout* (measured on podman 5: that is where podman build writes) -- which is the one
# thing the script must never let reach its own stdout, because ensure_image()'s stdout
# is read back as the image ref.
source "$TESTS/lib.sh"
PROJ="$WORK/proj"; mkdir -p "$PROJ"; cd "$PROJ" || exit 1
BIN="$WORK/bin"; mkdir -p "$BIN"
cat > "$BIN/podman" <<'EOS'
#!/usr/bin/env bash
case "${1:-} ${2:-}" in
"image exists") exit 1 ;;
"build -t")
# what the build was invoked with, so a credential smuggled into a --build-arg
# shows up in the transcript and the canary test below fails
printf 'STUB-BUILD-ARGS: %s\n' "$*"
# the packages the build was actually handed, read back out of --build-arg, so this
# transcript names what the policy under test asked for instead of a fixed guess
pkgs=""
for a in "$@"; do
case "$a" in AISB_EXTRA_PACKAGES=*) pkgs="${a#*=}" ;; esac
done
printf 'Trying to pull docker.io/library/ubuntu:24.04...\n'
printf 'Getting image source signatures\n'
printf 'STEP 1/3: FROM docker.io/library/ubuntu:24.04\n'
printf 'STEP 2/3: RUN apt-get update && apt-get install -y%s\n' "${pkgs:+ $pkgs}"
for ((i = 0; i < "${AISB_STUB_NOISE:-20}"; i++)); do
printf 'Get:%d http://archive.ubuntu.com/ubuntu noble/main amd64 curl amd64 8.5.0-1 [11 MB]\n' "$i"
sleep "${AISB_STUB_NOISE_DELAY:-0}"
done
sleep "${AISB_STUB_SLEEP:-0}"
if [[ "${AISB_STUB_RC:-0}" != 0 ]]; then
printf 'Reading package lists...\n'
for p in $pkgs; do printf 'E: Unable to locate package %s\n' "$p"; done
printf 'Error: building at STEP "RUN apt-get install": while running runtime: exit status 100\n'
exit "${AISB_STUB_RC}"
fi
printf '%s\n' '--> 1234567890ab'
printf 'STEP 3/3: CMD ["bash", "-li"]\n'
printf 'COMMIT localhost/ai-sandbox:stub\n'
printf 'Successfully tagged localhost/ai-sandbox:stub\n'
printf 'abc123def\n'
;;
*) exit 0 ;;
esac
EOS
chmod +x "$BIN/podman"
export PATH="$BIN:$PATH"
# node_sha256() fetches from nodejs.org; a unit test never touches a registry.
export AISB_NODE_SHA256=0000000000000000000000000000000000000000000000000000000000000000
# The transcript path the last run announced, which is the only way a test is supposed
# to find it -- and naming it is itself part of the feedback under test.
last_transcript() { printf '%s' "$OUT" | sed -n 's/^ai-sandbox: build: [^ ]* (transcript: \(.*\))$/\1/p' | head -1; }
log_says() { # log_says <needle> <label>
local f; f="$(last_transcript)"
if [[ -n "$f" && -r "$f" ]] && grep -qF -- "$1" "$f"; then _ok "$2"
else _bad "$2" "not in ${f:-no transcript was named}"; fi
}
log_lacks() { # log_lacks <needle> <label>
local f; f="$(last_transcript)"
if [[ -n "$f" && -r "$f" ]] && grep -qF -- "$1" "$f"; then _bad "$2" "present in $f"
else _ok "$2"; fi
}
t_name "a build says what it is doing instead of sitting silent"
sb_env AISB_INSTALL="rustc,cmake" build
assert_rc 0 "ai-sandbox build runs against the stub podman"
assert_contains "building the sandbox image with 2 extra package(s): rustc cmake" \
"it names the extra packages, which is why this run builds and the last did not"
assert_contains "one build per policy change" "and says the build is not repeated"
assert_contains "build: [1/3] FROM" "one line per build step, in order"
assert_contains "build: [2/3] RUN apt-get" "including the slow one"
assert_contains "build: [3/3]" "to the last step"
assert_contains "Trying to pull" "the base download before the first step is visible too"
assert_contains "COMMIT localhost/ai-sandbox:stub" "and the commit that names the image"
assert_contains "built localhost/ai-sandbox:" "and that it finished"
t_name "the transcript is kept in full, and named"
assert_contains "transcript: $WORK/state/build-sandbox-" "the run says where the full log is"
log_says "STUB-BUILD-ARGS:" "the log starts at the beginning of the build"
log_says "Get:1 http://archive.ubuntu.com" "the apt output that was filtered out is in the log"
log_says "Successfully tagged" "and so is the end of it"
t_name "apt's own noise is not what the terminal gets"
assert_absent "Get:1 http://archive.ubuntu.com" "a hundred lines of package lists stay in the log"
assert_absent "abc123def" "and so does the image id podman prints at the end"
t_name "the build transcript never becomes part of the image ref"
# The reason for the routing: ensure_image() is called as $(...), so podman build's
# stdout would come back spliced into the ref and into every 'podman run' after it.
rc=0; "$AISB" build > "$WORK/out" 2> "$WORK/err" || rc=$?
assert_eq 0 "$rc" "the build itself succeeded"
if [[ -s "$WORK/out" ]]; then
_bad "not one line of it lands on stdout" "$(head -2 "$WORK/out")"
else
_ok "not one line of it lands on stdout"
fi
if grep -qxE 'ai-sandbox: built [A-Za-z0-9./:@_+-]+ in [0-9]+s' "$WORK/err"; then
_ok "the ref it reports is still one clean line"
else
_bad "the ref it reports is still one clean line" "$(grep built "$WORK/err" | head -2)"
fi
t_name "a build with nothing new to show still says it is working (filtered output)"
# apt streaming 6 lines at 400ms apart is two and a half seconds of output the progress
# view deliberately drops: the heartbeat has to come from the clock, not from a read timeout.
sb_env AISB_BUILD_QUIET_SECS=1 AISB_STUB_NOISE=6 AISB_STUB_NOISE_DELAY=0.4 build
assert_rc 0 "a build that only takes a couple of seconds still succeeds"
assert_contains "still on [2/3]" "it names the step it is waiting on"
assert_contains "total)" "and how long the build has taken"
assert_absent "Get:3 http://archive.ubuntu.com" "without giving up the filtering to do it"
t_name "a build producing no output at all says it is working"
sb_env AISB_BUILD_QUIET_SECS=1 AISB_STUB_NOISE=0 AISB_STUB_SLEEP=3 build
assert_rc 0 "a build with a silent step succeeds"
assert_contains "still on [2/3]" "the heartbeat covers a step that says nothing either"
t_name "AISB_BUILD_OUTPUT=full puts the whole transcript on screen"
sb_env AISB_BUILD_OUTPUT=full build
assert_rc 0 "full output is not an error"
assert_contains "Get:1 http://archive.ubuntu.com" "the raw apt lines are shown as they come"
assert_contains "Successfully tagged" "all the way to the end"
assert_absent "still on [2/3]" "and nothing is summarised, so no heartbeat is needed"
t_name "AISB_BUILD_OUTPUT=quiet keeps the terminal clean and the log complete"
sb_env AISB_BUILD_OUTPUT=quiet build
assert_rc 0 "quiet is not an error"
assert_absent "STEP 2/3" "no step lines"
assert_absent "Get:1 http://archive.ubuntu.com" "no apt output"
assert_contains "transcript: $WORK/state/build-sandbox-" "it still says where the build went"
assert_contains "built localhost/ai-sandbox:" "and that it finished, with the time it took"
log_says "Get:1 http://archive.ubuntu.com" "quiet does not mean the transcript is lost"
t_name "a failed build names the package apt could not find"
sb_env AISB_STUB_RC=100 AISB_INSTALL="not-a-real-package" build
assert_rc 1 "a failed build fails the run"
assert_contains "no such apt package: not-a-real-package" \
"it names the package, instead of pointing at a file to go read"
assert_contains "E: Unable to locate package" "the failing lines are quoted back even in progress mode"
assert_contains "Error: building at STEP" "and the step that died is one of them"
assert_absent "built localhost/ai-sandbox" "it does not claim a build that did not happen"
t_name "a build mode that does not exist is reported, not obeyed blindly"
sb_env AISB_BUILD_OUTPUT=loud AISB_BUILD_QUIET_SECS=nope build
assert_rc 0 "a bad value is not fatal"
assert_contains "AISB_BUILD_OUTPUT='loud' is not progress, full or quiet" "it says the mode is not usable"
assert_contains "AISB_BUILD_QUIET_SECS='nope' is not a number" "and the same for the threshold"
assert_contains "build: [1/3]" "and it carries on with the default view"
t_name "nothing a build prints or writes carries a credential value"
CANARY="canary-token-value-4f2a9c"
sb_env AI_API_TOKEN="$CANARY" AISB_INSTALL="curl,jq" build
assert_rc 0 "the build ran"
assert_absent "$CANARY" "the token value is not printed"
log_lacks "$CANARY" "the build transcript holds no credential either"