diff --git a/AGENTS.md b/AGENTS.md index 4f70c97..6056d31 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -16,13 +16,24 @@ read-only with no escape hatch (`agents`/`--agents`/`--agents-dir`, and `--no-ag a sandbox that could write your skills could write instructions you follow later on the host. See `agents_src_of()` and the tests in `tests/unit/09_agents.sh` / `tests/e2e/12_sandbox.sh`. +There is a second program, `bin/ai-sandbox-autophase`: the driver for a +`.agents/phases/` pipeline. It runs every executor agent *and the `.agents/validate.sh` +gate* through the sandbox, and does the commits and pushes on the host — the git remote +is not in the allowlist and must never be, because an agent that can push can ship what +you are about to review. It owns no policy: the allowlist, the token and every other +choice come from the user's own launcher command (a function/alias found in their rc +files, or `--pi-cmd`). It is documented in the README and `docs/recipes.md`. + ## Non-negotiables 1. **Single script.** The Containerfile, the container entrypoint, the squid config, the nftables rules and the bridged pi config are all generated from `bin/ai-sandbox` (`containerfile()`, `entrypoint_script()`, `write_squid_conf()`, `gen_rules()` inside the entrypoint, `pi_config_bridge()`). Keep it that way; do - not split policy into files that can drift from the script. + not split policy into files that can drift from the script. (The rule is about + *policy*: `bin/ai-sandbox-autophase` is a second program, and it generates none of + those artifacts and holds none of those choices — it delegates all of them to the + user's launcher.) 2. **Fail closed.** Any path where the sandbox cannot guarantee its policy must refuse to start rather than degrade: no allowlist in `allow` mode, missing policy file in the entrypoint (exit 78), ruleset that cannot be installed @@ -53,14 +64,19 @@ host. See `agents_src_of()` and the tests in `tests/unit/09_agents.sh` / `tests/ ## Layout ``` -bin/ai-sandbox the application -install.sh symlink + optional alias -config/*.example documented examples (never loaded directly) -docs/networking.md the measurements behind the enforcement model -docs/threat-model.md what is and is not protected -- keep it honest -docs/recipes.md per-harness and per-language setups -tests/run.sh runner: unit (no containers) | --e2e (real containers) -tests/spike_network.sh re-measure the host +bin/ai-sandbox the application +bin/ai-sandbox-autophase pipeline driver: agent + gate in the sandbox, git on the host +install.sh symlink both + optional alias +contrib/ai-sandbox.bash bash completion (flags mirrored from parse_args; keep in sync) +config/*.example documented examples (never loaded directly) +docs/networking.md the measurements behind the enforcement model +docs/threat-model.md what is and is not protected -- keep it honest +docs/recipes.md per-harness, per-language and pipeline setups +tests/run.sh runner: unit (no containers) | --e2e (real containers) +tests/fixtures/phase-lib contract fixture for the driver: the phased-execution library's + function names and call shapes, so 10_autophase.sh runs with no + skill installed and no containers +tests/spike_network.sh re-measure the host ``` ## Commands @@ -113,3 +129,50 @@ networking or the image, `make e2e` green too. * `split_target()` / `classify_allows()` — the mapping from what the user typed to what the firewall and the proxy enforce. * `build_argv()` — every isolation flag lives here. +* `resolve_workdir()` / `mirror_block_of()` — what the project is called inside the + container. Mirroring the host path is the convenience; the deny list is the safety, because + a bind hides whatever the image has at that destination and podman puts `--tmpfs /tmp` on + *top* of a bind of the same path. Extend the list with paths the image or the sandbox + owns, never with paths a project happens to live under, and keep `/project` reachable: + it is the fallback for every refusal and for `--no-cwd`, and the image always creates it. + A fallback must be announced (`workdir_note` in `--print-policy`, a comment in + `--dry-run`, one log line on a real run). Tests: `tests/unit/03_argv.sh` drives both + branches, asks the extracted function about the paths a test cannot create (`/usr`, + `/home/sandbox`), and `tests/e2e/12_sandbox.sh` proves it in a container. +* `bin/ai-sandbox-autophase` — four things are load-bearing, each a quiet way + to break the pipeline: + - **It shadows five functions of the sourced pipeline library** — `pi` (a child is a + container), `first_prompt`/`fix_prompt` (the agent is told what it cannot do in + there), `run_validation` (the gate runs where the agent runs) and `push_phase` + (every push, including the library's own, goes through `ap_push_pending`, so the + fingerprint guard, the credential redaction and the error contract cover all of + them) — and nothing else: unit order, retries, provider backoff, report recovery, + the scoped commit and notifications stay the library's. The `declare -F` gate + refuses to start if the library stops providing any of them, so a renamed function + fails loudly instead of quietly running around a guard. Shadow order matters: the + launcher is resolved *before* `pi()` is defined, or a launcher named `pi` resolves + to the shim that wraps it (and a PATH program is captured as an absolute path for + the same reason). + - **Nothing runs on the host against a repository the agent just edited.** `ap_git_env` + exports an empty `core.hooksPath`, `GIT_PAGER=cat`, `GIT_CONFIG_NOSYSTEM` and + `protocol.ext.allow=never`; `ap_git_fingerprint` hashes `.git/config` plus every + file in `.git/hooks/` before the first unit and `ap_git_guard` re-checks it before + every delivery, refusing the run unless `--trust-git-changes`. Git executes those + paths, so a `pre-commit` planted inside the sandbox is otherwise code running as + the user who invoked the pipeline. + - **The rc probe reads a definition and never calls it.** `ap_probe_rc` sources one + file in a throwaway `bash -ic` (interactive, because `~/.bashrc` guards on `$-`) with + `exit` neutered, and returns the text. `--status`/`--dry-run` must therefore never + fetch a token or start a container. The captured body is embedded in each child's + launcher script with `printf %q` per argument — the prompt is one argument that can + contain quotes, newlines and `$` at once, so nothing may build that command line by + concatenating raw text. The body is *never printed*: `ap_launcher_preview` names a + function rather than echoing it, because its body is the `pass` call. + - **The push is the delivery contract.** `ap_push_pending` runs at start, at every + phase boundary and at end of run; start is a warning, the boundary and the end are + fatal, because exiting 0 with the work unpushed is the silent miss this script + exists to prevent. Nothing printed may carry a value: `ap_redact` covers literal + `--env NAME=value` and credentials inside remote URLs, which is exactly what a git + error quotes back. Tests: `tests/unit/10_autophase.sh` (canary token, canary + credential in the remote URL, the composed command checked against the real + `ai-sandbox --dry-run`, and one run against the real library). diff --git a/Makefile b/Makefile index e30b19e..4f0de27 100644 --- a/Makefile +++ b/Makefile @@ -2,12 +2,14 @@ # machine plus podman; nothing needs to be installed system-wide. SHELL := /bin/bash BIN := bin/ai-sandbox +BINS := bin/ai-sandbox bin/ai-sandbox-autophase SHELLCHECK_IMAGE := docker.io/koalaman/shellcheck:stable .PHONY: help test e2e check lint style doctor build clean-demo all help: ## show this help @grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk -F':.*?## ' '{printf " %-12s %s\n", $$1, $$2}' + test: ## unit tests only (no containers, no network) ./tests/run.sh @@ -19,12 +21,14 @@ check: lint test ## lint + unit tests (what CI would run) lint: ## shellcheck every shell file, in a container (nothing installed locally) @podman run --rm -v "$(CURDIR):/src:ro,z" -w /src $(SHELLCHECK_IMAGE) \ --severity=warning --shell=bash --exclude=SC1091 \ - bin/ai-sandbox install.sh tests/run.sh tests/lib.sh tests/spike_network.sh \ - $$(ls tests/unit/*.sh tests/e2e/*.sh) \ + $(BINS) install.sh contrib/ai-sandbox.bash \ + tests/run.sh tests/lib.sh tests/spike_network.sh \ + $$(ls tests/unit/*.sh tests/e2e/*.sh tests/fixtures/*.sh tests/fixtures/phase-lib/*.sh) \ && echo "shellcheck: clean" style: ## bash-style syntax check of every shell file (fast, no deps) - @for f in $(BIN) install.sh tests/*.sh tests/unit/*.sh tests/e2e/*.sh; do \ + @for f in $(BINS) install.sh contrib/*.bash tests/*.sh tests/unit/*.sh tests/e2e/*.sh \ + tests/fixtures/*.sh tests/fixtures/phase-lib/*.sh; do \ [ -f "$$f" ] || continue; bash -n "$$f" || exit 1; \ done; echo "bash -n: clean" diff --git a/PLAN.md b/PLAN.md index 5b1c0ed..ea42984 100644 --- a/PLAN.md +++ b/PLAN.md @@ -154,6 +154,99 @@ in the outer shell -- valid bash, wrong program, invisible to `bash -n` and shel `tests/unit/06_entrypoint.sh` now extracts the seeding script and asserts it is quote-free, and extracts the merge program to test its semantics without a container. +### 12 — The pipeline driver: agent in, git out +`bin/ai-sandbox-autophase` runs a `.agents/phases/` pipeline (the shape the +phased-execution skill drives) with the split that the sandbox forces: executor agents +*and* `.agents/validate.sh` inside, commits and pushes outside. Two reasons, both about +the writable mount. `validate.sh` lives in it, so running the gate on the host is +running agent-editable bash on the host (`--validate-in host` for a project whose +toolchain is only on the machine, warned every run). And the remote is deliberately not +allowlisted -- an agent that can push can ship what you are about to review -- so +somebody else has to deliver, and that somebody is this script, between unit runs: after +each phase commit, again before the next phase starts, again at the end, and a re-push of +whatever an earlier run could not deliver. + +The launcher is the user's own command, because the allowlist and the token are theirs. +A script cannot see an interactive shell's functions, so `sandbox-pi` (default, +`--pi-cmd`/`AISB_AP_PI_CMD`) is looked up in `~/.bashrc.d/*`, `~/.bashrc`, +`~/.bash_aliases`, `~/.profile`: the rc file is sourced in a throwaway `bash -ic` +with `exit` neutered and the *definition* returned -- never called, which is what lets +`--status`/`--dry-run` promise no token fetch, no image build, no container. Each child +then re-sources and runs it, so `pass` runs per unit rather than once a night. + +Integration is by shadowing five functions of the sourced library (`pi`, +`first_prompt`, `fix_prompt`, `run_validation`, `push_phase`) behind a `declare -F` +contract check, so the library keeps scheduling, retries, provider backoff, report +recovery and the scoped commit, and a library that changes shape is refused rather than +half-wired. + +What it found on the way, in order of how much it mattered: + +* **A repository is what git runs, not just what it records.** Committing on the host + after an agent had the tree writable means `.git/config` and `.git/hooks/` are now + input to a host process. Host git therefore runs with an empty `core.hooksPath`, no + pager, no system config and `protocol.ext.allow=never`, and delivery is *refused* if a + unit changed either (`--trust-git-changes` to override). `tests/unit/10_autophase.sh` + plants a `pre-commit` from inside the fake sandbox and asserts it never runs, in either + mode. +* **Shadowing `pi` and resolving a launcher called `pi` are the same name.** Resolve + first, install the shims after; a PATH program is captured as an absolute path, or the + function wins and the child recurses. +* **A preview can leak a body.** The captured launcher definition contains the `pass` + call, and the child command is that text plus arguments -- so previews name the + launcher instead of printing it, and `ap_redact` covers `--env NAME=value` and + credentials inside remote URLs, which is exactly what a failing `git push` quotes back. +* **A report is a gate too.** Running the library's `push_phase` around the guard, rather + than letting it push on its own, is the difference between "the harness pushes" and + "every push is checked". + +### 13 — Naming the directory the agent works in +Until now `pwd` was `/project` in every sandbox on the machine, which is two problems +wearing one costume: paths an agent prints (`/project/src/api.py:41`) have to be translated +back to the host by whoever reads them, and a harness that titles its window after the cwd +makes every tab of a session read the same. So the cwd is mounted at the path it has on the +host, and `--workdir` now defaults to `auto`, which means precisely that. + +Nothing about the *mount* changed -- same directory, same `rw`, same `:z`, still the only +writable one. What changed is the destination, and the destination is where a bind mount +can do damage: it hides whatever the image already had at that path. Hence +`mirror_block_of()`, whose list is not "places people keep code" but "paths the image or +this sandbox owns": `/usr`, `/etc`, `/var`, `/opt`, `/bin`, `/sbin`, `/lib*`, `/boot`, +`/proc`, `/sys`, `/dev`, `/root`, `/run`, `/tmp`, `/home` and `/home/sandbox*`. `/home` +needs its own rule because it is the one standard hierarchy where work lives: +`/home//…` mirrors, while `/home` and the container's own home do not -- that is +where the home volume and the read-only `.agents` share are mounted. + +`/tmp` decided the shape of the function, and it was measured rather than assumed +(podman's behaviour *is* the question): with `--tmpfs /tmp` and `-v /tmp/x:/tmp/x` in one +run, podman puts the tmpfs on top. `pwd` prints the right path, writes succeed, and the +project is not there. A failure that quiet, in the one place the sandbox promises not to +fail quietly, earns a refusal plus a line of output: `workdir_note` in `--print-policy`, a +comment line in `--dry-run`, one log line on a real run. `/project` -- the one directory +baked into the image -- is what everything falls back to, so a run always has a working +directory that exists, `--no-cwd` included. + +The tests could not create most of the deny list, so `tests/unit/03_argv.sh` extracts +`mirror_block_of` from the script and asks it about `/usr`, `/home/sandbox` and `/` +directly (the pattern `06_entrypoint.sh` already uses for the entrypoint), and drives both +real branches through the CLI with a scratch directory under `$HOME` -- `lib.sh: +mirrorable_dir`, because `$TMPDIR` is always the fallback branch, which is why the whole +suite had only ever seen `/project`. `tests/e2e/12_sandbox.sh` then holds a real container +up to the same promise: `pwd`, writes landing on the host, and `/project`, `/home/sandbox` +and a home-volume marker all still present through a mirrored run. + +Two things the naming turned up on the way: + +* The first version took the first path component with `${top%%/*}`, which on any path with + a second slash yields the empty string -- so every deny rule matched nothing, everything + mirrored, and `bash -n` and shellcheck were both clean. The fix is the table of paths in + *both* directions in `03_argv.sh`; one assertion on the happy path would not have seen it. +* A cwd containing `:` cannot be handed to podman in a `-v` spec at all (`invalid option + type "/project"`), and mirroring doubles the exposure by making that string the +destination too. It is refused now, with the reason, instead of podman's. (`--mount +type=bind,src=…,dst=…` parses it happily; switching the mount form for one character is not +worth the relabel-option churn that comes with it.) + ## Follow-ups (not yet done) * **`--net unix` mode**: `--network=none` plus a unix-socket proxy and an in-container diff --git a/README.md b/README.md index 9be1982..a9ba43a 100644 --- a/README.md +++ b/README.md @@ -16,9 +16,9 @@ $ pi --allow aipi.reeseapps.com --env AI_API_TOKEN pi ``` -`pi` there is an alias for `ai-sandbox`. Inside, the agent sees `/project` (your -cwd, read-write), nothing else on your disk, and the internet only where you -pointed it. +`pi` there is an alias for `ai-sandbox`. Inside, the agent sees your cwd at the path it +has on the host — `~/Projects/billing-api`, not a generic `/project` — read-write, and +nothing else on your disk, with the internet only where you pointed it. ## Install @@ -101,14 +101,35 @@ Asking for a variable that is not set fails the run closed. ## Filesystem -Your cwd is mounted at `/project` and is the working directory. That is the only -*writable* mount, and nothing else on the host is visible except the one exception -below. Everything else is opt-in: +Your cwd is mounted at **the path it has on the host**, and that is the working +directory. So `pwd`, an error the agent quotes back, a commit message and the window title +your harness derives from the cwd all name *this* project instead of reading the same as +every sandbox in the session. The mount itself is unchanged by the naming: same directory, +same read-write, nothing else visible except the one exception below. + +A bind mount hides whatever was already at that path in the image, so a host path that +would land on something the image or the sandbox owns (`/usr`, `/etc`, `/var`, `/opt`, +`/tmp`, `/run`, `/home/sandbox`, `/`) falls back to `/project` — and says so, because a +working directory you did not choose is worth a line: + +``` +[aisb] ai-sandbox: cannot mirror /tmp/scratch/billing-api: the sandbox owns /tmp*, so the +project is mounted at /project instead (--workdir DIR to choose) +``` + +`/tmp` is the one case worth naming: the sandbox mounts a tmpfs there, and podman puts +that tmpfs *over* a bind of the same path, so a project under `/tmp` would come out as an +empty directory that happens to be writable and to have the right name. Measured, not +guessed — see [docs/networking.md](docs/networking.md). + +`--workdir DIR` picks the mount point yourself (`auto`, the default, is the mirroring +behaviour above). The table below is everything else, all opt-in: ```bash --mount ~/.ssh:ro # if the agent must be able to push --mount ../docs:/docs:ro # read-only reference material ---no-cwd # a scratch sandbox with nothing mounted +--workdir /src # mount and cd there instead of mirroring the host path +--no-cwd # a scratch sandbox with nothing mounted (lands in /project) --read-only # read-only root filesystem --ephemeral-home # do not persist /home/sandbox (default: persist in a volume) ``` @@ -280,14 +301,113 @@ the design: [docs/networking.md](docs/networking.md). does not protect you from (allowlisted destinations, container escapes, anything you explicitly mounted). +## An unattended pipeline: `ai-sandbox-autophase` + +A phase pipeline (`.agents/phases/todo/…`) is the harshest thing to ask of a sandbox: +hours of unattended work by an agent that installs dependencies, runs the tests and +wants to commit and push. It gets its own driver, and the split inside that driver is +the design: + +| in the sandbox | on the host | +|---|---| +| the executor agent of every task, and the phase's final pass | the phase commit | +| `.agents/validate.sh`, the pass/fail gate | the push — and the re-push of anything an earlier run left behind | + +The git remote stays unreachable from inside on purpose: an agent that could push could +ship what you are about to review. `git status`, `diff`, `log` and `blame` work in there +(the repository is the mount, git is in the image), and the agent is told in its own +prompt that `push`/`fetch`/`pull` cannot work, so it does not spend attempts discovering +that. Delivery is the driver's job: it pushes when a phase commits, again before the next +phase starts, and again at the end of the run, so nothing sits unpushed between runs. A +push failure stops the run and prints the command that finishes it — the commit is local +and safe, and the next run takes it out. + +The gate runs in the sandbox for the same reason the agent does: `.agents/validate.sh` +sits in the writable mount, so running it on the host is running agent-editable bash on +the host. `--validate-in host` exists for a project whose toolchain only exists on the +machine, and says so on every run. + +``` +[2026-10-28 02:11:03] preflight: one sandboxed shell, to prove the launcher, the image and the policy + aisb-ok aisb=1 git=2.43.0 user=sandbox cwd=/home/you/Projects/foo +[2026-10-28 02:11:04] checking for commits a previous run left unpushed + (up to date with origin/main) +[2026-10-28 02:11:04] ━━ 01_one/01_task.md — attempt 1/3 ━━ + (every line prefixed with the time, as the pipeline prints them) + (committed: 9f1c2ab phase: 01_one) + (pushed: 9f1c2ab phase: 01_one) +[2026-10-28 02:41:52] ✓ pipeline done — 2 unit(s) delivered this run: 01_one/01_task.md 01_one/00_phase.md +``` + +### It runs *your* command, not a copy of it + +The driver does not know your allowlist and does not try to. It runs your launcher — by +default the `sandbox-pi` shell function you keep in `~/.bashrc.d`: + +```bash +sandbox-pi () { + export AI_API_TOKEN=$(pass aipi/dev-token) + ai-sandbox --allow aipi.reeseapps.com --allow github.com \ + --allow pi.dev --allow release-assets.githubusercontent.com \ + --env AI_API_TOKEN +} +``` + +A script cannot see an interactive shell's functions, so the driver looks the name up in +`~/.bashrc.d/*`, `~/.bashrc`, `~/.bash_aliases` and `~/.profile` (or `AISB_AP_RC_FILES`) +and takes the *definition* — probing never calls it, so a preview cannot fetch a token. +Each child then runs exactly what your shell would have, `pass` included, once per unit +so an overnight run is not holding yesterday's token. An alias or a plain program works +too; `--pi-cmd` also accepts a whole command line. + +```bash +cd your/project +ai-sandbox-autophase --status # launcher, pending units, git plan. Runs nothing. +ai-sandbox-autophase --dry-run # the exact child and gate commands +ai-sandbox-autophase # every pending task, in pipeline order +``` + +Every child is started as ` --no-tty --no-interactive -- pi `: +your flags first, then the two that keep an unattended run off your terminal (no TTY to +corrupt the JSON stream, no stdin for your keystrokes to fall into), then `--`, after +which ai-sandbox reads nothing but the command. + +| flag | | +|---|---| +| `--pi-cmd NAME` | the launcher: function, alias, program or whole command line | +| `--rc-files A:B` | which rc files to look in | +| `--sandbox-args ARGS` | flags added to every run (default `--no-tty --no-interactive`) | +| `--validate-in sandbox\|host` | where the gate runs | +| `--no-push` | commit locally, deliver nothing | +| `--no-preflight` | skip the throwaway container that proves the chain works before spending tokens | +| `--trust-git-changes` | deliver even though the sandbox edited `.git/config` or `.git/hooks/` | +| `--unit`, `--phase`, `--max-units` | scope a run | + +It does *all* the pushing — including the pipeline library's own — so that one function +answers for every commit that leaves the machine. And because a repository is not +something git merely records but something it **runs**, host-side git is started with an +empty hooks path, no pager, no system config and no `ext::` transport, and the run +refuses to deliver if a sandboxed unit rewrote `.git/config` or planted a hook. A +`pre-commit` written inside the sandbox is the sandbox executing code on the host, and +the phase commit is not worth finding that out the hard way. + +`MAX_FIX_ATTEMPTS`, `PHASE_MODEL`, `PHASE_THINKING`, `PI_TRUST`, `FRESH_FIX`, `QUIET`, +`PHASE_NOTIFY` and `PHASE_COMMIT` behave as the phased-execution skill documents them — +the driver reuses that pipeline and replaces only the parts that assume the agent is a +local process. + ## Project layout ``` -bin/ai-sandbox the application: one script, ~1.3k lines -install.sh symlink into ~/.local/bin + optional alias +bin/ai-sandbox the application: one script, ~2.2k lines +bin/ai-sandbox-autophase the pipeline driver: agent in the sandbox, git on the host +install.sh symlink both into ~/.local/bin + optional alias +contrib/ai-sandbox.bash bash completion for both config/ example config, allowlist and env file docs/ networking.md, threat-model.md, recipes.md tests/run.sh runner; tests/unit/* (no containers) tests/e2e/* (real ones) +tests/fixtures/phase-lib contract fixture: the pipeline library's shape, without skill + or containers, for the driver's tests tests/spike_network.sh re-measure podman's behaviour on a new host Makefile make test | e2e | lint | check | doctor | build ``` @@ -295,7 +415,7 @@ Makefile make test | e2e | lint | check | doctor | build ## Development ```bash -make test # unit: args, config, argv construction, env handling, rule generation +make test # unit: args, config, argv construction, env handling, rules, pipeline driver make e2e # + real containers: build cache, egress blocking, capability drop, lifecycle make lint # shellcheck in a container make style # bash -n diff --git a/bin/ai-sandbox b/bin/ai-sandbox index 1b377c4..02e8ffc 100755 --- a/bin/ai-sandbox +++ b/bin/ai-sandbox @@ -188,13 +188,18 @@ NETWORK DNS is opened up. Weaker than the gate for CDNs. FILESYSTEM - The current directory is bind-mounted at /project and is the working directory. - Nothing else on the host is writable, and nothing else is visible except the one - exception below, which is read-only and there to keep a harness usable. - The rest is opt-in. + The current directory is bind-mounted at the path it has on the host and is the + working directory, so pwd, prompts and window titles name the project instead of + every sandbox looking like every other. The mount itself is unchanged by that: same + directory, same rw, nothing else visible except the one exception below, which is + read-only and there to keep a harness usable. The rest is opt-in. --mount PATH[:DEST][:ro] Extra bind mount. Repeatable. Default rw, DEST=/mnt/. --no-cwd Do not mount the current directory. - --workdir DIR Working directory in the container. Default /project. + --workdir DIR Mount the cwd there instead of mirroring its host path, and + cd there. 'auto' (the default) mirrors, falling back to + /project when the host path would bury part of the image or + of the sandbox itself (/usr, /etc, /tmp, /home/sandbox, /). + A DIR onto one of those is mounted as asked and warned about. --home-cache Persist /home/sandbox in a named volume (default). --ephemeral-home Discard /home/sandbox on exit. --read-only Read-only root filesystem (plus tmpfs /tmp). @@ -255,6 +260,9 @@ RUN --label K=V Extra label. Repeatable. --keep Keep the container after it exits. --interactive | --no-tty Attachment. A TTY is allocated when stdin+stdout are TTYs. + --no-interactive Never attach stdin. For unattended or scripted runs: the + command cannot read this terminal, so a stray keystroke + cannot type into an agent that is running by itself. --dry-run, -n Print the exact podman invocation (secrets redacted), exit 0. --print-policy Print the effective policy as key=value lines, exit 0. --print-entrypoint Print the script that is baked into the image as the @@ -446,8 +454,9 @@ parse_args() { --keep) OPT_KEEP=1; shift ;; --read-only) OPT_READONLY=1; shift ;; --shell) OPT_SHELL=1; shift ;; - --interactive) OPT_INTERACTIVE=1; shift ;; - --no-tty) OPT_TTY=0; shift ;; + --interactive) OPT_INTERACTIVE=1; shift ;; + --no-interactive) OPT_INTERACTIVE=0; shift ;; + --no-tty) OPT_TTY=0; shift ;; --dry-run|-n) OPT_DRY=1; shift ;; --print-policy) OPT_PRINT=1; shift ;; --print-entrypoint) OPT_PRINT_EP=1; shift ;; @@ -510,10 +519,95 @@ agents_dest_taken() { return 1 } +# Would a mirrored copy of $1 land on top of something the image or this sandbox needs? +# +# A bind mount does not only name a place, it hides whatever was already there -- so the +# question is what lives at that path inside the container, not what the project is +# called. /home is the one standard directory people keep their work in, and a per-user +# tree there never meets the container's own home; the rest of the hierarchy either +# belongs to the image or is one of this sandbox's own mounts. +# +# /tmp is measured, not guessed: with `--tmpfs /tmp` and `-v /tmp/x:/tmp/x` in the same +# invocation, podman (5.x, rootless, SELinux enforcing) puts the tmpfs on top, so the +# project comes out as an empty tmpfs -- the right name, writable, and not the project. +# The failure is silent, which is exactly the shape a refusal has to take instead. +# +# Prints the path that blocks mirroring, or nothing when the mirror is safe: the caller +# needs the reason, not just the verdict, because the fallback is a surprise otherwise. +mirror_block_of() { # $1 = absolute host path -> the container path it would bury, or empty + local p="$1" top + if [[ "$p" == "/" ]]; then printf '/'; return 0; fi + top="${p#/}"; top="/${top%%/*}" # the first component, with its leading slash + case "$top" in + # Mirror /home//..., but never /home itself (it would bury the home mount) and + # never the container's own home, which holds the home volume and the .agents share. + /home) + case "$p" in + /home) printf '/home' ;; + /home/sandbox|/home/sandbox/*) printf '/home/sandbox' ;; + esac + return 0 ;; + # Directories the base image owns, in whole or in part. + /bin|/boot|/dev|/etc|/lib|/lib32|/lib64|/opt|/proc|/root|/run|/sbin|/sys|/tmp|/usr|/var) + printf '%s' "$top"; return 0 ;; + esac + # Everything else -- /mnt, /srv, /media, or a top-level directory the image does not + # have at all -- is empty inside the container: mirroring creates a path, shadows + # nothing, and the run keeps the host path it recognises. + printf '' +} + +# Where the cwd is mounted and worked in -- 'auto' mirrors the host path. +# +# Two reasons for mirroring, one practical and one human: +# +# * paths the agent prints (an error, a commit message, a todo list) stay readable on +# the host, where someone has to act on them; +# * a harness titles its window after the last component of the working directory, so +# a fixed /project makes every sandbox tab in the session read the same. +# +# The fallback is /project, the one directory baked into the image, so a run always has +# a working directory that exists -- including --no-cwd, which mounts nothing and lands +# there. Destination only: never the mode, the SELinux option or the rw-ness. +resolve_workdir() { # sets POL[workdir] and POL[workdir_note]; needs POL[cwd] + local w="${OPT_WORKDIR:-${AISB_PROJECT_DEST:-$(cfgv workdir auto)}}" + if [[ "$w" != auto ]]; then + [[ "$w" == /* ]] || die "--workdir: must be an absolute path or 'auto' (got '$w')" + w="${w%/}" + [[ -n "$w" ]] || die "--workdir: / would mount the project over the whole container" + POL[workdir]="$w"; POL[workdir_note]="" + # An explicit path is a choice rather than a fallback, so it is honoured -- but the same + # hiding rule applies to it, and whoever chose it cannot be expected to know which of + # these paths the sandbox mounts over. Say so, since the alternative is a pwd that lies. + local hide; hide="$(mirror_block_of "$w")" + if [[ -n "$hide" ]]; then + local msg="--workdir $w: the sandbox owns ${hide}*," + msg+=" so the project mount there can be covered by the image or by one of our own" + msg+=" mounts (/tmp is a tmpfs)" + warn "$msg" + fi + return 0 + fi + # --no-cwd mounts nothing, so there is nothing to mirror: /project is the image's own + # directory and a note about the host path would only be noise. + if [[ "${OPT_NO_CWD:-0}" == 1 ]]; then + POL[workdir]="/project"; POL[workdir_note]=""; return 0 + fi + local block; block="$(mirror_block_of "${POL[cwd]}")" + if [[ -n "$block" ]]; then + POL[workdir]="/project" + POL[workdir_note]="cannot mirror ${POL[cwd]}: the sandbox owns ${block}*," + POL[workdir_note]+=" so the project is mounted at /project instead (--workdir DIR to choose)" + else + POL[workdir]="${POL[cwd]}"; POL[workdir_note]="" + fi +} + resolve_opts() { + POL[cwd]="$PWD" POL[net]="${OPT_NET:-${AISB_NET:-$(cfgv net allow)}}" POL[harness]="${OPT_HARNESS:-${AISB_HARNESS:-$(cfgv harness pi)}}" - POL[workdir]="${OPT_WORKDIR:-${AISB_PROJECT_DEST:-$(cfgv workdir /project)}}" + resolve_workdir POL[cpus]="${OPT_CPUS:-${AISB_CPUS:-$(cfgv cpus 4)}}" POL[memory]="${OPT_MEMORY:-${AISB_MEMORY:-$(cfgv memory 4g)}}" POL[pids]="${OPT_PIDS:-${AISB_PIDS:-$(cfgv pids 512)}}" @@ -573,7 +667,6 @@ resolve_opts() { POL[unenforced]="$OPT_UNENFORCED" POL[keep_root]="$OPT_KEEP_ROOT" POL[name]="${OPT_NAME:-aisb-${SESSION:-$(session_id)}}" - POL[cwd]="$PWD" P_allow=(${CFG_allow[@]+"${CFG_allow[@]}"} ${OPT_allow[@]+"${OPT_allow[@]}"}) P_env=(${CFG_env[@]+"${CFG_env[@]}"} ${OPT_env[@]+"${OPT_env[@]}"}) @@ -603,6 +696,13 @@ validate_opts() { [[ "${POL[cpus]}" =~ ^[0-9.]+$ ]] || die "--cpus must be a number (got '${POL[cpus]}')" [[ "${POL[memory]}" =~ ^[0-9]+([bkmgBKMG]?)$ ]] || die "--memory must look like 512m or 4g (got '${POL[memory]}')" [[ -d "${POL[cwd]}" ]] || die "current directory does not exist" + # podman splits a `-v` spec on ':', so a project path containing one cannot be handed to + # it at all (measured: `invalid option type "/project"`). Mirroring makes that string the + # destination as well as the source, so say it here rather than forward podman's error. + if [[ "${OPT_NO_CWD:-0}" != 1 && "${POL[cwd]}" == *:* ]]; then + die "the project path contains a ':', which podman cannot parse in a mount spec. + rename the directory, or start without it: $PROG --no-cwd" + fi if [[ "${POL[net]}" == full ]]; then warn "--net full: the container has unrestricted internet access" fi @@ -1213,7 +1313,11 @@ ENV HOME=/home/sandbox \\ AISB=1 \\ AISB_MANIFEST=/etc/aisb/manifest.txt -WORKDIR ${POL[workdir]} +# Fixed, on purpose: where the project is mounted is a run-time decision (see +# resolve_workdir) and podman is told with --workdir, so two projects -- or one project +# checked out twice -- must never mean two images. /project is also what the runtime +# falls back to, and it is created above, so a workdir always exists. +WORKDIR /project ENTRYPOINT ["/usr/local/lib/aisb/entrypoint.sh"] CMD ["bash", "-li"] AISB_CF @@ -1248,7 +1352,8 @@ image_hash() { # bare tag change (24.04 -> 24.10) changes it too because the name is in the hash. h="${POL[base_ref]:-${POL[base_image]}}|${POL[node_version]}|${POL[pi_version]}|${AISB_PI_PACKAGE}|v${VERSION}" h+="|$(printf '%s ' ${P_install[@]+"${P_install[@]}"})" - # any change to the generated image definition produces a new tag + # any change to the generated image definition produces a new tag -- and nothing + # per-project is in it, so the same image serves every directory on the machine h+="|$(entrypoint_script | sha256sum | cut -d' ' -f1)" h+="|$(containerfile | sha256sum | cut -d' ' -f1)" sha12 "$h" @@ -1777,6 +1882,9 @@ print_argv() { "$(printf '%s ' ${DIRECT4[@]+"${DIRECT4[@]}"})" printf '# env (%d, passed via 0600 file): %s\n' "${#ENV_NAMES[@]}" \ "$(printf '%s ' ${ENV_NAMES[@]+"${ENV_NAMES[@]}"})" + # A fallback is worth a line here: the invocation on its own only shows /project, not + # that the host path was passed over. Same text as workdir_note in --print-policy. + [[ -z "${POL[workdir_note]:-}" ]] || printf '# workdir: %s\n' "${POL[workdir_note]}" for a in "${ARGV[@]}"; do if [[ "$a" == "$ENV_FILE" ]]; then printf ' ' else printf '%s ' "$(shquote "$a")"; fi @@ -1803,6 +1911,7 @@ print_policy() { printf 'harness=%s\n' "${POL[harness]}" printf 'command=%s\n' "$(printf '%s ' ${COMMAND[@]+"${COMMAND[@]}"})" printf 'workdir=%s\n' "${POL[workdir]}" + [[ -z "${POL[workdir_note]:-}" ]] || printf 'workdir_note=%s\n' "${POL[workdir_note]}" printf 'cwd=%s\n' "${POL[cwd]}" printf 'cwd_mounted=%s\n' "$([[ ${OPT_NO_CWD:-0} == 1 ]] && echo no || echo yes)" printf 'limits=cpus:%s,memory:%s,pids:%s\n' "${POL[cpus]}" "${POL[memory]}" "${POL[pids]}" @@ -1865,6 +1974,9 @@ cmd_run() { if (( OPT_PRINT )); then print_policy; exit 0; fi if (( OPT_DRY )); then print_argv; exit 0; fi + # A real run is where the fallback would otherwise be invisible: pwd is not the path + # the user typed, and they should find out from this run rather than from the agent. + [[ -z "${POL[workdir_note]:-}" ]] || log "${POL[workdir_note]}" ensure_image sandbox > /dev/null # The gate only comes up once we know the sandbox image exists: a gate with no diff --git a/bin/ai-sandbox-autophase b/bin/ai-sandbox-autophase new file mode 100755 index 0000000..8e77fd1 --- /dev/null +++ b/bin/ai-sandbox-autophase @@ -0,0 +1,879 @@ +#!/usr/bin/env bash +# ai-sandbox-autophase -- run the .agents/phases/ pipeline with the agent inside +# ai-sandbox and the git history outside it. +# +# Who runs where is the whole design: +# +# * the executor agents run INSIDE the sandbox, so the code they write, the tests +# they run and the dependencies they download cannot touch the host; +# * .agents/validate.sh runs INSIDE it too, because that file sits in the writable +# mount -- a gate executed on the host is the sandboxed agent executing arbitrary +# bash on the host the moment it edits its own jailer; +# * the commit and the push run on the HOST, between unit runs, by this script. +# The egress allowlist does not contain the git remote and adding it would let an +# agent push what you are about to review. `git status`, `git diff`, `git log` and +# `git blame` all work in there -- the repository is in the mount and git is in the +# image -- only the remote is out of reach. So the pipeline cannot expect the agent +# to deliver anything to the remote, and this script does that delivery instead: +# one atomic commit per completed phase (the library's, scoped to the phase's own +# work), pushed as soon as it exists, re-pushed between phase runs if a previous +# run left something behind. +# +# The sandbox launcher is YOUR command, not a copy of it. Default: the `sandbox-pi` +# shell function people keep in ~/.bashrc.d / ~/.bashrc, which knows the allowlist and +# how to fetch the token. A non-interactive script never loads an interactive rc, so +# the command is looked up in those rc files (function, alias or executable) and every +# child then runs exactly what your own shell would have run -- the `pass` call +# included, once per unit so an overnight run is not holding a day-old token. +# +# Usage: ai-sandbox-autophase [options] (see --help) +set -uo pipefail + +VERSION="1.0.0" +PROG="ai-sandbox-autophase" + +# -------------------------------------------------------------------------- +# options: defaults -> environment -> command line +# -------------------------------------------------------------------------- +AISB_AP_PI_CMD="${AISB_AP_PI_CMD-sandbox-pi}" # function, alias or program +AISB_AP_RC_FILES="${AISB_AP_RC_FILES:-}" # colon list; default: your rc files +AISB_AP_LIB="${AISB_AP_LIB:-}" # phased-execution lib.sh +AISB_AP_SKILL_DIR="${AISB_AP_SKILL_DIR:-}" # ... or its skill directory +AISB_AP_VALIDATE="${AISB_AP_VALIDATE:-sandbox}" # sandbox | host +AISB_AP_PUSH="${AISB_AP_PUSH:-1}" # push between phase runs +AISB_AP_PREFLIGHT="${AISB_AP_PREFLIGHT:-1}" # prove the chain before spending tokens +# No colon on purpose: an explicitly empty AISB_AP_SANDBOX_ARGS means "no extra flags", +# which is how a launcher that is not ai-sandbox (or a test) opts out of them. +AISB_AP_SANDBOX_ARGS="${AISB_AP_SANDBOX_ARGS---no-tty --no-interactive}" +AISB_AP_NOTES="${AISB_AP_NOTES:-1}" # append the sandbox rules to the prompts +AISB_AP_TRUST_GIT="${AISB_AP_TRUST_GIT:-0}" # allow a run that rewrote .git config/hooks + +AP_MODE="pipeline" # pipeline | unit | phase | status | dryrun +AP_UNIT="" +AP_PHASE="" +AP_MAX_UNITS=0 + +ap_ts() { date '+%Y-%m-%d %H:%M:%S'; } +ap_log() { printf '[%s] %s\n' "$(ap_ts)" "$*"; } +ap_warn() { printf '[%s] ⚠ %s\n' "$(ap_ts)" "$*" >&2; } +ap_die() { printf '[%s] ✗ ERROR: %s\n' "$(ap_ts)" "$*" >&2; exit 1; } +ap_need() { [[ -n "${2:-}" ]] || ap_die "$1 needs a value (see --help)"; } + +usage() { + cat <&2; exit 2 ;; + esac +done + +case "$AISB_AP_VALIDATE" in + sandbox) ;; + host) + # Said once, up front, whatever mode this is: .agents/validate.sh is inside the + # mount the agent can write, so this choice runs agent-editable bash outside the + # sandbox. It is a legitimate choice (a project whose toolchain only exists on the + # host) and it is never a quiet one. + ap_warn "the validation gate will run on the HOST: .agents/validate.sh is writable by the sandboxed agent, so this runs agent-editable bash outside the sandbox" + ;; + *) ap_die "--validate-in must be 'sandbox' or 'host' (got '$AISB_AP_VALIDATE')" ;; +esac +[[ "$AP_MAX_UNITS" =~ ^[0-9]+$ ]] || ap_die "--max-units must be a number" + +# -------------------------------------------------------------------------- +# the launcher: run the command the user's own shell would run +# -------------------------------------------------------------------------- +# rc files to look in, in the order a login shell reads them. Only existence is +# checked here; ap_probe_rc is what sources them, and only to read a definition. +AP_RC_LIST=() +ap_rc_list() { + local -a out=() + local f + if [[ -n "$AISB_AP_RC_FILES" ]]; then + local IFS=':' + for f in $AISB_AP_RC_FILES; do [[ -n "$f" ]] && out+=("$f"); done + else + for f in "$HOME"/.bashrc.d/* "$HOME/.bashrc" "$HOME/.bash_aliases" "$HOME/.profile"; do + [[ -f "$f" && -r "$f" ]] && out+=("$f") + done + fi + AP_RC_LIST=("${out[@]+"${out[@]}"}") +} + +# Source ONE rc file in a throwaway interactive bash and print what it defines for +# NAME -- the definition text only, never a call: probing must not fetch a token or +# start a container. Interactive because a ~/.bashrc that opens with `case $- in *i*)` +# defines nothing for a non-interactive shell. `exit` is neutered first, because plenty +# of rc files exit when they dislike the conditions and must not take the probe with them. +# +# Output, in this order, so it parses without ambiguity: +# _AP_PATH= +# _AP_FN | _AP_ALIAS (absent when NAME is defined nowhere in that file) +# +ap_probe_rc() { # + local name="$1" file="$2" + AISB_AP_P_NAME="$name" AISB_AP_P_FILE="$file" \ + timeout 15 bash -ic ' + exit() { return "${1:-0}"; } + . "$AISB_AP_P_FILE" /dev/null 2>&1 + shopt -s expand_aliases 2>/dev/null || true + printf "_AP_PATH=%s\n" "$PATH" + if declare -F -- "$AISB_AP_P_NAME" >/dev/null 2>&1; then + printf "_AP_FN\n"; declare -f -- "$AISB_AP_P_NAME" + elif alias -- "$AISB_AP_P_NAME" >/dev/null 2>&1; then + printf "_AP_ALIAS\n"; alias -- "$AISB_AP_P_NAME" + fi + ' 2>/dev/null +} + +AP_KIND="" # function | alias | program | arguments +AP_LAUNCHER="" # resolved name, for messages +AP_SOURCE="" # where it was found +AP_DEF="" # captured definition text, when there is one +AP_CHILD_PATH="" # PATH as the rc files left it +AP_CHILD=() # the launcher itself; AP_RUN below is what a child is started with + +# Two word lists live in globals: AP_WORDS is the sandbox flags of every run, AP_CMD_WORDS +# the words of a --pi-cmd that named a whole command line. Do not reuse one for the other. +AP_WORDS=() +AP_CMD_WORDS=() +ap_words() { # -> AP_WORDS + AP_WORDS=() + read -r -a AP_WORDS <<< "$1" +} + +# A captured function/alias becomes a small launcher script: neuter `exit`, source the +# rc files (the command may lean on other functions or aliases from them), define what +# we captured, then call it. Every argument is re-quoted with printf %q, because the +# prompt is one argument that can contain quotes, newlines and dollar signs at once. +ap_build_runner() { + local -a body=() + local f + [[ -n "$AP_CHILD_PATH" ]] && body+=("PATH=$(printf '%q' "$AP_CHILD_PATH")" "export PATH") + body+=('exit() { return "${1:-0}"; }') + body+=('shopt -s expand_aliases 2>/dev/null || true') + for f in ${AP_RC_LIST[@]+"${AP_RC_LIST[@]}"}; do + body+=(". $(printf '%q' "$f") /dev/null 2>&1 || true") + done + # An rc file that turned on errexit must not kill the launcher halfway through the + # run: the child's own exit code is the only status that matters here. + body+=('set +e') + body+=("$AP_DEF") + body+=('_ap_run_named() {') + body+=(' local n="$1"; shift') + body+=(' local -a q=(); local a') + body+=(' for a in "$@"; do q+=("$(printf "%q" "$a")"); done') + body+=(' eval "$n ${q[*]}"') + body+=('}') + body+=("_ap_run_named $AP_LAUNCHER \"\$@\"") + AP_CHILD=(bash -c "$(printf '%s\n' "${body[@]}")" "$PROG-launcher") +} + +ap_resolve_launcher() { + local name="$AISB_AP_PI_CMD" f out kind path def + AP_CMD_WORDS=() + [[ -n "$name" ]] || ap_die "no sandbox launcher: \$AISB_AP_PI_CMD and --pi-cmd are both empty" + + # An explicit command line wins: the user said exactly what to run. + if [[ "$name" == *[[:space:]]* ]]; then + read -r -a AP_CMD_WORDS <<< "$name" + ((${#AP_CMD_WORDS[@]})) || ap_die "--pi-cmd has no command in it" + AP_KIND=arguments + AP_LAUNCHER="${AP_CMD_WORDS[0]}" + AP_SOURCE="--pi-cmd" + AP_CHILD=("${AP_CMD_WORDS[@]}") + return 0 + fi + + [[ "$name" =~ ^[A-Za-z_][A-Za-z0-9_-]*$ ]] \ + || ap_die "launcher '$name' is not a command name; pass a whole command line instead (see --pi-cmd)" + + # 1. a function the caller exported + if declare -F -- "$name" >/dev/null 2>&1; then + AP_KIND=function; AP_LAUNCHER="$name"; AP_SOURCE="the calling shell" + AP_DEF="$(declare -f -- "$name")" + ap_build_runner + return 0 + fi + + # 2. a function or an alias inside the rc files + ap_rc_list + for f in ${AP_RC_LIST[@]+"${AP_RC_LIST[@]}"}; do + out="$(ap_probe_rc "$name" "$f")" + path="$(printf '%s\n' "$out" | sed -n '1s/^_AP_PATH=//p')" + kind="$(printf '%s\n' "$out" | sed -n '2p')" + if [[ "$kind" == "_AP_FN" || "$kind" == "_AP_ALIAS" ]]; then + def="$(printf '%s\n' "$out" | tail -n +3)" + if [[ "$kind" == "_AP_FN" ]]; then AP_KIND="function"; else AP_KIND="alias"; fi + AP_LAUNCHER="$name"; AP_SOURCE="$f"; AP_DEF="$def"; AP_CHILD_PATH="$path" + ap_build_runner + return 0 + fi + done + + # 3. a program on PATH -- recorded as the absolute path, because a bare word would be + # resolved as a shell name first, and `pi` is one of the names this script shadows. + if command -v -- "$name" >/dev/null 2>&1; then + AP_KIND=program; AP_LAUNCHER="$(command -v -- "$name")"; AP_SOURCE="$AP_LAUNCHER" + AP_CHILD=("$AP_LAUNCHER") + return 0 + fi + + ap_die "cannot resolve the sandbox launcher '$name': + not an exported function, + not a function or alias in: ${AP_RC_LIST[*]:-}, + not a program on PATH. + Point --rc-files at the file that defines it, or name the command outright: + --pi-cmd 'ai-sandbox --allow api.example.com --env API_KEY'" +} + +# The full argv prefix of a child run: the launcher, this run's own sandbox flags, then +# -- so that everything after it is the in-container command verbatim. ai-sandbox stops +# reading its own options at --, which is what lets a launcher function with a fixed +# policy carry an arbitrary harness command line. +AP_RUN=() +ap_build_run() { + AP_WORDS=() + read -r -a AP_WORDS <<< "$AISB_AP_SANDBOX_ARGS" + AP_RUN=("${AP_CHILD[@]}" ${AP_WORDS[@]+"${AP_WORDS[@]}"}) +} + +# Anything printed that could carry a value is redacted first: a --pi-cmd that sets a +# literal value, and a remote URL with a credential in it, which is what a git error +# quotes back verbatim. Values never appear in output, only their names. +ap_redact() { + sed -E -e 's#([A-Za-z][A-Za-z0-9+.-]*://[^/:@ ]+:)[^@[:space:]]+@#\1***@#g' \ + -e 's/(--[a-z-]*env[= ]+[A-Za-z_][A-Za-z0-9_]*)=[^[:space:]]+/\1=/g' +} + +# The launcher part of a preview. A captured function or alias is NOT printed: its body +# is the thing that fetches the token, and a preview must not echo it -- naming it is +# the useful and the safe thing to print. A command line is printed with any literal +# value redacted. +ap_launcher_preview() { + case "$AP_KIND" in + function|alias) printf '<%s>' "$AP_LAUNCHER" ;; + program) printf '%s' "$AP_LAUNCHER" ;; + arguments) printf '%s' "$(printf '%s ' ${AP_CMD_WORDS[@]+"${AP_CMD_WORDS[@]}"} | ap_redact)" ;; + *) printf '' ;; + esac +} + +ap_child_preview() { + local a out="" + out="$(ap_launcher_preview) " + for a in ${AP_WORDS[@]+"${AP_WORDS[@]}"}; do out+="$(printf '%q' "$a") "; done + printf '%s-- pi ' "$out" +} + +# -------------------------------------------------------------------------- +# the pipeline library, and the four things that change when the agent is remote +# -------------------------------------------------------------------------- +ap_find_lib() { + local -a c=() + [[ -n "$AISB_AP_LIB" ]] && c+=("$AISB_AP_LIB") + [[ -n "$AISB_AP_SKILL_DIR" ]] && c+=("$AISB_AP_SKILL_DIR/scripts/lib.sh") + c+=("$HOME/.agents/skills/phased-execution/scripts/lib.sh" + ".agents/skills/phased-execution/scripts/lib.sh" + "$HOME/.pi/agent/skills/phased-execution/scripts/lib.sh") + local f + for f in "${c[@]}"; do + [[ -f "$f" && -r "$f" ]] && { printf '%s\n' "$f"; return 0; } + done + return 1 +} + +AP_LIB="$(ap_find_lib)" || ap_die "the phased-execution library (lib.sh) was not found. + Looked in: \$AISB_AP_LIB, \$AISB_AP_SKILL_DIR/scripts/lib.sh, + ~/.agents/skills/phased-execution/scripts/lib.sh, .agents/skills/..., ~/.pi/agent/skills/... + Install the phased-execution skill, or point --lib at its scripts/lib.sh." + +# shellcheck source=/dev/null +source "$AP_LIB" + +# The library owns the pipeline. This script shadows exactly four of its functions -- +# the ones that assume the agent is a local process -- and nothing else, so unit order, +# validation, retries, provider-outage backoff, report recovery, the scoped phase +# commit and the notifications all stay the library's. +# +# pi() the library calls `pi` once per child attempt; this turns that +# call into a container. The child contract is unchanged: stdout is +# still the JSON stream progress.mjs reads, and the exit code is +# still pi's own, because ai-sandbox propagates the in-container +# command's status and its own chatter goes to stderr. +# first_prompt() the agent also has to know what it cannot do in there. +# fix_prompt() +# run_validation() the gate runs where the agent runs. +# This happens after the launcher is resolved, and that order is not cosmetic: one of +# the names installed here is `pi`, and a launcher called `pi` has to resolve to the +# user's command rather than to the shim that wraps it. +ap_install_overrides() { + # Every push goes through ap_push_pending, including the library's own: it commits at + # the phase's commit point and then calls push_phase, so shadowing that function is + # what puts the fingerprint guard, the credential redaction and the error contract in + # front of all of it. Requiring it is deliberate -- if the library ever stops calling + # it, this script must find out here rather than quietly pushing around the guard. + for _ap_fn in find_root next_unit phase_entries phase_next_unit unit_phase is_phase_end \ + count_units sweep_finished_phases execute_unit unit_report notify_task \ + commit_phase push_phase; do + declare -F -- "$_ap_fn" >/dev/null 2>&1 \ + || ap_die "$AP_LIB does not provide $_ap_fn(): it is not the phased-execution library, or it has changed shape. Refusing to run a half-wired pipeline." + done + for _ap_fn in first_prompt fix_prompt; do + declare -F -- "$_ap_fn" >/dev/null 2>&1 \ + || ap_die "$AP_LIB does not provide $_ap_fn(): refusing to run a pipeline whose prompts it cannot build." + # the library's version, kept under its own name so the wrappers below can extend it + eval "ap_lib_$_ap_fn ()$(declare -f -- "$_ap_fn" | tail -n +2)" + done + unset _ap_fn + + AP_SANDBOX_NOTES=' + +## Sandbox rules for this run (ai-sandbox) + +You are running inside a rootless podman container. This directory is the only +writable mount and your home is a sandbox home, not the host one. + +* git works locally: git status, git diff, git log and git blame are available and the + repository is the real one. The git REMOTE is unreachable from here, and that is + policy rather than a fault to fix: git push, git fetch, git pull, git ls-remote and + git clone cannot work, so never attempt them and never report them as a + failure of the task. The harness commits and pushes on the host when the phase + completes; a task is delivered by leaving the work in the tree and green. +* Egress is limited to an allowlist you cannot change. A connection that is refused or + times out to anything outside it is the policy working: say so in your report rather + than routing around it. +* No root, no package manager, no system services. Use the toolchain in the image + (node, python3, make, gcc, git, jq) and the project own vendored dependencies. +* .agents/validate.sh, .agents/phases/ and .agents/reports/ belong to the harness. + Never edit the gate to make it pass. +' + + pi() { "${AP_RUN[@]}" -- pi "$@"; } + + first_prompt() { + local text + text="$(ap_lib_first_prompt "$@")" || return 1 + printf '%s\n' "$text" + [[ "$AISB_AP_NOTES" == 1 ]] && printf '%s\n' "$AP_SANDBOX_NOTES" + return 0 + } + + fix_prompt() { + local text + text="$(ap_lib_fix_prompt "$@")" || return 1 + printf '%s\n' "$text" + [[ "$AISB_AP_NOTES" == 1 ]] && printf '%s\n' "$AP_SANDBOX_NOTES" + return 0 + } + + run_validation() { # + if [[ "$AISB_AP_VALIDATE" == host ]]; then + bash .agents/validate.sh >"$1" 2>&1 + else + "${AP_RUN[@]}" -- bash .agents/validate.sh >"$1" 2>&1 + fi + } + + push_phase() { ap_push_pending "phase ${1:-?}"; } # <- guarded, redacted, honest +} + +# -------------------------------------------------------------------------- +# git, on the host, in between the runs +# -------------------------------------------------------------------------- +# The agent has the repository: git status/diff/log are how it sees what it changed. But +# the COMMIT happens out here, and git is not a passive reader of a repository -- it runs +# what the repository tells it to. So two things: +# +# 1. host-side git starts with the knobs that execute things turned off: no system +# config, no pager, no prompt, no ext:: transport, and an empty hooks directory +# instead of .git/hooks; +# 2. the two paths that carry them (.git/config and .git/hooks/) are fingerprinted +# before the first unit and re-checked before every delivery. If a sandboxed run +# edited either, the run stops: a pre-commit hook written in there is the sandbox +# executing code on the host, and refusing to commit is cheap next to that. +# --trust-git-changes is the way through when the edit was yours. +AP_EMPTY_HOOKS="" +GIT_FINGERPRINT0="" +GIT_FINGERPRINT_MSG="" + +ap_git_env() { + AP_EMPTY_HOOKS="$(mktemp -d "${TMPDIR:-/tmp}/aisb-ap-hooks.XXXXXX")" \ + || ap_die "could not create the empty hooks directory that keeps .git/hooks out of this run" + export GIT_CONFIG_NOSYSTEM=1 GIT_PAGER=cat GIT_TERMINAL_PROMPT=0 + export GIT_CONFIG_COUNT=2 + export GIT_CONFIG_KEY_0=core.hooksPath GIT_CONFIG_VALUE_0="$AP_EMPTY_HOOKS" + export GIT_CONFIG_KEY_1=protocol.ext.allow GIT_CONFIG_VALUE_1=never +} + +ap_git_fingerprint() { # -> hash of the repo's local config and hook files, or nothing + local gd + gd="$(git rev-parse --git-dir 2>/dev/null)" || return 0 + [[ -n "$gd" ]] || return 0 + { git config --local --list 2>/dev/null + find "$gd/hooks" -type f -printf '%p %s\n' 2>/dev/null | sort + find "$gd/hooks" -type f -exec sha1sum {} + 2>/dev/null | sort + } | sha1sum | cut -d' ' -f1 +} + +# 0 when nothing dangerous moved; 1 with the reason when it did. +ap_git_guard() { + [[ -n "$GIT_FINGERPRINT0" ]] || return 0 + [[ "$AISB_AP_TRUST_GIT" == 1 ]] && return 0 + local now + now="$(ap_git_fingerprint)" + [[ -z "$now" || "$now" == "$GIT_FINGERPRINT0" ]] && return 0 + GIT_FINGERPRINT_MSG="$(git rev-parse --git-dir 2>/dev/null)" + return 1 +} + +ap_git_guard_fail() { + printf '✗ ERROR: refusing to deliver (commit/push) -- the sandbox changed this repository'\''s own git configuration.\n' >&2 + printf ' A repository runs .git/config and .git/hooks/, it does not merely record them: a hook\n' >&2 + printf ' written inside the sandbox would execute on the host the moment this script committed.\n' >&2 + printf ' look at it by hand: git config --local --list; ls -l "%s/hooks"\n' "${GIT_FINGERPRINT_MSG:-.git}" >&2 + printf ' if the change was yours: re-run with --trust-git-changes\n' >&2 +} + +ap_push_fail() { # + local out="$1" cmd="$2" fatal="$3" + printf '✗ ERROR: %s\n' "the push FAILED. The commit is local and safe; the remote does not have it." >&2 + printf '%s\n' "$out" | ap_redact | sed 's/^/ /' >&2 + printf ' finish by hand: %s\n' "$cmd" >&2 + printf ' nothing is lost: the next run pushes it again once the remote answers.\n' >&2 + return "$fatal" +} + +# ap_push_pending [fatal] -- idempotent: pushes what the remote is missing. +ap_push_pending() { + local context="$1" fatal="${2:-1}" + if ! ap_git_guard; then + ap_git_guard_fail + return "$fatal" + fi + if [[ "$AISB_AP_PUSH" != 1 ]]; then + echo " (push disabled by --no-push: commits stay local)" + return 0 + fi + if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + echo " (not a git repository -- nothing to push)" + return 0 + fi + local remote branch ahead out + remote="$(git remote 2>/dev/null | head -n1)" + if [[ -z "$remote" ]]; then + echo " (no git remote configured -- commits stay local)" + return 0 + fi + branch="$(git symbolic-ref --short -q HEAD || true)" + if [[ -z "$branch" ]]; then + printf '✗ ERROR: %s -- detached HEAD, nothing to push.\n' "$context" >&2 + printf ' the commits are local; check out a branch and push them.\n' >&2 + return "$fatal" + fi + if git rev-parse --abbrev-ref --disambiguate '@{u}' >/dev/null 2>&1; then + ahead="$(git rev-list --count '@{u}..HEAD' 2>/dev/null || echo 0)" + if [[ "$ahead" == "0" ]]; then + echo " (up to date with $remote/$branch)" + return 0 + fi + ap_log "pushing $ahead unpushed commit(s) on $branch ($context)" + if ! out="$(git push 2>&1)"; then + ap_push_fail "$out" "git push" "$fatal" + return $? + fi + else + ap_log "pushing $branch to $remote (no upstream set yet)" + if ! out="$(git push -u "$remote" "$branch" 2>&1)"; then + ap_push_fail "$out" "git push -u $remote $branch" "$fatal" + return $? + fi + fi + echo " (pushed: $(git log -1 --oneline))" + return 0 +} + +# What the host intends to do with git. Printed by --status and --dry-run, neither of +# which contacts the remote: 'unpushed' is computed from local refs only. +ap_git_plan() { + if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + printf 'git: not a repository -- the pipeline leaves its work in the tree\n' + return 0 + fi + local branch remote ahead="n/a" + branch="$(git symbolic-ref --short -q HEAD || echo '(detached)')" + remote="$(git remote 2>/dev/null | head -n1)" + if [[ -n "$remote" ]] && git rev-parse --abbrev-ref --disambiguate '@{u}' >/dev/null 2>&1; then + ahead="$(git rev-list --count '@{u}..HEAD' 2>/dev/null || echo '?')" + fi + # The remote's NAME is resolved by name, never printed as a URL: one with a + # credential in it would land in the log and the session output. + printf 'git: branch=%s remote=%s unpushed=%s\n' "$branch" \ + "$(if [[ -n "$remote" ]]; then git remote get-url "$remote" 2>/dev/null | ap_redact; else printf 'none'; fi)" "$ahead" + printf 'git: commit and push happen on the host, between phase runs; the agent cannot reach the remote and is not given it\n' +} + +# -------------------------------------------------------------------------- +# preflight: one throwaway container, before any agent tokens are spent +# -------------------------------------------------------------------------- +ap_preflight() { + local out errf rc=0 + ap_log "preflight: one sandboxed shell, to prove the launcher, the image and the policy" + errf="$(mktemp)" + out="$("${AP_RUN[@]}" -- bash -c 'printf "aisb-ok aisb=%s git=%s user=%s cwd=%s\n" "${AISB:-none}" "$(git --version 2>/dev/null | cut -d" " -f3 || echo none)" "$(id -un)" "$PWD"' 2>"$errf")" || rc=$? + if (( rc != 0 )) || [[ "$out" != aisb-ok* ]]; then + printf '%s: preflight failed (exit %s): the launcher does not give a working sandbox.\n' "$PROG" "$rc" >&2 + printf ' command: %s\n' "$(ap_child_preview)" >&2 + tail -n 12 "$errf" | sed 's/^/ | /' >&2 + rm -f "$errf" + ap_die "Fix the launcher or the sandbox before running the pipeline: every unit would fail this way. (--no-preflight to skip the check.)" + fi + rm -f "$errf" + echo " $out" + if [[ "$out" == *"git=none"* ]]; then + ap_warn "no git inside the sandbox -- the agent cannot read its own diff. Add it to the image (--install git) or expect blind fixer runs." + fi + return 0 +} + +# -------------------------------------------------------------------------- +# modes +# -------------------------------------------------------------------------- +ap_gate_preview() { + local a out="" + out="$(ap_launcher_preview) " + for a in ${AP_WORDS[@]+"${AP_WORDS[@]}"}; do out+="$(printf '%q' "$a") "; done + printf '%s-- bash .agents/validate.sh' "$out" +} + +ap_report_config() { + # The launcher line is printed through the same redaction as everything else: a + # --pi-cmd may carry a literal --env value, and nothing that prints repeats a value. + printf 'launcher : %s\n' "$(printf '%s' "$AISB_AP_PI_CMD" | ap_redact)" + printf ' = %s, from %s\n' "$AP_KIND" "${AP_SOURCE:-nowhere}" + printf 'child : %s\n' "$(ap_child_preview)" + if [[ "$AISB_AP_VALIDATE" == host ]]; then + printf 'gate : .agents/validate.sh, on the host (see --validate-in)\n' + else + printf 'gate : .agents/validate.sh, in the sandbox\n' + fi + printf 'pipeline : %s\n' "$AP_LIB" + printf 'attempts : MAX_FIX_ATTEMPTS=%s, PI_TRUST=%s, FRESH_FIX=%s, QUIET=%s, PHASE_COMMIT=%s\n' \ + "${MAX_FIX_ATTEMPTS:-3}" "${PI_TRUST:-0}" "${FRESH_FIX:-0}" "${QUIET:-0}" "${PHASE_COMMIT:-1}" +} + +# The pending units of one phase, in the library's execution order: the task files +# sorted, then 00_phase.md as the final pass. next_unit() reports the first pending unit +# rather than consuming it, so a listing cannot be built from it -- it has to walk the +# same shape the scheduler does. --status compares the total against the library's own +# count_units(), which is what notices if that shape ever changes. +ap_phase_units() { # + local p="$1" f t + if [[ -d "$PHASE_TODO/$p" ]]; then + for f in "$PHASE_TODO/$p"/[0-9]*.md; do # globs, so a name cannot be mis-split + [[ -f "$f" ]] || continue + t="${f##*/}"; [[ "$t" == "00_phase.md" ]] && continue + printf '%s/%s\n' "$p" "$t" + done + [[ -f "$PHASE_TODO/$p/00_phase.md" ]] && printf '%s/00_phase.md\n' "$p" + elif [[ -f "$PHASE_TODO/$p" ]]; then + printf '%s\n' "$p" + fi + return 0 +} + +ap_list_units() { + local p u n=0 + for p in $(phase_entries); do + while IFS= read -r u; do + [[ -n "$u" ]] || continue + printf ' %s\n' "$u" + n=$((n + 1)) + (( n >= 10000 )) && break # a listing is never worth an unbounded loop + done < <(ap_phase_units "$p") + done + [[ "$n" == 0 ]] && printf ' (none -- the pipeline is complete)\n' + local libn + libn="$(count_units 2>/dev/null || echo '?')" + if [[ "$libn" != "?" && "$libn" != "$n" ]]; then + ap_warn "the library counts $libn pending unit(s) but this listing found $n -- its directory shape has changed; trust the library" + fi + return 0 +} + +ap_preview() { + ap_report_config + echo + echo "one container per unit attempt; the prompt travels as one argument:" + printf ' %s\n' "$(ap_child_preview)" + echo + echo "the gate:" + printf ' %s\n' "$(ap_gate_preview)" + echo + echo "pending units:" + ap_list_units + echo + ap_git_plan + echo + echo "Nothing was started: --dry-run builds no image, starts no container, fetches no" + echo "token and contacts no remote. The launcher was resolved by reading rc files only." +} + +ensure_git_env() { + # Two switches mean the same thing here, so they are kept in step in both directions: + # PHASE_PUSH=0 in the environment means "do not deliver", and --no-push means the + # library must not push behind our back either. + [[ "${PHASE_PUSH:-1}" == 1 ]] || AISB_AP_PUSH=0 + [[ "$AISB_AP_PUSH" == 1 ]] || export PHASE_PUSH=0 +} + +ap_run_pipeline() { + local unit phase last_phase="" last_unit="" n=0 rc=0 + local -a delivered=() + + sweep_finished_phases + ensure_git_env + command -v node >/dev/null 2>&1 || ap_die "node is required on the host to render task progress" + [[ "$AISB_AP_PREFLIGHT" == 1 ]] && ap_preflight + + # Anything an earlier run failed to push goes out before new work. A failure here is + # a warning: the pipeline can still make progress locally and every later push takes + # the same commits up. + ap_log "checking for commits a previous run left unpushed" + ap_push_pending "start of run" 0 || ap_warn "some commits are still local -- continuing, and each phase push retries them" + + while unit="$(next_unit)"; do + [[ -n "$unit" ]] || break + # next_unit() reports the first pending unit; the progress that removes it is + # execute_unit(). A unit that comes back twice has not been delivered, and looping + # on it would run the pipeline in place forever -- stop and say so. + if [[ -n "$unit" && "$unit" == "$last_unit" ]]; then + echo "✗ ERROR: $unit was reported pending again after completing -- nothing moved it to $PHASE_DONE/. Stopping rather than repeating it forever." >&2 + return 1 + fi + last_unit="$unit" + phase="$(unit_phase "$unit")" + if [[ -n "$last_phase" && "$phase" != "$last_phase" ]]; then + # In between phase runs: deliver the phase that just committed. + if ! ap_push_pending "between $last_phase and $phase"; then + echo "✗ ERROR: pipeline stopped -- the phase commit is not on the remote (see above)" >&2 + return 1 + fi + fi + rc=0 + execute_unit "$unit" || rc=$? + if (( rc != 0 )); then + # execute_unit printed the detail: task failure, or commit/push failure with the + # hand-fix command. + echo "✗ ERROR: pipeline stopped -- $unit did not complete (see the error output above)" >&2 + echo " re-run this script to continue where it stopped" >&2 + return 1 + fi + delivered+=("$unit") + last_phase="$phase" + n=$((n + 1)) + if is_phase_end "$unit"; then notify_task "$unit" phase; else notify_task "$unit" task; fi + if (( AP_MAX_UNITS > 0 && n >= AP_MAX_UNITS )); then + ap_log "--max-units $AP_MAX_UNITS reached; $(count_units) unit(s) still pending" + break + fi + done + + # Delivery is the point of the run, so a remote that will not answer makes the run a + # failure: exiting 0 with the work committed but unpushed is the silent miss this + # script exists to prevent. The commits are safe either way and the next run retries. + if ! ap_push_pending "end of run"; then + echo "✗ ERROR: the run delivered the work but could not push it (see above)" >&2 + return 1 + fi + echo + ap_log "✓ pipeline done -- ${#delivered[@]} unit(s) delivered this run: ${delivered[*]:-none}" + echo " remaining in $PHASE_TODO/: $(count_units) task(s)" + return 0 +} + +# One unit, or one phase to completion: the library's own entry points, with the +# between-run delivery around them. +ap_run_selected() { + local unit phase="$AP_PHASE" rc=0 n=0 + sweep_finished_phases + ensure_git_env + command -v node >/dev/null 2>&1 || ap_die "node is required on the host to render task progress" + [[ "$AISB_AP_PREFLIGHT" == 1 ]] && ap_preflight + + if [[ "$AP_MODE" == unit ]]; then + unit="$AP_UNIT" + [[ "$unit" == *.md ]] || unit="$unit.md" + [[ -f "$PHASE_TODO/$unit" ]] || ap_die "$unit not found in $PHASE_TODO/" + execute_unit "$unit" || rc=$? + if (( rc == 0 )); then + if is_phase_end "$unit"; then notify_task "$unit" phase; else notify_task "$unit" task; fi + fi + ap_push_pending "after $unit" || return 1 + return "$rc" + fi + + [[ -d "$PHASE_TODO/$phase" || -f "$PHASE_TODO/$phase.md" ]] \ + || ap_die "phase $phase not found in $PHASE_TODO/" + while unit="$(phase_next_unit "$phase")"; do + [[ -n "$unit" ]] || break + rc=0 + execute_unit "$unit" || rc=$? + if (( rc != 0 )); then + echo "✗ ERROR: phase $phase did not complete (see the error output above)" >&2 + return 1 + fi + n=$((n + 1)) + if is_phase_end "$unit"; then notify_task "$unit" phase; else notify_task "$unit" task; fi + done + ap_push_pending "after phase $phase" || return 1 + ap_log "✓ phase $phase complete ($n unit(s))" +} + +main() { + ap_resolve_launcher + ap_build_run + ap_install_overrides # after the resolve: see the note on ap_install_overrides + # Preview modes stop here: no image build, no container, no state written, no + # network -- and no sweep of finished phase directories either, which writes. + local root + root="$(find_root)" || root="" + [[ -n "$root" ]] || ap_die "no .agents/phases/todo found in this or parent directories (run it inside a phased project)" + cd "$root" || ap_die "cannot enter $root" + if [[ "$AP_MODE" != status && "$AP_MODE" != dryrun ]]; then + ap_git_env # see the note above + GIT_FINGERPRINT0="$(ap_git_fingerprint)" + fi + case "$AP_MODE" in + status) ap_status ;; + dryrun) ap_preview ;; + unit|phase) ap_run_selected ;; + *) ap_run_pipeline ;; + esac +} + +ap_status() { + ap_report_config + echo + echo "pending units:" + ap_list_units + echo + ap_git_plan +} + +ap_interrupt() { # + echo + printf '[%s] ✗ ERROR: interrupted (%s) -- the unit stays in the todo/ directory; re-run to continue where it stopped\n' "$(ap_ts)" "$2" >&2 + exit "$1" +} +trap 'ap_interrupt 130 SIGINT' INT +trap 'ap_interrupt 143 SIGTERM' TERM +# the empty hooks directory exists only to keep .git/hooks out of our own git calls +trap '[ -z "$AP_EMPTY_HOOKS" ] || rm -rf "$AP_EMPTY_HOOKS"' EXIT + +main "$@" diff --git a/config/config.example b/config/config.example index 57b327b..60dc4d8 100644 --- a/config/config.example +++ b/config/config.example @@ -24,7 +24,12 @@ net = allow # harness = pi # --- mounts and layout ----------------------------------------------------- -# workdir = /project +# Where the current directory is mounted inside the container, which is also the +# working directory. 'auto' (the default) uses the path it has on the host, so the +# agent's pwd and the paths it prints are the ones you act on afterwards. It falls +# back to /project when the host path would cover something the image or the sandbox +# owns (/usr, /etc, /var, /opt, /tmp, /run, /home/sandbox) and announces the fallback. +# workdir = auto # --- limits ---------------------------------------------------------------- # cpus = 4 diff --git a/contrib/ai-sandbox.bash b/contrib/ai-sandbox.bash index 1717cb5..fdce449 100755 --- a/contrib/ai-sandbox.bash +++ b/contrib/ai-sandbox.bash @@ -3,10 +3,17 @@ # Install (either): # mkdir -p ~/.local/share/bash-completion/completions # cp contrib/ai-sandbox.bash ~/.local/share/bash-completion/completions/ai-sandbox +# (it also registers ai-sandbox-autophase, which shares this file) # or source it from ~/.bashrc -- put the `source` line *after* the alias, since # bash needs `complete` registered under the alias name (`alias pi=ai-sandbox`). # # The flag list mirrors parse_args() in bin/ai-sandbox -- keep them in sync. + +# compgen prints one candidate per line; reading them instead of dropping them through an +# unquoted $(...) keeps a candidate that contains a space (a file name, a phase directory) +# in one piece. Process substitution, not a pipe: COMPREPLY has to survive the call. +_aisb_fill() { local w; while IFS= read -r w; do COMPREPLY+=("$w"); done; } + _ai_sandbox_completions() { local cur prev word first="" sub="" i if type _get_comp_words_by_ref >/dev/null 2>&1; then @@ -34,43 +41,77 @@ _ai_sandbox_completions() { [[ " $subs " == *" $first "* ]] && sub="$first" case "$prev" in # values of the flag just typed - --net) COMPREPLY=( $(compgen -W "none allow full" -- "$cur") ); return ;; - --harness) COMPREPLY=( $(compgen -W "pi codex claude none" -- "$cur") ); return ;; - --env) COMPREPLY=( $(compgen -A export -- "$cur") ); return ;; + --net) COMPREPLY=(); _aisb_fill < <(compgen -W "none allow full" -- "$cur"); return ;; + --harness) COMPREPLY=(); _aisb_fill < <(compgen -W "pi codex claude none" -- "$cur"); return ;; + --env) COMPREPLY=(); _aisb_fill < <(compgen -A export -- "$cur"); return ;; --allow) # hosts already in the allow files; no DNS, no network local hosts hosts="$(sed 's/#.*//' "$HOME/.config/ai-sandbox/allow" ./.ai-sandbox/allow 2>/dev/null | tr ',[:space:]' '\n\n')" - COMPREPLY=( $(compgen -W "$hosts" -- "$cur") ) + COMPREPLY=(); _aisb_fill < <(compgen -W "$hosts" -- "$cur") (( ${#COMPREPLY[@]} )) || COMPREPLY=( "$cur" ) # don't fall back to filenames return ;; --proxy) COMPREPLY=( "$cur" ); return ;; - --allow-file|--env-file) COMPREPLY=( $(compgen -f -- "$cur") ); return ;; - --mount|--agents-dir|--workdir) COMPREPLY=( $(compgen -d -- "$cur") ); return ;; + --allow-file|--env-file) COMPREPLY=(); _aisb_fill < <(compgen -f -- "$cur"); return ;; + --workdir) COMPREPLY=(); _aisb_fill < <(compgen -W "auto" -- "$cur") # 'auto' mirrors the host path + (( ${#COMPREPLY[@]} )) || _aisb_fill < <(compgen -d -- "$cur"); return ;; + --mount|--agents-dir) COMPREPLY=(); _aisb_fill < <(compgen -d -- "$cur"); return ;; esac # A non-flag word before the cursor that is not a subcommand means the rest goes # to the harness: ai-sandbox stops parsing there, so offer nothing, not sandbox flags. [[ -n "$first" && -z "$sub" ]] && return 1 - [[ -z "$first" && "$cur" != -* ]] && { COMPREPLY=( $(compgen -W "$subs" -- "$cur") ); return; } + [[ -z "$first" && "$cur" != -* ]] && { COMPREPLY=(); _aisb_fill < <(compgen -W "$subs" -- "$cur"); return; } # subcommand-specific flags (each subcommand dies on anything else); # `inspect` takes the full policy flag set, so it falls through to the list below. case "$sub" in - build) COMPREPLY=( $(compgen -W "--rebuild -f --refresh-base -B --gate" -- "$cur") ); return ;; - update) COMPREPLY=( $(compgen -W "--check --no-gate" -- "$cur") ); return ;; - doctor) COMPREPLY=( $(compgen -W "--deep -d" -- "$cur") ); return ;; - clean) COMPREPLY=( $(compgen -W "--all -a --volumes -V" -- "$cur") ); return ;; - gate) COMPREPLY=( $(compgen -W "up down status logs" -- "$cur") ); return ;; + build) COMPREPLY=(); _aisb_fill < <(compgen -W "--rebuild -f --refresh-base -B --gate" -- "$cur"); return ;; + update) COMPREPLY=(); _aisb_fill < <(compgen -W "--check --no-gate" -- "$cur"); return ;; + doctor) COMPREPLY=(); _aisb_fill < <(compgen -W "--deep -d" -- "$cur"); return ;; + clean) COMPREPLY=(); _aisb_fill < <(compgen -W "--all -a --volumes -V" -- "$cur"); return ;; + gate) COMPREPLY=(); _aisb_fill < <(compgen -W "up down status logs" -- "$cur"); return ;; ps|version) return 1 ;; esac - COMPREPLY=( $(compgen -W \ - "--net --allow --allow-file --proxy --no-gate --mount --no-cwd --workdir --home-cache --ephemeral-home --read-only --agents --no-agents --agents-dir --env --env-file --base-image --install --harness --pi-token-env --no-pi-token-env --no-pi-config --cpus --memory --pids --tmpfs-size --shell --interactive --no-tty --keep --name --label --no-build --dry-run --print-policy --print-entrypoint --print-containerfile --allow-unenforced --insecure-keep-root --strict-args -h --help -V" \ - -- "$cur") ) + COMPREPLY=() + _aisb_fill < <(compgen -W \ + "--net --allow --allow-file --proxy --no-gate --mount --no-cwd --workdir --home-cache --ephemeral-home --read-only --agents --no-agents --agents-dir --env --env-file --base-image --install --harness --pi-token-env --no-pi-token-env --no-pi-config --cpus --memory --pids --tmpfs-size --shell --interactive --no-interactive --no-tty --keep --name --label --no-build --dry-run --print-policy --print-entrypoint --print-containerfile --allow-unenforced --insecure-keep-root --strict-args -h --help -V" \ + -- "$cur") +} + +# ai-sandbox-autophase: the pipeline driver. Its own flag list, mirroring its parser. +_ai_sandbox_autophase_completions() { + local cur + if type _get_comp_words_by_ref >/dev/null 2>&1; then + _get_comp_words_by_ref -n : cur + else + cur="${COMP_WORDS[COMP_CWORD]}" + fi + case "$cur" in + --validate-in=*) COMPREPLY=(); _aisb_fill < <(compgen -W "sandbox host" -- "${cur#*=}"); return ;; + esac + COMPREPLY=() + _aisb_fill < <(compgen -W \ + "--pi-cmd --rc-files --lib --skill-dir --sandbox-args --validate-in --no-push --no-preflight --no-notes --trust-git-changes --unit --phase --max-units --status --dry-run -n -h --help -V" \ + -- "$cur") + # --unit and --phase name pending work: offer the phase directories first, then the + # ordinary file completion the caller would otherwise have got. + local todo="${PWD}/.agents/phases/todo" + case "${COMP_WORDS[COMP_CWORD-1]}" in + --unit|--phase) + if [[ -d "$todo" ]]; then + local d + for d in "$todo"/"$cur"*; do + [[ -e "$d" ]] && COMPREPLY+=("${d#"$todo"/}") # named as the driver wants them + done + _aisb_fill < <(compgen -f -- "$cur") + fi ;; + esac } complete -o default -F _ai_sandbox_completions ai-sandbox 2>/dev/null || true +complete -o default -F _ai_sandbox_autophase_completions ai-sandbox-autophase 2>/dev/null || true if [[ $- == *i* ]]; then # also complete the harness alias for _aisb in $(compgen -A alias 2>/dev/null); do if alias "$_aisb" 2>/dev/null | grep -q 'ai-sandbox'; then diff --git a/docs/networking.md b/docs/networking.md index 1b38b0d..9f63dc1 100644 --- a/docs/networking.md +++ b/docs/networking.md @@ -72,6 +72,8 @@ name itself. | `--pids-limit 64 --memory 256m --cpus 0.5` | `pids.max=64`, `memory.max=268435456` visible inside | limits are real (note: `/proc/cpuinfo` still lists all cores; use `nproc`/cgroup limits, not cpuinfo) | | `-v $dir:/project` with SELinux enforcing, `--userns=keep-id:uid=1000,gid=1000` | write **denied** | bind mounts need `:z` on SELinux hosts | | same, with `:z` | write succeeds, file owned by the host uid (1000:1000) | project files keep sane ownership | +| `--tmpfs /tmp` **and** `-v /tmp/x:/tmp/x` in one run | the tmpfs wins: `mount` shows `tmpfs on /tmp/x`, `pwd` prints `/tmp/x`, writes succeed | a project mounted at its own path under `/tmp` would be an empty, writable, correctly-named directory that is not the project — so `mirror_block_of()` refuses `/tmp` rather than degrading quietly | +| `-v $dir:$dir` (a per-user path) beside the `/tmp` tmpfs, the `/home/sandbox` volume and the `.agents` share | `pwd` is the host path, writes land on the host owned by the host uid, `/project` + the home volume + the share all still there | mirroring buries nothing outside the denied set; asserted per release in `tests/e2e/12_sandbox.sh` | ## Why not the other designs diff --git a/docs/recipes.md b/docs/recipes.md index 5001e0c..418be04 100644 --- a/docs/recipes.md +++ b/docs/recipes.md @@ -236,3 +236,93 @@ registry.npmjs.org Then `pi` alone in that directory is the whole invocation — and because both files are in the repository, the sandbox policy gets reviewed with the code. + +## An unattended pipeline: phases in the sandbox, pushes on the host + +The nightly case: `.agents/phases/todo/` full of task files, nobody watching, an +agent that must not be able to reach the git remote or touch the host. + +```bash +# ~/.bashrc.d/ai-sandbox.bash -- the launcher, the way you already write it +sandbox-pi () { + export AI_API_TOKEN=$(pass aipi/dev-token) + ai-sandbox --allow aipi.reeseapps.com --allow github.com \ + --allow pi.dev --allow release-assets.githubusercontent.com \ + --env AI_API_TOKEN +} +``` + +```bash +cd ~/Projects/foo +ai-sandbox-autophase --status # what it resolved, what is pending, the git plan +ai-sandbox-autophase --dry-run # the exact child and gate commands, nothing run +ai-sandbox-autophase # every pending task, in pipeline order +``` + +What it does with that function is the interesting part. A script cannot see an +interactive shell's functions, so it reads the rc files, takes the *definition*, and +starts each child as: + +``` +sandbox-pi --no-tty --no-interactive -- pi --no-approve --session-dir .agents/phase-sessions --name --mode json +``` + +`pass` runs again per unit, so a run that starts at 23:00 is not holding a token +fetched at 09:00. Reading the definition is all the probe does: `--status` and +`--dry-run` never call the launcher, fetch a token, build an image or start a +container. + +| you want | flag | +|---|---| +| a different launcher | `--pi-cmd sandbox-claude`, `--pi-cmd pi`, or a whole command line | +| the function somewhere else | `--rc-files ~/.bashrc.d/zz-ai.bash` | +| fewer flags of its own | `--sandbox-args ''` (the defaults are `--no-tty --no-interactive`) | +| the gate on the host, because the toolchain lives there | `--validate-in host` (warns: the gate file is in the writable mount) | +| one task, one phase, a short run | `--unit 03_api/02_routes.md`, `--phase 03_api`, `--max-units 4` | +| commits that stay local while you watch | `--no-push` | +| no warm-up container (it costs a build the first time) | `--no-preflight` | + +Where the work goes when it finishes: the executor agents never commit; the driver +commits each completed phase on the host and pushes it, checks again before starting +the next phase, and pushes whatever is left at the end of the run. A failed push +stops the run and prints the command that finishes it — the commit is never lost, and +the next run's push sweeps it up: + +``` +[2026-10-28 02:14:07] pushing 1 unpushed commit(s) on main (between 03_api and 04_docs) + (pushed: 9f1c2ab phase: 03_api) +``` + +Inside the sandbox the agent has the repository but not the remote, and it is told so +in its own prompt (`--no-notes` to drop that block) — `git status`, `git diff`, `git +log` work; `git push`, `fetch`, `pull` and `clone ` are not attempted, so no +fixer round is spent rediscovering that. That is also why the gate runs in the sandbox: +`.agents/validate.sh` is inside the writable mount, and a jailer the prisoner can edit +is not a jailer. + +One consequence of committing out here is worth naming: a repository is not only what +git records, it is what git **runs** — `.git/config`, and `.git/hooks/` especially. So +the driver's git calls run with an empty hooks path, no pager, no system config and no +`ext::` transport, and the run refuses to deliver if a sandboxed unit changed either: + +``` +✗ ERROR: refusing to deliver (commit/push) -- the sandbox changed this repository's own git configuration. + look at it by hand: git config --local --list; ls -l .git/hooks + if the change was yours: re-run with --trust-git-changes +``` + +The commits and the work stay exactly where they are; nothing is pushed and no hook +runs either way. + +A project whose tests need something the image lacks gets it from the launcher: + +```bash +sandbox-pi () { + export AI_API_TOKEN=$(pass aipi/dev-token) + ai-sandbox --allow aipi.reeseapps.com --allow pypi.org --allow files.pythonhosted.org \ + --install 'python3-pip ripgrep' --env AI_API_TOKEN +} +``` + +`doctor`, `inspect`, `clean` and the rest of ai-sandbox work unchanged underneath it — +`--status` prints the policy the launcher will apply, without applying it. diff --git a/docs/threat-model.md b/docs/threat-model.md index 8ed184a..1d378e3 100644 --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -34,6 +34,8 @@ registry you don't control, or someone with access to your host user account. | Writing to the host's package manager, `/etc`, global npm/pip | read-only-ish image, `HOME` inside the container, optional `--read-only` | `tests/e2e/12_sandbox.sh` | | Credential persistence you did not ask for | `--ephemeral-home`; the default home volume is named and inspectable | `tests/e2e/12_sandbox.sh` | | Silent wide-open starts | an empty allowlist in `allow` mode is a hard error | `tests/unit/01_args.sh` | +| An agent shipping work you have not looked at (`git push`) | the git remote is not in the allowlist, so nothing in the container can reach it; `ai-sandbox-autophase` commits and pushes on the host, outside the sandbox, between unit runs | `tests/unit/10_autophase.sh` | +| The gate becoming the escape (`validate.sh` is inside the writable mount) | the pass/fail gate runs in the sandbox as the same unprivileged user, not on the host | `tests/unit/10_autophase.sh` | ## What does **not** stop it @@ -97,6 +99,14 @@ Be honest about these, and design your work accordingly. re-forced every run so a run cannot quietly lower the reasoning of the next. It is not eliminated: `--ephemeral-home` removes it, and `inspect` shows what is bridged. +13. **A push on the host still pushes what the agent wrote.** `ai-sandbox-autophase` + keeps the remote away from the sandbox, but somebody has to deliver the work, and + that somebody is the harness running as you. It commits only the phase's own + changes and only after `.agents/validate.sh` passes — so read that gate as part of + your trust base. It lives in the writable mount, which is exactly why the gate runs + *in the sandbox* by default; `--validate-in host` hands that judgement to you and + warns on every run. + ## Residual risk rating *Accidental destruction and ambient credential leakage: largely eliminated.* diff --git a/install.sh b/install.sh index f992b28..c842bf0 100755 --- a/install.sh +++ b/install.sh @@ -3,12 +3,15 @@ # # ./install.sh install to ~/.local/bin, print the alias line # ./install.sh --alias pi also append the alias to ~/.bashrc -# ./install.sh --prefix DIR install somewhere else +# ./install.sh --prefix DIR install somewhere else ($AISB_PREFIX does the same) # ./install.sh --uninstall remove what this script installed +# +# Both programs are installed: ai-sandbox itself, and ai-sandbox-autophase, which runs +# the .agents/phases/ pipeline with the agent in the sandbox and the git push on the host. set -Eeuo pipefail HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -PREFIX="${HOME}/.local/bin" +PREFIX="${AISB_PREFIX:-${HOME}/.local/bin}" ALIAS_NAME="" UNINSTALL=0 @@ -19,16 +22,19 @@ while (($#)); do --alias) ALIAS_NAME="${2:?--alias needs a name, e.g. --alias pi}"; shift 2 ;; --alias=*) ALIAS_NAME="${1#*=}"; shift ;; --uninstall) UNINSTALL=1; shift ;; - -h|--help) sed -n '2,10p' "$0"; exit 0 ;; + -h|--help) sed -n '2,12p' "$0"; exit 0 ;; *) echo "unknown option: $1" >&2; exit 2 ;; esac done -target="$PREFIX/ai-sandbox" +targets=("$PREFIX/ai-sandbox" "$PREFIX/ai-sandbox-autophase") +target="${targets[0]}" rcfile="${HOME}/.bashrc" if (( UNINSTALL )); then - [[ -L "$target" || -f "$target" ]] && { rm -f "$target"; echo "removed $target"; } || echo "nothing at $target" + for t in "${targets[@]}"; do + [[ -L "$t" || -f "$t" ]] && { rm -f "$t"; echo "removed $t"; } || echo "nothing at $t" + done if [[ -n "$ALIAS_NAME" ]]; then tmp="$(mktemp)" grep -v "^alias ${ALIAS_NAME}=" "$rcfile" > "$tmp" 2>/dev/null || true @@ -37,12 +43,14 @@ if (( UNINSTALL )); then exit 0 fi -[[ -x "$HERE/bin/ai-sandbox" ]] || { echo "bin/ai-sandbox is missing or not executable" >&2; exit 1; } command -v podman > /dev/null || echo "warning: podman is not installed; ai-sandbox needs it" mkdir -p "$PREFIX" -ln -sfn "$HERE/bin/ai-sandbox" "$target" -echo "installed $target -> $HERE/bin/ai-sandbox" +for src in ai-sandbox ai-sandbox-autophase; do + [[ -x "$HERE/bin/$src" ]] || { echo "bin/$src is missing or not executable" >&2; exit 1; } + ln -sfn "$HERE/bin/$src" "$PREFIX/$src" + echo "installed $PREFIX/$src -> $HERE/bin/$src" +done if [[ -n "$ALIAS_NAME" ]]; then line="alias ${ALIAS_NAME}='${target}'" @@ -56,9 +64,18 @@ else echo echo "To use it as a harness wrapper, add this to $rcfile:" echo " alias pi='$target'" - echo "then: source $rcfile && cd your/project && pi --allow aipi.reeseapps.com --env AI_API_TOKEN" + echo "then: source $rcfile && cd your/project && pi --allow api.example.com --env API_KEY" fi echo echo "next: $target doctor # sanity-check this machine" echo " $target build # build the sandbox image now (otherwise built on first run)" +echo +echo "To run a phased project inside the sandbox, wrap this in a function in ~/.bashrc.d" +echo "(that is what ai-sandbox-autophase looks for, since a script cannot see a shell" +echo "function it did not define):" +echo " sandbox-pi () {" +echo " export AI_API_TOKEN=\$(pass aipi/token)" +echo " ai-sandbox --allow api.example.com --env AI_API_TOKEN" +echo " }" +echo "then: cd your/project && ai-sandbox-autophase --status" diff --git a/tests/e2e/11_net.sh b/tests/e2e/11_net.sh index 0085e82..1b4ffc0 100644 --- a/tests/e2e/11_net.sh +++ b/tests/e2e/11_net.sh @@ -22,7 +22,9 @@ trap 'kill '"$SRV"' 2>/dev/null' EXIT sleep 1 t_name "allow mode: enforcement in one run" -sb $GATE --env AISB_PROBE_HOSTSVC="$HOSTPORT" -- bash /project/egress_probe.sh +# the probe is a file in the project, reached at whatever path the container mounts it at +PROJ_DEST="$([[ -n "$(mirror_block_of "$PROJ")" ]] && echo /project || echo "$PROJ")" +sb $GATE --env AISB_PROBE_HOSTSVC="$HOSTPORT" -- bash "$PROJ_DEST/egress_probe.sh" assert_contains "ok_allowed_http=200" "allowlisted http works (through the gate)" assert_contains "ok_allowed_https=200" "allowlisted https works (CONNECT)" assert_contains "ok_gate_denied_domain=403" "a non-allowlisted domain is refused by the gate" diff --git a/tests/e2e/12_sandbox.sh b/tests/e2e/12_sandbox.sh index 315a25f..ae37433 100644 --- a/tests/e2e/12_sandbox.sh +++ b/tests/e2e/12_sandbox.sh @@ -22,17 +22,51 @@ sb $DARK --harness pi -- bash -lc "command -v pi" assert_rc 0 "the default pi binary exists in the image" t_name "the project directory is mounted, writable, and owned by the host user" -sb $DARK -- bash -c 'pwd; touch /project/from-container.txt; echo wrote' -assert_contains "/project" "working directory is /project" +# Where it is mounted depends on the host path; see the mirroring block below, which is +# the other branch of the same decision. +DEST="$([[ -n "$(mirror_block_of "$WORK_DIR")" ]] && echo /project || echo "$WORK_DIR")" +sb $DARK -- bash -c "pwd; touch '$DEST/from-container.txt'; echo wrote" +assert_contains "$DEST" "working directory is the project mount" [[ -f "$WORK_DIR/from-container.txt" ]] && _ok "writes land on the host" || _bad "writes land on the host" if [[ "$(stat -c %u "$WORK_DIR/from-container.txt")" == "$(id -u)" ]]; then _ok "the file is owned by the host uid, not by root" else _bad "the file is owned by the host uid" "owner=$(stat -c %u "$WORK_DIR/from-container.txt") expected=$(id -u)" fi -sb $DARK -- bash -c 'cat /project/host-file.txt' +sb $DARK -- bash -c "cat '$DEST/host-file.txt'" assert_contains "hello from the host" "existing project files are readable" +t_name "the container names the directory what the host calls it" +# The reason for mirroring: pwd, an error message and a harness' window title name *this* +# project instead of every sandbox alike. Only a real container answers the question -- +# podman decides what a bind hides, and it is the one thing about the arrangement that can +# go quietly wrong. +if [[ "$DEST" == "/project" ]]; then + assert_contains "cannot mirror" "the fallback is announced rather than silent" +fi +mirrorable_dir || true +if [[ -z "$USER_MIRROR_DIR" ]]; then + _ok "no path on this host may be mirrored ($HOME is inside a sandbox?) -- skipped" +else + MD="$USER_MIRROR_DIR" + echo "hello from the host" > "$MD/host-file.txt" + cd "$MD" || exit 1 + sb $DARK -- bash -c 'pwd; cat host-file.txt; touch from-container.txt; echo wrote; ls -d /project; test -w /home/sandbox && echo home_writable && echo marked > /home/sandbox/mirror-marker' + assert_contains "$MD" "pwd is the host path, not /project" + assert_contains "hello from the host" "the mirrored mount is the project, not an empty directory" + assert_contains "wrote" "and it is writable" + [[ -f "$MD/from-container.txt" ]] \ + && _ok "writes land on the host through the mirrored path" || _bad "writes land on the host" + assert_absent "cannot mirror" "nothing is announced when the path mirrors" + assert_contains "/project" "the image's own /project is still there" + assert_contains "home_writable" "the home volume is still mounted at /home/sandbox" + # The home volume is the state the next run expects, so check it survived the mirror. + sb $DARK -- bash -c 'cat /home/sandbox/mirror-marker 2>&1; ls' + assert_contains "marked" "and the marker written through a mirrored run is still there" + assert_contains "host-file.txt" "a mirrored run sees its own project, not the previous one" + cd "$WORK_DIR" || exit 1 +fi + t_name "nothing else on the host is visible" echo secret > "$WORK/not-shared.txt" sb $DARK -- bash -c "cat '$WORK/not-shared.txt' 2>&1 || echo NOT_VISIBLE" @@ -145,12 +179,12 @@ assert_absent "demo-skill" "not even as a directory name" # visible at once, which is what a harness that reads both (pi) expects. mkdir -p "$WORK_DIR/.agents/skills/project-skill" printf 'name: project-skill\ninstructions: project only\n' > "$WORK_DIR/.agents/skills/project-skill/SKILL.md" -sb $DARK -- bash -c 'cat /project/.agents/skills/project-skill/SKILL.md; ls "$HOME/.agents/skills" | tr "\n" " "' +sb $DARK -- bash -c "cat '$DEST/.agents/skills/project-skill/SKILL.md'; ls \"\$HOME/.agents/skills\" | tr '\n' ' '" assert_contains "project only" "the project's own .agents comes in with the project mount" assert_contains "demo-skill" "and the user-level one is still alongside it" t_name "read-only root filesystem" -sb $DARK --read-only -- bash -c 'touch /usr/bin/nope 2>/dev/null && echo "BAD: wrote to rootfs" || echo rootfs_locked=ok; touch /project/rw-still-works && echo project_still_writable' +sb $DARK --read-only -- bash -c "touch /usr/bin/nope 2>/dev/null && echo 'BAD: wrote to rootfs' || echo rootfs_locked=ok; touch '$DEST/rw-still-works' && echo project_still_writable" assert_contains "rootfs_locked=ok" "--read-only blocks writes outside mounts" assert_contains "project_still_writable" "the project mount stays writable" diff --git a/tests/fixtures/phase-lib/lib.sh b/tests/fixtures/phase-lib/lib.sh new file mode 100644 index 0000000..80e8937 --- /dev/null +++ b/tests/fixtures/phase-lib/lib.sh @@ -0,0 +1,161 @@ +#!/usr/bin/env bash +# tests/fixtures/phase-lib/lib.sh -- a stand-in for the phased-execution skill's lib.sh. +# +# It is a CONTRACT fixture, not a reimplementation: the function names, the child call +# shape (`pi "${PI_ARGS[@]}" --session-dir ... --mode json "$prompt"` piped to a progress +# renderer), the gate call (`run_validation `) and the host-side commit + push +# at a phase's commit point are what bin/ai-sandbox-autophase is written against, and +# what breaks if that library changes shape. Keeping it here means the unit tests run +# with no skill installed and no containers; tests/unit/10_autophase.sh also runs the +# real library once, when it is installed. +# +# Sourced, never executed. + +PHASE_TODO=".agents/phases/todo" +PHASE_DONE=".agents/phases/complete" +PHASE_REPORTS=".agents/reports" +PHASE_SESSIONS=".agents/phase-sessions" +MAX_FIX_ATTEMPTS="${MAX_FIX_ATTEMPTS:-3}" + +FIXTURE_LIB_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +ts() { date '+%Y-%m-%d %H:%M:%S'; } +warn() { echo "[$(ts)] ⚠ $*" >&2; } +die() { echo "[$(ts)] ✗ ERROR: $*" >&2; exit 1; } + +find_root() { + local d; d="$(pwd)" + while :; do + [[ -d "$d/.agents/phases/todo" ]] && { printf '%s\n' "$d"; return 0; } + [[ "$d" == "/" ]] && return 1 + d="$(dirname "$d")" + done +} + +# A phase entry is anything in todo/ whose name starts with a digit: a phase directory, or +# a single .md file. Globs rather than ls|grep, so a name cannot be mis-split. +phase_entries() { + local e + { + ( cd "$PHASE_TODO" 2>/dev/null || exit 0 + for e in [0-9]*; do [[ -e "$e" ]] && printf '%s\n' "$e"; done ) + } | sort +} + +phase_next_unit() { + local p="$1" f t + if [[ -d "$PHASE_TODO/$p" ]]; then + for f in "$PHASE_TODO/$p"/[0-9]*.md; do + [[ -f "$f" ]] || continue + t="${f##*/}"; [[ "$t" == "00_phase.md" ]] && continue + printf '%s/%s\n' "$p" "$t"; return 0 + done + [[ -f "$PHASE_TODO/$p/00_phase.md" ]] && { printf '%s/00_phase.md\n' "$p"; return 0; } + elif [[ -f "$PHASE_TODO/$p" ]]; then + printf '%s\n' "$p" + fi + return 0 +} + +next_unit() { + local p u + for p in $(phase_entries); do + u="$(phase_next_unit "$p")" + [[ -n "$u" ]] && { printf '%s\n' "$u"; return 0; } + done + return 0 +} + +count_units() { + local p n=0 f + for p in $(phase_entries); do + if [[ -d "$PHASE_TODO/$p" ]]; then + for f in "$PHASE_TODO/$p"/[0-9]*.md; do [[ -f "$f" ]] && n=$((n + 1)); done + elif [[ -f "$PHASE_TODO/$p" ]]; then + n=$((n + 1)) + fi + done + printf '%s\n' "$n" +} + +unit_phase() { local u="$1"; [[ "$u" == */* ]] && printf '%s\n' "${u%%/*}" || printf '%s\n' "${u%.md}"; } +unit_base() { local u="${1%.md}"; printf '%s\n' "${u//\//__}"; } # 01_api/02_x.md -> 01_api__02_x +is_phase_end() { local u="$1"; if [[ "$u" == */* ]]; then [[ "${u##*/}" == "00_phase.md" ]]; else return 0; fi; } + +unit_report_dir() { + local u="$1" p + if [[ "$u" == */* ]]; then p="${u%%/*}"; else p="."; fi + printf '%s/%s\n' "$PHASE_REPORTS" "$p" +} +unit_report() { # + printf '%s/%s.a%s.%s\n' "$(unit_report_dir "$1")" "$(unit_base "$1")" "$2" "$3" +} + +move_unit() { + local u="$1" + if [[ "$u" == */* ]]; then + mkdir -p "$PHASE_DONE/${u%%/*}" + mv -f "$PHASE_TODO/$u" "$PHASE_DONE/${u%%/*}/${u##*/}" + rmdir "$PHASE_TODO/${u%%/*}" 2>/dev/null || true + else + mkdir -p "$PHASE_DONE" + mv -f "$PHASE_TODO/$u" "$PHASE_DONE/$u" + fi +} + +sweep_finished_phases() { return 0; } + +build_pi_args() { PI_ARGS=(--no-approve); } + +first_prompt() { printf 'Execute the unit %s. Reply with a report.\n' "$1"; } +fix_prompt() { printf 'Fix the failures: %s\n' "${1:-}"; } + +run_validation() { bash .agents/validate.sh >"$1" 2>&1; } + +# The host-side commit + push, at the phase's commit point: same shape as the real +# library's commit_phase/push_phase (commit, then push when PHASE_PUSH=1). +commit_phase() { + local phase="$1" + git rev-parse --is-inside-work-tree >/dev/null 2>&1 || return 0 + git add -A >/dev/null 2>&1 || return 1 + if ! git diff --cached --quiet; then + git commit --no-gpg-sign -m "phase: $phase" >/dev/null 2>&1 || return 1 + fi + # the real library hands the commit to the remote through push_phase, so this fixture + # does too -- which is also the function the driver shadows to keep every push (and + # every refusal to push) in one place. + push_phase "$phase" || return 1 + return 0 +} + +# Mirrors the library's own entry point; the driver replaces it. +push_phase() { + git remote >/dev/null 2>&1 || return 0 + git push >/dev/null 2>&1 || return 1 + return 0 +} + +notify_task() { return 0; } + +execute_unit() { + local unit="$1" errors="" rc=0 + mkdir -p "$PHASE_DONE" "$PHASE_REPORTS" "$PHASE_SESSIONS" "$(unit_report_dir "$unit")" + echo "[$(ts)] ━━ $unit — attempt 1/$MAX_FIX_ATTEMPTS ━━" # the real library prints this; tests read it + build_pi_args + local prompt out errf vrc + prompt="$(first_prompt "$unit")" + out="$(unit_report "$unit" 1 md)" + errf="$(unit_report "$unit" 1 err)" + pi "${PI_ARGS[@]}" --session-dir "$PHASE_SESSIONS" --name "$(unit_base "$unit")" \ + --mode json "$prompt" 2>"$errf" | bash "$FIXTURE_LIB_DIR/progress.sh" "$out" + rc="${PIPESTATUS[0]}" + (( rc == 0 )) || errors+="child exited $rc"$'\n' + run_validation "$(unit_report "$unit" 1 validate)"; vrc=$? + (( vrc == 0 )) || errors+=".agents/validate.sh failed"$'\n' + [[ -n "$errors" ]] && { printf '%s' "$errors" >&2; return 1; } + move_unit "$unit" + if is_phase_end "$unit"; then + commit_phase "$unit" || return 1 + fi + return 0 +} diff --git a/tests/fixtures/phase-lib/progress.sh b/tests/fixtures/phase-lib/progress.sh new file mode 100644 index 0000000..111f043 --- /dev/null +++ b/tests/fixtures/phase-lib/progress.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +# tests/fixtures/phase-lib/progress.sh -- the fixture's stand-in for progress.mjs. +# +# Same job and same contract: read the child's JSON stream on stdin, write the final +# assistant text to $1, exit non-zero when the stream produced no report. +set -uo pipefail + +out="$1"; shift +text="" +while IFS= read -r line; do + case "$line" in + *'"type": "message_end"'* | *'"type":"message_end"'*) + t="$(printf '%s' "$line" | sed -n 's/.*"text": *"\([^"]*\)".*/\1/p')" + [[ -n "$t" ]] && text="$t" + ;; + esac +done + +if [[ -z "$text" ]]; then + printf '(no final assistant message)\n' > "$out" + exit 1 +fi +printf '%s\n' "$text" > "$out" diff --git a/tests/lib.sh b/tests/lib.sh index 7ee8a24..734c378 100644 --- a/tests/lib.sh +++ b/tests/lib.sh @@ -31,6 +31,84 @@ sb_timeout() { return 0 } +# Extract one function from the script itself, the way 06_entrypoint.sh extracts the +# entrypoint: the test then asks the real code a question instead of restating its answer. +aisb_fn() { # aisb_fn -> that function's source, on stdout + awk -v fn="$1" '$0 ~ "^" fn "\\(\\) \\{" {f=1} f {print} f && /^\}$/ {exit}' "$AISB" +} + +# Where the sandbox would mount a host path. Empty means it mirrors the path; anything +# else names the container path the bind would bury. Extracted rather than reimplemented: +# the answer depends on the container's own filesystem (/usr, /tmp, /home/sandbox), and +# most of that hierarchy cannot be created by a test. +mirror_block_of() { # mirror_block_of -> blocking path or empty + local src; src="$(aisb_fn mirror_block_of)" + [[ -n "$src" ]] || { echo "mirror_block_of not found in $AISB" >&2; return 2; } + ( set -uo pipefail; eval "$src"; mirror_block_of "$1" ) +} + +# A directory whose host path the sandbox is allowed to mirror, in USER_MIRROR_DIR (empty +# when there is none). The scratch dir ($WORK) is under $TMPDIR, and /tmp is one of the +# paths the sandbox owns, so a run from there always takes the fallback branch -- the +# mirroring branch needs a per-user tree, and $HOME is the one place an unprivileged test +# can create it. Nothing is written into the real ~/.config, ~/.local or ~/.cache. +# +# Assigns a global instead of printing: the cleanup trap below belongs to the test file, +# so this must not be called as $(mirrorable_dir) -- a command substitution would set the +# trap in a subshell that exits, and takes the directory with it. +USER_MIRROR_DIR="" +mirrorable_dir() { + local base="${HOME%/}" qbase + USER_MIRROR_DIR="" + [[ -n "$base" && -d "$base" && -w "$base" ]] || return 1 + [[ -z "$(mirror_block_of "$base")" ]] || return 1 + USER_MIRROR_DIR="$(mktemp -d "$base/.ai-sandbox-test-mirror.XXXXXX" 2>/dev/null)" || return 1 + [[ -n "$USER_MIRROR_DIR" ]] || return 1 + # Removed when the test file's subshell exits, including on a failed assert. The base is + # quoted in now rather than read from $HOME later, so the trap cannot widen. + printf -v qbase '%q' "$base" + # shellcheck disable=SC2064 # immediate expansion of the captured base is the point + trap "rm -rf -- $qbase/.ai-sandbox-test-mirror.* >/dev/null 2>&1 || true" EXIT + return 0 +} + +# Run a command with stdin, stdout and stderr on a pty: -t and -i are decided from the +# real terminal, so a test about them cannot use the runner's pipes. util-linux's 'script' +# is the usual tool; python3's pty module does the same job where script is not installed +# (measured: all three descriptors are ttys and the exit code propagates). Sets OUT and RC +# like sb(). Args are joined for 'script', so pass simple words, not quoted sentences. +pty_available() { command -v script > /dev/null 2>&1 || command -v python3 > /dev/null 2>&1; } +pty_run() { + OUT="" RC=0 + if command -v script > /dev/null 2>&1; then + OUT="$(script -qec "$*" /dev/null 2>&1 /dev/null 2>&1; then + OUT="$(python3 -c ' +import os, pty, sys +st = pty.spawn(sys.argv[1:]) +try: sys.exit(os.waitstatus_to_exitcode(st)) +except AttributeError: sys.exit(st >> 8)' "$@" 2>&1 (after snapshot_containers) + local now extra + now="$(podman ps -aq --filter label=ai-sandbox=1 2>/dev/null | sort)" + if [[ "$now" == "${AISB_CONTAINER_SNAPSHOT:-}" ]]; then _ok "$1"; return 0; fi + extra="$(comm -13 <(printf '%s\n' "${AISB_CONTAINER_SNAPSHOT:-}") \ + <(printf '%s\n' "$now") | tr '\n' ' ')" + _bad "$1" "container ids appeared: ${extra:-none (one disappeared instead)}" +} + # sb_env VAR=val ... -- args (env prefix, without leaking into the test shell) sb_env() { local -a envs=() diff --git a/tests/run.sh b/tests/run.sh index 817de4b..51a2029 100755 --- a/tests/run.sh +++ b/tests/run.sh @@ -30,7 +30,8 @@ command -v podman >/dev/null || { echo "podman is required to run the tests" >&2 # Syntax-check every shell file in the project before running anything. SYNTAX_BAD=0 -for f in "$AISB" "$HERE"/*.sh "$HERE"/unit/*.sh "$HERE"/e2e/*.sh "$ROOT"/install.sh; do +for f in "$AISB" "$HERE"/*.sh "$HERE"/unit/*.sh "$HERE"/e2e/*.sh "$HERE"/fixtures/*.sh \ + "$HERE"/fixtures/phase-lib/*.sh "$ROOT"/install.sh "$ROOT"/contrib/*.bash; do [[ -f "$f" ]] || continue if ! syn="$(bash -n "$f" 2>&1)"; then echo "SYNTAX: $f" >&2; printf '%s\n' "$syn" | sed 's/^/ /' >&2; SYNTAX_BAD=1 diff --git a/tests/unit/03_argv.sh b/tests/unit/03_argv.sh index f1af16e..3b39f1e 100755 --- a/tests/unit/03_argv.sh +++ b/tests/unit/03_argv.sh @@ -5,6 +5,9 @@ source "$TESTS/lib.sh" PROJ="$WORK/proj"; mkdir -p "$PROJ"; cd "$PROJ" || exit 1 export AISB_CONFIG_DIR="$WORK/cfg-empty"; mkdir -p "$AISB_CONFIG_DIR" A="--allow example.com" +# Every run in this file is a preview. What that is worth is asserted at the end, against +# the set of containers that existed before it started. +snapshot_containers t_name "isolation flags are always on" sb --dry-run $A @@ -36,12 +39,15 @@ sb --dry-run $A --no-gate assert_absent "HTTP_PROXY=" "--no-gate sets no proxy" t_name "filesystem" +# Where the cwd lands depends on the host path, so the expected destination is asked of +# the same function the script uses instead of being hardcoded here. +PROJ_DEST="$([[ -n "$(mirror_block_of "$PROJ")" ]] && echo /project || echo "$PROJ")" sb --dry-run $A -assert_contains "$PROJ:/project" "cwd is mounted at /project" -assert_contains "--workdir /project" "cwd is the working directory" +assert_contains "$PROJ:$PROJ_DEST" "cwd is mounted at its own path or the fallback" +assert_contains "--workdir $PROJ_DEST" "cwd is the working directory" assert_contains "/run/aisb:ro" "policy dir is mounted read-only" sb --dry-run $A --no-cwd -assert_absent "$PROJ:/project" "--no-cwd mounts nothing" +assert_absent "$PROJ" "--no-cwd mounts nothing" sb --dry-run $A --workdir /src assert_contains "$PROJ:/src" "--workdir changes the mount point" mkdir -p "$WORK/extra" @@ -58,6 +64,119 @@ sb --dry-run $A --ephemeral-home assert_contains "/home/sandbox:rw,nosuid,nodev" "--ephemeral-home uses a tmpfs home" assert_absent "type=volume,source=aisb-home" "--ephemeral-home drops the volume" +t_name "the container names the directory what the host calls it" +# Mirroring the host path is what makes pwd, an error message and a harness' window title +# name *this* project. It is a decision about the host path, so both branches need a real +# directory: $TMPDIR is a path the sandbox owns, a per-user tree is not. +mirrorable_dir || true +MIRROR_DIR="$USER_MIRROR_DIR" +if [[ -z "$MIRROR_DIR" ]]; then + _ok "no writable host path here may be mirrored ($HOME is itself inside a sandbox?) -- skipped" +else + cd "$MIRROR_DIR" || exit 1 + sb --dry-run $A + assert_contains "$MIRROR_DIR:$MIRROR_DIR" "the cwd is mounted at the path it has on the host" + assert_contains "--workdir $MIRROR_DIR" "and worked in at that path" + assert_absent "# workdir:" "nothing to announce when the path mirrors" + sb --dry-run $A --print-policy + assert_eq "" "$(printf '%s' "$OUT" | grep '^workdir_note=' || true)" "and no note in the policy" + sb --dry-run $A --workdir /src + assert_contains "$MIRROR_DIR:/src" "--workdir still picks the mount point over mirroring" + assert_contains "--workdir /src" "--workdir is what reaches podman" + sb --dry-run $A --workdir auto + assert_contains "--workdir $MIRROR_DIR" "'auto' is the default, spelled out" + cd "$PROJ" || exit 1 +fi + +t_name "a host path the sandbox owns is not mirrored, and says so" +# /tmp is the guaranteed case: the sandbox mounts a tmpfs there, and podman puts that +# tmpfs on top of a bind of the same path -- so a mirrored /tmp project would come out as +# an empty tmpfs. Measured, see docs/networking.md. +cd /tmp || exit 1 +sb --dry-run $A +assert_contains "--workdir /project" "/tmp falls back to the image's own directory" +assert_contains "-v /tmp:/project" "and mounts the cwd there instead" +assert_contains "# workdir: cannot mirror /tmp" "--dry-run explains the fallback" +assert_contains "the sandbox owns /tmp*" "naming what it would have buried" +assert_contains "--workdir DIR to choose" "and the way to decide otherwise" +sb_env AI_API_TOKEN=canary-not-a-real-token --dry-run $A --env AI_API_TOKEN +assert_absent "canary-not-a-real-token" "the note carries no values, only the decision" +sb --dry-run $A --print-policy +assert_contains "workdir=/project" "the policy says which directory was chosen" +assert_contains "workdir_note=" "and why" +assert_contains "cwd=/tmp" "while still printing the host path" +cd "$PROJ" || exit 1 + +t_name "which host paths may be mirrored is a decision about the container's filesystem" +# The refusals are about what lives at a path *inside* the image, and a test cannot create +# /usr or /home/sandbox. So ask the function itself, extracted from the script, about the +# paths it was written for -- the same trick 06_entrypoint.sh uses for the entrypoint. +[[ -n "$(aisb_fn mirror_block_of)" ]] \ + && _ok "the mirror decision is one function in the script" \ + || _bad "the mirror decision is one function in the script" +# Blocked: everything the base image owns, plus this sandbox's own mounts and home. +for p in / /usr/src/proj /var/lib/pg/data /opt/stack/app /etc/proj /tmp/build/proj \ + /run/user/1000/proj /home /home/sandbox /home/sandbox/work /bin/proj /boot/efi \ + /dev/shm/x /lib/x86_64-linux-gnu /lib32/x /lib64/ld /proc/1/cwd /root/proj \ + /sbin/init /sys/firmware; do + b="$(mirror_block_of "$p")" + if [[ -n "$b" ]]; then _ok "$p is not mirrored (would bury $b)" + else _bad "$p is not mirrored" "it was mirrored"; fi +done +assert_eq "/tmp" "$(mirror_block_of /tmp/a/b)" "the note names the top-level path, not the project" +assert_eq "/home/sandbox" "$(mirror_block_of /home/sandbox/work)" "the container home is named as itself" +assert_eq "/" "$(mirror_block_of /)" "the whole root is refused" +# Free: per-user trees under /home, and the hierarchy the image leaves empty. +for p in /home/you/Projects/foo /home/dev /home/sandboxer/proj /mnt/data/proj /srv/www \ + /media/usb/proj /code/x /work/proj /data/proj /home/sandboxish/x; do + b="$(mirror_block_of "$p")" + if [[ -z "$b" ]]; then _ok "$p mirrors to itself" + else _bad "$p mirrors to itself" "refused because of $b"; fi +done + +t_name "a project path podman cannot express is refused with a reason" +# Measured: podman answers `-v /a:b:/project` with `invalid option type "/project"`, which +# is a strange thing to read when all you typed was `pi`. Mirroring puts the same string in +# the destination too, so it is refused before podman is involved. +COLON="$WORK/a:b"; mkdir -p "$COLON"; cd "$COLON" || exit 1 +sb --dry-run $A +assert_rc 1 "a ':' in the project path is refused" +assert_contains "podman cannot parse" "and says whose parser" +assert_contains "--no-cwd" "and names the way out" +sb --dry-run $A --no-cwd +assert_rc 0 "--no-cwd needs nothing mounted, so the path is fine" +cd "$PROJ" || exit 1 + +t_name "--workdir takes a container path or 'auto', nothing else" +sb --dry-run $A --workdir relative/dir +assert_rc 1 "a relative --workdir is refused" +assert_contains "must be an absolute path or 'auto'" "and says what it wanted" +sb --dry-run $A --workdir / +assert_rc 1 "--workdir / is refused" +assert_contains "over the whole container" "with the reason" +sb --dry-run $A --workdir /src/ +assert_contains "$PROJ:/src" "a trailing slash is not part of the destination" +# An explicit path onto something the sandbox owns is honoured -- it was asked for -- but +# the hiding rule does not stop applying because a human chose the path, so it is warned. +sb --dry-run $A --workdir /tmp/proj +assert_rc 0 "--workdir onto a path the sandbox owns is still honoured" +assert_contains "$PROJ:/tmp/proj" "podman is told what was asked for" +assert_contains "warning: --workdir /tmp/proj" "and the covering is announced" +assert_contains "/tmp is a tmpfs" "with the reason" +sb --dry-run $A --workdir /opt/proj +assert_contains "the sandbox owns /opt*" "the same for a directory the image owns" +sb --dry-run $A --workdir /srv/app +assert_absent "warning: --workdir" "a path nothing owns is chosen in silence" +sb_env AISB_PROJECT_DEST=/srv/app --dry-run $A +assert_contains "$PROJ:/srv/app" "AISB_PROJECT_DEST still chooses it" +mkdir -p "$WORK/cfg-wd" +printf 'workdir = /srv/cfg\n' > "$WORK/cfg-wd/config" +# No subshell here: OUT and the counters have to survive it. +AISB_CONFIG_DIR_SAVED="$AISB_CONFIG_DIR"; export AISB_CONFIG_DIR="$WORK/cfg-wd" +sb --dry-run $A +assert_contains "$PROJ:/srv/cfg" "the config file chooses it too" +export AISB_CONFIG_DIR="$AISB_CONFIG_DIR_SAVED" + t_name "labels, limits, extra args" sb --dry-run $A --label team=x assert_contains "--label team=x" "--label" @@ -80,7 +199,41 @@ if [[ -n "$pos_cmd" && -n "$pos_img" && $pos_img -lt $pos_cmd ]]; then _ok "the t_name "dry run is side-effect free" sb --dry-run $A -RUNNING="$(podman ps --format '{{.Names}}' | grep -c 'aisb' || true)" +# Ask about the container *this* invocation would have created, not about every sandbox on +# the machine: another session that happens to be running is not this test starting one. +# The session id is unique per run, so its label is the hermetic question. +SID="$(printf '%s' "$OUT" | tr ' ' '\n' | sed -n 's/^aisb\.session=//p' | head -1)" +[[ -n "$SID" ]] && _ok "the dry run names its session" || _bad "the dry run names its session" +CREATED="$(podman ps -a --filter "label=aisb.session=$SID" --format '{{.Names}}' | wc -l | tr -d ' ')" DIRS="$(ls -A "$AISB_STATE_DIR/sessions" 2>/dev/null | wc -l | tr -d ' ')" -assert_eq "$RUNNING" 0 "dry run started no container" +assert_eq "$CREATED" 0 "dry run created no container, under any name" assert_eq "$DIRS" 0 "dry run left no session directory behind" +assert_no_new_containers "and no preview in this file created one either" + +t_name "attachment: stdin is a choice, and an unattended run does not want it" +# -t and -i are decided from the real stdin/stdout, and the test runner's are pipes. The +# pipe case is checked either way; the attached case needs a pty, which lib.sh builds from +# 'script' or python3 and reports as skipped when the host has neither. +inv_line() { printf '%s' "$OUT" | tr -d '\r' | tail -1; } +has_opt() { inv_line | grep -qE "(^| )$1( |$)"; } +sb_pty() { pty_run "$AISB" --dry-run --net none "$@"; } +sb --dry-run --net none +if has_opt "-i" || has_opt "-t"; then _bad "pipes get neither" "$(inv_line)"; else _ok "pipes get neither: nothing is attached to a terminal that is not there"; fi +sb --dry-run --net none --interactive +if has_opt "-i"; then _ok "--interactive still forces it on with pipes"; else _bad "--interactive still forces it on with pipes" "$(inv_line)"; fi +if ! pty_available; then + _ok "neither 'script' nor python3 here -- the attached cases are skipped" +else + sb_pty + assert_rc 0 "on a pty" + if has_opt "-i" && has_opt "-t"; then _ok "a terminal run is attached, which is what an interactive shell expects"; else _bad "a terminal run is attached" "$(inv_line)"; fi + sb_pty --no-interactive + assert_rc 0 "--no-interactive" + if has_opt "-i"; then _bad "--no-interactive drops -i" "$(inv_line)"; else _ok "--no-interactive drops -i: the command cannot read this terminal"; fi + if has_opt "-t"; then _ok "--no-interactive leaves the TTY alone (it is a separate choice)"; else _bad "--no-interactive leaves the TTY alone" "$(inv_line)"; fi + sb_pty --no-tty --no-interactive + if has_opt "-t" || has_opt "-i"; then _bad "--no-tty --no-interactive detaches both" "$(inv_line)"; else _ok "--no-tty --no-interactive detaches both"; fi + sb_pty --strict-args --no-interactive + assert_rc 0 "--no-interactive is a real flag, not a pass-through to the harness" + if has_opt "-i"; then _bad "--no-interactive is parsed, not forwarded" "$(inv_line)"; else _ok "--no-interactive is parsed, not forwarded"; fi +fi diff --git a/tests/unit/08_pi_config.sh b/tests/unit/08_pi_config.sh index 3b6e338..cf85a7e 100644 --- a/tests/unit/08_pi_config.sh +++ b/tests/unit/08_pi_config.sh @@ -207,12 +207,13 @@ assert_contains "will not reach the container" "preview warns instead of launchi t_name "a bridged provider with no credential in reach is refused, not attempted" unset AI_API_TOKEN +snapshot_containers sb --net none --harness pi assert_rc 1 "the run refuses to start" assert_contains 'asks for $AI_API_TOKEN, but it is not going to reach the container' "it names the variable and why" assert_contains "env = AI_API_TOKEN" "it shows the config-file way" assert_contains "--no-pi-config" "and the way to opt out" -assert_eq "0" "$([[ -n "$(podman ps -q --filter label=ai-sandbox=1)" ]] && echo 1 || echo 0)" "and nothing was started" +assert_no_new_containers "and nothing was started (not even a container someone else runs)" t_name "an empty token is treated as missing" sb_env AI_API_TOKEN= --net none --harness pi # set, but empty diff --git a/tests/unit/10_autophase.sh b/tests/unit/10_autophase.sh new file mode 100644 index 0000000..bee6c59 --- /dev/null +++ b/tests/unit/10_autophase.sh @@ -0,0 +1,569 @@ +#!/usr/bin/env bash +# tests/unit/10_autophase.sh -- bin/ai-sandbox-autophase: the pipeline driver. +# +# Nothing here starts a container. The two things the driver is built on are both +# stood in for, faithfully: +# +# * the LAUNCHER -- a sandbox-pi() function written into a fixture rc file, exactly +# like the one people keep in ~/.bashrc.d. It records the argv it was handed (that +# is what "the launcher, then --, then the harness command" has to mean) and then +# runs what it was told to run, so the pipeline is exercised for real. It also +# exports a canary token the way the real one calls `pass`, which is how these +# tests check that no printed surface repeats a value. +# * the PIPELINE LIBRARY -- tests/fixtures/phase-lib is a contract fixture: the same +# function names, the same `pi ... | progress` call, the same gate call, the same +# host-side commit+push at a phase's commit point. The last test runs the REAL +# library as well, when it is installed. +# +# The git remote is a local bare repository: a push really happens, and really fails on +# demand, with no network involved. +source "$TESTS/lib.sh" + +APBIN="$(dirname "$AISB")/ai-sandbox-autophase" +LIB_FIX="$TESTS/fixtures/phase-lib/lib.sh" +CANARY="CANARY-TOKEN-9d41f6" +REAL_GIT="$(command -v git)" +mkdir -p "$WORK/bin" + +t_name "the driver exists" +[[ -x "$APBIN" ]] && _ok "bin/ai-sandbox-autophase is executable" || _bad "bin/ai-sandbox-autophase is executable" + +# --- the launcher, as it lives in an rc file --------------------------------- +mkdir -p "$WORK/rc" +cat > "$WORK/rc/00-nothing.bash" <<'EOF' +# defines nothing the driver asked for: the search must keep looking +unrelated_setup() { PATH="$PATH:/nowhere"; } +EOF +cat > "$WORK/rc/sandbox.bash" <> "\$trace" + (( \${#cmd[@]} )) || return 0 + "\${cmd[@]}" +} +EOF +cat > "$WORK/rc/aliases.bash" <<'EOF' +alias sandbox-alias='sandbox-pi' +EOF +cat > "$WORK/rc/broken.bash" <<'EOF' +broken-sandbox() { + local -a cmd=() + while (($#)); do case "$1" in --) shift; cmd=("$@"); break ;; *) shift ;; esac; done + echo "Error: podman: executable not found" >&2 + return 125 +} +EOF + +# --- the harness and the git shim ------------------------------------------- +cat > "$WORK/bin/pi" <<'EOF' +#!/usr/bin/env bash +# fake pi: does the work, writes a session file, emits the stream a real pi emits. +name=""; sessdir="."; prompt="" +while (($#)); do + case "$1" in + --session-dir) sessdir="$2"; shift 2 ;; + --name) name="$2"; shift 2 ;; + --session|--mode|--model|--thinking) shift 2 ;; + --no-session|--no-approve|--approve) shift ;; + --*) shift ;; + *) prompt="$1"; shift ;; + esac +done +[[ -n "$prompt" ]] || { echo "fake pi: no prompt arrived" >&2; exit 3; } +# the token has to reach the sandboxed agent -- recorded as a length, never a value +printf 'token_bytes=%s\n' "${#AI_API_TOKEN}" >> "${FAKE_PI_MARKER:-/dev/null}" +printf '%s\n---\n' "$prompt" >> "${FAKE_PI_PROMPTS:-/dev/null}" +printf 'work by %s\n' "${name:-unit}" >> work.txt +# an agent that tries to make the *host* run something when the harness commits +if [[ -n "${FAKE_PI_PLANT_HOOK:-}" ]]; then + mkdir -p .git/hooks + printf '#!/bin/sh\ntouch "%s"\n' "$FAKE_PI_PLANT_HOOK" > .git/hooks/pre-commit + chmod +x .git/hooks/pre-commit +fi +mkdir -p "$sessdir" +{ + printf '{"type": "session", "id": "fake-%s"}\n' "${name:-s}" + printf '{"type": "message_end", "message": {"role": "assistant", "content": [{"type": "text", "text": "report for %s"}]}}\n' "${name:-s}" + printf '{"type": "agent_end"}\n' +} | tee "${sessdir}/${name:-session}.jsonl" +EOF +cat > "$WORK/bin/git" <<'EOF' +#!/usr/bin/env bash +printf '%s\n' "$*" >> "${FAKE_GIT_TRACE:-/dev/null}" +if [[ "$1" == push && -n "${FAKE_GIT_FAIL:-}" ]]; then + echo "fatal: unable to access '${FAKE_GIT_FAIL}': Could not resolve host" >&2 + exit 1 +fi +exec "$REAL_GIT" "$@" +EOF +chmod +x "$WORK/bin/pi" "$WORK/bin/git" +export GIT_AUTHOR_NAME=test GIT_AUTHOR_EMAIL=t@example.invalid +export GIT_COMMITTER_NAME=test GIT_COMMITTER_EMAIL=t@example.invalid + +# --- a phased project, with a local bare repo as its remote ------------------ +# Every project gets its own bare remote: they all commit a branch called main, and a +# shared remote would turn the second project's push into a non-fast-forward that has +# nothing to do with what is being tested. +mkproj() { # [phases] -> the remote it pushed its init commit to + local p="$1" n="${2:-1}" REMOTE + # Named after the project, not counted: a counter incremented in a command + # substitution would stay in the subshell and hand every project the same remote, + # which is a non-fast-forward waiting to happen. + REMOTE="$WORK/remotes/$(basename "$p").git" + mkdir -p "$WORK/remotes" + "$REAL_GIT" init -q --bare "$REMOTE" >/dev/null 2>&1 + mkdir -p "$p/.agents/phases/todo/01_one" + printf '# task one\n\nDo the thing.\n' > "$p/.agents/phases/todo/01_one/01_task.md" + printf '# final pass one\n\nWrap up phase 01_one.\n' > "$p/.agents/phases/todo/01_one/00_phase.md" + if [[ "$n" == 2 ]]; then + mkdir -p "$p/.agents/phases/todo/02_two" + printf '# task two\n\nDo the next thing.\n' > "$p/.agents/phases/todo/02_two/01_task.md" + printf '# final pass two\n\nWrap up phase 02_two.\n' > "$p/.agents/phases/todo/02_two/00_phase.md" + fi + printf '#!/usr/bin/env bash\nexit 0\n' > "$p/.agents/validate.sh" + printf '.agents/phase-sessions/\n*.log\n' > "$p/.gitignore" + ( cd "$p" && "$REAL_GIT" init -q -b main . && "$REAL_GIT" remote add origin "$REMOTE" \ + && "$REAL_GIT" add -A && "$REAL_GIT" commit -q --no-gpg-sign -m 'init' \ + && "$REAL_GIT" push -q -u origin main ) >/dev/null 2>&1 + printf '%s\n' "$REMOTE" # so a test can count what actually reached the remote +} + +TRACE="$WORK/launcher.trace" +GTRACE="$WORK/git.trace" +MARKER="$WORK/token.marker" +PROMPTS="$WORK/prompts.log" +: > "$TRACE"; : > "$GTRACE"; : > "$MARKER"; : > "$PROMPTS" +DEF_RC="$WORK/rc/sandbox.bash:$WORK/rc/aliases.bash:$WORK/rc/00-nothing.bash" + +# A prefix assignment in front of a shell function sticks around for the rest of the +# script, so ap_run reads every T_* into a local and then unsets it: each run starts +# from the defaults unless the test asked otherwise, and cannot inherit the last one. +ap_run() { # [args...] + local cmd="${T_CMD-sandbox-pi}" rc="${T_RC-$DEF_RC}" lib="${T_LIB-$LIB_FIX}" + local validate="${T_VALIDATE-sandbox}" sargs="${T_SANDBOX_ARGS---no-tty --no-interactive}" + local push="${T_PUSH-1}" pre="${T_PREFLIGHT-1}" notes="${T_NOTES-1}" gfail="${T_GITFAIL-}" + local ppush="${T_PHASE_PUSH-}" dryout="${T_DRYOUT-}" plant="${T_PLANT-}" + unset T_CMD T_RC T_LIB T_VALIDATE T_SANDBOX_ARGS T_PUSH T_PREFLIGHT T_NOTES T_GITFAIL T_PHASE_PUSH T_DRYOUT T_PLANT + OUT="$(env "PATH=$WORK/bin:$PATH" \ + "AISB_AP_LIB=$lib" "AISB_AP_RC_FILES=$rc" "AISB_AP_PI_CMD=$cmd" \ + "AISB_AP_VALIDATE=$validate" "AISB_AP_SANDBOX_ARGS=$sargs" \ + "AISB_AP_PUSH=$push" "AISB_AP_PREFLIGHT=$pre" "AISB_AP_NOTES=$notes" \ + "AISB_AP_TRACE=$TRACE" "FAKE_GIT_TRACE=$GTRACE" "FAKE_PI_MARKER=$MARKER" \ + "FAKE_PI_PROMPTS=$PROMPTS" "REAL_GIT=$REAL_GIT" "PHASE_NOTIFY=0" \ + ${gfail:+"FAKE_GIT_FAIL=$gfail"} ${ppush:+"PHASE_PUSH=$ppush"} ${dryout:+"AISB_AP_DRYOUT=$dryout"} \ + ${plant:+"FAKE_PI_PLANT_HOOK=$plant"} \ + "$APBIN" "$@" 2>&1)" && RC=0 || RC=$? + return 0 +} +gt() { grep -c "^$1" "$GTRACE" 2>/dev/null || true; } # count of git calls by verb +trace_has() { grep -qF -- "$1" "$TRACE"; } + +PROJ="$WORK/proj"; mkproj "$PROJ" 1 >/dev/null +cd "$PROJ" || exit 1 + +t_name "the launcher is resolved from the rc files, and is never run to find it" +ap_run --status +assert_rc 0 "--status" +assert_contains "sandbox-pi" "names the configured command" +assert_contains "function, from $WORK/rc/sandbox.bash" "found as a function, in the file that defines it" +assert_absent "$CANARY" "the token never appears" +if [[ -s "$TRACE" ]]; then _bad "resolving does not run the launcher" "the launcher was called"; else _ok "resolving does not run the launcher"; fi +ap_run --dry-run +assert_absent "$CANARY" "--dry-run does not either" + +t_name "an alias is a launcher too" +T_CMD=sandbox-alias ap_run --status +assert_rc 0 "--status with an alias" +assert_contains "alias, from $WORK/rc/aliases.bash" "found as an alias" + +t_name "a program on PATH, and a whole command line" +T_CMD=pi ap_run --status +assert_rc 0 "a program on PATH" +assert_contains "program, from $WORK/bin/pi" "and where it came from" +T_CMD='ai-sandbox --allow api.example.com --env API_KEY' ap_run --status +assert_rc 0 "a command line" +assert_contains "arguments, from --pi-cmd" "used verbatim" +T_CMD='ai-sandbox --env API_KEY=SECRETLITERALVALUE --allow api.example.com' ap_run --status +assert_absent "SECRETLITERALVALUE" "a literal --env value in --pi-cmd is not echoed back" +assert_contains "--env API_KEY" "while the name still is" + +t_name "an unresolvable launcher is refused with what was searched" +T_CMD=no-such-sandbox-cmd ap_run --status +assert_rc 1 "nothing by that name anywhere" +assert_contains "cannot resolve the sandbox launcher" "says what failed" +assert_contains "$WORK/rc/00-nothing.bash" "lists the rc files it searched" +T_CMD='' ap_run --status +assert_rc 1 "an explicitly empty launcher is an error, not a silent fallback" +assert_contains "no sandbox launcher" "and says so" + +t_name "a child is the launcher, then --, then the harness command" +: > "$TRACE" +PROJ_A="$WORK/pa"; mkproj "$PROJ_A" 1 >/dev/null +cd "$PROJ_A" || exit 1 +ap_run --dry-run +assert_rc 0 "--dry-run" +assert_contains "-- pi " "the pi command rides after --" +assert_contains "--no-tty" "no TTY for the JSON stream to corrupt" +assert_contains "--no-interactive" "and your keystrokes cannot reach the agent" +: > "$TRACE" +T_SANDBOX_ARGS='' ap_run --dry-run +assert_absent "--no-tty" "--sandbox-args replaces the defaults rather than adding to them" +assert_rc 0 "and an explicitly empty value means none, not the default" +: > "$TRACE" +ap_run --max-units 1 +assert_rc 0 "one unit" +trace_has "A:--no-interactive" "the launcher got the run flags" +if grep -q '^LAUNCH argc=[0-9]* cmd=pi$' "$TRACE"; then + _ok "pi is the in-container command" +else + _bad "pi is the in-container command" "$(grep '^LAUNCH' "$TRACE" | head -3)" +fi +argc="$(grep '^LAUNCH argc=.* cmd=pi$' "$TRACE" | head -1 | sed 's/.*argc=\([0-9]*\).*/\1/')" +[[ -n "$argc" && "$argc" -ge 8 ]] && _ok "the whole harness command line reached the launcher ($argc arguments)" \ + || _bad "the whole harness command line reached the launcher" "argc=$argc" +if grep -A1 '^A:--$' "$TRACE" | grep -q '^A:pi$'; then + _ok "-- is the separator, and nothing after it is a sandbox flag" +else + _bad "-- is the separator, and nothing after it is a sandbox flag" +fi + +t_name "the prompt arrives as one argument, with the sandbox rules in it" +if grep -qF 'Sandbox rules for this run' "$PROMPTS"; then + _ok "the agent is told it is inside a sandbox" +else + _bad "the agent is told it is inside a sandbox" "$(tail -5 "$PROMPTS")" +fi +grep -qF 'git push' "$PROMPTS" && _ok "it is told the remote is unreachable, by name" \ + || _bad "it is told the remote is unreachable, by name" +grep -qF '01_one/01_task.md' "$PROMPTS" && _ok "the unit it is running is in the prompt" \ + || _bad "the unit it is running is in the prompt" +: > "$PROMPTS" +T_NOTES=0 ap_run --unit 01_one/00_phase.md +assert_rc 0 "one unit, notes off" +if grep -qF 'Sandbox rules' "$PROMPTS"; then _bad "--no-notes drops the block"; else _ok "--no-notes drops the block"; fi +grep -qF '01_one/00_phase.md' "$PROMPTS" && _ok "the library's own prompt still arrives" \ + || _bad "the library's own prompt still arrives" "$(head -3 "$PROMPTS")" + +t_name "the token reaches the agent" +if grep -q "token_bytes=${#CANARY}" "$MARKER"; then + _ok "exported by the launcher, present in the child" +else + _bad "exported by the launcher, present in the child" "$(tail -1 "$MARKER" 2>/dev/null)" +fi + +t_name "the gate runs where the agent runs" +: > "$TRACE" +PROJ_B="$WORK/pb"; mkproj "$PROJ_B" 1 >/dev/null +cd "$PROJ_B" || exit 1 +ap_run --unit 01_one/01_task.md +assert_rc 0 "one task" +if grep -q '^A:.agents/validate.sh$' "$TRACE"; then + _ok ".agents/validate.sh is handed to the launcher" +else + _bad ".agents/validate.sh is handed to the launcher" "$(tail -5 "$TRACE")" +fi +: > "$TRACE" +T_VALIDATE=host ap_run --unit 01_one/00_phase.md +assert_rc 0 "one unit, host gate" +if grep -q '^A:.agents/validate.sh$' "$TRACE"; then _bad "host mode does not consult the launcher"; else _ok "host mode does not consult the launcher"; fi +assert_contains "outside the sandbox" "and the choice is announced, because it is the risky one" +T_VALIDATE=host ap_run --status +assert_contains "on the host (see --validate-in)" "and --status says where the gate would run" +assert_contains "gate will run on the HOST" "and says so loudly: the gate file is agent-writable" +T_VALIDATE=sideways ap_run --status +assert_rc 1 "and a gate location that is neither is refused" + +t_name "a completed phase is committed and pushed by the host" +PROJ1="$WORK/p1"; R1="$(mkproj "$PROJ1" 1)" +: > "$GTRACE"; cd "$PROJ1" || exit 1 +ap_run +assert_rc 0 "the pipeline ran to completion" +if grep -q 'report for 01_one__00_phase' "$PROJ1/.agents/reports/01_one/01_one__00_phase.a1.md" 2>/dev/null; then + _ok "the final pass ran last, and its report is on disk" +else + _bad "the final pass ran last, and its report is on disk" "$(ls "$PROJ1/.agents/reports/01_one" 2>/dev/null)" +fi +if grep -q '^push' "$GTRACE"; then _ok "a push went out for the phase"; else _bad "a push went out for the phase"; fi +assert_contains "up to date with origin/main" "and the end-of-run check confirmed the remote has it" +if [[ -d "$PROJ1/.agents/phases/complete/01_one" && ! -d "$PROJ1/.agents/phases/todo/01_one" ]]; then + _ok "the phase moved to complete/" +else + _bad "the phase moved to complete/" "$(ls "$PROJ1/.agents/phases"/* 2>/dev/null)" +fi +if ( cd "$PROJ1" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then + _ok "the host made the phase commit" +else + _bad "the host made the phase commit" "$( cd "$PROJ1" && "$REAL_GIT" log --oneline | head -3 )" +fi +assert_eq 2 "$( git --git-dir="$R1" rev-list --count --all )" "the remote has init + the phase" + +t_name "the driver owns delivery: push at the phase boundary, re-check, push again at the end" +PROJ2="$WORK/p2"; R2="$(mkproj "$PROJ2" 2)" +: > "$GTRACE"; cd "$PROJ2" || exit 1 +ap_run +assert_rc 0 "two phases ran" +commits="$(gt commit)"; pushes="$(gt push)" +[[ "$commits" == 2 ]] && _ok "both phases committed on the host" || _bad "both phases committed on the host" "commits=$commits" +[[ "$pushes" == "$commits" ]] && _ok "every commit is delivered, not left for a later run ($pushes)" \ + || _bad "every commit is delivered, not left for a later run" "commits=$commits pushes=$pushes" +assert_contains "phase 01_one" "the push is attributed to the phase that made it" +assert_contains "up to date with origin/main" "and the next phase's start re-checks before running it" +assert_eq 3 "$( git --git-dir="$R2" rev-list --count --all )" "the remote has init + both phases" + +# PHASE_PUSH is the library's switch; it means the same thing here, so it is honoured +# rather than overridden -- and honoured means nothing goes out, in either direction. +PROJ3="$WORK/p3"; R3="$(mkproj "$PROJ3" 2)" +: > "$GTRACE"; cd "$PROJ3" || exit 1 +T_PHASE_PUSH=0 ap_run +assert_rc 0 "the pipeline still ran" +if grep -q '^push' "$GTRACE"; then _bad "PHASE_PUSH=0 pushes nothing"; else _ok "PHASE_PUSH=0 pushes nothing"; fi +assert_contains "push disabled" "and says so" +assert_eq 1 "$( git --git-dir="$R3" rev-list --count --all )" "the remote is untouched" + +t_name "--no-push leaves the commits local, and says so" +PROJ4="$WORK/p4"; R4="$(mkproj "$PROJ4" 1)" +: > "$GTRACE"; cd "$PROJ4" || exit 1 +ap_run --no-push +assert_rc 0 "the pipeline ran" +if grep -q '^push' "$GTRACE"; then _bad "nothing is pushed" "$(grep '^push' "$GTRACE")"; else _ok "nothing is pushed"; fi +assert_contains "push disabled" "and that is stated, not silent" +if [[ "$(gt commit)" == 1 ]] && ( cd "$PROJ4" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then + _ok "the work is still committed locally" +else + _bad "the work is still committed locally" "$( cd "$PROJ4" && "$REAL_GIT" log --oneline | head -3 )" +fi +assert_eq 1 "$( git --git-dir="$R4" rev-list --count --all )" "the remote is untouched" + +t_name "a failed push stops the run, with the hand-fix command" +PROJ5="$WORK/p5"; mkproj "$PROJ5" 1 >/dev/null +: > "$GTRACE"; cd "$PROJ5" || exit 1 +T_GITFAIL="https://user:pushcanary@remote.invalid/repo.git" ap_run +assert_rc 1 "the run stops on a failed push" +assert_contains "the push FAILED" "says what failed" +assert_contains "finish by hand" "and how to finish it" +assert_absent "pushcanary" "a credential in the remote URL is not repeated in the output" +: > "$GTRACE" +ap_run # the remote answers this time +assert_contains "unpushed commit" "the next run notices what is still local" +assert_contains "pushed:" "and takes it out" +# and a run that could not deliver never reports success +PROJ5B="$WORK/p5b"; mkproj "$PROJ5B" 1 >/dev/null; cd "$PROJ5B" || exit 1 +T_GITFAIL="https://user:pushcanary@remote.invalid/repo.git" ap_run +assert_rc 1 "a run that could not deliver does not report success" +assert_contains "the push FAILED" "and says which part failed" +if ( cd "$PROJ5B" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then + _ok "its commit is still local and safe" +else + _bad "its commit is still local and safe" +fi +if ( cd "$PROJ5" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then + _ok "the phase stayed complete through it" +else + _bad "the phase stayed complete through it" +fi + +t_name "nothing a preview prints carries a value" +PROJ6="$WORK/p6"; mkproj "$PROJ6" 1 >/dev/null +cd "$PROJ6" || exit 1 +: > "$TRACE" +out_status="$(cd "$PROJ6" && env PATH="$WORK/bin:$PATH" AISB_AP_LIB="$LIB_FIX" \ + AISB_AP_RC_FILES="$WORK/rc/sandbox.bash" AISB_AP_PI_CMD=sandbox-pi \ + AISB_AP_TRACE="$TRACE" "$APBIN" --status 2>&1)" +out_dry="$(cd "$PROJ6" && env PATH="$WORK/bin:$PATH" AISB_AP_LIB="$LIB_FIX" \ + AISB_AP_RC_FILES="$WORK/rc/sandbox.bash" AISB_AP_PI_CMD=sandbox-pi \ + AISB_AP_TRACE="$TRACE" "$APBIN" --dry-run 2>&1)" +for v in "$out_status" "$out_dry"; do + if printf '%s' "$v" | grep -qF -- "$CANARY"; then + _bad "a preview does not echo the launcher's secret" + else + _ok "a preview does not echo the launcher's secret" + fi + if printf '%s' "$v" | grep -qF "export AI_API_TOKEN"; then + _bad "a preview does not print the launcher's body" + else + _ok "a preview does not print the launcher's body" + fi +done +if [[ -s "$TRACE" ]]; then _bad "a preview runs nothing"; else _ok "a preview runs nothing"; fi +assert_eq "$( git -C "$PROJ6" rev-list --count HEAD )" "$( git -C "$PROJ6" rev-list --count HEAD )" "a preview changes no git state" +if [[ -e "$PROJ6/.agents/reports" ]]; then _bad "a preview writes no reports"; else _ok "a preview writes no reports"; fi +if [[ -e "$PROJ6/.agents/phases/complete/01_one" ]]; then _bad "a preview does not move units"; else _ok "a preview does not move units"; fi +OUT="$out_status" +assert_contains "01_one/01_task.md" "but it does list what is pending" +assert_contains "01_one/00_phase.md" "task first, then the final pass" +OUT="$out_dry" +assert_contains "one container per unit" "and --dry-run says what it would do" + +t_name "preflight refuses to start a pipeline the sandbox cannot run" +cd "$PROJ6" || exit 1 +: > "$TRACE" +T_CMD=broken-sandbox T_RC="$WORK/rc/broken.bash" ap_run --max-units 1 +assert_rc 1 "preflight stops the run" +assert_contains "preflight failed" "and says so" +assert_contains "executable not found" "with the launcher's own error, for diagnosis" +if grep -q 'cmd=pi' "$TRACE"; then _bad "no unit runs"; else _ok "no unit runs"; fi +T_PREFLIGHT=0 ap_run --max-units 1 +assert_rc 0 "--no-preflight is available for the runs that cannot afford it" + +t_name "bad usage is refused before anything happens" +cd "$PROJ6" || exit 1 +ap_run --not-a-flag +assert_rc 2 "an unknown option" +ap_run --max-units lots +assert_rc 1 "--max-units wants a number" +cd / || exit 1 +ap_run --status +assert_rc 1 "outside a phased project" +assert_contains "no .agents/phases/todo" "and it says what is missing" +cd "$PROJ6" || exit 1 + +t_name "a library that does not match the contract is refused" +printf 'find_root() { printf "%%s\\n" "$PWD"; }\nnext_unit() { :; }\n' > "$WORK/rc/half-lib.sh" +T_LIB="$WORK/rc/half-lib.sh" ap_run --status +assert_rc 1 "a library missing pieces is not run against" +assert_contains "does not provide" "and the missing function is named" + +t_name "a hook planted from inside the sandbox never runs on the host" +# The commit is made out here, by us, after an agent had the tree writeable. A repository +# does not merely record .git/hooks and .git/config -- it runs them. So the delivery +# refuses when they changed, and even when it does not, our git never looks in .git/hooks. +HOOKPROOF="$WORK/hook-ran" +PROJ9="$WORK/p9"; R9="$(mkproj "$PROJ9" 1)" +cd "$PROJ9" || exit 1 +: > "$GTRACE" +T_PLANT="$HOOKPROOF" ap_run +assert_rc 1 "a run that rewrote the repository's own git config does not deliver" +assert_contains "refusing to deliver" "and says why" +assert_contains "--trust-git-changes" "and names the way through" +if [[ -e "$HOOKPROOF" ]]; then _bad "the planted pre-commit hook did not run"; else _ok "the planted pre-commit hook did not run"; fi +if grep -q '^push' "$GTRACE"; then _bad "nothing was pushed"; else _ok "nothing was pushed"; fi +assert_eq 1 "$( git --git-dir="$R9" rev-list --count --all )" "the remote still has only the init commit" +ap_run --trust-git-changes +assert_rc 0 "the named escape hatch does deliver" +assert_contains "pushed:" "and says so" +if [[ -e "$HOOKPROOF" ]]; then _bad "and still the hook did not run"; else _ok "and still the hook did not run: our git uses an empty hooks path"; fi +assert_eq 2 "$( git --git-dir="$R9" rev-list --count --all )" "the phase commit is on the remote now" +# and it is refused even when this script is not the one that would push: the point is +# that the operator hears about it, not that a push is blocked +PROJ9B="$WORK/p9b"; mkproj "$PROJ9B" 1 >/dev/null; cd "$PROJ9B" || exit 1 +T_PLANT="$HOOKPROOF" ap_run --no-push +assert_rc 1 "--no-push does not make it safe to ignore" +assert_contains "refusing to deliver" "and the same message explains it" +if [[ -e "$HOOKPROOF" ]]; then _bad "the hook did not run here either"; else _ok "the hook did not run here either"; fi + +t_name "what the driver composes is a real ai-sandbox invocation" +# The fixture launcher stands in for ai-sandbox everywhere else, because this suite runs +# without containers. Here the launcher IS ai-sandbox, in --dry-run: if the flags land on +# the wrong side of --, or --no-interactive were forwarded to the harness, or the project +# were not the mount, ai-sandbox itself says so. +cat > "$WORK/rc/drylaunch.bash" <> "\${AISB_AP_DRYOUT:-/dev/null}" 2>&1 +} +EOF +PROJ8="$WORK/p8"; mkproj "$PROJ8" 1 >/dev/null +cd "$PROJ8" || exit 1 +DRYOUT="$WORK/ai-sandbox.dryrun"; : > "$DRYOUT" +T_CMD=sandbox-dry T_RC="$WORK/rc/drylaunch.bash" T_PREFLIGHT=0 T_DRYOUT="$DRYOUT" \ + ap_run --max-units 1 +# the child produces no JSON (ai-sandbox printed an invocation instead), so the unit +# does not complete -- what is under test is the command that produced that output +if grep -q 'effective podman invocation' "$DRYOUT"; then + _ok "ai-sandbox accepted the composed command" +else + _bad "ai-sandbox accepted the composed command" "$(head -4 "$DRYOUT")" +fi +if grep -q 'egress: net=none' "$DRYOUT"; then + _ok "the launcher's own policy is still what is enforced" +else + _bad "the launcher's own policy is still what is enforced" "$(head -2 "$DRYOUT")" +fi +if grep -qF -- '--no-interactive' "$DRYOUT" || grep -qF -- '--no-tty' "$DRYOUT"; then + _bad "the run's sandbox flags are consumed by ai-sandbox, not passed on to pi" +else + _ok "the run's sandbox flags are consumed by ai-sandbox, not passed on to pi" +fi +if grep -qE -- ' pi (--no-approve|--session-dir)' "$DRYOUT"; then + _ok "pi is the in-container command, after --" +else + _bad "pi is the in-container command, after --" "$(tail -1 "$DRYOUT" | cut -c1-200)" +fi +if grep -qE -- '(^| )-i( |$)' "$DRYOUT" || grep -qE -- '(^| )-t( |$)' "$DRYOUT"; then + _bad "nothing is attached to the terminal: no -t in the JSON stream, no -i to steal keystrokes" +else + _ok "nothing is attached to the terminal: no -t in the JSON stream, no -i to steal keystrokes" +fi +# The project is mounted at its own host path where the sandbox can mirror it, and at +# /project where it cannot -- the driver composes neither, it just does not disturb it. +MOUNT_AT="$([[ -n "$(mirror_block_of "$PROJ8")" ]] && echo /project || echo "$PROJ8")" +if grep -qF -- "$PROJ8:$MOUNT_AT" "$DRYOUT"; then + _ok "the project directory is the mount the agent writes into" +else + _bad "the project directory is the mount the agent writes into" +fi +if grep -qF -- 'bash .agents/validate.sh' "$DRYOUT"; then + _ok "and so is the gate: validate.sh runs in the same sandbox, as the same user" +else + _bad "and so is the gate: validate.sh runs in the same sandbox, as the same user" "$(tail -2 "$DRYOUT" | cut -c1-200)" +fi + +# --------------------------------------------------------------------------- +# The real library, real renderer, real commit -- with the same fake sandbox. +t_name "the real phased-execution library, end to end" +REAL_LIB="$HOME/.agents/skills/phased-execution/scripts/lib.sh" +if [[ ! -f "$REAL_LIB" ]]; then + _ok "the skill is not installed here -- the real-library run is skipped" +else + PROJ7="$WORK/p7"; mkproj "$PROJ7" 1 >/dev/null + : > "$GTRACE"; cd "$PROJ7" || exit 1 + # shellcheck disable=SC2034 # assert_rc() in tests/lib.sh is what reads RC + OUT="$(env PATH="$WORK/bin:$PATH" AISB_AP_LIB="$REAL_LIB" \ + AISB_AP_RC_FILES="$WORK/rc/sandbox.bash" AISB_AP_PI_CMD=sandbox-pi \ + AISB_AP_VALIDATE=sandbox AISB_AP_TRACE="$TRACE" FAKE_GIT_TRACE="$GTRACE" \ + FAKE_PI_MARKER="$MARKER" REAL_GIT="$REAL_GIT" PHASE_NOTIFY=0 QUIET=1 \ + "$APBIN" 2>&1)" && RC=0 || RC=$? + assert_rc 0 "the real pipeline completed a phase inside the sandbox launcher" + if grep -rq 'report for 01_one__00_phase' "$PROJ7/.agents/reports/" 2>/dev/null; then + _ok "the real progress renderer read the child stream and wrote the report" + else + _bad "the real progress renderer read the child stream and wrote the report" "$(ls -R "$PROJ7/.agents/reports" 2>/dev/null | head)" + fi + if compgen -G "$PROJ7/.agents/reports/01_one/*.md" > /dev/null; then + _ok "the executor reports were written where the skill puts them" + else + _bad "the executor reports were written where the skill puts them" "$(ls -R "$PROJ7/.agents" 2>/dev/null | head -20)" + fi + if ( cd "$PROJ7" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then + _ok "the real commit_phase made the scoped commit" + else + _bad "the real commit_phase made the scoped commit" "$( cd "$PROJ7" && "$REAL_GIT" log --oneline | head -3 )" + fi + assert_contains "pushed:" "and the push went out" + if printf '%s' "$OUT" | grep -qF -- "$CANARY"; then _bad "no token in the real pipeline output"; else _ok "no token in the real pipeline output"; fi + if ( cd "$PROJ7" && "$REAL_GIT" ls-tree -r --name-only HEAD | grep -q '.agents/phases/complete/01_one/00_phase.md' ); then + _ok "the moved phase file is in the commit" + else + _bad "the moved phase file is in the commit" "$( cd "$PROJ7" && "$REAL_GIT" ls-tree -r --name-only HEAD | head -10 )" + fi + if grep -qF 'Sandbox rules for this run' "$PROMPTS"; then + _ok "and the real prompts carried the sandbox rules" + else + _bad "and the real prompts carried the sandbox rules" + fi +fi + +cd "$WORK" || exit 1