ducoterra
8ee535f6ae
The image ships pi with no provider and no preferences, so a sandboxed run
meant configuring the model twice. ai-sandbox now reads the host
~/.pi/agent/{models,settings}.json and rewrites them instead of copying them:
every provider apiKey becomes "$AI_API_TOKEN" (pi resolves an env reference
itself, so no value is written to disk), '!command' header values are dropped
rather than executed, settings that point into $HOME are dropped because that
path is not mounted, and auth.json / models-store.json / sessions / extensions
are never read.
The rewritten pair is seeded into /home/sandbox/.pi/agent at container start,
owned by uid 1000 and mode 600, and is treated as managed: a host change is
re-seeded, an edit made inside the sandbox is kept and reported, a deletion
takes the host version again.
The token follows the existing environment rules -- one variable, passed by
name through the 0600 env file, announced when auto-passed. A bridged run
whose token variable is unset or empty is refused instead of starting a pi
that cannot reach its model, and the bridge never widens the allowlist: it
warns with the exact --allow to add.
Docs and examples now use a self-hosted gateway (https://aipi.reeseapps.com)
instead of vendor API endpoints, since that is the only thing this tool is
used against. 391 assertions pass (make lint, ./tests/run.sh --e2e).
LLM-Generated: true
142 lines
9.6 KiB
Markdown
142 lines
9.6 KiB
Markdown
# PLAN
|
|
|
|
Execution plan for `ai-sandbox`, with the state each phase leaves behind.
|
|
Every phase ends with a gate that must pass before the next one starts.
|
|
Status as of this writing: **phases 1-9 complete**, follow-ups listed at the end.
|
|
|
|
| # | Phase | Gate | Status |
|
|
|---|-------|------|--------|
|
|
| 0 | Measure the host before designing | `tests/spike_network.sh` results recorded in `docs/networking.md` | ✅ done — killed the sidecar-on-internal-network design, found `pasta --map-host-loopback` + in-container nftables |
|
|
| 1 | Skeleton: CLI, config, preview modes | `make test` (args, config, precedence) | ✅ done |
|
|
| 2 | Image build | build, cache, content-hash tag, contents asserted | ✅ done (`tests/e2e/10_image.sh`) |
|
|
| 3 | Enforcement in the container | generated ruleset shape + real-container capability checks | ✅ done (`tests/unit/06_entrypoint.sh`, `tests/e2e/11_net.sh`) |
|
|
| 4 | Egress gate | allowlisted host reachable, everything else refused, gate lifecycle | ✅ done (`tests/e2e/11_net.sh`) |
|
|
| 5 | Environment passing | deny-by-default, `0600` delivery, redaction, fail-closed on unset | ✅ done (`tests/unit/04_env.sh`, `tests/e2e/12_sandbox.sh`) |
|
|
| 6 | Limits, lifecycle, observability | exit codes, cleanup, caps, session log, `--keep` | ✅ done (`tests/e2e/12_sandbox.sh`) |
|
|
| 7 | Harness defaults, docs, portability sweep | alias UX, README/threat model, shellcheck clean, no personal paths | ✅ done |
|
|
| 8 | Supply chain: pinned base, TTL refresh, audit trail | digest in `FROM` and in the content hash; `update`/`--check`; labels + in-image package manifest | ✅ done (`tests/unit/07_base.sh`, `tests/e2e/10_image.sh`) |
|
|
| 9 | Pi config bridge: same provider inside, credential by env | host `models.json`/`settings.json` rewritten not copied; canary secret nowhere in state; seeding, refresh and edit-presence asserted in a real container | ✅ done (`tests/unit/08_pi_config.sh`, `tests/e2e/12_sandbox.sh`) |
|
|
|
|
## Phase detail
|
|
|
|
### 0 — Measure first (this is why the design looks unusual)
|
|
Ran `tests/spike_network.sh`. Three results decided the architecture:
|
|
containers on one user-defined network cannot talk to each other rootless here
|
|
(so no sidecar proxy on an internal network); `--network=host` works rootless and
|
|
has internet (so a host-loopback gate is possible); `pasta:--map-host-loopback,<addr>`
|
|
makes a container's connection to `<addr>` land on the host's loopback; and
|
|
`--cap-add=NET_ADMIN` lets root *inside* the container install nftables that
|
|
genuinely gate the pasta path. Also learned bind mounts need `:z` under SELinux.
|
|
|
|
### 1 — Skeleton
|
|
`parse_args` with the alias-friendly rule (sandbox flags until the first
|
|
positional/unknown flag; those become arguments to the harness; `--` replaces the
|
|
command), config files at two levels, `AISB_*` overrides, `--dry-run` /
|
|
`--print-policy` / `--print-entrypoint` / `--print-containerfile`.
|
|
Precedence is built-in < config file < env < flag, and the built-in defaults live
|
|
only in `resolve_opts()` — materialising them at the top of the script silently
|
|
made the environment beat config files, which is a bug worth remembering.
|
|
|
|
### 2 — Image
|
|
Generated Containerfile, tag = content hash of base image + versions + install
|
|
list + entrypoint + Containerfile itself, so a policy change never reuses a stale
|
|
image and variants coexist. Node tarballs are checksum-verified against
|
|
nodejs.org. `pi` is baked in so the `pi` alias works after one build.
|
|
|
|
### 3 — Enforcement
|
|
`entrypoint_script()`: read policy → render `inet aisb` with `policy drop` →
|
|
`nft -f` → verify the default-drop rule exists → `setpriv` drop to uid 1000 with
|
|
`--inh-caps=-all --bounding-set=-all --no-new-privs`. Refuses to start (77/78)
|
|
rather than run unenforced.
|
|
|
|
### 4 — Gate
|
|
Per-session squid container on `--network=host`, bound to `127.0.0.1:<free port>`,
|
|
config generated and bind-mounted read-only, `http_access deny all` last.
|
|
Sandbox gets proxy env + a ruleset that allows nothing else out. Torn down with
|
|
the session unless `gate up` created a named one.
|
|
|
|
### 5-7 — Environment, lifecycle, polish
|
|
`0600` env file delivery and redaction everywhere; limits and caps asserted from
|
|
inside; `--keep` leaves the container *and* the session dir for forensics;
|
|
`clean` reaps (`--all` for running ones, `--volumes[=GLOB]` for unused home caches,
|
|
never a volume a container still references); `doctor` verifies the host including a
|
|
real bind/uid round-trip; docs; shellcheck-clean in a container.
|
|
|
|
`/home/sandbox` is a named volume so npm and tool caches survive between runs. It
|
|
outlives the image, which is the one kind of staleness this design cannot hash away:
|
|
change the image's home layout and existing volumes keep the old files, so
|
|
`ensure_home_volume()` creates it labelled and `clean --volumes` is the documented way
|
|
to drop it.
|
|
|
|
### 8 — Supply chain
|
|
The hash made *our* changes traceable but said nothing about the base: `FROM
|
|
ubuntu:24.04` is a moving pointer, `podman build`'s pull policy is *missing*, and
|
|
`ensure_image()` returns early when the tag exists — so even `build --rebuild` reused
|
|
the cached base. Now the tag is resolved to a digest (`skopeo`, falling back to
|
|
`podman pull` unless `AISB_BASE_PULL_FALLBACK=0`), recorded per-ref under
|
|
`$AISB_CACHE_DIR/base/`, and used in `FROM` *and* in `image_hash()`: a refreshed base
|
|
yields a new image rather than mutating one that may be running.
|
|
|
|
Two rules make it survivable rather than merely strict. **A stale pin beats a mutable
|
|
tag** — offline, or with `AISB_BASE_AUTO_UPDATE=0`, the recorded digest is used, and
|
|
only a build with no digest at all is allowed, loudly marked `UNPINNED` in the output,
|
|
in the image labels and in `doctor`. **Freshness is a TTL** (`AISB_BASE_TTL_DAYS=7`,
|
|
`0` every build, `-1` never) rather than a background job, so nothing surprises you
|
|
mid-session and `update` stays an explicit, dated event. Both images carry
|
|
`aisb.base-digest`/`aisb.node`/`aisb.pi-version` labels and
|
|
`/etc/aisb/manifest.txt` with resolved apt versions, which is how you answer "which
|
|
squid was enforcing this?" a month later. `--print-*` stays cache-only and
|
|
deterministic; `inspect` is the surface that asks the machine now.
|
|
|
|
One bug worth remembering: `POL[base_digest]="$(base_pin …)"` looked right and
|
|
silently produced `unpinned` labels, because the subshell's assignment was discarded.
|
|
Helpers return text; callers assign.
|
|
|
|
### 9 — Pi configuration bridge
|
|
The image ships `pi` with no provider and no preferences, so the first run asked you to
|
|
configure the model twice. It now reads `~/.pi/agent/{models,settings}.json` and
|
|
rewrites them: every provider `apiKey` becomes `"$AI_API_TOKEN"` (pi resolves an env
|
|
reference itself, so no value is ever written to disk), `!command` header values are
|
|
dropped rather than executed, settings pointing into `$HOME` are dropped because that
|
|
path is not mounted, and `auth.json`/`models-store.json`/sessions/extensions are never
|
|
read. The pair lands in `/home/sandbox/.pi/agent` at container start, owned by uid 1000,
|
|
mode 600, and is a managed file: a host change is re-seeded, a sandbox edit is kept and
|
|
reported, a deletion takes the host version again.
|
|
|
|
Two decisions worth keeping. The token follows the existing env rules — one variable,
|
|
passed by name through the `0600` file, announced when auto-passed — and a bridged run
|
|
where that variable is unset *or empty* is refused instead of starting a pi that cannot
|
|
reach its model, because "sandbox is broken" is a worse bug report than an explicit
|
|
refusal. And the bridge never widens the allowlist; it warns with the exact `--allow`
|
|
to add, since an agent silently unable to answer looks like a network failure.
|
|
|
|
## Follow-ups (not yet done)
|
|
|
|
* **`--net unix` mode**: `--network=none` plus a unix-socket proxy and an in-container
|
|
TCP→socket forwarder. Structurally stronger (no IP stack at all); needs a story
|
|
for tools that resolve names themselves.
|
|
* **Re-resolve `--no-gate` domains** on a timer, or resolve through the gate and pin
|
|
the returned answers, so long-running sessions do not go stale against CDNs.
|
|
* **Idle gate reaping** beyond `clean`: a `--gate-ttl` label check on start.
|
|
* **`--allow-from-lockfile`**: derive a first-cut allowlist from
|
|
`package.json`/`requirements.txt` registries so the first run is not a blank page.
|
|
* **Quarantine mode for untrusted repos**: `--net none` + read-only mount +
|
|
`--ephemeral-home` as a single flag, plus a warning if `.git` hooks exist.
|
|
* **`--record`**: optional pcap/CONNECT-log capture of a session for review.
|
|
* **`--mount` for agent skills/plugins**: first-class flag instead of a generic mount.
|
|
* Packaging: a `quadlet`/systemd unit for a long-lived named gate.
|
|
* Signature verification for the base where the registry signs it (cosign/sigstore),
|
|
so a pin is authenticated rather than merely immutable; and an optional local
|
|
mirror/`AISB_SKOPEO`-compatible offline source for air-gapped hosts.
|
|
* `update --auto` as a timer unit, if TTL-on-next-run turns out to be too lazy for
|
|
security response — deliberately not done yet, since a background rebuild that
|
|
surprises a running session is its own hazard.
|
|
* **Scoped sandbox tokens**: if the gateway can mint a short-lived, per-sandbox key,
|
|
`--pi-token-env` should ask for one rather than forwarding the long-lived host
|
|
credential. The bridge deliberately does not guess a minting endpoint.
|
|
* **Bridging extensions/skills on request**: `--pi-mount-ext` or similar for the
|
|
extension paths named in `enabledExtensionPaths`, which are host paths and therefore
|
|
currently dropped; needs the same explicit-grant shape as `--mount`.
|
|
* **`models.json` review surface**: `inspect` could print the exact diff between host
|
|
config and bridged config, so a change is reviewable before a run rather than after.
|