ducoterra
0242ab5b5e
A tag is a moving pointer, and three things conspired to make the base immutable-in-practice while mutable-in-theory: image_hash() hashed the string "ubuntu:24.04" rather than what it resolves to, ensure_image() returns early when the tag exists, and podman build's pull policy is "missing" -- so even 'build --rebuild' rebuilt on the cached base. A Canonical patch simply never arrived, and nothing said so. Now the tag is resolved to a registry digest (skopeo, falling back to a pull unless AISB_BASE_PULL_FALLBACK=0), recorded per-ref under $AISB_CACHE_DIR/base/ at 0600, and used in FROM *and* in image_hash(): refreshing the base produces a new image instead of mutating one a sandbox may be running. Freshness is a TTL (AISB_BASE_TTL_DAYS=7, 0 every build, -1 never) plus an explicit 'update' / 'update --check'. Two rules keep it usable rather than merely strict. A stale pin beats a mutable tag: offline or with AISB_BASE_AUTO_UPDATE=0 the recorded digest is still used, and only a build with no digest at all is allowed -- announced as UNPINNED, labelled that way, warned about by doctor. And preview modes stay deterministic: --print-* never reach the network, inspect is the surface that asks the machine now. Both images carry aisb.base-digest / .node / .pi-package / .pi-version labels plus /etc/aisb/manifest.txt of resolved apt versions, which answers "which squid was enforcing this?" after the build log is gone. Tags are named after the base instead of the hardcoded u24- prefix. Bug found on the way: POL[base_digest]="$(base_pin ...)" silently produced unpinned labels, because an assignment inside a command substitution is discarded with the subshell. Helpers return text; callers assign. Also fixed: the manifest step used jq on npm's dependency objects as if they were strings, and a missing Node checksum used to skip verification quietly -- it now warns. 311 assertions pass (unit + e2e), shellcheck clean. LLM-Generated: true
7.4 KiB
PLAN
Execution plan for ai-sandbox, with the state each phase leaves behind.
Every phase ends with a gate that must pass before the next one starts.
Status as of this writing: phases 1-8 complete, follow-ups listed at the end.
| # | Phase | Gate | Status |
|---|---|---|---|
| 0 | Measure the host before designing | tests/spike_network.sh results recorded in docs/networking.md |
✅ done — killed the sidecar-on-internal-network design, found pasta --map-host-loopback + in-container nftables |
| 1 | Skeleton: CLI, config, preview modes | make test (args, config, precedence) |
✅ done |
| 2 | Image build | build, cache, content-hash tag, contents asserted | ✅ done (tests/e2e/10_image.sh) |
| 3 | Enforcement in the container | generated ruleset shape + real-container capability checks | ✅ done (tests/unit/06_entrypoint.sh, tests/e2e/11_net.sh) |
| 4 | Egress gate | allowlisted host reachable, everything else refused, gate lifecycle | ✅ done (tests/e2e/11_net.sh) |
| 5 | Environment passing | deny-by-default, 0600 delivery, redaction, fail-closed on unset |
✅ done (tests/unit/04_env.sh, tests/e2e/12_sandbox.sh) |
| 6 | Limits, lifecycle, observability | exit codes, cleanup, caps, session log, --keep |
✅ done (tests/e2e/12_sandbox.sh) |
| 7 | Harness defaults, docs, portability sweep | alias UX, README/threat model, shellcheck clean, no personal paths | ✅ done |
| 8 | Supply chain: pinned base, TTL refresh, audit trail | digest in FROM and in the content hash; update/--check; labels + in-image package manifest |
✅ done (tests/unit/07_base.sh, tests/e2e/10_image.sh) |
Phase detail
0 — Measure first (this is why the design looks unusual)
Ran tests/spike_network.sh. Three results decided the architecture:
containers on one user-defined network cannot talk to each other rootless here
(so no sidecar proxy on an internal network); --network=host works rootless and
has internet (so a host-loopback gate is possible); pasta:--map-host-loopback,<addr>
makes a container's connection to <addr> land on the host's loopback; and
--cap-add=NET_ADMIN lets root inside the container install nftables that
genuinely gate the pasta path. Also learned bind mounts need :z under SELinux.
1 — Skeleton
parse_args with the alias-friendly rule (sandbox flags until the first
positional/unknown flag; those become arguments to the harness; -- replaces the
command), config files at two levels, AISB_* overrides, --dry-run /
--print-policy / --print-entrypoint / --print-containerfile.
Precedence is built-in < config file < env < flag, and the built-in defaults live
only in resolve_opts() — materialising them at the top of the script silently
made the environment beat config files, which is a bug worth remembering.
2 — Image
Generated Containerfile, tag = content hash of base image + versions + install
list + entrypoint + Containerfile itself, so a policy change never reuses a stale
image and variants coexist. Node tarballs are checksum-verified against
nodejs.org. pi is baked in so the pi alias works after one build.
3 — Enforcement
entrypoint_script(): read policy → render inet aisb with policy drop →
nft -f → verify the default-drop rule exists → setpriv drop to uid 1000 with
--inh-caps=-all --bounding-set=-all --no-new-privs. Refuses to start (77/78)
rather than run unenforced.
4 — Gate
Per-session squid container on --network=host, bound to 127.0.0.1:<free port>,
config generated and bind-mounted read-only, http_access deny all last.
Sandbox gets proxy env + a ruleset that allows nothing else out. Torn down with
the session unless gate up created a named one.
5-7 — Environment, lifecycle, polish
0600 env file delivery and redaction everywhere; limits and caps asserted from
inside; --keep leaves the container and the session dir for forensics;
clean reaps (--all for running ones, --volumes[=GLOB] for unused home caches,
never a volume a container still references); doctor verifies the host including a
real bind/uid round-trip; docs; shellcheck-clean in a container.
/home/sandbox is a named volume so npm and tool caches survive between runs. It
outlives the image, which is the one kind of staleness this design cannot hash away:
change the image's home layout and existing volumes keep the old files, so
ensure_home_volume() creates it labelled and clean --volumes is the documented way
to drop it.
8 — Supply chain
The hash made our changes traceable but said nothing about the base: FROM ubuntu:24.04 is a moving pointer, podman build's pull policy is missing, and
ensure_image() returns early when the tag exists — so even build --rebuild reused
the cached base. Now the tag is resolved to a digest (skopeo, falling back to
podman pull unless AISB_BASE_PULL_FALLBACK=0), recorded per-ref under
$AISB_CACHE_DIR/base/, and used in FROM and in image_hash(): a refreshed base
yields a new image rather than mutating one that may be running.
Two rules make it survivable rather than merely strict. A stale pin beats a mutable
tag — offline, or with AISB_BASE_AUTO_UPDATE=0, the recorded digest is used, and
only a build with no digest at all is allowed, loudly marked UNPINNED in the output,
in the image labels and in doctor. Freshness is a TTL (AISB_BASE_TTL_DAYS=7,
0 every build, -1 never) rather than a background job, so nothing surprises you
mid-session and update stays an explicit, dated event. Both images carry
aisb.base-digest/aisb.node/aisb.pi-version labels and
/etc/aisb/manifest.txt with resolved apt versions, which is how you answer "which
squid was enforcing this?" a month later. --print-* stays cache-only and
deterministic; inspect is the surface that asks the machine now.
One bug worth remembering: POL[base_digest]="$(base_pin …)" looked right and
silently produced unpinned labels, because the subshell's assignment was discarded.
Helpers return text; callers assign.
Follow-ups (not yet done)
--net unixmode:--network=noneplus a unix-socket proxy and an in-container TCP→socket forwarder. Structurally stronger (no IP stack at all); needs a story for tools that resolve names themselves.- Re-resolve
--no-gatedomains on a timer, or resolve through the gate and pin the returned answers, so long-running sessions do not go stale against CDNs. - Idle gate reaping beyond
clean: a--gate-ttllabel check on start. --allow-from-lockfile: derive a first-cut allowlist frompackage.json/requirements.txtregistries so the first run is not a blank page.- Quarantine mode for untrusted repos:
--net none+ read-only mount +--ephemeral-homeas a single flag, plus a warning if.githooks exist. --record: optional pcap/CONNECT-log capture of a session for review.--mountfor agent skills/plugins: first-class flag instead of a generic mount.- Packaging: a
quadlet/systemd unit for a long-lived named gate. - Signature verification for the base where the registry signs it (cosign/sigstore),
so a pin is authenticated rather than merely immutable; and an optional local
mirror/
AISB_SKOPEO-compatible offline source for air-gapped hosts. update --autoas a timer unit, if TTL-on-next-run turns out to be too lazy for security response — deliberately not done yet, since a background rebuild that surprises a running session is its own hazard.