add ai-sandbox: rootless podman sandbox for agentic development
A single bash script that runs a coding harness (pi by default) inside a rootless Ubuntu 24.04 container with only the current directory mounted at /project, an explicit egress allowlist and an explicit environment: nothing is passed in that was not named on the command line or in a config file. Network enforcement here is empirical rather than assumed (tests/spike_network.sh, docs/networking.md). The original design -- a sidecar proxy on an internal network -- does not work rootless on this host, so allow mode runs pasta with --map-host-loopback, installs an inet aisb ruleset inside the container that drops every egress path except the gate's, and relays through a squid gate bound to the host's loopback that resolves names itself and refuses anything unlisted. The entrypoint checks that the default-drop rule really exists before setpriv drops to uid 1000 with an empty capability set, and exits 77/78 rather than run unenforced. The Containerfile, the entrypoint, the squid config generator and the ruleset renderer all live in bin/ai-sandbox, and the image is tagged by content hash, so a policy change can never silently reuse a stale image. Tests are pure bash: 161 assertions start no container and touch no network, 256 run real ones, including proof that a raw socket, a stripped proxy env and DNS all stay blocked while an allowlisted host works over both http and CONNECT. LLM-Generated: true
This commit is contained in:
@@ -0,0 +1,20 @@
|
|||||||
|
root = true
|
||||||
|
|
||||||
|
[*]
|
||||||
|
charset = utf-8
|
||||||
|
end_of_line = lf
|
||||||
|
insert_final_newline = true
|
||||||
|
trim_trailing_whitespace = true
|
||||||
|
indent_style = space
|
||||||
|
indent_size = 2
|
||||||
|
|
||||||
|
[*.sh]
|
||||||
|
indent_style = space
|
||||||
|
indent_size = 2
|
||||||
|
shell_variant = bash
|
||||||
|
|
||||||
|
[*.{md,yml,yaml}]
|
||||||
|
indent_size = 2
|
||||||
|
|
||||||
|
[Makefile]
|
||||||
|
indent_style = tab
|
||||||
+17
@@ -0,0 +1,17 @@
|
|||||||
|
# secrets and local policy overrides: never commit these
|
||||||
|
env
|
||||||
|
.env
|
||||||
|
*.env
|
||||||
|
config/env
|
||||||
|
.ai-sandbox/local
|
||||||
|
|
||||||
|
# test and build scratch
|
||||||
|
/tmp/
|
||||||
|
*.log
|
||||||
|
/tests/.work/
|
||||||
|
|
||||||
|
# editors
|
||||||
|
.vscode/
|
||||||
|
.idea/
|
||||||
|
*.swp
|
||||||
|
.DS_Store
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
# AGENTS.md
|
||||||
|
|
||||||
|
Working notes for anyone (human or agent) changing this repository.
|
||||||
|
|
||||||
|
## What this project is
|
||||||
|
|
||||||
|
One bash script, `bin/ai-sandbox`, that runs another program — usually an LLM
|
||||||
|
coding harness — inside a rootless podman container with a strict, explicit policy:
|
||||||
|
only the current directory mounted, no inherited environment, no root or
|
||||||
|
capabilities inside the container, resource caps, and egress limited to an
|
||||||
|
allowlist that the sandboxed process cannot lift.
|
||||||
|
|
||||||
|
## Non-negotiables
|
||||||
|
|
||||||
|
1. **Single script.** The Containerfile, the container entrypoint, the squid
|
||||||
|
config and the nftables rules are all generated from `bin/ai-sandbox`
|
||||||
|
(`containerfile()`, `entrypoint_script()`, `write_squid_conf()`, `gen_rules()`
|
||||||
|
inside the entrypoint). Keep it that way; do not split policy into files that
|
||||||
|
can drift from the script.
|
||||||
|
2. **Fail closed.** Any path where the sandbox cannot guarantee its policy must
|
||||||
|
refuse to start rather than degrade: no allowlist in `allow` mode, missing
|
||||||
|
policy file in the entrypoint (exit 78), ruleset that cannot be installed
|
||||||
|
(exit 77), `--env NAME` where NAME is unset. Escape hatches exist but must be
|
||||||
|
explicitly named (`--allow-unenforced`, `--insecure-keep-root`, `--net full`).
|
||||||
|
3. **Never leak a secret.** Values go through a `0600` env file, never argv.
|
||||||
|
Nothing that prints (`--dry-run`, `--print-policy`, the session log, error
|
||||||
|
messages) may echo a value. If you add output, add a test that a canary value
|
||||||
|
does not appear in it.
|
||||||
|
4. **No hardcoded machine paths, usernames or ports.** Everything is overridable
|
||||||
|
via `AISB_*` or a config file; defaults must work for any user on any host.
|
||||||
|
5. **Measure, don't assume, about podman.** Rootless networking varies by version
|
||||||
|
and distro. Run `tests/spike_network.sh` on a new host and update
|
||||||
|
`docs/networking.md` when reality disagrees with the design.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
bin/ai-sandbox the application
|
||||||
|
install.sh symlink + optional alias
|
||||||
|
config/*.example documented examples (never loaded directly)
|
||||||
|
docs/networking.md the measurements behind the enforcement model
|
||||||
|
docs/threat-model.md what is and is not protected -- keep it honest
|
||||||
|
docs/recipes.md per-harness and per-language setups
|
||||||
|
tests/run.sh runner: unit (no containers) | --e2e (real containers)
|
||||||
|
tests/spike_network.sh re-measure the host
|
||||||
|
```
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make test # unit tests -- fast, no containers, no network
|
||||||
|
make e2e # plus real container/egress tests (needs network + the image)
|
||||||
|
make lint # shellcheck in a container
|
||||||
|
make style # bash -n on every shell file
|
||||||
|
make check # lint + unit
|
||||||
|
./bin/ai-sandbox --dry-run --allow example.com --print-policy # introspection
|
||||||
|
```
|
||||||
|
|
||||||
|
Before finishing a change: `make check` green, and if you touched enforcement,
|
||||||
|
networking or the image, `make e2e` green too.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
* bash ≥ 4.4, `set -Eeuo pipefail` in the main script; `set -uo pipefail` in the
|
||||||
|
test runner (assertions report, they do not abort).
|
||||||
|
* Option flow is always: defaults → config files → `AISB_*` env → CLI flags.
|
||||||
|
The built-in defaults live in `resolve_opts()`; the top of the script only sets
|
||||||
|
variables that have no config-file counterpart. Breaking this is the single
|
||||||
|
easiest way to introduce a precedence bug.
|
||||||
|
* `POL[...]` holds resolved policy, `P_*` arrays resolved lists, `OPT_*` raw flags,
|
||||||
|
`CFG_*` config-file values. Do not read raw env vars deep in the code.
|
||||||
|
* Preview paths (`--dry-run`, `--print-policy`, `--print-entrypoint`,
|
||||||
|
`--print-containerfile`) must not build images, start containers or write state.
|
||||||
|
* Tests use `t_name` + `assert_*` from `tests/lib.sh`; each file gets its own
|
||||||
|
`AISB_STATE_DIR`/`AISB_CACHE_DIR`/`AISB_CONFIG_DIR`. Never let a test touch the
|
||||||
|
real `~/.config`, `~/.local/state` or `~/.cache`, and never override `HOME`
|
||||||
|
(podman storage lives there).
|
||||||
|
|
||||||
|
## Areas that deserve the most care
|
||||||
|
|
||||||
|
* `entrypoint_script()` — the enforcement half. It is rendered into the image and
|
||||||
|
runs as root there. A bug here is a security bug. `tests/unit/06_entrypoint.sh`
|
||||||
|
checks the generated rule shape; `tests/e2e/11_net.sh` checks it in a container.
|
||||||
|
* `split_target()` / `classify_allows()` — the mapping from what the user typed to
|
||||||
|
what the firewall and the proxy enforce.
|
||||||
|
* `build_argv()` — every isolation flag lives here.
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
# ai-sandbox -- development tasks. Everything runs with the tools already on this
|
||||||
|
# machine plus podman; nothing needs to be installed system-wide.
|
||||||
|
SHELL := /bin/bash
|
||||||
|
BIN := bin/ai-sandbox
|
||||||
|
SHELLCHECK_IMAGE := docker.io/koalaman/shellcheck:stable
|
||||||
|
|
||||||
|
.PHONY: help test e2e check lint style doctor build clean-demo all
|
||||||
|
help: ## show this help
|
||||||
|
@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk -F':.*?## ' '{printf " %-12s %s\n", $$1, $$2}'
|
||||||
|
|
||||||
|
test: ## unit tests only (no containers, no network)
|
||||||
|
./tests/run.sh
|
||||||
|
|
||||||
|
e2e: ## everything, including real containers and real egress tests
|
||||||
|
./tests/run.sh --e2e
|
||||||
|
|
||||||
|
check: lint test ## lint + unit tests (what CI would run)
|
||||||
|
|
||||||
|
lint: ## shellcheck every shell file, in a container (nothing installed locally)
|
||||||
|
@podman run --rm -v "$(CURDIR):/src:ro,z" -w /src $(SHELLCHECK_IMAGE) \
|
||||||
|
--severity=warning --shell=bash --exclude=SC1091 \
|
||||||
|
bin/ai-sandbox install.sh tests/run.sh tests/lib.sh tests/spike_network.sh \
|
||||||
|
$$(ls tests/unit/*.sh tests/e2e/*.sh) \
|
||||||
|
&& echo "shellcheck: clean"
|
||||||
|
|
||||||
|
style: ## bash-style syntax check of every shell file (fast, no deps)
|
||||||
|
@for f in $(BIN) install.sh tests/*.sh tests/unit/*.sh tests/e2e/*.sh; do \
|
||||||
|
[ -f "$$f" ] || continue; bash -n "$$f" || exit 1; \
|
||||||
|
done; echo "bash -n: clean"
|
||||||
|
|
||||||
|
doctor: ## verify this machine can run the sandbox
|
||||||
|
@$(BIN) doctor
|
||||||
|
|
||||||
|
build: ## build the sandbox and gate images
|
||||||
|
@$(BIN) build --gate
|
||||||
|
|
||||||
|
clean-demo: ## remove sandbox containers, gates and stale session files
|
||||||
|
@$(BIN) clean --all
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
# PLAN
|
||||||
|
|
||||||
|
Execution plan for `ai-sandbox`, with the state each phase leaves behind.
|
||||||
|
Every phase ends with a gate that must pass before the next one starts.
|
||||||
|
Status as of this writing: **phases 1-7 complete**, follow-ups listed at the end.
|
||||||
|
|
||||||
|
| # | Phase | Gate | Status |
|
||||||
|
|---|-------|------|--------|
|
||||||
|
| 0 | Measure the host before designing | `tests/spike_network.sh` results recorded in `docs/networking.md` | ✅ done — killed the sidecar-on-internal-network design, found `pasta --map-host-loopback` + in-container nftables |
|
||||||
|
| 1 | Skeleton: CLI, config, preview modes | `make test` (args, config, precedence) | ✅ done |
|
||||||
|
| 2 | Image build | build, cache, content-hash tag, contents asserted | ✅ done (`tests/e2e/10_image.sh`) |
|
||||||
|
| 3 | Enforcement in the container | generated ruleset shape + real-container capability checks | ✅ done (`tests/unit/06_entrypoint.sh`, `tests/e2e/11_net.sh`) |
|
||||||
|
| 4 | Egress gate | allowlisted host reachable, everything else refused, gate lifecycle | ✅ done (`tests/e2e/11_net.sh`) |
|
||||||
|
| 5 | Environment passing | deny-by-default, `0600` delivery, redaction, fail-closed on unset | ✅ done (`tests/unit/04_env.sh`, `tests/e2e/12_sandbox.sh`) |
|
||||||
|
| 6 | Limits, lifecycle, observability | exit codes, cleanup, caps, session log, `--keep` | ✅ done (`tests/e2e/12_sandbox.sh`) |
|
||||||
|
| 7 | Harness defaults, docs, portability sweep | alias UX, README/threat model, shellcheck clean, no personal paths | ✅ done |
|
||||||
|
|
||||||
|
## Phase detail
|
||||||
|
|
||||||
|
### 0 — Measure first (this is why the design looks unusual)
|
||||||
|
Ran `tests/spike_network.sh`. Three results decided the architecture:
|
||||||
|
containers on one user-defined network cannot talk to each other rootless here
|
||||||
|
(so no sidecar proxy on an internal network); `--network=host` works rootless and
|
||||||
|
has internet (so a host-loopback gate is possible); `pasta:--map-host-loopback,<addr>`
|
||||||
|
makes a container's connection to `<addr>` land on the host's loopback; and
|
||||||
|
`--cap-add=NET_ADMIN` lets root *inside* the container install nftables that
|
||||||
|
genuinely gate the pasta path. Also learned bind mounts need `:z` under SELinux.
|
||||||
|
|
||||||
|
### 1 — Skeleton
|
||||||
|
`parse_args` with the alias-friendly rule (sandbox flags until the first
|
||||||
|
positional/unknown flag; those become arguments to the harness; `--` replaces the
|
||||||
|
command), config files at two levels, `AISB_*` overrides, `--dry-run` /
|
||||||
|
`--print-policy` / `--print-entrypoint` / `--print-containerfile`.
|
||||||
|
Precedence is built-in < config file < env < flag, and the built-in defaults live
|
||||||
|
only in `resolve_opts()` — materialising them at the top of the script silently
|
||||||
|
made the environment beat config files, which is a bug worth remembering.
|
||||||
|
|
||||||
|
### 2 — Image
|
||||||
|
Generated Containerfile, tag = content hash of base image + versions + install
|
||||||
|
list + entrypoint + Containerfile itself, so a policy change never reuses a stale
|
||||||
|
image and variants coexist. Node tarballs are checksum-verified against
|
||||||
|
nodejs.org. `pi` is baked in so the `pi` alias works after one build.
|
||||||
|
|
||||||
|
### 3 — Enforcement
|
||||||
|
`entrypoint_script()`: read policy → render `inet aisb` with `policy drop` →
|
||||||
|
`nft -f` → verify the default-drop rule exists → `setpriv` drop to uid 1000 with
|
||||||
|
`--inh-caps=-all --bounding-set=-all --no-new-privs`. Refuses to start (77/78)
|
||||||
|
rather than run unenforced.
|
||||||
|
|
||||||
|
### 4 — Gate
|
||||||
|
Per-session squid container on `--network=host`, bound to `127.0.0.1:<free port>`,
|
||||||
|
config generated and bind-mounted read-only, `http_access deny all` last.
|
||||||
|
Sandbox gets proxy env + a ruleset that allows nothing else out. Torn down with
|
||||||
|
the session unless `gate up` created a named one.
|
||||||
|
|
||||||
|
### 5-7 — Environment, lifecycle, polish
|
||||||
|
`0600` env file delivery and redaction everywhere; limits and caps asserted from
|
||||||
|
inside; `--keep` leaves the container *and* the session dir for forensics;
|
||||||
|
`clean` reaps (`--all` for running ones, `--volumes[=GLOB]` for unused home caches,
|
||||||
|
never a volume a container still references); `doctor` verifies the host including a
|
||||||
|
real bind/uid round-trip; docs; shellcheck-clean in a container.
|
||||||
|
|
||||||
|
`/home/sandbox` is a named volume so npm and tool caches survive between runs. It
|
||||||
|
outlives the image, which is the one kind of staleness this design cannot hash away:
|
||||||
|
change the image's home layout and existing volumes keep the old files, so
|
||||||
|
`ensure_home_volume()` creates it labelled and `clean --volumes` is the documented way
|
||||||
|
to drop it.
|
||||||
|
|
||||||
|
## Follow-ups (not yet done)
|
||||||
|
|
||||||
|
* **`--net unix` mode**: `--network=none` plus a unix-socket proxy and an in-container
|
||||||
|
TCP→socket forwarder. Structurally stronger (no IP stack at all); needs a story
|
||||||
|
for tools that resolve names themselves.
|
||||||
|
* **Re-resolve `--no-gate` domains** on a timer, or resolve through the gate and pin
|
||||||
|
the returned answers, so long-running sessions do not go stale against CDNs.
|
||||||
|
* **Idle gate reaping** beyond `clean`: a `--gate-ttl` label check on start.
|
||||||
|
* **`--allow-from-lockfile`**: derive a first-cut allowlist from
|
||||||
|
`package.json`/`requirements.txt` registries so the first run is not a blank page.
|
||||||
|
* **Quarantine mode for untrusted repos**: `--net none` + read-only mount +
|
||||||
|
`--ephemeral-home` as a single flag, plus a warning if `.git` hooks exist.
|
||||||
|
* **`--record`**: optional pcap/CONNECT-log capture of a session for review.
|
||||||
|
* **`--mount` for agent skills/plugins**: first-class flag instead of a generic mount.
|
||||||
|
* Packaging: a `quadlet`/systemd unit for a long-lived named gate.
|
||||||
@@ -0,0 +1,182 @@
|
|||||||
|
# ai-sandbox
|
||||||
|
|
||||||
|
Run an autonomous coding agent against your real project, with the escape routes
|
||||||
|
closed: only the current directory is mounted, no ambient environment variables,
|
||||||
|
no root inside, hard resource caps, and outbound network limited to an explicit
|
||||||
|
allowlist that the agent cannot lift.
|
||||||
|
|
||||||
|
Single bash script, rootless podman, Ubuntu 24.04. No daemon, no k8s, no config
|
||||||
|
language.
|
||||||
|
|
||||||
|
```
|
||||||
|
$ cd ~/Projects/billing-api
|
||||||
|
$ pi --allow api.anthropic.com --env ANTHROPIC_API_KEY
|
||||||
|
[aisb] egress gate aisb-gate-7f3c1a -> 127.0.0.1:18084 (1 domains, 0 ip/cidr entries)
|
||||||
|
[aisb] egress net=allow enforce=nft proxy=[100.64.0.1:18084] direct=0 dns=0
|
||||||
|
pi
|
||||||
|
```
|
||||||
|
|
||||||
|
`pi` there is an alias for `ai-sandbox`. Inside, the agent sees `/project` (your
|
||||||
|
cwd, read-write), nothing else on your disk, and the internet only where you
|
||||||
|
pointed it.
|
||||||
|
|
||||||
|
## Install
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone <this repo> && cd ai-sandbox
|
||||||
|
./install.sh --alias pi # symlinks to ~/.local/bin and adds the alias to ~/.bashrc
|
||||||
|
ai-sandbox doctor # checks podman, rootless, subuid, pasta, SELinux, uid mapping
|
||||||
|
ai-sandbox build # optional: build the image now instead of on first run
|
||||||
|
make test # 160+ assertions, no containers needed
|
||||||
|
```
|
||||||
|
|
||||||
|
Requirements: podman ≥ 5 running rootless (with a `/etc/subuid` range for your
|
||||||
|
user), and either `pasta` (default in podman 5) or `slirp4netns`. Nothing else —
|
||||||
|
shellcheck, tests and the image build all run through podman.
|
||||||
|
|
||||||
|
## The alias, and how arguments are read
|
||||||
|
|
||||||
|
Sandbox flags are consumed until the first positional argument or unknown flag;
|
||||||
|
those become **arguments to the harness**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
alias pi='ai-sandbox'
|
||||||
|
|
||||||
|
pi # runs pi inside the sandbox
|
||||||
|
pi --allow api.anthropic.com --env ANTHROPIC_API_KEY --model sonnet
|
||||||
|
# ^^^^^ sandbox ^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^ for pi
|
||||||
|
pi --shell # a bash shell in the same sandbox
|
||||||
|
pi --net none -- make test # -- replaces the command entirely
|
||||||
|
```
|
||||||
|
|
||||||
|
`--strict-args` turns an unrecognised flag into an error instead of a pass-through,
|
||||||
|
which is what you want in scripts.
|
||||||
|
|
||||||
|
## Network policy
|
||||||
|
|
||||||
|
| mode | what it means |
|
||||||
|
|---|---|
|
||||||
|
| `--net allow` *(default)* | Only the allowlist is reachable. A netfilter ruleset inside the container plus a domain-filtering proxy on the host loopback. If the ruleset cannot be installed, the sandbox **refuses to start** (exit 77). |
|
||||||
|
| `--net none` | No network at all. |
|
||||||
|
| `--net full` | Nothing restricted. Prints a warning. Never the default. |
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ai-sandbox --allow api.anthropic.com --allow registry.npmjs.org --env ANTHROPIC_API_KEY
|
||||||
|
ai-sandbox --allow 10.20.0.5:5432 # a database: pinned IP and port
|
||||||
|
ai-sandbox --allow files.pythonhosted.org --allow pypi.org
|
||||||
|
ai-sandbox --no-gate --allow example.com # no proxy container: pin resolved IPs instead
|
||||||
|
ai-sandbox --proxy corp-proxy.internal:3128 # route through a proxy you already run
|
||||||
|
```
|
||||||
|
|
||||||
|
Domains match that name and anything below it, and are enforced by the gate at
|
||||||
|
CONNECT time, so CDNs and rotating IPs are not a problem. IP/CIDR entries become
|
||||||
|
direct netfilter rules for non-HTTP destinations (databases, etc.).
|
||||||
|
|
||||||
|
An allowlist is **required** in `allow` mode: an empty list is a hard error, never
|
||||||
|
a silent free-for-all. Keep it in the repo so the policy is reviewable:
|
||||||
|
|
||||||
|
```
|
||||||
|
# .ai-sandbox/allow (one entry per line, # comments fine)
|
||||||
|
api.anthropic.com
|
||||||
|
registry.npmjs.org
|
||||||
|
```
|
||||||
|
|
||||||
|
## Environment
|
||||||
|
|
||||||
|
Nothing is inherited except `TERM LANG LC_ALL LC_CTYPE TZ`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
--env ANTHROPIC_API_KEY # from your environment
|
||||||
|
--env FOO=bar # literal (shows up in ps; prefer the two below)
|
||||||
|
--env-file ~/.config/secrets # a 0600 NAME=VALUE file
|
||||||
|
```
|
||||||
|
|
||||||
|
Values are handed to podman through a `0600` file that is removed when the run
|
||||||
|
ends, so they never appear in `ps`, in the printed dry-run, or in the session log.
|
||||||
|
Asking for a variable that is not set fails the run closed.
|
||||||
|
|
||||||
|
## Filesystem
|
||||||
|
|
||||||
|
Your cwd is mounted at `/project` and is the working directory — that is the whole
|
||||||
|
default. Everything else is opt-in:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
--mount ~/.ssh:ro # if the agent must be able to push
|
||||||
|
--mount ../docs:/docs:ro # read-only reference material
|
||||||
|
--no-cwd # a scratch sandbox with nothing mounted
|
||||||
|
--read-only # read-only root filesystem
|
||||||
|
--ephemeral-home # do not persist /home/sandbox (default: persist in a volume)
|
||||||
|
```
|
||||||
|
|
||||||
|
Files the agent writes land on your disk owned by **your** uid, not root
|
||||||
|
(`--userns=keep-id`, verified by `doctor`).
|
||||||
|
|
||||||
|
## Image
|
||||||
|
|
||||||
|
Built once per distinct policy from `ubuntu:24.04`, tagged by content hash, reused
|
||||||
|
afterwards: git, curl, jq, build tools, python3, Node 22 (checksum-verified
|
||||||
|
against nodejs.org) and the `pi` CLI. `--install PKG` adds packages and produces a
|
||||||
|
different tag, so variants coexist. `--base-image` swaps the base.
|
||||||
|
|
||||||
|
## Inspecting a run
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ai-sandbox --dry-run --allow example.com --env KEY # exact podman invocation, secrets redacted
|
||||||
|
ai-sandbox --print-policy # effective policy as key=value lines
|
||||||
|
ai-sandbox --print-entrypoint # the enforcement script baked into the image
|
||||||
|
ai-sandbox doctor [--deep] # host checks; --deep tests egress for real
|
||||||
|
ai-sandbox ps | clean [--all] [--volumes[=GLOB]] | inspect | gate up|status|logs|down
|
||||||
|
```
|
||||||
|
|
||||||
|
State lives under `~/.local/state/ai-sandbox` (session dirs, `log` of every run
|
||||||
|
with no secret values, build logs). Config in `~/.config/ai-sandbox`.
|
||||||
|
|
||||||
|
`/home/sandbox` is a named volume (`aisb-home`) so npm and tool caches survive
|
||||||
|
between runs. It outlives the image, so after changing the image you may be looking
|
||||||
|
at a stale home: `ai-sandbox clean --volumes` reclaims unused ones. It only deletes
|
||||||
|
volumes in the `aisb-` namespace that no container references, and `--volumes=GLOB`
|
||||||
|
narrows it to one name.
|
||||||
|
|
||||||
|
## Why it is hard to undo from inside
|
||||||
|
|
||||||
|
The container starts as root with `CAP_NET_ADMIN`, the entrypoint installs the
|
||||||
|
egress ruleset, and then it execs your command through
|
||||||
|
|
||||||
|
```
|
||||||
|
setpriv --reuid=1000 --regid=1000 --clear-groups --inh-caps=-all --bounding-set=-all --no-new-privs
|
||||||
|
```
|
||||||
|
|
||||||
|
No root, no capabilities, `no-new-privileges`. Flushing the ruleset needs
|
||||||
|
`CAP_NET_ADMIN`; `unshare -n` needs `CAP_SYS_ADMIN`; setuid helpers cannot hand
|
||||||
|
either back. Both are asserted against a real container in
|
||||||
|
[`tests/e2e/11_net.sh`](tests/e2e/11_net.sh). Details and the measurements behind
|
||||||
|
the design: [docs/networking.md](docs/networking.md).
|
||||||
|
|
||||||
|
**Read [docs/threat-model.md](docs/threat-model.md)** — it says plainly what this
|
||||||
|
does not protect you from (allowlisted destinations, container escapes, anything
|
||||||
|
you explicitly mounted).
|
||||||
|
|
||||||
|
## Project layout
|
||||||
|
|
||||||
|
```
|
||||||
|
bin/ai-sandbox the application: one script, ~1.3k lines
|
||||||
|
install.sh symlink into ~/.local/bin + optional alias
|
||||||
|
config/ example config, allowlist and env file
|
||||||
|
docs/ networking.md, threat-model.md, recipes.md
|
||||||
|
tests/run.sh runner; tests/unit/* (no containers) tests/e2e/* (real ones)
|
||||||
|
tests/spike_network.sh re-measure podman's behaviour on a new host
|
||||||
|
Makefile make test | e2e | lint | check | doctor | build
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make test # unit: args, config, argv construction, env handling, rule generation
|
||||||
|
make e2e # + real containers: build cache, egress blocking, capability drop, lifecycle
|
||||||
|
make lint # shellcheck in a container
|
||||||
|
make style # bash -n
|
||||||
|
```
|
||||||
|
|
||||||
|
The Containerfile, the container entrypoint and the squid config are all generated
|
||||||
|
from `bin/ai-sandbox`, so the script remains the single source of truth
|
||||||
|
(`--print-containerfile`, `--print-entrypoint`).
|
||||||
Executable
+1459
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,22 @@
|
|||||||
|
# Egress allowlist -- copy to ~/.config/ai-sandbox/allow (user-wide) or to
|
||||||
|
# ./.ai-sandbox/allow in a repository (checked in, so the policy is reviewable).
|
||||||
|
#
|
||||||
|
# One entry per line. '#' starts a comment. Blank lines are ignored.
|
||||||
|
# In --net allow mode at least one entry is required; an empty list is an error.
|
||||||
|
#
|
||||||
|
# domain matched by the gate, including subdomains, at CONNECT time
|
||||||
|
# domain:port same, and a direct netfilter rule for that port
|
||||||
|
# 10.0.0.5:5432 pinned host and port (databases, internal services)
|
||||||
|
# 10.20.0.0/24 pinned subnet (default ports 80,443 unless :port given)
|
||||||
|
# [fd00::5]:8443 pinned ipv6
|
||||||
|
|
||||||
|
api.anthropic.com
|
||||||
|
registry.npmjs.org
|
||||||
|
|
||||||
|
# typical additions, by kind:
|
||||||
|
# api.openai.com
|
||||||
|
# github.com # git push, if you also mount ~/.ssh
|
||||||
|
# pypi.org
|
||||||
|
# files.pythonhosted.org
|
||||||
|
# index.crates.io
|
||||||
|
# proxy.golang.org
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
# config for ai-sandbox -- copy to ~/.config/ai-sandbox/config and edit
|
||||||
|
#
|
||||||
|
# Format: key = value. '#' starts a comment.
|
||||||
|
# Precedence: built-in default < this file (user) < ./.ai-sandbox/config (repo)
|
||||||
|
# < AISB_* environment variable < command line flag.
|
||||||
|
#
|
||||||
|
# Keys listed under "repeatable" accumulate; every other key replaces.
|
||||||
|
|
||||||
|
# --- network ---------------------------------------------------------------
|
||||||
|
# none | allow | full. allow is the safe default and requires an allowlist.
|
||||||
|
net = allow
|
||||||
|
|
||||||
|
# Address that stands for the host's loopback inside the container
|
||||||
|
# (--network=pasta:--map-host-loopback,<addr>). Change only if it collides
|
||||||
|
# with a real network you need to reach.
|
||||||
|
# gate_addr = 100.64.0.1
|
||||||
|
|
||||||
|
# Preferred first port for the per-session egress gate; the next free port is
|
||||||
|
# used if it is busy.
|
||||||
|
# gate_port = 18081
|
||||||
|
|
||||||
|
# --- harness ---------------------------------------------------------------
|
||||||
|
# Default in-container command: pi | codex | claude | none | "<your command>"
|
||||||
|
# harness = pi
|
||||||
|
|
||||||
|
# --- mounts and layout -----------------------------------------------------
|
||||||
|
# workdir = /project
|
||||||
|
|
||||||
|
# --- limits ----------------------------------------------------------------
|
||||||
|
# cpus = 4
|
||||||
|
# memory = 4g
|
||||||
|
# pids = 512
|
||||||
|
# tmpfs_size = 1g
|
||||||
|
|
||||||
|
# --- image -----------------------------------------------------------------
|
||||||
|
# base_image = docker.io/library/ubuntu:24.04
|
||||||
|
# node_version = 22.23.4
|
||||||
|
# pi_version = 1.4.0
|
||||||
|
|
||||||
|
# --- behaviour -------------------------------------------------------------
|
||||||
|
# SELinux bind-mount option: ':z' (shared), ':Z' (private) or empty to disable.
|
||||||
|
# Defaults to ':z' when getenforce exists.
|
||||||
|
# selinux = :z
|
||||||
|
|
||||||
|
# Persist /home/sandbox in a named volume so harness logins and tool caches
|
||||||
|
# survive between runs. 0 = use a throwaway tmpfs instead.
|
||||||
|
# home_cache = 1
|
||||||
|
|
||||||
|
# Appended verbatim to the podman invocation (power users; word-split).
|
||||||
|
# extra_args = --annotation io.k8s.description="agent sandbox"
|
||||||
|
|
||||||
|
# --- repeatable keys -------------------------------------------------------
|
||||||
|
# These may appear more than once, and are merged with --allow / --env / etc.
|
||||||
|
#
|
||||||
|
# allow = api.anthropic.com
|
||||||
|
# allow = registry.npmjs.org:443
|
||||||
|
# allow = 10.20.0.5:5432
|
||||||
|
#
|
||||||
|
# env = ANTHROPIC_API_KEY
|
||||||
|
# env = CI_JOB_TOKEN
|
||||||
|
#
|
||||||
|
# mount = ~/.ssh:ro
|
||||||
|
# mount = ../docs:/docs:ro
|
||||||
|
#
|
||||||
|
# install = postgresql-client
|
||||||
|
#
|
||||||
|
# label = team=platform
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
# Copy to a file only you can read, then: ai-sandbox --env-file ~/.config/ai-sandbox/env
|
||||||
|
#
|
||||||
|
# chmod 600 ~/.config/ai-sandbox/env
|
||||||
|
#
|
||||||
|
# One NAME=VALUE per line. '#' comments and blank lines are ignored.
|
||||||
|
# Nothing here is passed to the container unless you ask for it with --env or
|
||||||
|
# --env-file, so listing a variable is not the same as granting it.
|
||||||
|
#
|
||||||
|
# ANTHROPIC_API_KEY=sk-ant-...
|
||||||
|
# OPENAI_API_KEY=sk-...
|
||||||
|
# ANTHROPIC_MODEL=sonnet
|
||||||
@@ -0,0 +1,107 @@
|
|||||||
|
# Networking: how `--net allow` actually holds
|
||||||
|
|
||||||
|
Everything in this file was measured on the reference host (Fedora 44, podman
|
||||||
|
5.8.9, rootless, `pasta` as the rootless network helper, SELinux enforcing) with
|
||||||
|
[`tests/spike_network.sh`](../tests/spike_network.sh). Re-run that script on a new
|
||||||
|
machine before trusting the model; podman's rootless networking is the part of the
|
||||||
|
stack most likely to differ.
|
||||||
|
|
||||||
|
## The two mechanisms
|
||||||
|
|
||||||
|
`--net allow` stacks two independent controls:
|
||||||
|
|
||||||
|
1. **A netfilter ruleset inside the container's own network namespace.**
|
||||||
|
The container starts with `--cap-add=NET_ADMIN` and the image's entrypoint runs
|
||||||
|
as root *only long enough* to install an `inet aisb` table with
|
||||||
|
`policy drop` on the output hook, then it execs the real command through
|
||||||
|
|
||||||
|
```
|
||||||
|
setpriv --reuid=1000 --regid=1000 --clear-groups --inh-caps=-all --bounding-set=-all --no-new-privs
|
||||||
|
```
|
||||||
|
|
||||||
|
After that exec the command has no capabilities and no root. Removing a rule
|
||||||
|
needs `CAP_NET_ADMIN`, opening a new network namespace needs `CAP_SYS_ADMIN`,
|
||||||
|
and a setuid helper can't hand either back because `no-new-privileges` is set on
|
||||||
|
the container. The ruleset is therefore a one-way door.
|
||||||
|
|
||||||
|
2. **An egress gate on the host loopback.** With `pasta`, a container can be given
|
||||||
|
a magic address that *means* the host's loopback:
|
||||||
|
|
||||||
|
```
|
||||||
|
--network=pasta:--map-host-loopback,100.64.0.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Connecting to `100.64.0.1:PORT` inside the container lands on the host's
|
||||||
|
`127.0.0.1:PORT`. The gate is a squid container started with `--network=host`
|
||||||
|
(so it has internet and the host's resolver) bound to `127.0.0.1:<free port>`,
|
||||||
|
with `http_access deny all` after the allowlist. The sandbox gets
|
||||||
|
`HTTP(S)_PROXY` pointing at `100.64.0.1:<that port>`, and the netfilter ruleset
|
||||||
|
allows nothing else out.
|
||||||
|
|
||||||
|
The gate itself runs as the image's unprivileged `proxy` user (`--user proxy`,
|
||||||
|
`--cap-drop=ALL`, `pid_filename /tmp/...`): squid normally starts as root and drops
|
||||||
|
privileges itself, which needs SETUID/SETGID we would rather not grant. Binding a
|
||||||
|
high loopback port does not need them. squid 6 treats obsolete directives as hard
|
||||||
|
errors, so the generated config avoids `dns_v4_first` and `max_mem_obj_size` and does
|
||||||
|
not redeclare the built-in `manager` ACL.
|
||||||
|
|
||||||
|
Together: the harness can only talk to the gate, and the gate only relays to
|
||||||
|
domains you allowlisted, resolved by the gate itself. CDNs, SNI and CONNECT-time
|
||||||
|
filtering all work because squid terminates the CONNECT request and resolves the
|
||||||
|
name itself.
|
||||||
|
|
||||||
|
## Measurements
|
||||||
|
|
||||||
|
| Probe | Result | Consequence |
|
||||||
|
|---|---|---|
|
||||||
|
| `--network=<internal>` container, egress | blocked (no default route) | a blackout network is available |
|
||||||
|
| `--network=<internal>` container, DNS | blocked | internal mode is a real blackout |
|
||||||
|
| container → container on the same user-defined network | **does not work** (`Failed to connect ... Could not connect to server`) | rules out the "sandbox + sidecar proxy on a shared internal network" design |
|
||||||
|
| `--network=host` in a rootless container | works, has internet | makes the host-loopback gate possible |
|
||||||
|
| `--network=pasta` (default), egress | works | pasta is a plain internet tap, no filtering |
|
||||||
|
| container → host `127.0.0.1` via `--map-host-loopback,100.64.0.1` | **works** | the gate address |
|
||||||
|
| `--map-host-loopback,127.0.0.1` | rejected by passt (`Invalid address to remap to host`) | the magic address must not itself be loopback |
|
||||||
|
| `--network=pasta:--map-gw` | works (the gateway address already maps to the host by default) | `--map-host-loopback` chosen for explicitness |
|
||||||
|
| squid as root with `--cap-drop=ALL` | `setgid: (1) Operation not permitted`, gate dies | run the gate as `proxy` from the start |
|
||||||
|
| squid 6 with `dns_v4_first` / `max_mem_obj_size` | `ERROR: Directive ... is obsolete`, config refused | generate only current directives |
|
||||||
|
| sandbox in allow mode, `curl` with the proxy env removed | connection times out | netfilter, not just `HTTP_PROXY`, is doing the gating |
|
||||||
|
| sandbox in allow mode, raw TCP socket to a literal IP, DNS to 8.8.8.8 | both blocked | nothing bypasses the ruleset by ignoring proxy variables |
|
||||||
|
| root in a rootless container, `nft add table` with podman's default caps | **fails** (`Operation not permitted`) | NET_ADMIN is not in podman's default cap set, so it must be added deliberately |
|
||||||
|
| root + `--cap-add=NET_ADMIN`, drop `tcp dport 80` inside the container | egress to port 80 **times out** | in-container netfilter really does gate the pasta path |
|
||||||
|
| `--cap-drop=ALL`, `nft add table` | fails | a non-privileged process cannot install or remove rules |
|
||||||
|
| `--pids-limit 64 --memory 256m --cpus 0.5` | `pids.max=64`, `memory.max=268435456` visible inside | limits are real (note: `/proc/cpuinfo` still lists all cores; use `nproc`/cgroup limits, not cpuinfo) |
|
||||||
|
| `-v $dir:/project` with SELinux enforcing, `--userns=keep-id:uid=1000,gid=1000` | write **denied** | bind mounts need `:z` on SELinux hosts |
|
||||||
|
| same, with `:z` | write succeeds, file owned by the host uid (1000:1000) | project files keep sane ownership |
|
||||||
|
|
||||||
|
## Why not the other designs
|
||||||
|
|
||||||
|
* **Internal network + sidecar proxy container.** The obvious design, and it is
|
||||||
|
what most write-ups suggest. On this host, two containers on the same
|
||||||
|
user-defined network cannot reach each other at all, so the sandbox could never
|
||||||
|
reach its own proxy. Rejected on measurement, not taste.
|
||||||
|
* **`--network=none` plus a unix-socket proxy.** Structurally the strongest (no IP
|
||||||
|
stack at all) and worth revisiting, but only `curl` and a handful of tools speak
|
||||||
|
`--unix-socket`; everything else (git, npm, pip, node's fetch) would need a
|
||||||
|
TCP-to-socket forwarder inside the container, and DNS would need its own path.
|
||||||
|
The current design gets the same practical containment with the two controls above.
|
||||||
|
* **DNS filtering only.** A resolver allowlist is trivially bypassed by connecting
|
||||||
|
to a literal IP, and it breaks the SNI-based allowlist that real agent traffic
|
||||||
|
needs. Used only as the fallback in `--no-gate` mode, where it is paired with
|
||||||
|
IP-pinned netfilter rules.
|
||||||
|
* **`--network=slirp4netns`.** Not installed here; `pasta` is the default helper in
|
||||||
|
podman 5 and is what the code targets (it falls back to whatever helper podman
|
||||||
|
picks, since the code only ever asks for the `pasta` driver).
|
||||||
|
|
||||||
|
## Failure modes we chose
|
||||||
|
|
||||||
|
* If the ruleset cannot be installed (no NET_ADMIN, no `nft`, weird kernel) the
|
||||||
|
container **refuses to start** with exit 77 rather than running unfiltered.
|
||||||
|
`--allow-unenforced` exists for debugging and says so in the flag name.
|
||||||
|
* If the policy file is missing the entrypoint exits 78 before running anything.
|
||||||
|
* The gate container is created per session, bound to loopback only, and removed
|
||||||
|
when the sandboxed command exits. A gate that outlives its session would be an
|
||||||
|
open door with nobody watching it. `gate up` is the explicit, named exception.
|
||||||
|
* In gate mode DNS is **closed** inside the container. That is deliberate: it stops
|
||||||
|
a sandboxed process from resolving names and then talking to them behind the
|
||||||
|
proxy's back. Tools that insist on resolving names themselves need
|
||||||
|
`--allow <ip/cidr>` entries instead.
|
||||||
+146
@@ -0,0 +1,146 @@
|
|||||||
|
# Recipes
|
||||||
|
|
||||||
|
Concrete invocations. `pi` below is whatever you aliased to `ai-sandbox`.
|
||||||
|
|
||||||
|
## A first session
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/Projects/myapp
|
||||||
|
ai-sandbox doctor # is this machine ready?
|
||||||
|
pi --allow api.anthropic.com --env ANTHROPIC_API_KEY
|
||||||
|
```
|
||||||
|
|
||||||
|
Same thing, but remembered by the repo (so teammates review the policy, not just
|
||||||
|
your shell history):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p .ai-sandbox
|
||||||
|
printf 'api.anthropic.com\n' > .ai-sandbox/allow
|
||||||
|
pi --env ANTHROPIC_API_KEY
|
||||||
|
```
|
||||||
|
|
||||||
|
## Per harness
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# pi (baked into the image)
|
||||||
|
pi --allow api.anthropic.com --env ANTHROPIC_API_KEY
|
||||||
|
|
||||||
|
# pi with a proxy/harness you pass explicitly
|
||||||
|
pi --harness 'pi --model sonnet' --allow api.anthropic.com --env ANTHROPIC_API_KEY
|
||||||
|
|
||||||
|
# OpenAI Codex CLI (install it into the image with --install or use a custom base)
|
||||||
|
pi --allow chatgpt.com --allow api.openai.com --env OPENAI_API_KEY
|
||||||
|
|
||||||
|
# Claude Code
|
||||||
|
pi --allow api.anthropic.com --env ANTHROPIC_API_KEY --env CLAUDE_CODE_OAUTH_TOKEN
|
||||||
|
|
||||||
|
# just a shell in the sandbox
|
||||||
|
pi --shell
|
||||||
|
|
||||||
|
# any other build/test command, no network needed
|
||||||
|
pi --net none -- make test
|
||||||
|
```
|
||||||
|
|
||||||
|
## Per ecosystem
|
||||||
|
|
||||||
|
Only the endpoints the tool actually needs. Everything else stays blocked, so a
|
||||||
|
typosquatted package from an unexpected host fails loudly instead of quietly
|
||||||
|
working.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# npm / node
|
||||||
|
pi --allow registry.npmjs.org --allow nodejs.org --allow github.com
|
||||||
|
|
||||||
|
# python: pypi needs both of these (the CDN that serves the files is separate)
|
||||||
|
pi --allow pypi.org --allow files.pythonhosted.org
|
||||||
|
|
||||||
|
# rust
|
||||||
|
pi --allow crates.io --allow static.crates.io --allow index.crates.io
|
||||||
|
|
||||||
|
# go
|
||||||
|
pi --allow proxy.golang.org --allow sum.golang.org
|
||||||
|
|
||||||
|
# docker/podman images from within a build
|
||||||
|
pi --allow registry-1.docker.io --allow auth.docker.io --allow production.cloudflare.docker.com
|
||||||
|
|
||||||
|
# a project that also needs its dev database (non-HTTP: pinned IP + port)
|
||||||
|
pi --allow api.anthropic.com --allow 10.20.0.5:5432 --env ANTHROPIC_API_KEY
|
||||||
|
```
|
||||||
|
|
||||||
|
## Git
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# fetch/push over https with a token in the environment
|
||||||
|
pi --allow github.com --env GH_TOKEN -- git push origin main
|
||||||
|
|
||||||
|
# ssh: the key has to be mounted deliberately (an explicit, reviewable grant)
|
||||||
|
pi --allow github.com --mount ~/.ssh:ro -- git push origin main
|
||||||
|
|
||||||
|
# read-only view of a sibling repo for context
|
||||||
|
pi --allow api.anthropic.com --env ANTHROPIC_API_KEY --mount ../docs:/docs:ro
|
||||||
|
```
|
||||||
|
|
||||||
|
## Tightening a run
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# nothing persistent, nothing readable, nothing writable outside the repo
|
||||||
|
pi --net none --ephemeral-home --read-only -- make build
|
||||||
|
|
||||||
|
# an entirely blank sandbox for untrusted code
|
||||||
|
pi --net none --no-cwd --ephemeral-home --read-only -- bash
|
||||||
|
|
||||||
|
# CI-ish: refuse to start if the policy or environment is not exactly right
|
||||||
|
pi --strict-args --net allow --allow-file .ai-sandbox/allow --env-file .ci/env -- npm ci
|
||||||
|
```
|
||||||
|
|
||||||
|
## Long-lived gate shared by several sandboxes
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ai-sandbox gate up --allow api.anthropic.com --allow registry.npmjs.org
|
||||||
|
# name=aisb-gate
|
||||||
|
# port=18081
|
||||||
|
# proxy_env=export HTTPS_PROXY=http://100.64.0.1:18081
|
||||||
|
|
||||||
|
ai-sandbox --proxy 100.64.0.1:18081 --allow api.anthropic.com --shell
|
||||||
|
ai-sandbox gate logs # what did it allow/deny?
|
||||||
|
ai-sandbox gate down
|
||||||
|
```
|
||||||
|
|
||||||
|
## Debugging
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ai-sandbox --dry-run --allow example.com --env KEY # the exact podman invocation
|
||||||
|
ai-sandbox --print-policy # effective policy, no secrets
|
||||||
|
ai-sandbox --print-entrypoint | less # the enforcement script
|
||||||
|
ai-sandbox --keep --net none -- false # leave the container for forensics
|
||||||
|
ai-sandbox ps
|
||||||
|
AISB_GATE_KEEP=1 pi --allow example.com --shell # keep the gate after exit
|
||||||
|
ai-sandbox clean --all
|
||||||
|
|
||||||
|
# "the ruleset would not install" (exit 77): check the host
|
||||||
|
podman info | grep -i rootless
|
||||||
|
grep "$(id -un)" /etc/subuid
|
||||||
|
```
|
||||||
|
|
||||||
|
## Making it the default for a project
|
||||||
|
|
||||||
|
`.ai-sandbox/config`:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
net = allow
|
||||||
|
harness = pi
|
||||||
|
cpus = 8
|
||||||
|
memory = 8g
|
||||||
|
mount = ../specs:/specs:ro
|
||||||
|
env = ANTHROPIC_API_KEY
|
||||||
|
```
|
||||||
|
|
||||||
|
`.ai-sandbox/allow`:
|
||||||
|
|
||||||
|
```
|
||||||
|
api.anthropic.com
|
||||||
|
registry.npmjs.org
|
||||||
|
```
|
||||||
|
|
||||||
|
Then `pi` alone in that directory is the whole invocation — and because both files
|
||||||
|
are in the repository, the sandbox policy gets reviewed with the code.
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# Threat model
|
||||||
|
|
||||||
|
Read this before relying on `ai-sandbox`. It is a **seatbelt, not a jail**: it
|
||||||
|
raises the cost of an agent's mistakes and of a moderately compromised prompt
|
||||||
|
context, on a machine you still log into as yourself.
|
||||||
|
|
||||||
|
## Who we assume is hostile
|
||||||
|
|
||||||
|
The modelled adversary is **the agent you are running**: an LLM harness with your
|
||||||
|
write access to one directory, capable of running arbitrary shell commands, and
|
||||||
|
vulnerable to prompt injection from the files it reads, the web pages it fetches,
|
||||||
|
and the package changelogs it installs. We assume it will try to escape when a
|
||||||
|
prompt injection tells it to.
|
||||||
|
|
||||||
|
We do **not** model a kernel-level adversary, a hostile container image from a
|
||||||
|
registry you don't control, or someone with access to your host user account.
|
||||||
|
|
||||||
|
## What stops working (defence in depth, most to least load-bearing)
|
||||||
|
|
||||||
|
| Attack | Stopped by | Evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| Editing / deleting anything outside the project directory | only `$PWD` is mounted; the rest of the host is not in the mount namespace | `tests/e2e/12_sandbox.sh` |
|
||||||
|
| Reading `~/.ssh`, `~/.aws`, `~/.config/gcloud`, browser profiles | same as above; those paths do not exist in the container | `tests/e2e/12_sandbox.sh` |
|
||||||
|
| Exfiltrating your environment (API keys for *other* services) | nothing is passed through unless named with `--env` | `tests/unit/04_env.sh`, `tests/e2e/12_sandbox.sh` |
|
||||||
|
| Passing secrets through the process list to other users | values travel in a `0600` file, never in argv | `tests/e2e/12_sandbox.sh` |
|
||||||
|
| Egress to an arbitrary host (curl, python, a raw socket, a bespoke exfil tool) | netfilter `policy drop` inside the container's namespace | `tests/e2e/11_net.sh` |
|
||||||
|
| Egress through the proxy to a non-allowlisted domain, including HTTPS | squid's `http_access deny all` after the allowlist | `tests/e2e/11_net.sh` |
|
||||||
|
| Deleting or weakening the egress ruleset | the command runs with no capabilities; `CAP_NET_ADMIN` was dropped across `exec` | `tests/e2e/11_net.sh` |
|
||||||
|
| `unshare -n` to get a fresh namespace with its own routes | requires `CAP_SYS_ADMIN`, which the command does not have | `tests/e2e/11_net.sh` |
|
||||||
|
| Privilege escalation via a setuid binary in the image | `--security-opt no-new-privileges` + `--cap-drop=ALL` | container config, asserted in `tests/e2e/10_image.sh` |
|
||||||
|
| Reaching other host services on loopback | only the gate's port is routable through the mapped loopback address | `tests/e2e/11_net.sh` |
|
||||||
|
| Fork bombs, memory bombs, runaway builds | `--pids-limit`, `--memory`, `--cpus`, `--tmpfs` sizes | `tests/e2e/10_image.sh` |
|
||||||
|
| Writing to the host's package manager, `/etc`, global npm/pip | read-only-ish image, `HOME` inside the container, optional `--read-only` | `tests/e2e/12_sandbox.sh` |
|
||||||
|
| Credential persistence you did not ask for | `--ephemeral-home`; the default home volume is named and inspectable | `tests/e2e/12_sandbox.sh` |
|
||||||
|
| Silent wide-open starts | an empty allowlist in `allow` mode is a hard error | `tests/unit/01_args.sh` |
|
||||||
|
|
||||||
|
## What does **not** stop it
|
||||||
|
|
||||||
|
Be honest about these, and design your work accordingly.
|
||||||
|
|
||||||
|
1. **Anything you allowlist is reachable, in both directions.** Allow
|
||||||
|
`api.github.com` and a prompt-injected agent can push your code to a repository
|
||||||
|
it controls, using the token you deliberately handed it. The allowlist limits
|
||||||
|
*where*, not *what*. Prefer least-privilege, read-only, short-lived tokens.
|
||||||
|
2. **The project directory is genuinely writable.** A bad `rm -rf`, a force push, a
|
||||||
|
corrupted lockfile all still happen — just confined to that directory. Keep the
|
||||||
|
tree committed (`ai-sandbox doctor` warns you when it is dirty).
|
||||||
|
3. **Kernel and runtime escapes.** A container is not a VM. A container escape
|
||||||
|
vulnerability in runc/crun or the kernel gets an attacker to your user account.
|
||||||
|
For genuinely untrusted code, use a VM or a disposable machine.
|
||||||
|
4. **`--net full` and the escape hatches do what they say.** `--allow-unenforced`,
|
||||||
|
`--insecure-keep-root`, `--mount`, `AISB_EXTRA_ARGS`, `--net full` all weaken the
|
||||||
|
posture. They are named to be uncomfortable on purpose; nothing uses them
|
||||||
|
implicitly.
|
||||||
|
5. **Side channels.** Timing, resource consumption, and "the agent typed X into a
|
||||||
|
form on a site you allowed" are out of scope.
|
||||||
|
6. **A cached home volume can hold credentials.** If a harness logs in inside the
|
||||||
|
sandbox, the token lives in the `aisb-home` volume, and any later sandbox run
|
||||||
|
using that volume can read it. `--ephemeral-home` per run, or a dedicated
|
||||||
|
`AISB_HOME_VOLUME` per trust level.
|
||||||
|
7. **DNS is closed in gate mode, open in `--no-gate` mode.** With `--no-gate`, name
|
||||||
|
lookups leave the container; that is a modest data channel and a fingerprint of
|
||||||
|
what you are doing. Egress to non-allowlisted IPs stays blocked either way.
|
||||||
|
8. **The gate sees everything that flows through it** — including any credentials
|
||||||
|
in proxied requests. That is the point of an egress proxy; the squid config logs
|
||||||
|
to its own container log, and `gate logs` shows them. Do not put secrets in URLs.
|
||||||
|
9. **Supply chain inside the allowlist.** `registry.npmjs.org` allowlisted means any
|
||||||
|
package, including a typosquatted one, can be fetched. Allowlisting a domain is
|
||||||
|
not a security review of what you install from it.
|
||||||
|
10. **Rootless podman still means shared kernel and shared UID namespace.** Files
|
||||||
|
the container writes are owned by *you*. There is no uid boundary between
|
||||||
|
"agent's file" and "your file" beyond ownership metadata and your own discipline.
|
||||||
|
|
||||||
|
## Residual risk rating
|
||||||
|
|
||||||
|
*Accidental destruction and ambient credential leakage: largely eliminated.*
|
||||||
|
*Deliberate data exfiltration over an allowlisted path: mitigated, not prevented.*
|
||||||
|
*Container escape to the host: out of scope, accept it or use a VM.*
|
||||||
Executable
+64
@@ -0,0 +1,64 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# install.sh -- put ai-sandbox on PATH and wire up an alias.
|
||||||
|
#
|
||||||
|
# ./install.sh install to ~/.local/bin, print the alias line
|
||||||
|
# ./install.sh --alias pi also append the alias to ~/.bashrc
|
||||||
|
# ./install.sh --prefix DIR install somewhere else
|
||||||
|
# ./install.sh --uninstall remove what this script installed
|
||||||
|
set -Eeuo pipefail
|
||||||
|
|
||||||
|
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
PREFIX="${HOME}/.local/bin"
|
||||||
|
ALIAS_NAME=""
|
||||||
|
UNINSTALL=0
|
||||||
|
|
||||||
|
while (($#)); do
|
||||||
|
case "$1" in
|
||||||
|
--prefix) PREFIX="${2:?--prefix needs a directory}"; shift 2 ;;
|
||||||
|
--prefix=*) PREFIX="${1#*=}"; shift ;;
|
||||||
|
--alias) ALIAS_NAME="${2:?--alias needs a name, e.g. --alias pi}"; shift 2 ;;
|
||||||
|
--alias=*) ALIAS_NAME="${1#*=}"; shift ;;
|
||||||
|
--uninstall) UNINSTALL=1; shift ;;
|
||||||
|
-h|--help) sed -n '2,10p' "$0"; exit 0 ;;
|
||||||
|
*) echo "unknown option: $1" >&2; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
target="$PREFIX/ai-sandbox"
|
||||||
|
rcfile="${HOME}/.bashrc"
|
||||||
|
|
||||||
|
if (( UNINSTALL )); then
|
||||||
|
[[ -L "$target" || -f "$target" ]] && { rm -f "$target"; echo "removed $target"; } || echo "nothing at $target"
|
||||||
|
if [[ -n "$ALIAS_NAME" ]]; then
|
||||||
|
tmp="$(mktemp)"
|
||||||
|
grep -v "^alias ${ALIAS_NAME}=" "$rcfile" > "$tmp" 2>/dev/null || true
|
||||||
|
mv "$tmp" "$rcfile"; echo "removed the ${ALIAS_NAME} alias from $rcfile"
|
||||||
|
fi
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
[[ -x "$HERE/bin/ai-sandbox" ]] || { echo "bin/ai-sandbox is missing or not executable" >&2; exit 1; }
|
||||||
|
command -v podman > /dev/null || echo "warning: podman is not installed; ai-sandbox needs it"
|
||||||
|
|
||||||
|
mkdir -p "$PREFIX"
|
||||||
|
ln -sfn "$HERE/bin/ai-sandbox" "$target"
|
||||||
|
echo "installed $target -> $HERE/bin/ai-sandbox"
|
||||||
|
|
||||||
|
if [[ -n "$ALIAS_NAME" ]]; then
|
||||||
|
line="alias ${ALIAS_NAME}='${target}'"
|
||||||
|
if grep -qxF "$line" "$rcfile" 2>/dev/null; then
|
||||||
|
echo "alias already present in $rcfile"
|
||||||
|
else
|
||||||
|
printf '\n# added by ai-sandbox install.sh\n%s\n' "$line" >> "$rcfile"
|
||||||
|
echo "added to $rcfile: $line"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo
|
||||||
|
echo "To use it as a harness wrapper, add this to $rcfile:"
|
||||||
|
echo " alias pi='$target'"
|
||||||
|
echo "then: source $rcfile && cd your/project && pi --allow api.anthropic.com --env ANTHROPIC_API_KEY"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "next: $target doctor # sanity-check this machine"
|
||||||
|
echo " $target build # build the sandbox image now (otherwise built on first run)"
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/e2e/10_image.sh -- the image itself: build, cache, contents, fail-closed entrypoint.
|
||||||
|
source "$TESTS/lib.sh"
|
||||||
|
|
||||||
|
PROJ="$WORK/proj"; mkdir -p "$PROJ"; cd "$PROJ" || exit 1
|
||||||
|
|
||||||
|
t_name "build"
|
||||||
|
sb build
|
||||||
|
assert_rc 0 "ai-sandbox build succeeds"
|
||||||
|
IMG="$("$AISB" --print-policy --net none 2>/dev/null | grep '^image=' | cut -d= -f2)"
|
||||||
|
[[ -n "$IMG" ]] && _ok "image ref resolved: $IMG" || _bad "image ref resolved"
|
||||||
|
podman image exists "$IMG" && _ok "image exists in podman storage" || _bad "image exists"
|
||||||
|
|
||||||
|
t_name "build is cached and tagged by content"
|
||||||
|
START=$SECONDS
|
||||||
|
sb build
|
||||||
|
assert_rc 0 "rebuild succeeds"
|
||||||
|
(( SECONDS - START < 15 )) && _ok "second build reused the cache ($((SECONDS - START))s)" \
|
||||||
|
|| _bad "second build reused the cache ($((SECONDS - START))s)"
|
||||||
|
IMG2="$("$AISB" --print-policy --net none 2>/dev/null | grep '^image=' | cut -d= -f2)"
|
||||||
|
assert_eq "$IMG" "$IMG2" "same inputs, same tag"
|
||||||
|
sb --print-policy --net none --install hexd > /dev/null
|
||||||
|
IMG3="$("$AISB" --print-policy --net none --install hexd | grep '^image=' | cut -d= -f2)"
|
||||||
|
[[ "$IMG3" != "$IMG" ]] && _ok "extra packages produce a different tag" || _bad "extra packages change the tag"
|
||||||
|
|
||||||
|
t_name "the entrypoint fails closed without a policy"
|
||||||
|
podman run --rm --network none --cap-drop=ALL "$IMG" true > "$WORK/fc.out" 2>&1
|
||||||
|
assert_eq 78 "$?" "no /run/aisb policy -> exit 78, nothing ran"
|
||||||
|
grep -q 'policy file' "$WORK/fc.out" && _ok "it says why" || _bad "it says why: $(cat "$WORK/fc.out")"
|
||||||
|
|
||||||
|
t_name "image contents"
|
||||||
|
POL="$WORK/pol"; mkdir -p "$POL"
|
||||||
|
printf 'AISB_NET=none\nAISB_UID=1000\nAISB_GID=1000\nAISB_ALLOW_ICMP=1\nAISB_ALLOW_DNS=0\n' > "$POL/policy.env"
|
||||||
|
: > "$POL/direct4.list"; : > "$POL/direct6.list"
|
||||||
|
# podman takes mount options as a comma list: ":ro,:z" not ":ro:z"
|
||||||
|
SEL="${AISB_SELINUX:-:z}"; SELC="${SEL#:}"
|
||||||
|
run_in() {
|
||||||
|
podman run --rm --network none --cap-drop=ALL --cap-add=setuid --cap-add=setgid --cap-add=setpcap \
|
||||||
|
--userns "keep-id:uid=1000,gid=1000" -u 0:0 \
|
||||||
|
--security-opt no-new-privileges \
|
||||||
|
-v "$POL:/run/aisb:ro${SELC:+,$SELC}" -v "$PROJ:/project$SEL" -w /project \
|
||||||
|
"$IMG" bash -lc "$1" 2>&1
|
||||||
|
}
|
||||||
|
OUT="$(run_in 'id -u')"; assert_eq "1000" "$(printf '%s' "$OUT" | tail -1)" "drops to uid 1000"
|
||||||
|
OUT="$(run_in 'grep CapEff /proc/self/status | cut -f2')"
|
||||||
|
assert_eq "0000000000000000" "$(printf '%s' "$OUT" | tail -1)" "no capabilities left for the command"
|
||||||
|
OUT="$(run_in 'node --version')"
|
||||||
|
printf '%s' "$OUT" | grep -qE '^v2[2-9]' && _ok "node $(printf '%s' "$OUT" | tail -1)" || _bad "node 22+: $OUT"
|
||||||
|
OUT="$(run_in 'command -v pi')"
|
||||||
|
printf '%s' "$OUT" | grep -q '/pi' && _ok "the pi harness is installed" || _bad "pi installed: $OUT"
|
||||||
|
OUT="$(run_in 'nft --version && setpriv --version | head -1 && git --version')"
|
||||||
|
assert_contains "nftables" "nftables present"
|
||||||
|
assert_contains "git version" "git present"
|
||||||
|
OUT="$(run_in 'pwd')"
|
||||||
|
assert_eq "/project" "$(printf '%s' "$OUT" | tail -1)" "working directory is /project"
|
||||||
|
|
||||||
|
t_name "the container cannot see the host filesystem"
|
||||||
|
SECRET="$WORK/parent-secret.txt"; echo "do not read" > "$SECRET"
|
||||||
|
OUT="$(run_in "cat '$SECRET' 2>&1 || echo NOT_VISIBLE")"
|
||||||
|
assert_contains "NOT_VISIBLE" "files outside the mount are unreachable"
|
||||||
|
OUT="$(run_in 'ls / | tr "\n" " "')"
|
||||||
|
assert_absent "$(id -un)" "no host home directories in /"
|
||||||
|
|
||||||
|
t_name "resource caps are visible from inside"
|
||||||
|
OUT="$(podman run --rm --network none --cap-drop=ALL --cap-add=setuid --cap-add=setgid --cap-add=setpcap --pids-limit 64 --memory 128m \
|
||||||
|
--userns "keep-id:uid=1000,gid=1000" -u 0:0 \
|
||||||
|
-v "$POL:/run/aisb:ro${SELC:+,$SELC}" "$IMG" \
|
||||||
|
bash -lc 'cat /sys/fs/cgroup/pids.max; cat /sys/fs/cgroup/memory.max' 2>&1)"
|
||||||
|
assert_contains "64" "pids.max = 64"
|
||||||
|
assert_contains "134217728" "memory.max = 128m"
|
||||||
@@ -0,0 +1,107 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/e2e/11_net.sh -- the promise of the project: only the allowlist is reachable.
|
||||||
|
#
|
||||||
|
# Needs outbound internet on the host; skips (rc 0) when there is none.
|
||||||
|
source "$TESTS/lib.sh"
|
||||||
|
|
||||||
|
PROJ="$WORK/proj"; mkdir -p "$PROJ"; cp "$TESTS/fixtures/egress_probe.sh" "$PROJ/"
|
||||||
|
cd "$PROJ" || exit 1
|
||||||
|
|
||||||
|
if ! curl -m8 -sS -o /dev/null http://example.com 2>/dev/null; then
|
||||||
|
echo " (skipped: no outbound internet on this host, so egress cannot be tested)"
|
||||||
|
summary; exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
GATE="--net allow --allow example.com"
|
||||||
|
|
||||||
|
# a service on the host's loopback that the sandbox must not be able to reach
|
||||||
|
HOSTPORT="$(python3 -c 'import socket;s=socket.socket();s.bind(("127.0.0.1",0));print(s.getsockname()[1]);s.close()')"
|
||||||
|
python3 -m http.server "$HOSTPORT" --bind 127.0.0.1 --directory "$WORK" >/dev/null 2>&1 &
|
||||||
|
SRV=$!
|
||||||
|
trap 'kill '"$SRV"' 2>/dev/null' EXIT
|
||||||
|
sleep 1
|
||||||
|
|
||||||
|
t_name "allow mode: enforcement in one run"
|
||||||
|
sb $GATE --env AISB_PROBE_HOSTSVC="$HOSTPORT" -- bash /project/egress_probe.sh
|
||||||
|
assert_contains "ok_allowed_http=200" "allowlisted http works (through the gate)"
|
||||||
|
assert_contains "ok_allowed_https=200" "allowlisted https works (CONNECT)"
|
||||||
|
assert_contains "ok_gate_denied_domain=403" "a non-allowlisted domain is refused by the gate"
|
||||||
|
assert_contains "ok_gate_denied_https=" "https CONNECT to a non-allowlisted host is refused"
|
||||||
|
assert_contains "ok_direct_egress_blocked=domain" "ignoring the proxy env does not open a direct route"
|
||||||
|
assert_contains "ok_direct_egress_blocked=rawip" "a literal IP is not routable either"
|
||||||
|
assert_contains "ok_raw_socket_blocked" "a raw socket cannot connect out"
|
||||||
|
assert_contains "ok_host_service_blocked" "only the gate's port is reachable on the mapped loopback address"
|
||||||
|
assert_contains "ok_dns_blocked" "DNS is closed: names cannot be resolved behind the proxy's back"
|
||||||
|
|
||||||
|
t_name "the ruleset cannot be lifted from inside"
|
||||||
|
assert_contains "uid=1000" "the harness runs unprivileged"
|
||||||
|
assert_contains "capeff=0000000000000000" "no capabilities remain for it"
|
||||||
|
assert_contains "ok_rules_hidden" "it cannot read the ruleset"
|
||||||
|
assert_contains "ok_rules_locked" "it cannot delete the ruleset"
|
||||||
|
assert_contains "ok_netns_locked" "it cannot open its own network namespace"
|
||||||
|
assert_contains "ok_still_enforced=403" "and enforcement still holds afterwards"
|
||||||
|
|
||||||
|
t_name "an IP entry is allowed through the gate as well"
|
||||||
|
sb --net allow --allow example.com --allow 1.1.1.1:80 -- bash -lc '
|
||||||
|
echo "ip_entry_code=$(curl -m10 -sS -o /dev/null -w %{http_code} http://1.1.1.1 2>&1)"
|
||||||
|
echo "domain_code=$(curl -m10 -sS -o /dev/null -w %{http_code} http://example.org 2>&1)"'
|
||||||
|
assert_absent "ip_entry_code=403" "an allowlisted IP is not refused by the gate"
|
||||||
|
assert_contains "domain_code=403" "while an unlisted domain still is"
|
||||||
|
|
||||||
|
t_name "--net none is dark"
|
||||||
|
sb --net none -- bash -lc '
|
||||||
|
curl -m5 -sS -o /dev/null http://example.com 2>/dev/null && echo BAD_open || echo ok_dark
|
||||||
|
ip -4 -o addr show 2>/dev/null | grep -c global | sed "s/^/addrs=/"
|
||||||
|
echo "capeff=$(grep CapEff /proc/self/status | cut -f2)"'
|
||||||
|
assert_contains "ok_dark" "no egress at all"
|
||||||
|
assert_contains "addrs=0" "no global addresses in the namespace"
|
||||||
|
assert_contains "capeff=0000000000000000" "no capabilities either"
|
||||||
|
|
||||||
|
t_name "--no-gate: IPs resolved up front, everything else dropped"
|
||||||
|
sb --net allow --no-gate --allow example.com -- bash -lc '
|
||||||
|
echo "allowed=$(curl -m15 -sS -o /dev/null -w %{http_code} http://example.com 2>&1)"
|
||||||
|
curl -m8 -sS -o /dev/null http://example.org 2>/dev/null && echo BAD_not_blocked || echo ok_blocked
|
||||||
|
echo "dns=$(: </dev/tcp/8.8.8.8/53 2>/dev/null && echo open || echo closed)"'
|
||||||
|
assert_contains "allowed=200" "the resolved address of an allowlisted name works"
|
||||||
|
assert_contains "ok_blocked" "anything else is dropped by netfilter"
|
||||||
|
|
||||||
|
t_name "the gate does not outlive the session"
|
||||||
|
BEFORE_GATES="$(podman ps -a --filter label=aisb.role=gate -q | wc -l | tr -d ' ')"
|
||||||
|
sb $GATE -- true
|
||||||
|
AFTER_GATES="$(podman ps -a --filter label=aisb.role=gate -q | wc -l | tr -d ' ')"
|
||||||
|
assert_eq "$BEFORE_GATES" "$AFTER_GATES" "no gate container is left behind by a run"
|
||||||
|
LEFT="$(ls -A "$AISB_STATE_DIR/sessions" 2>/dev/null | wc -l | tr -d ' ')"
|
||||||
|
assert_eq "0" "$LEFT" "no session directory is left behind"
|
||||||
|
|
||||||
|
t_name "a named gate outlives a session on purpose"
|
||||||
|
AISB_GATE_NAME=aisb-test-gate "$AISB" gate up --allow example.com > "$WORK/gate.out" 2>&1
|
||||||
|
OUT="$(cat "$WORK/gate.out")"
|
||||||
|
assert_rc 0 "gate up succeeds"
|
||||||
|
assert_contains "port=" "and reports the port"
|
||||||
|
PORT="$(printf '%s' "$OUT" | grep '^port=' | cut -d= -f2)"
|
||||||
|
if [[ -n "$PORT" ]]; then
|
||||||
|
code="$(curl -sS -m6 -x "http://127.0.0.1:$PORT" -o /dev/null -w '%{http_code}' http://example.com/ 2>/dev/null || echo 000)"
|
||||||
|
assert_eq "200" "$code" "the named gate proxies an allowlisted host"
|
||||||
|
code="$(curl -sS -m6 -x "http://127.0.0.1:$PORT" -o /dev/null -w '%{http_code}' http://example.org/ 2>/dev/null || echo 000)"
|
||||||
|
assert_eq "403" "$code" "and refuses a non-allowlisted host"
|
||||||
|
fi
|
||||||
|
AISB_GATE_NAME=aisb-test-gate "$AISB" gate down > /dev/null 2>&1
|
||||||
|
assert_eq "0" "$(podman ps -a --filter name=aisb-test-gate -q | wc -l | tr -d ' ')" "gate down removes it"
|
||||||
|
|
||||||
|
t_name "a sandbox can use a named gate instead of starting its own"
|
||||||
|
AISB_GATE_NAME=aisb-test-gate2 "$AISB" gate up --allow example.com > "$WORK/gate2.out" 2>&1
|
||||||
|
PORT2="$(grep '^port=' "$WORK/gate2.out" | cut -d= -f2)"
|
||||||
|
sb --net allow --proxy "100.64.0.1:${PORT2}" --allow example.com -- bash -lc '
|
||||||
|
echo "via_proxy=$(curl -m15 -sS -o /dev/null -w %{http_code} http://example.com 2>&1)"'
|
||||||
|
assert_contains "via_proxy=200" "--proxy routes through the named gate"
|
||||||
|
AISB_GATE_NAME=aisb-test-gate2 "$AISB" gate down > /dev/null 2>&1
|
||||||
|
|
||||||
|
t_name "a bad gate config fails closed rather than running wide open"
|
||||||
|
# point the sandbox at a port where nothing answers: the run must not silently
|
||||||
|
# proceed with a broken egress path.
|
||||||
|
sb --net allow --proxy 100.64.0.1:1 -- bash -lc '
|
||||||
|
code=$(curl -m8 -sS -o /dev/null -w %{http_code} http://example.com 2>/dev/null); echo "code=$code"
|
||||||
|
exit 0'
|
||||||
|
assert_contains "code=000" "with a dead proxy, egress simply fails (no bypass)"
|
||||||
|
|
||||||
|
kill "$SRV" 2>/dev/null
|
||||||
@@ -0,0 +1,154 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/e2e/12_sandbox.sh -- lifecycle, filesystem and environment behaviour of a real run.
|
||||||
|
source "$TESTS/lib.sh"
|
||||||
|
|
||||||
|
WORK_DIR="$WORK/project"; mkdir -p "$WORK_DIR"; cd "$WORK_DIR" || exit 1
|
||||||
|
echo "hello from the host" > "$WORK_DIR/host-file.txt"
|
||||||
|
export AISB_CANARY_IN=canary-value-42
|
||||||
|
unset AISB_CANARY_OUT
|
||||||
|
DARK="--net none"
|
||||||
|
|
||||||
|
t_name "the command runs and its exit code comes back"
|
||||||
|
sb $DARK -- bash -c 'echo ran; exit 0'
|
||||||
|
assert_rc 0 "exit 0 propagates"; assert_contains "ran" "stdout reaches the host"
|
||||||
|
sb $DARK -- bash -c 'exit 3'
|
||||||
|
assert_rc 3 "exit 3 propagates"
|
||||||
|
sb $DARK -- bash -c 'exit 127'
|
||||||
|
assert_rc 127 "exit 127 propagates"
|
||||||
|
|
||||||
|
t_name "default command follows the harness"
|
||||||
|
sb $DARK --harness none -- true >/dev/null; assert_rc 0 "harness none is accepted"
|
||||||
|
sb $DARK --harness pi -- bash -lc "command -v pi"
|
||||||
|
assert_rc 0 "the default pi binary exists in the image"
|
||||||
|
|
||||||
|
t_name "the project directory is mounted, writable, and owned by the host user"
|
||||||
|
sb $DARK -- bash -c 'pwd; touch /project/from-container.txt; echo wrote'
|
||||||
|
assert_contains "/project" "working directory is /project"
|
||||||
|
[[ -f "$WORK_DIR/from-container.txt" ]] && _ok "writes land on the host" || _bad "writes land on the host"
|
||||||
|
if [[ "$(stat -c %u "$WORK_DIR/from-container.txt")" == "$(id -u)" ]]; then
|
||||||
|
_ok "the file is owned by the host uid, not by root"
|
||||||
|
else
|
||||||
|
_bad "the file is owned by the host uid" "owner=$(stat -c %u "$WORK_DIR/from-container.txt") expected=$(id -u)"
|
||||||
|
fi
|
||||||
|
sb $DARK -- bash -c 'cat /project/host-file.txt'
|
||||||
|
assert_contains "hello from the host" "existing project files are readable"
|
||||||
|
|
||||||
|
t_name "nothing else on the host is visible"
|
||||||
|
echo secret > "$WORK/not-shared.txt"
|
||||||
|
sb $DARK -- bash -c "cat '$WORK/not-shared.txt' 2>&1 || echo NOT_VISIBLE"
|
||||||
|
assert_contains "NOT_VISIBLE" "the parent directory is not mounted"
|
||||||
|
sb $DARK -- bash -c 'ls /root 2>&1 | head -1; test -e /etc/shadow && echo has_shadow'
|
||||||
|
assert_absent "$(hostname)" "the host filesystem layout is gone"
|
||||||
|
|
||||||
|
t_name "environment: deny by default, allow on request"
|
||||||
|
sb $DARK -- bash -c 'echo "canary=[${AISB_CANARY_IN:-unset}]"; echo "home=$HOME"; echo "aisb=$AISB"'
|
||||||
|
assert_contains "canary=[unset]" "a host variable does not leak in"
|
||||||
|
assert_contains "home=/home/sandbox" "HOME is the container's own"
|
||||||
|
assert_contains "aisb=1" "the sandbox marks itself"
|
||||||
|
sb $DARK --env AISB_CANARY_IN -- bash -c 'echo "canary=[$AISB_CANARY_IN]"'
|
||||||
|
assert_contains "canary=[canary-value-42]" "--env passes exactly what was asked for"
|
||||||
|
sb $DARK --env AISB_CANARY_OUT -- bash -c 'true'
|
||||||
|
assert_rc 1 "an unset --env variable fails the run closed"
|
||||||
|
|
||||||
|
t_name "the value never appears in a command line"
|
||||||
|
( sb $DARK --env AISB_CANARY_IN -- bash -c 'sleep 2' >/dev/null 2>&1 ) &
|
||||||
|
RUNPID=$!
|
||||||
|
sleep 1.5
|
||||||
|
# Every process the value could legitimately have been handed to: the container
|
||||||
|
# runtime. argv[0] is matched exactly so the test's own shell (whose command line
|
||||||
|
# contains this very string) can never be the thing that matches.
|
||||||
|
# (the bracket keeps this grep from matching its own command line)
|
||||||
|
LEAKY="$(ps -ewwo comm=,args= | awk '{a=$1; sub(".*/","",a)} a ~ /^(podman|crun|conmon|catatonit)$/' \
|
||||||
|
| grep -c 'canar[y]-value-42' || true)"
|
||||||
|
assert_eq "0" "$LEAKY" "no podman/crun/conmon argv carries the secret"
|
||||||
|
wait $RUNPID 2>/dev/null
|
||||||
|
|
||||||
|
t_name "the env file is 0600 while running and gone afterwards"
|
||||||
|
( sb $DARK --env AISB_CANARY_IN -- bash -c 'sleep 2' >/dev/null 2>&1 ) &
|
||||||
|
RUNPID2=$!
|
||||||
|
sleep 1.5
|
||||||
|
ENVF="$(find "$AISB_STATE_DIR/sessions" -name env -print -quit 2>/dev/null)"
|
||||||
|
if [[ -n "$ENVF" ]]; then
|
||||||
|
assert_file_mode "$ENVF" 600 "the env file is owner-read/write only"
|
||||||
|
grep -q 'AISB_CANARY_IN=canary-value-42' "$ENVF" && _ok "and it holds the value we asked for" || _bad "env file content"
|
||||||
|
else
|
||||||
|
_bad "the env file is owner-read/write only" "no env file found while the container was running"
|
||||||
|
fi
|
||||||
|
wait $RUNPID2 2>/dev/null
|
||||||
|
sb $DARK --env AISB_CANARY_IN -- true
|
||||||
|
LEFT="$(find "$AISB_STATE_DIR/sessions" -name env 2>/dev/null | wc -l | tr -d ' ')"
|
||||||
|
assert_eq "0" "$LEFT" "the env file is deleted when the run ends, even with --keep"
|
||||||
|
sb $DARK --keep --env AISB_CANARY_IN -- true
|
||||||
|
LEFT2="$(find "$AISB_STATE_DIR/sessions" -name env 2>/dev/null | wc -l | tr -d ' ')"
|
||||||
|
assert_eq "0" "$LEFT2" "--keep keeps the container, not the secret"
|
||||||
|
|
||||||
|
t_name "extra mounts"
|
||||||
|
mkdir -p "$WORK/ro-src" "$WORK/rw-src"; echo data > "$WORK/ro-src/file"
|
||||||
|
sb $DARK --mount "$WORK/ro-src:/ro:ro" --mount "$WORK/rw-src:/rw" -- bash -c '
|
||||||
|
cat /ro/file
|
||||||
|
touch /rw/created && echo rw_ok
|
||||||
|
touch /ro/writable 2>/dev/null && echo "BAD: ro mount is writable" || echo ro_locked=ok
|
||||||
|
'
|
||||||
|
assert_contains "data" "read-only mount is readable"
|
||||||
|
assert_contains "rw_ok" "read-write mount is writable"
|
||||||
|
assert_contains "ro_locked=ok" "ro really means ro"
|
||||||
|
|
||||||
|
t_name "the home directory"
|
||||||
|
sb $DARK --ephemeral-home -- bash -c 'echo hi > "$HOME/scratch"; ls -A "$HOME" | tr "\n" " "'
|
||||||
|
assert_contains "scratch" "home is writable with an ephemeral mount"
|
||||||
|
sb $DARK --home-cache -- bash -c 'echo cached > "$HOME/cached-marker"'
|
||||||
|
sb $DARK --home-cache -- bash -c 'cat "$HOME/cached-marker"'
|
||||||
|
assert_contains "cached" "the home cache volume persists across runs"
|
||||||
|
sb $DARK --ephemeral-home -- bash -c 'test -f "$HOME/cached-marker" && echo LEAK || echo isolated'
|
||||||
|
assert_contains "isolated" "an ephemeral home does not see it"
|
||||||
|
|
||||||
|
t_name "read-only root filesystem"
|
||||||
|
sb $DARK --read-only -- bash -c 'touch /usr/bin/nope 2>/dev/null && echo "BAD: wrote to rootfs" || echo rootfs_locked=ok; touch /project/rw-still-works && echo project_still_writable'
|
||||||
|
assert_contains "rootfs_locked=ok" "--read-only blocks writes outside mounts"
|
||||||
|
assert_contains "project_still_writable" "the project mount stays writable"
|
||||||
|
|
||||||
|
t_name "lifecycle"
|
||||||
|
BEFORE="$(podman ps -a --filter label=ai-sandbox=1 -q | wc -l | tr -d ' ')"
|
||||||
|
sb $DARK -- true
|
||||||
|
AFTER="$(podman ps -a --filter label=ai-sandbox=1 -q | wc -l | tr -d ' ')"
|
||||||
|
assert_eq "$BEFORE" "$AFTER" "the container is removed when the command exits"
|
||||||
|
sb $DARK --keep --name aisb-keeptest -- true
|
||||||
|
podman ps -a --format '{{.Names}}' | grep -qx aisb-keeptest && _ok "--keep leaves the container" || _bad "--keep leaves the container"
|
||||||
|
SESSIONS_KEPT="$(ls -d "$AISB_STATE_DIR"/sessions/* 2>/dev/null | wc -l | tr -d ' ')"
|
||||||
|
[[ "$SESSIONS_KEPT" -ge 1 ]] && _ok "--keep leaves the session dir for inspection" || _bad "--keep leaves the session dir"
|
||||||
|
sb clean --all > /dev/null
|
||||||
|
GONE="$(podman ps -a --filter name=aisb-keeptest -q | wc -l | tr -d ' ')"
|
||||||
|
assert_eq "0" "$GONE" "clean removes leftovers"
|
||||||
|
|
||||||
|
t_name "clean --volumes reclaims the home cache, carefully"
|
||||||
|
podman volume create not-aisb-decoy > /dev/null 2>&1
|
||||||
|
podman volume create aisb-other-dangling > /dev/null 2>&1
|
||||||
|
AISB_HOME_VOLUME=aisb-test-home sb $DARK -- true
|
||||||
|
if podman volume exists aisb-test-home > /dev/null 2>&1; then _ok "the run created its home cache volume"; else _bad "the run created its home cache volume"; fi
|
||||||
|
sb clean --volumes=aisb-test-home > /dev/null
|
||||||
|
if podman volume exists aisb-test-home > /dev/null 2>&1; then _bad "clean --volumes=NAME reclaimed the unused volume"; else _ok "clean --volumes=NAME reclaimed the unused volume"; fi
|
||||||
|
if podman volume exists aisb-other-dangling > /dev/null 2>&1; then _ok "an unmatched volume was left alone"; else _bad "an unmatched volume was left alone"; fi
|
||||||
|
|
||||||
|
# a live sandbox's home is never reclaimed while it runs
|
||||||
|
AISB_HOME_VOLUME=aisb-test-home2 sb $DARK --name aisb-vollive -- bash -c 'sleep 25' > /dev/null 2>&1 &
|
||||||
|
for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20; do
|
||||||
|
podman ps --filter name=aisb-vollive -q | grep -q . && break
|
||||||
|
sleep 0.5
|
||||||
|
done
|
||||||
|
AISB_HOME_VOLUME=aisb-test-home2 sb clean --volumes=aisb-test-home2 > /dev/null
|
||||||
|
if podman volume exists aisb-test-home2 > /dev/null 2>&1; then _ok "a running sandbox keeps its home volume"; else _bad "a running sandbox keeps its home volume"; fi
|
||||||
|
podman rm -f aisb-vollive > /dev/null 2>&1
|
||||||
|
AISB_HOME_VOLUME=aisb-test-home2 sb clean --volumes=aisb-test-home2 > /dev/null
|
||||||
|
if podman volume exists aisb-test-home2 > /dev/null 2>&1; then _bad "it is reclaimed once nothing references it"; else _ok "it is reclaimed once nothing references it"; fi
|
||||||
|
podman volume rm not-aisb-decoy aisb-other-dangling > /dev/null 2>&1
|
||||||
|
|
||||||
|
t_name "the session log records what ran, without secrets"
|
||||||
|
LOG="$(cat "$AISB_STATE_DIR/log" 2>/dev/null || true)"
|
||||||
|
printf '%s' "$LOG" | grep -q 'net=none' && _ok "the log records the run" || _bad "the log records the run"
|
||||||
|
printf '%s' "$LOG" | grep -q 'canary-value-42' && _bad "secrets must never reach the log" || _ok "no secrets in the log"
|
||||||
|
|
||||||
|
t_name "doctor"
|
||||||
|
sb doctor
|
||||||
|
assert_rc 0 "doctor reports no problems"
|
||||||
|
assert_contains "bind mount + uid mapping" "doctor verifies uid mapping"
|
||||||
|
assert_contains "rootless" "doctor verifies rootless podman"
|
||||||
Vendored
+84
@@ -0,0 +1,84 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/fixtures/egress_probe.sh -- runs *inside* the sandbox and reports, one
|
||||||
|
# marker per line, what is and is not reachable. Kept as a file so the outer test
|
||||||
|
# does not have to nest five layers of quoting.
|
||||||
|
#
|
||||||
|
# Markers start with ok_/BAD_ so the caller can grep for either.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
ALLOW="${AISB_PROBE_ALLOW:-example.com}"
|
||||||
|
OTHER="${AISB_PROBE_OTHER:-example.org}"
|
||||||
|
RAWIP="${AISB_PROBE_RAWIP:-1.1.1.1}"
|
||||||
|
GATE="${AISB_PROBE_GATE:-100.64.0.1}"
|
||||||
|
HOSTSVC="${AISB_PROBE_HOSTSVC:-18099}"
|
||||||
|
|
||||||
|
# 1. allowlisted destination, through the proxy
|
||||||
|
code="$(curl -m15 -sS -o /dev/null -w '%{http_code}' "http://$ALLOW" 2>/dev/null)"
|
||||||
|
[[ "$code" == 200 ]] && echo "ok_allowed_http=$code" || echo "BAD_allowed_http=$code"
|
||||||
|
|
||||||
|
# 2. allowlisted destination over https (CONNECT)
|
||||||
|
code="$(curl -m15 -sS -o /dev/null -w '%{http_code}' "https://$ALLOW" 2>/dev/null)"
|
||||||
|
[[ "$code" == 200 ]] && echo "ok_allowed_https=$code" || echo "BAD_allowed_https=$code"
|
||||||
|
|
||||||
|
# 3. non-allowlisted domain: the gate must answer 403
|
||||||
|
code="$(curl -m10 -sS -o /dev/null -w '%{http_code}' "http://$OTHER" 2>/dev/null)"
|
||||||
|
[[ "$code" == 403 ]] && echo "ok_gate_denied_domain=$code" || echo "BAD_gate_denied_domain=$code"
|
||||||
|
# a denied CONNECT is refused by the proxy: curl reports 000/502, never 200
|
||||||
|
code="$(curl -m10 -sS -o /dev/null -w '%{http_code}' "https://$OTHER" 2>/dev/null)"
|
||||||
|
[[ "$code" != 200 ]] && echo "ok_gate_denied_https=$code" || echo "BAD_gate_denied_https=$code"
|
||||||
|
|
||||||
|
# 4. same, but pretending the caller ignores the proxy env: the netfilter ruleset
|
||||||
|
# is what stops it. Anything other than a connection failure is a leak.
|
||||||
|
nop () { env -u http_proxy -u https_proxy -u all_proxy -u HTTP_PROXY -u HTTPS_PROXY -u ALL_PROXY "$@"; }
|
||||||
|
if nop curl -m8 -sS -o /dev/null "http://$OTHER" 2>/dev/null; then
|
||||||
|
echo "BAD_direct_egress=domain"
|
||||||
|
else
|
||||||
|
echo "ok_direct_egress_blocked=domain"
|
||||||
|
fi
|
||||||
|
if nop curl -m8 -sS -o /dev/null "http://$RAWIP" 2>/dev/null; then
|
||||||
|
echo "BAD_direct_egress=rawip"
|
||||||
|
else
|
||||||
|
echo "ok_direct_egress_blocked=rawip"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 5. a raw socket (no proxy semantics at all) must not connect either
|
||||||
|
python3 - "$RAWIP" "$GATE" <<'PY'
|
||||||
|
import socket, sys
|
||||||
|
rawip, gate = sys.argv[1], sys.argv[2]
|
||||||
|
for label, host, port in (("rawip", rawip, 80), ("cdnish", "93.184.216.34", 80), ("dns", "8.8.8.8", 53)):
|
||||||
|
s = socket.socket(); s.settimeout(6)
|
||||||
|
try:
|
||||||
|
s.connect((host, port)); print(f"BAD_raw_socket={label}")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"ok_raw_socket_blocked={label}:{type(e).__name__}")
|
||||||
|
finally:
|
||||||
|
s.close()
|
||||||
|
PY
|
||||||
|
|
||||||
|
# 6. an unrelated service on the host's loopback must be unreachable through the
|
||||||
|
# mapped address: only the gate's own port is routable there. Tested without
|
||||||
|
# the proxy, because a proxied request dies at the squid and would exit 0 on a
|
||||||
|
# 403 -- which proves the proxy works, not that the route is closed.
|
||||||
|
if nop curl -m6 -sS -o /dev/null "http://$GATE:$HOSTSVC/" 2>/dev/null; then
|
||||||
|
echo "BAD_host_service_reachable=$HOSTSVC"
|
||||||
|
else
|
||||||
|
echo "ok_host_service_blocked=$HOSTSVC"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 7. DNS: gate mode keeps it shut, so nothing can resolve names behind the proxy
|
||||||
|
if getent hosts "$OTHER" > /dev/null 2>&1; then
|
||||||
|
echo "BAD_dns_resolved=$OTHER"
|
||||||
|
else
|
||||||
|
echo "ok_dns_blocked=$OTHER"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 8. who are we, and can we touch the ruleset?
|
||||||
|
echo "uid=$(id -u)"
|
||||||
|
echo "capeff=$(grep CapEff /proc/self/status | cut -f2)"
|
||||||
|
if nft list table inet aisb > /dev/null 2>&1; then echo "BAD_rules_readable"; else echo "ok_rules_hidden"; fi
|
||||||
|
if nft delete table inet aisb > /dev/null 2>&1; then echo "BAD_rules_deleted"; else echo "ok_rules_locked"; fi
|
||||||
|
if unshare -n true > /dev/null 2>&1; then echo "BAD_new_netns"; else echo "ok_netns_locked"; fi
|
||||||
|
|
||||||
|
# 9. and after all that, enforcement is still in place
|
||||||
|
code="$(curl -m10 -sS -o /dev/null -w '%{http_code}' "http://$OTHER" 2>/dev/null)"
|
||||||
|
[[ "$code" == 403 ]] && echo "ok_still_enforced=$code" || echo "BAD_still_enforced=$code"
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/lib.sh -- tiny assertion helpers for the ai-sandbox test suite.
|
||||||
|
#
|
||||||
|
# Every test file gets: AISB (path to the script), WORK (scratch dir), and uses
|
||||||
|
# sb() to run the sandbox with fully redirected XDG dirs so nothing touches the
|
||||||
|
# real machine.
|
||||||
|
|
||||||
|
PASS=0
|
||||||
|
FAIL=0
|
||||||
|
FAILED_NAMES=()
|
||||||
|
_current=""
|
||||||
|
|
||||||
|
t_name() { _current="$1"; printf '\n\033[1m# %s\033[0m\n' "$1"; }
|
||||||
|
|
||||||
|
_ok() { PASS=$((PASS + 1)); printf ' \033[32mok\033[0m %s\n' "$1"; }
|
||||||
|
_bad() { FAIL=$((FAIL + 1)); FAILED_NAMES+=("$_current :: $1"); printf ' \033[31mFAIL\033[0m %s\n' "$1"; shift || true;
|
||||||
|
[[ -n "${1:-}" ]] && printf ' %s\n' "$1"; return 0; }
|
||||||
|
|
||||||
|
# sb [args...] -> sets RC, OUT (stdout+stderr)
|
||||||
|
sb() {
|
||||||
|
OUT="$("$AISB" "$@" 2>&1)" && RC=0 || RC=$?
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# sb_env VAR=val ... -- args (env prefix, without leaking into the test shell)
|
||||||
|
sb_env() {
|
||||||
|
local -a envs=()
|
||||||
|
while (($#)) && [[ "$1" != -* && "$1" == *=* ]]; do envs+=("$1"); shift; done
|
||||||
|
OUT="$(env "${envs[@]}" "$AISB" "$@" 2>&1)" && RC=0 || RC=$?
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_rc() { # assert_rc <expected> <label>
|
||||||
|
if [[ "$RC" == "$1" ]]; then _ok "$2"
|
||||||
|
else _bad "$2" "expected rc=$1 got rc=$RC; output: $(printf '%s' "$OUT" | head -3 | tr '\n' ' ')"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_contains() { # assert_contains <needle> <label>
|
||||||
|
if printf '%s' "$OUT" | grep -qF -- "$1"; then _ok "$2"
|
||||||
|
else _bad "$2" "missing: '$1' in: $(printf '%s' "$OUT" | head -5 | tr '\n' ' ')"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_absent() { # assert_absent <needle> <label>
|
||||||
|
if printf '%s' "$OUT" | grep -qF -- "$1"; then
|
||||||
|
_bad "$2" "unexpectedly present: '$1'"
|
||||||
|
else _ok "$2"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_eq() { # assert_eq <expected> <actual> <label>
|
||||||
|
if [[ "$1" == "$2" ]]; then _ok "$3"
|
||||||
|
else _bad "$3" "expected '$1' got '$2'"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_count() { # assert_count <expected> <needle> <label>
|
||||||
|
local n
|
||||||
|
n="$(printf '%s' "$OUT" | grep -oF -- "$2" | wc -l | tr -d ' ')"
|
||||||
|
if [[ "$n" == "$1" ]]; then _ok "$3"
|
||||||
|
else _bad "$3" "expected $1 occurrence(s) of '$2', got $n"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_file_mode() { # assert_file_mode <path> <mode> <label>
|
||||||
|
local m; m="$(stat -c '%a' "$1" 2>/dev/null || echo missing)"
|
||||||
|
if [[ "$m" == "$2" ]]; then _ok "$3"; else _bad "$3" "expected mode $2 got $m"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
summary() {
|
||||||
|
printf '\n------------------------------------------------------------\n'
|
||||||
|
[[ -n "${AISB_TEST_COUNTS:-}" ]] && printf '%s %s\n' "$PASS" "$FAIL" >> "$AISB_TEST_COUNTS"
|
||||||
|
if (( FAIL )); then
|
||||||
|
printf '\033[31m%d passed, %d FAILED\033[0m\n' "$PASS" "$FAIL"
|
||||||
|
local f; for f in "${FAILED_NAMES[@]}"; do printf ' - %s\n' "$f"; done
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
printf '\033[32m%d passed, 0 failed\033[0m\n' "$PASS"
|
||||||
|
}
|
||||||
Executable
+87
@@ -0,0 +1,87 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/run.sh -- the ai-sandbox test runner.
|
||||||
|
#
|
||||||
|
# ./tests/run.sh unit tests (no containers started, no network)
|
||||||
|
# ./tests/run.sh --e2e everything, including real containers and egress tests
|
||||||
|
# ./tests/run.sh --only 04 just the files matching 04
|
||||||
|
# ./tests/run.sh --verbose show each test file's raw output as it runs
|
||||||
|
#
|
||||||
|
# Every test file runs with its own XDG state so nothing here touches the real
|
||||||
|
# machine's config, state or cache directories.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT="$(dirname "$HERE")"
|
||||||
|
AISB="${AISB:-$ROOT/bin/ai-sandbox}"
|
||||||
|
export AISB
|
||||||
|
|
||||||
|
WITH_E2E=0 ONLY="" VERBOSE=0
|
||||||
|
while (($#)); do
|
||||||
|
case "$1" in
|
||||||
|
--e2e|-e) WITH_E2E=1; shift ;;
|
||||||
|
--only) ONLY="${2:-}"; shift 2 ;;
|
||||||
|
--verbose|-v) export VERBOSE=1; shift ;; # lib.sh reads VERBOSE
|
||||||
|
--help|-h) sed -n '2,12p' "$0"; exit 0 ;;
|
||||||
|
*) echo "unknown option: $1" >&2; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
command -v podman >/dev/null || { echo "podman is required to run the tests" >&2; exit 2; }
|
||||||
|
|
||||||
|
# Syntax-check every shell file in the project before running anything.
|
||||||
|
SYNTAX_BAD=0
|
||||||
|
for f in "$AISB" "$HERE"/*.sh "$HERE"/unit/*.sh "$HERE"/e2e/*.sh "$ROOT"/install.sh; do
|
||||||
|
[[ -f "$f" ]] || continue
|
||||||
|
if ! syn="$(bash -n "$f" 2>&1)"; then
|
||||||
|
echo "SYNTAX: $f" >&2; printf '%s\n' "$syn" | sed 's/^/ /' >&2; SYNTAX_BAD=1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
(( SYNTAX_BAD )) && exit 2
|
||||||
|
|
||||||
|
FILES=( "$HERE"/unit/*.sh )
|
||||||
|
(( WITH_E2E )) && FILES+=( "$HERE"/e2e/*.sh )
|
||||||
|
if [[ -n "$ONLY" ]]; then
|
||||||
|
mapfile -t FILES < <(printf '%s\n' "${FILES[@]}" | grep "/[^/]*$ONLY")
|
||||||
|
fi
|
||||||
|
|
||||||
|
BASE="$(mktemp -d "${TMPDIR:-/tmp}/ai-sandbox-tests.XXXXXX")"
|
||||||
|
COUNTS="$BASE/counts"
|
||||||
|
: > "$COUNTS"
|
||||||
|
trap 'chmod -R u+rwX "$BASE" 2>/dev/null; rm -rf "$BASE" 2>/dev/null' EXIT
|
||||||
|
|
||||||
|
echo "ai-sandbox test suite"
|
||||||
|
echo " script : $AISB"
|
||||||
|
echo " mode : $([[ $WITH_E2E == 1 ]] && echo 'unit + e2e (real containers)' || echo 'unit only (add --e2e for containers)')"
|
||||||
|
echo " files : ${#FILES[@]}"
|
||||||
|
echo
|
||||||
|
|
||||||
|
FAILED_FILES=()
|
||||||
|
for f in "${FILES[@]}"; do
|
||||||
|
name="$(basename "$f")"
|
||||||
|
W="$BASE/$name"; mkdir -p "$W"/{state,cache,cfg}
|
||||||
|
echo "=================================================================="
|
||||||
|
echo "== $name"
|
||||||
|
echo "=================================================================="
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
(
|
||||||
|
export AISB_STATE_DIR="$W/state" AISB_CACHE_DIR="$W/cache" AISB_CONFIG_DIR="$W/cfg"
|
||||||
|
export WORK="$W" TESTS="$HERE" AISB_TEST_COUNTS="$COUNTS"
|
||||||
|
# NOTE: HOME is deliberately not overridden -- podman storage lives there.
|
||||||
|
unset AISB_ALLOW AISB_ENV AISB_MOUNT AISB_INSTALL AISB_NET AISB_EXTRA_ARGS AISB_SESSION_OVERRIDE
|
||||||
|
source "$f"
|
||||||
|
summary
|
||||||
|
) || FAILED_FILES+=("$name")
|
||||||
|
echo
|
||||||
|
done
|
||||||
|
|
||||||
|
TOTAL_P=0 TOTAL_F=0
|
||||||
|
while read -r p fl; do TOTAL_P=$((TOTAL_P + p)); TOTAL_F=$((TOTAL_F + fl)); done < "$COUNTS"
|
||||||
|
|
||||||
|
echo "=================================================================="
|
||||||
|
if ((${#FAILED_FILES[@]})); then
|
||||||
|
printf '\033[31mFAILED\033[0m %d passed, %d failed across %d file(s)\n' "$TOTAL_P" "$TOTAL_F" "${#FAILED_FILES[@]}"
|
||||||
|
printf ' failing files: %s\n' "${FAILED_FILES[*]}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
printf '\033[32mALL PASS\033[0m %d assertions, %d file(s), %s mode\n' \
|
||||||
|
"$TOTAL_P" "${#FILES[@]}" "$([[ $WITH_E2E == 1 ]] && echo unit+e2e || echo unit)"
|
||||||
Executable
+79
@@ -0,0 +1,79 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/spike_network.sh -- empirical verification of the network enforcement primitives.
|
||||||
|
#
|
||||||
|
# Run on a machine with rootless podman. Output is captured into docs/networking.md.
|
||||||
|
# Nothing here is a unit test; it is evidence that the enforcement model holds on the
|
||||||
|
# host it was run on. Safe: creates and removes its own podman network + containers.
|
||||||
|
#
|
||||||
|
# ./tests/spike_network.sh | tee /tmp/spike.txt
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
NET="${AISB_SPIKE_NET:-aisb-spike}"
|
||||||
|
IMG="${AISB_SPIKE_IMG:-docker.io/library/ubuntu:24.04}"
|
||||||
|
CURL_IMG="${AISB_SPIKE_CURL_IMG:-docker.io/curlimages/curl:latest}"
|
||||||
|
UID_N="$(id -u)"
|
||||||
|
|
||||||
|
say() { printf '\n=== %s ===\n' "$*"; }
|
||||||
|
cleanup() {
|
||||||
|
podman rm -f aisb-spike-a aisb-spike-b aisb-spike-gate >/dev/null 2>&1 || true
|
||||||
|
podman network rm "$NET" >/dev/null 2>&1 || true
|
||||||
|
}
|
||||||
|
cleanup
|
||||||
|
# "Connection refused" => L3 reachable (nothing listening). "Could not connect"/timeout => not reachable.
|
||||||
|
probe() { # probe <ctr> <ip> -> prints REFUSED|UNREACHABLE
|
||||||
|
local out
|
||||||
|
out="$(timeout 25 podman run --rm --network="$1" "$CURL_IMG" -m3 -o /dev/null "http://$2:1/" 2>&1 || true)"
|
||||||
|
case "$out" in
|
||||||
|
*"Connection refused"*) echo "REFUSED (reachable)" ;;
|
||||||
|
*) echo "UNREACHABLE :: $(printf '%s' "$out" | tr '\n' ' ' | cut -c1-90)" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
say "0. host"
|
||||||
|
printf 'podman : %s\n' "$(podman --version)"
|
||||||
|
printf 'rootless: %s\n' "$(podman info --format '{{.Host.Security.Rootless}}')"
|
||||||
|
printf 'uid : %s subuid: %s\n' "$UID_N" "$(grep -c "^$(id -un):" /etc/subuid 2>/dev/null || echo 0)"
|
||||||
|
printf 'helpers : pasta=%s slirp4netns=%s\n' "$(command -v pasta || echo no)" "$(command -v slirp4netns || echo no)"
|
||||||
|
printf 'state : %s\n' "$(ls "/run/user/$UID_N/netns" 2>/dev/null | tr '\n' ' ')"
|
||||||
|
printf 'bridges : %s\n' "$(ip -br -o link show type bridge 2>/dev/null | awk '{print $1}' | tr '\n' ' ')"
|
||||||
|
|
||||||
|
say "1. internal network: egress + DNS must fail"
|
||||||
|
podman network create --internal "$NET" >/dev/null || { echo "SKIP: cannot create network"; exit 0; }
|
||||||
|
printf 'internal subnet: %s\n' "$(podman network inspect "$NET" --format '{{range .plugins}}{{if .subnet}}{{.subnet}} gw={{.gateway}}{{end}}{{end}}')"
|
||||||
|
timeout 60 podman run --rm --network="$NET" "$IMG" sh -c '
|
||||||
|
cat /etc/resolv.conf | sed "s/^/ resolv: /"
|
||||||
|
getent hosts example.com >/dev/null 2>&1 && echo " DNS: RESOLVED (!!! BAD)" || echo " DNS: blocked"
|
||||||
|
timeout 4 sh -c "echo > /dev/tcp/93.184.216.34/80" 2>/dev/null && echo " RAW-IP EGRESS: WORKED (!!! BAD)" || echo " RAW-IP EGRESS: blocked"
|
||||||
|
' 2>&1 | sed 's/^/ /'
|
||||||
|
|
||||||
|
say "2. default (podman) network: egress must work"
|
||||||
|
timeout 60 podman run --rm "$CURL_IMG" -sS -m6 -o /dev/null -w ' http://example.com -> HTTP %{http_code}\n' http://example.com 2>&1 | sed 's/^/ /'
|
||||||
|
|
||||||
|
say "3. container placement: where do the veths live?"
|
||||||
|
timeout 60 podman run -d --name aisb-spike-a --network="$NET" "$IMG" sleep 120 >/dev/null
|
||||||
|
timeout 60 podman run -d --name aisb-spike-b --network="$NET" "$IMG" sleep 120 >/dev/null
|
||||||
|
IP_A=$(podman inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}' aisb-spike-a | awk '{print $1}')
|
||||||
|
printf ' container A internal ip: %s\n' "$IP_A"
|
||||||
|
printf ' A -> B : %s\n' "$(probe "$NET" "$(podman inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' aisb-spike-b)")"
|
||||||
|
|
||||||
|
say "4. gate candidate A: --network=host can it reach the internal subnet?"
|
||||||
|
timeout 60 podman run --rm --network=host "$CURL_IMG" -m3 -o /dev/null "http://$IP_A:1/" 2>&1 | sed 's/^/ host-net ctr -> internal ip: /'
|
||||||
|
printf ' host-net ctr has egress : %s\n' "$(timeout 60 podman run --rm --network=host "$CURL_IMG" -sS -m6 -o /dev/null -w '%{http_code}' http://example.com 2>&1 | tail -1)"
|
||||||
|
|
||||||
|
say "5. gate candidate B: multi-attach (internal + internet) egress + DNS"
|
||||||
|
timeout 60 podman run --rm --network="$NET" --network=podman "$CURL_IMG" -sS -m6 -o /dev/null -w ' HTTP %{http_code}\n' http://example.com 2>&1 | sed 's/^/ /'
|
||||||
|
|
||||||
|
say "6. pasta option passthrough (does podman forward args to pasta?)"
|
||||||
|
timeout 60 podman run --rm --network=pasta:--map-gw "$CURL_IMG" -sS -m6 -o /dev/null -w ' pasta:--map-gw HTTP %{http_code}\n' http://example.com 2>&1 | tail -2 | sed 's/^/ /'
|
||||||
|
|
||||||
|
say "7. resource caps observable from inside"
|
||||||
|
timeout 60 podman run --rm --pids-limit 64 --memory 256m --cpus 0.5 "$IMG" sh -c \
|
||||||
|
'printf " pids.max=%s mem.max=%s\n" "$(cat /sys/fs/cgroup/pids.max 2>/dev/null)" "$(cat /sys/fs/cgroup/memory.max 2>/dev/null)"; grep -c ^processor /proc/cpuinfo | sed "s/^/ cpuinfo nproc=/"' 2>&1 | sed 's/^//'
|
||||||
|
|
||||||
|
say "8. userns keep-id: file ownership written to a host bind"
|
||||||
|
D="$(mktemp -d)"; trap 'rm -rf "$D"' EXIT
|
||||||
|
timeout 60 podman run --rm --userns=keep-id:uid=1000,gid=1000 -u 1000:1000 -v "$D:/project" -w /project "$IMG" \
|
||||||
|
sh -c 'id -u; id -g; touch /project/owned && echo ok' 2>&1 | sed 's/^/ ctr: /'
|
||||||
|
printf ' on host: %s\n' "$(stat -c '%u:%g %n' "$D/owned" 2>/dev/null || echo "NOT CREATED")"
|
||||||
|
|
||||||
|
cleanup
|
||||||
Executable
+42
@@ -0,0 +1,42 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/unit/01_args.sh -- argument handling.
|
||||||
|
source "$TESTS/lib.sh"
|
||||||
|
|
||||||
|
t_name "help / version"
|
||||||
|
sb --help; assert_rc 0 "--help exits 0"; assert_contains "NETWORK" "--help documents the network section"
|
||||||
|
sb --version; assert_rc 0 "--version exits 0"
|
||||||
|
sb version; assert_contains "ai-sandbox" "version subcommand prints the name"
|
||||||
|
|
||||||
|
t_name "flag -> harness pass-through"
|
||||||
|
A="--allow example.com" # allow mode is the default and fails closed without this
|
||||||
|
sb --dry-run $A --model sonnet; assert_rc 0 "unknown flag is accepted"; assert_contains "pi --model sonnet" "unknown flag and the rest are handed to pi"
|
||||||
|
sb --dry-run $A file.txt --more; assert_contains "pi file.txt --more" "positional starts the command"
|
||||||
|
sb --dry-run $A -- echo hi; assert_contains "echo hi" "after -- the command runs exactly as given"
|
||||||
|
assert_absent "pi echo hi" "after -- the harness is replaced, not prefixed"
|
||||||
|
assert_contains "--network pasta:--map-host-loopback" "and a --net after -- is not mistaken for a sandbox flag"
|
||||||
|
sb --dry-run $A --shell; assert_contains "bash -li" "--shell runs bash"
|
||||||
|
sb --dry-run $A --harness codex; assert_contains "codex" "--harness codex selects the command"
|
||||||
|
sb --dry-run $A --harness none; assert_contains "bash -li" "--harness none drops to a shell"
|
||||||
|
sb --dry-run --net none -- echo hi; assert_contains "echo hi" "--net= form before the command still parses"
|
||||||
|
|
||||||
|
t_name "strict args"
|
||||||
|
sb --strict-args --model sonnet; assert_rc 1 "strict mode rejects unknown flags"
|
||||||
|
assert_contains "unknown option" "strict mode says 'unknown option'"
|
||||||
|
sb --strict-args --al foo; assert_contains "did you mean --allow" "strict mode suggests a near miss"
|
||||||
|
|
||||||
|
t_name "value validation"
|
||||||
|
sb --net bogus --dry-run; assert_rc 1 "bad --net rejected"; assert_contains "--net must be none, allow or full" "names the valid values"
|
||||||
|
sb --pids abc --dry-run; assert_rc 1 "bad --pids rejected"
|
||||||
|
sb --cpus x4 --dry-run; assert_rc 1 "bad --cpus rejected"
|
||||||
|
sb --memory huge --dry-run; assert_rc 1 "bad --memory rejected"
|
||||||
|
sb --allow; assert_rc 1 "--allow with no value"
|
||||||
|
sb --allow-file /nope/missing; assert_rc 1 "--allow-file must exist"
|
||||||
|
sb --mount /nope/missing --dry-run; assert_rc 1 "--mount path must exist"
|
||||||
|
|
||||||
|
t_name "fail-closed network policy"
|
||||||
|
sb --dry-run; assert_rc 1 "allow mode with no allowlist refuses to start"
|
||||||
|
assert_contains "Refusing to start wide open" "explains that it refuses to be wide open"
|
||||||
|
sb --dry-run --net none; assert_rc 0 "--net none needs no allowlist"; assert_contains "--network none" "--net none maps to podman --network none"
|
||||||
|
sb --dry-run --net full; assert_rc 0 "--net full is allowed (explicit opt-in)"
|
||||||
|
assert_contains "unrestricted internet" "--net full warns"
|
||||||
|
sb --dry-run --allow example.com; assert_rc 0 "one allow entry is enough"
|
||||||
Executable
+78
@@ -0,0 +1,78 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/unit/02_config.sh -- config files, allowlists and precedence.
|
||||||
|
source "$TESTS/lib.sh"
|
||||||
|
|
||||||
|
CFG="$WORK/cfg"
|
||||||
|
REPO="$WORK/repo"
|
||||||
|
mkdir -p "$CFG" "$REPO/.ai-sandbox"
|
||||||
|
export AISB_CONFIG_DIR="$CFG"
|
||||||
|
|
||||||
|
cd "$REPO" || exit 1
|
||||||
|
|
||||||
|
t_name "allow files are merged"
|
||||||
|
cat > "$CFG/allow" <<'EOF'
|
||||||
|
# user-wide allowlist
|
||||||
|
api.openai.com
|
||||||
|
|
||||||
|
registry.npmjs.org # trailing comment
|
||||||
|
EOF
|
||||||
|
cat > "$REPO/.ai-sandbox/allow" <<'EOF'
|
||||||
|
pypi.org
|
||||||
|
EOF
|
||||||
|
sb --print-policy --net allow
|
||||||
|
assert_rc 0 "print-policy runs"
|
||||||
|
assert_contains "api.openai.com" "user allowlist entry present"
|
||||||
|
assert_contains "registry.npmjs.org" "user entry with trailing comment present"
|
||||||
|
assert_contains "pypi.org" "repo allowlist entry present"
|
||||||
|
|
||||||
|
t_name "dedup"
|
||||||
|
sb --print-policy --net allow --allow pypi.org --allow pypi.org
|
||||||
|
ALLOW_LINE="$(printf '%s' "$OUT" | grep '^allow=')"
|
||||||
|
N="$(printf '%s' "$ALLOW_LINE" | grep -o 'pypi.org' | wc -l | tr -d ' ')"
|
||||||
|
assert_eq "1" "$N" "duplicate allow entries collapse into one"
|
||||||
|
|
||||||
|
t_name "key = value config"
|
||||||
|
cat > "$CFG/config" <<'EOF'
|
||||||
|
net = none
|
||||||
|
harness = none
|
||||||
|
cpus = 7
|
||||||
|
memory = 1536m
|
||||||
|
pids = 77
|
||||||
|
selinux = :Z
|
||||||
|
EOF
|
||||||
|
sb --print-policy
|
||||||
|
assert_contains "net=none" "config file sets net"
|
||||||
|
assert_contains "harness=none" "config file sets harness"
|
||||||
|
assert_contains "cpus:7" "config file sets cpus"
|
||||||
|
assert_contains "memory:1536m" "config file sets memory"
|
||||||
|
assert_contains "pids:77" "config file sets pids"
|
||||||
|
assert_contains "selinux=:Z" "config file sets selinux"
|
||||||
|
|
||||||
|
t_name "precedence: flag > env > config file"
|
||||||
|
sb --print-policy --net allow
|
||||||
|
assert_contains "net=allow" "flag beats config file"
|
||||||
|
sb_env AISB_NET=full --print-policy
|
||||||
|
assert_contains "net=full" "AISB_* env beats config file"
|
||||||
|
sb_env AISB_NET=full --print-policy --net none
|
||||||
|
assert_contains "net=none" "flag beats AISB_* env"
|
||||||
|
|
||||||
|
t_name "AISB_ALLOW / AISB_ENV lists"
|
||||||
|
sb_env AISB_ALLOW=a.example.com,b.example.com:8443 --print-policy --net none
|
||||||
|
assert_contains "a.example.com" "AISB_ALLOW entry 1"
|
||||||
|
assert_contains "b.example.com:8443" "AISB_ALLOW entry 2 with port"
|
||||||
|
sb_env AISB_ENV=HOME_X,TMPDIR --print-policy --net none
|
||||||
|
assert_contains "env_names=HOME_X TMPDIR" "AISB_ENV populates env names"
|
||||||
|
assert_contains "env_missing=HOME_X" "unset names are reported by --print-policy"
|
||||||
|
|
||||||
|
t_name "repo config wins per-directory"
|
||||||
|
cat > "$REPO/.ai-sandbox/config" <<'EOF'
|
||||||
|
pids = 42
|
||||||
|
EOF
|
||||||
|
sb --print-policy --net none
|
||||||
|
assert_contains "pids:42" "repo config overrides user config"
|
||||||
|
|
||||||
|
t_name "unknown keys warn but do not break"
|
||||||
|
echo 'bogus_key = 1' >> "$CFG/config"
|
||||||
|
sb --print-policy --net none
|
||||||
|
assert_contains "unknown key: bogus_key" "unknown key warns"
|
||||||
|
assert_rc 0 "unknown key is not fatal"
|
||||||
Executable
+86
@@ -0,0 +1,86 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/unit/03_argv.sh -- the policy actually reaches the podman invocation.
|
||||||
|
source "$TESTS/lib.sh"
|
||||||
|
|
||||||
|
PROJ="$WORK/proj"; mkdir -p "$PROJ"; cd "$PROJ" || exit 1
|
||||||
|
export AISB_CONFIG_DIR="$WORK/cfg-empty"; mkdir -p "$AISB_CONFIG_DIR"
|
||||||
|
A="--allow example.com"
|
||||||
|
|
||||||
|
t_name "isolation flags are always on"
|
||||||
|
sb --dry-run $A
|
||||||
|
assert_contains "--cap-drop=ALL" "all capabilities dropped"
|
||||||
|
assert_contains "no-new-privileges" "no new privileges"
|
||||||
|
assert_contains "--userns keep-id" "uid mapped back to the host user"
|
||||||
|
assert_contains "--pids-limit 512" "pid limit"
|
||||||
|
assert_contains "--memory 4g" "memory limit"
|
||||||
|
assert_contains "--cpus 4" "cpu limit"
|
||||||
|
assert_contains "/tmp:rw,nosuid,nodev" "/tmp is a nosuid tmpfs"
|
||||||
|
assert_contains "--hostname aisb" "hostname"
|
||||||
|
assert_contains "--rm" "container is removed on exit"
|
||||||
|
assert_contains "--stop-signal SIGTERM" "clean shutdown signal"
|
||||||
|
|
||||||
|
t_name "network modes"
|
||||||
|
sb --dry-run --net none
|
||||||
|
assert_contains "--network none" "none"
|
||||||
|
sb --dry-run --net full
|
||||||
|
assert_contains "--network pasta" "full uses pasta with no filter"
|
||||||
|
assert_absent "--cap-add=NET_ADMIN" "full mode needs no NET_ADMIN"
|
||||||
|
sb --dry-run $A
|
||||||
|
assert_contains "--network pasta:--map-host-loopback,100.64.0.1" "allow mode maps a host-loopback door for the gate"
|
||||||
|
assert_contains "--cap-add=NET_ADMIN" "allow mode grants NET_ADMIN to the entrypoint only"
|
||||||
|
assert_contains "HTTP_PROXY=http://100.64.0.1:18081" "proxy env points at the gate"
|
||||||
|
assert_contains "NO_PROXY=localhost" "localhost is never proxied"
|
||||||
|
sb --dry-run $A --proxy 10.9.8.7:3128
|
||||||
|
assert_contains "HTTP_PROXY=http://10.9.8.7:3128" "--proxy overrides the gate"
|
||||||
|
sb --dry-run $A --no-gate
|
||||||
|
assert_absent "HTTP_PROXY=" "--no-gate sets no proxy"
|
||||||
|
|
||||||
|
t_name "filesystem"
|
||||||
|
sb --dry-run $A
|
||||||
|
assert_contains "$PROJ:/project" "cwd is mounted at /project"
|
||||||
|
assert_contains "--workdir /project" "cwd is the working directory"
|
||||||
|
assert_contains "/run/aisb:ro" "policy dir is mounted read-only"
|
||||||
|
sb --dry-run $A --no-cwd
|
||||||
|
assert_absent "$PROJ:/project" "--no-cwd mounts nothing"
|
||||||
|
sb --dry-run $A --workdir /src
|
||||||
|
assert_contains "$PROJ:/src" "--workdir changes the mount point"
|
||||||
|
mkdir -p "$WORK/extra"
|
||||||
|
sb --dry-run $A --mount "$WORK/extra"
|
||||||
|
assert_contains "$WORK/extra:/mnt/extra" "--mount defaults to /mnt/<name>"
|
||||||
|
sb --dry-run $A --mount "$WORK/extra:/ex:ro"
|
||||||
|
assert_contains "$WORK/extra:/ex" "--mount with explicit destination"
|
||||||
|
assert_contains "ro" " ... and ro option present"
|
||||||
|
sb --dry-run $A --read-only
|
||||||
|
assert_contains "--read-only" "--read-only"
|
||||||
|
sb --dry-run $A
|
||||||
|
assert_contains "type=volume,source=aisb-home,target=/home/sandbox" "home cache volume by default"
|
||||||
|
sb --dry-run $A --ephemeral-home
|
||||||
|
assert_contains "/home/sandbox:rw,nosuid,nodev" "--ephemeral-home uses a tmpfs home"
|
||||||
|
assert_absent "type=volume,source=aisb-home" "--ephemeral-home drops the volume"
|
||||||
|
|
||||||
|
t_name "labels, limits, extra args"
|
||||||
|
sb --dry-run $A --label team=x
|
||||||
|
assert_contains "--label team=x" "--label"
|
||||||
|
assert_contains "aisb.net=allow" "session label"
|
||||||
|
sb --dry-run $A --cpus 12 --memory 8g --pids 99
|
||||||
|
assert_contains "--cpus 12" "custom cpus"
|
||||||
|
assert_contains "--memory 8g" "custom memory"
|
||||||
|
assert_contains "--pids-limit 99" "custom pids"
|
||||||
|
sb_env "AISB_EXTRA_ARGS=--label from=extra" --dry-run $A
|
||||||
|
assert_contains "--label from=extra" "AISB_EXTRA_ARGS is appended"
|
||||||
|
|
||||||
|
t_name "everything lands before the image; the command comes last"
|
||||||
|
sb --dry-run $A --label zzz=1 -- echo hello
|
||||||
|
line="$(printf '%s' "$OUT" | tail -1)"
|
||||||
|
pos_img="$(printf '%s' "$line" | tr ' ' '\n' | grep -n -m1 'localhost/ai-sandbox' | cut -d: -f1)"
|
||||||
|
pos_lbl="$(printf '%s' "$line" | tr ' ' '\n' | grep -n -m1 'zzz=1' | cut -d: -f1)"
|
||||||
|
pos_cmd="$(printf '%s' "$line" | tr ' ' '\n' | grep -n -m1 '^hello$' | cut -d: -f1)"
|
||||||
|
if [[ -n "$pos_img" && -n "$pos_lbl" && $pos_lbl -lt $pos_img ]]; then _ok "extra flags precede the image"; else _bad "extra flags precede the image" "img=$pos_img label=$pos_lbl"; fi
|
||||||
|
if [[ -n "$pos_cmd" && -n "$pos_img" && $pos_img -lt $pos_cmd ]]; then _ok "the command follows the image"; else _bad "the command follows the image" "img=$pos_img cmd=$pos_cmd"; fi
|
||||||
|
|
||||||
|
t_name "dry run is side-effect free"
|
||||||
|
sb --dry-run $A
|
||||||
|
RUNNING="$(podman ps --format '{{.Names}}' | grep -c 'aisb' || true)"
|
||||||
|
DIRS="$(ls -A "$AISB_STATE_DIR/sessions" 2>/dev/null | wc -l | tr -d ' ')"
|
||||||
|
assert_eq "$RUNNING" 0 "dry run started no container"
|
||||||
|
assert_eq "$DIRS" 0 "dry run left no session directory behind"
|
||||||
Executable
+65
@@ -0,0 +1,65 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/unit/04_env.sh -- environment passing and secret hygiene.
|
||||||
|
source "$TESTS/lib.sh"
|
||||||
|
|
||||||
|
PROJ="$WORK/proj"; mkdir -p "$PROJ"; cd "$PROJ" || exit 1
|
||||||
|
export AISB_CONFIG_DIR="$WORK/cfg-empty"; mkdir -p "$AISB_CONFIG_DIR"
|
||||||
|
export AISB_CANARY=hunter2-secret-value
|
||||||
|
export AISB_EMPTY_VAR=""
|
||||||
|
unset AISB_NOT_SET
|
||||||
|
|
||||||
|
t_name "deny by default"
|
||||||
|
sb --print-policy --net none
|
||||||
|
assert_eq "env_names=" "$(printf '%s' "$OUT" | grep '^env_names=')" "nothing is passed by default"
|
||||||
|
assert_absent "CANARY" "a host variable is not passed unless asked for"
|
||||||
|
|
||||||
|
t_name "explicit passing"
|
||||||
|
sb --print-policy --net none --env AISB_CANARY
|
||||||
|
assert_contains "env_names=AISB_CANARY" "named variable is queued"
|
||||||
|
assert_absent "hunter2" "the value never appears in --print-policy output"
|
||||||
|
sb --dry-run --net none --env AISB_CANARY
|
||||||
|
assert_absent "hunter2" "the value never appears in the printed podman invocation"
|
||||||
|
assert_contains "<env-file:0600>" "values travel through a 0600 env file, not the command line"
|
||||||
|
|
||||||
|
t_name "set-but-empty passes, unset fails closed"
|
||||||
|
sb --print-policy --net none --env AISB_EMPTY_VAR
|
||||||
|
assert_contains "env_names=AISB_EMPTY_VAR" "empty value still passes"
|
||||||
|
assert_absent "env_missing=AISB_EMPTY_VAR" "empty value is not 'missing'"
|
||||||
|
sb --print-policy --net none --env AISB_NOT_SET
|
||||||
|
assert_rc 0 "--print-policy warns instead of dying"
|
||||||
|
assert_contains "not set in the host environment" "the warning names the problem"
|
||||||
|
assert_contains "env_missing=AISB_NOT_SET" "env_missing reports it"
|
||||||
|
|
||||||
|
t_name "literal values"
|
||||||
|
sb --dry-run --net none --env FOO=bar
|
||||||
|
assert_contains "<env-file:0600>" "literal values also go through the env file"
|
||||||
|
assert_absent "bar" "literal values are not echoed into the invocation"
|
||||||
|
|
||||||
|
t_name "env files"
|
||||||
|
cat > "$WORK/envfile" <<'EOF'
|
||||||
|
# comment
|
||||||
|
AISENV_ONE=1
|
||||||
|
AISENV_TWO=with spaces
|
||||||
|
AISENV_THREE=has=equals
|
||||||
|
EOF
|
||||||
|
chmod 600 "$WORK/envfile"
|
||||||
|
sb --print-policy --net none --env-file "$WORK/envfile"
|
||||||
|
assert_contains "env_names=AISENV_ONE AISENV_TWO AISENV_THREE" "all three names queued"
|
||||||
|
sb_env AISENV_ONE=1 --print-policy --net none --env AISB_CANARY --env-file "$WORK/envfile"
|
||||||
|
assert_contains "AISENV_THREE" "env file entries pass"
|
||||||
|
assert_contains "AISB_CANARY" "and combine with --env"
|
||||||
|
|
||||||
|
t_name "loose env-file permissions warn"
|
||||||
|
chmod 644 "$WORK/envfile"
|
||||||
|
sb --print-policy --net none --env-file "$WORK/envfile"
|
||||||
|
assert_contains "chmod 600 recommended" "world-readable env file warns"
|
||||||
|
chmod 600 "$WORK/envfile"
|
||||||
|
|
||||||
|
t_name "invalid names and values"
|
||||||
|
sb --print-policy --net none --env 9BAD
|
||||||
|
assert_rc 1 "invalid variable name rejected"
|
||||||
|
sb --print-policy --net none --env-file "$WORK/does-not-exist"
|
||||||
|
assert_rc 1 "missing env file rejected"
|
||||||
|
export AISB_MULTILINE=$'one\ntwo'
|
||||||
|
sb --print-policy --net none --env AISB_MULTILINE
|
||||||
|
assert_rc 1 "multi-line values are rejected rather than silently truncated"
|
||||||
Executable
+48
@@ -0,0 +1,48 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/unit/05_policy.sh -- how allow entries are classified.
|
||||||
|
source "$TESTS/lib.sh"
|
||||||
|
|
||||||
|
PROJ="$WORK/proj"; mkdir -p "$PROJ"; cd "$PROJ" || exit 1
|
||||||
|
export AISB_CONFIG_DIR="$WORK/cfg-empty"; mkdir -p "$AISB_CONFIG_DIR"
|
||||||
|
|
||||||
|
direct_line() { printf '%s' "$1" | grep '^direct=' | cut -d= -f2-; }
|
||||||
|
|
||||||
|
t_name "domains go to the gate, not to netfilter"
|
||||||
|
sb --print-policy --net allow --allow api.openai.com
|
||||||
|
assert_contains "gate_domains=api.openai.com" "domain listed for the gate"
|
||||||
|
assert_eq "" "$(direct_line "$OUT")" "no netfilter entry for a domain in gate mode"
|
||||||
|
assert_contains "proxy=100.64.0.1:18081" "gate address is the proxy"
|
||||||
|
|
||||||
|
t_name "IP and CIDR entries become direct rules"
|
||||||
|
sb --print-policy --net allow --allow 10.0.0.5:5432
|
||||||
|
assert_eq "direct=10.0.0.5 5432" "$(printf '%s' "$OUT" | grep '^direct=')" "explicit host:port"
|
||||||
|
sb --print-policy --net allow --allow 10.1.0.0/16
|
||||||
|
assert_eq "direct=10.1.0.0/16 80,443" "$(printf '%s' "$OUT" | grep '^direct=')" "CIDR gets the default ports"
|
||||||
|
sb --print-policy --net allow --allow 10.0.0.5:5432 --allow 10.9.9.9
|
||||||
|
assert_eq "direct=10.0.0.5 5432;10.9.9.9 80,443" "$(printf '%s' "$OUT" | grep '^direct=')" "several direct entries"
|
||||||
|
|
||||||
|
t_name "ipv6"
|
||||||
|
sb --print-policy --net allow --allow "[::1]:8080"
|
||||||
|
assert_eq "direct6=::1 8080" "$(printf '%s' "$OUT" | grep '^direct6=')" "bracketed [ipv6]:port parses"
|
||||||
|
assert_eq "direct=" "$(printf '%s' "$OUT" | grep '^direct=')" "and stays out of the ipv4 rules"
|
||||||
|
assert_contains "gate_domains=" "policy printed for an ipv6 entry"
|
||||||
|
|
||||||
|
t_name "--no-gate resolves names up front"
|
||||||
|
sb --print-policy --net allow --no-gate --allow localhost
|
||||||
|
assert_contains "direct=127.0.0.1 80,443" "localhost resolves to a direct rule"
|
||||||
|
assert_contains "proxy=none" "--no-gate has no proxy"
|
||||||
|
sb --print-policy --net allow --no-gate --allow nope.invalid
|
||||||
|
assert_rc 1 "an unresolvable name is an error, not a silent no-op"
|
||||||
|
assert_contains "could not resolve" "and it says so"
|
||||||
|
|
||||||
|
t_name "ports"
|
||||||
|
sb --allow api.openai.com:not-a-port --print-policy
|
||||||
|
assert_rc 1 "non-numeric port rejected"
|
||||||
|
sb --print-policy --net allow --allow api.openai.com:8443
|
||||||
|
assert_contains "gate_domains=api.openai.com" "domain+port still routed through the gate"
|
||||||
|
|
||||||
|
t_name "mixed list"
|
||||||
|
sb --print-policy --net allow --allow api.openai.com --allow 10.0.0.5:5432 --allow files.pythonhosted.org
|
||||||
|
assert_contains "api.openai.com" "domain 1"
|
||||||
|
assert_contains "files.pythonhosted.org" "domain 2"
|
||||||
|
assert_contains "10.0.0.5 5432" "ip entry"
|
||||||
Executable
+71
@@ -0,0 +1,71 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tests/unit/06_entrypoint.sh -- the container-side enforcement script.
|
||||||
|
source "$TESTS/lib.sh"
|
||||||
|
|
||||||
|
PROJ="$WORK/proj"; mkdir -p "$PROJ"; cd "$PROJ" || exit 1
|
||||||
|
export AISB_CONFIG_DIR="$WORK/cfg-empty"; mkdir -p "$AISB_CONFIG_DIR"
|
||||||
|
|
||||||
|
# Extract the entrypoint exactly as it is baked into the image.
|
||||||
|
EP="$WORK/entrypoint.sh"
|
||||||
|
"$AISB" --print-entrypoint > "$EP" 2>"$WORK/ep.err"; RC=$?
|
||||||
|
assert_eq 0 "$RC" "the embedded entrypoint can be printed"
|
||||||
|
bash -n "$EP" && _ok "entrypoint is syntactically valid bash" || _bad "entrypoint is syntactically valid bash"
|
||||||
|
grep -q 'set -Eeuo pipefail' "$EP" && _ok "entrypoint is strict" || _bad "entrypoint is strict"
|
||||||
|
|
||||||
|
t_name "the image is described by the script itself"
|
||||||
|
sb --print-containerfile
|
||||||
|
assert_contains "FROM docker.io/library/ubuntu:24.04" "ubuntu 24.04 base"
|
||||||
|
assert_contains "nftables" "image ships nftables"
|
||||||
|
assert_contains "util-linux" "image ships setpriv (util-linux)"
|
||||||
|
assert_contains "sha256sum -c" "node tarball is checksum verified"
|
||||||
|
assert_contains "useradd" "image creates the sandbox user"
|
||||||
|
assert_contains 'ENTRYPOINT ["/usr/local/lib/aisb/entrypoint.sh"]' "entrypoint is baked in"
|
||||||
|
assert_contains "WORKDIR /project" "project is the default workdir"
|
||||||
|
|
||||||
|
t_name "generated ruleset shape"
|
||||||
|
# Run the rule generator outside a container (nft is not needed to render it).
|
||||||
|
POL="$WORK/mnt"; mkdir -p "$POL"
|
||||||
|
cat > "$POL/policy.env" <<EOF
|
||||||
|
AISB_NET=allow
|
||||||
|
AISB_UID=1000
|
||||||
|
AISB_GID=1000
|
||||||
|
AISB_UNENFORCED=0
|
||||||
|
AISB_KEEP_ROOT=0
|
||||||
|
AISB_ALLOW_ICMP=1
|
||||||
|
AISB_ALLOW_DNS=0
|
||||||
|
AISB_PROXY_IPS=100.64.0.1
|
||||||
|
AISB_PROXY_PORTS=18081
|
||||||
|
EOF
|
||||||
|
printf '10.0.0.5 5432\n10.1.0.0/16 80,443\n' > "$POL/direct4.list"
|
||||||
|
printf '::1 8080\n' > "$POL/direct6.list"
|
||||||
|
|
||||||
|
RULES="$AISB_STATE_DIR/rules.f"
|
||||||
|
# Render the ruleset by extracting the generator and feeding it the policy above.
|
||||||
|
GEN="$WORK/gen.sh"
|
||||||
|
awk '/^gen_rules\(\) \{/{f=1} f{print} f&&/^\}$/{exit}' "$EP" > "$GEN"
|
||||||
|
{ printf 'POLICY_DIR=%q\nAISB_PROXY_IPS="100.64.0.1"\nAISB_PROXY_PORTS="18081"\nAISB_ALLOW_ICMP=1\nAISB_ALLOW_DNS=0\n' "$POL"
|
||||||
|
printf 'gen_rules\n'; } >> "$GEN"
|
||||||
|
bash "$GEN" > "$RULES" 2>"$WORK/gen.err" && _ok "rules render" || _bad "rules render" "$(tail -3 "$WORK/gen.err")"
|
||||||
|
|
||||||
|
grep -q 'table inet aisb' "$RULES" && _ok "inet table (covers v4 and v6)" || _bad "inet table"
|
||||||
|
grep -q 'policy drop' "$RULES" && _ok "default policy is drop" || _bad "default policy drop"
|
||||||
|
grep -q 'aisb-default-drop' "$RULES" && _ok "explicit default-drop rule exists" || _bad "default-drop rule"
|
||||||
|
grep -q 'oifname "lo" accept' "$RULES" && _ok "loopback stays usable" || _bad "loopback accept"
|
||||||
|
grep -q 'ip daddr 10.0.0.5 tcp dport { 5432 }' "$RULES" && _ok "per-entry host:port rule" || _bad "host:port rule: $(grep -c . "$RULES") lines"
|
||||||
|
grep -q 'ip daddr 10.1.0.0/16 tcp dport { 80,443 }' "$RULES" && _ok "CIDR rule with default ports" || _bad "CIDR rule"
|
||||||
|
grep -q 'ip6 daddr ::1 tcp dport { 8080 }' "$RULES" && _ok "ipv6 rule" || _bad "ipv6 rule"
|
||||||
|
grep -q 'ip daddr 100.64.0.1 tcp dport 18081' "$RULES" && _ok "gate endpoint is allowed" || _bad "gate rule"
|
||||||
|
grep -q 'aisb-est' "$RULES" && _ok "established traffic accepted" || _bad "established rule"
|
||||||
|
grep -c 'accept comment "aisb-direct"' "$RULES" | grep -qx 3 && _ok "exactly 3 direct rules" || _bad "direct rule count: $(grep -c 'aisb-direct' "$RULES")"
|
||||||
|
|
||||||
|
t_name "no DNS hole when the gate is in front"
|
||||||
|
grep -E 'th dport 53' "$RULES" >/dev/null && _bad "DNS is closed in gate mode" || _ok "DNS is closed in gate mode"
|
||||||
|
|
||||||
|
t_name "privilege drop is the last thing that happens"
|
||||||
|
tail -5 "$EP" | grep -q 'setpriv' && _ok "exec setpriv at the end" || _bad "setpriv last: $(tail -3 "$EP" | tr '\n' ' ')"
|
||||||
|
grep -q -- '--bounding-set=-all' "$EP" && _ok "bounding set cleared" || _bad "bounding set cleared"
|
||||||
|
grep -q -- '--inh-caps=-all' "$EP" && _ok "inheritable caps cleared" || _bad "inheritable caps cleared"
|
||||||
|
grep -q -- '--no-new-privs' "$EP" && _ok "no new privileges" || _bad "no-new-privs"
|
||||||
|
grep -q -- '--reuid=' "$EP" && _ok "uid switched" || _bad "reuid"
|
||||||
|
grep -q 'fail "policy file' "$EP" && _ok "missing policy = refuse to start" || _bad "fail closed on missing policy"
|
||||||
|
grep -q 'refusing to start\|Refusing to run unenforced' "$EP" && _ok "missing NET_ADMIN = refuse to start" || _bad "fail closed without NET_ADMIN"
|
||||||
Reference in New Issue
Block a user