diff --git a/AGENTS.md b/AGENTS.md index 261d018..4f70c97 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -105,6 +105,11 @@ networking or the image, `make e2e` green too. * `entrypoint_script()` — the enforcement half. It is rendered into the image and runs as root there. A bug here is a security bug. `tests/unit/06_entrypoint.sh` checks the generated rule shape; `tests/e2e/11_net.sh` checks it in a container. + Its pi-seeding block is passed to `bash -c` inside single quotes, so a single quote + anywhere in there silently closes the argument and the rest runs in the *outer* shell + — valid bash, wrong program, clean `bash -n` and clean shellcheck. Nothing in it may + use `'`; jq programs there are written unquoted (`.defaultThinkingLevel=\$lvl`), and + `tests/unit/06_entrypoint.sh` fails the build if that ever changes. * `split_target()` / `classify_allows()` — the mapping from what the user typed to what the firewall and the proxy enforce. * `build_argv()` — every isolation flag lives here. diff --git a/PLAN.md b/PLAN.md index e6cf0a1..5b1c0ed 100644 --- a/PLAN.md +++ b/PLAN.md @@ -127,6 +127,33 @@ the same destination wins because podman refuses duplicates. A missing or unread directory is a silent skip like the pi bridge's; an explicit `--agents-dir` that resolves to nothing is a hard error. +### 11 — Thinking level as policy, and a settings merge that does not freeze +`defaultThinkingLevel` is now `xhigh` by default (`thinking`, `--thinking`, +`AISB_THINKING`; pi's own default is `medium`), written into the bridged settings and +re-applied by the entrypoint on every run, so a run cannot lower the reasoning of the runs +after it. `--thinking host` hands the key back to `settings.json`, `off` is pi's own +"no reasoning" level, and an unknown value is refused with the list to pick from. Nothing +is written for codex, claude or a command after `--`, and `--no-pi-config` declines the +policy along with the bridge. + +Getting there found a worse bug. `settings.json` was managed as one managed file, and pi +rewrites its own settings (a theme change, "seen the changelog"), so the first such write +matched nothing and froze *every* host preference out of the sandbox for good -- provider, model, +thinking budgets and the level included, with a real session left holding +`{"lastChangelogVersion": "1.4.0"}`. It is merged per key instead, against a copy of what +we last wrote (`.aisb-settings.json.wrote`): sandbox edits win per key, untouched keys +follow the host again, untouched keys the host dropped are dropped, keys pi invented +survive, and a file that is missing a base copy takes the host values -- which is what +repairs the volumes already frozen. Unparseable JSON is re-seeded with a warning (pi +cannot use it either); a file the container cannot *read* is reported with the fix, because +nothing inside the sandbox can repair it. + +One trap worth remembering: the seeding script is passed to `bash -c` in single quotes, so +a single quote anywhere inside it closes the argument early and the remainder becomes code +in the outer shell -- valid bash, wrong program, invisible to `bash -n` and shellcheck. +`tests/unit/06_entrypoint.sh` now extracts the seeding script and asserts it is quote-free, +and extracts the merge program to test its semantics without a container. + ## Follow-ups (not yet done) * **`--net unix` mode**: `--network=none` plus a unix-socket proxy and an in-container @@ -150,6 +177,9 @@ to nothing is a hard error. * **Scoped sandbox tokens**: if the gateway can mint a short-lived, per-sandbox key, `--pi-token-env` should ask for one rather than forwarding the long-lived host credential. The bridge deliberately does not guess a minting endpoint. +* **`--thinking` for the other harnesses**: codex and claude have their own reasoning + knobs; a policy that spans harnesses needs to know each one's config shape rather than + assume pi's. * **Bridging extensions/skills on request**: `--pi-mount-ext` or similar for the extension paths named in `enabledExtensionPaths`, which are host paths and therefore currently dropped; needs the same explicit-grant shape as `--mount`. diff --git a/README.md b/README.md index 48a6254..9be1982 100644 --- a/README.md +++ b/README.md @@ -154,14 +154,41 @@ purpose is to be unable to use it: | `apiKey: "sk-..."` or `apiKey: "!pass show ..."` | `apiKey: "$AI_API_TOKEN"` -- pi resolves it itself, no value is written to disk | | a header like `X-Trace: "!date +%s"` | dropped, and reported: a host command is not run in the sandbox | | `baseUrl: "https://aipi.reeseapps.com/v1"` | unchanged, along with every model, limit and thinking level | -| settings: provider, model, theme, thinking budgets, telemetry | unchanged | +| settings: provider, model, theme, thinking budgets, telemetry | unchanged, merged per key | | a settings value pointing into `~` | dropped, and reported: that path is not mounted | | `auth.json`, `models-store.json`, sessions, extensions | never touched | The rewritten pair is seeded into `/home/sandbox/.pi/agent` at container start, owned by -your uid, mode `600`. It is a managed file: a change on the host is picked up on the -next run, an edit made *inside* the sandbox is left alone, and deleting it inside -re-seeds from the host. +your uid, mode `600`. The two files are managed differently, because they are different +kinds of thing: + +| file | rule | why | +|---|---|---| +| `models.json` | replaced when missing or unchanged, left alone once you edit it in the sandbox | it is a catalog; a partial merge of a provider list is worse than either choice | +| `settings.json` | **merged per key** | pi rewrites its own settings — a theme change, "seen the changelog" — and managing the file whole meant that one write froze *every* host preference out of the sandbox permanently | + +So in `settings.json`: a key you change inside the sandbox stays yours, a key you never +touched follows the host again, a key the host drops disappears unless you had changed +it, and a key pi added (`lastChangelogVersion`) survives. + +### Thinking is xhigh, and it is policy + +`defaultThinkingLevel` is the one setting `ai-sandbox` insists on, because pi's own +default is `medium` and a sandbox run is usually the kind of work you want reasoned +about hard: + +```bash +pi --allow aipi.reeseapps.com # xhigh, whatever your host settings.json says +pi --thinking max ... # this run only +pi --thinking off ... # no reasoning, e.g. a mechanical refactor at scale +pi --thinking host ... # stop forcing it; your settings.json decides +``` + +`--thinking` is a policy, not a preference: it is written over the merged settings on +every run, so a run cannot set its own thinking level down for the runs after it. It is +`pi`-only — codex, claude and anything you put after `--` read their own config, and +ai-sandbox leaves that alone. `--no-pi-config` declines the policy too, since "pi starts +with its own defaults" is what that flag is for. The token itself follows the normal rule -- it is the one variable the bridged config names, so it is passed by name (never by value, and never into `ps`): diff --git a/bin/ai-sandbox b/bin/ai-sandbox index 77e3eee..1b377c4 100755 --- a/bin/ai-sandbox +++ b/bin/ai-sandbox @@ -42,6 +42,7 @@ AISB_PI_PACKAGE="${AISB_PI_PACKAGE:-@earendil-works/pi-coding-agent}" # AISB_TMPFS_SIZE AISB_BASE_IMAGE AISB_NODE_VERSION AISB_PI_VERSION # AISB_BASE_TTL_DAYS AISB_BASE_AUTO_UPDATE AISB_BASE_PULL_FALLBACK AISB_SKOPEO # AISB_SELINUX AISB_GATE_ADDR AISB_GATE_PORT AISB_HOME_CACHE AISB_EXTRA_ARGS +# AISB_PI_CONFIG AISB_PI_TOKEN_ENV AISB_PI_TOKEN_AUTO AISB_PI_AGENT_DIR AISB_THINKING # AISB_AGENTS AISB_AGENTS_DIR AISB_AGENTS_DEST AISB_UID="${AISB_UID:-1000}" AISB_GID="${AISB_GID:-1000}" @@ -233,6 +234,17 @@ PI CONFIG --no-pi-token-env Do not pass it automatically; then pass it with --env or the run is refused rather than starting a pi with no key. --no-pi-config Do not bridge at all; pi starts with its own defaults. + --thinking LEVEL Default thinking level in the sandbox: off, minimal, low, + medium, high, xhigh (the default) or host. It is policy, not + a preference: it is written over the bridged settings on + every run, so an edit inside the sandbox cannot quietly lower + it for the next one. 'host' leaves the key to your + settings.json. pi only -- codex, claude and your own command + read their own config and ai-sandbox does not touch it. + + settings.json is merged per key rather than managed as a whole file: keys you edit + inside the sandbox stay yours, every key you did not touch follows the host again, keys + the host dropped go away, and keys the sandbox invented survive. LIMITS --cpus N (4) --memory SIZE (4g) --pids N (512) --tmpfs-size SIZE (1g) @@ -274,7 +286,7 @@ CONFIG \$HOME/.config/ai-sandbox/{config,allow} and ./.ai-sandbox/{config,allow} Every default can also be set as an environment variable: AISB_NET, AISB_ALLOW (comma list), AISB_ENV (comma list), AISB_CPUS, AISB_MEMORY, - AISB_PIDS, AISB_HARNESS, AISB_EXTRA_ARGS, AISB_SELINUX, AISB_HOME_CACHE, + AISB_PIDS, AISB_HARNESS, AISB_EXTRA_ARGS, AISB_SELINUX, AISB_HOME_CACHE, AISB_THINKING, AISB_AGENTS, AISB_AGENTS_DIR, AISB_AGENTS_DEST, ... Base images: AISB_BASE_IMAGE, AISB_BASE_TTL_DAYS (7; 0 every build, -1 never), AISB_BASE_AUTO_UPDATE (0 keeps the recorded digest), AISB_BASE_PULL_FALLBACK @@ -315,7 +327,7 @@ cfg_read_kv() { # `key = value` file case "$key" in net|harness|workdir|cpus|memory|pids|tmpfs_size|base_image|node_version|pi_version|\ project_dest|selinux|home_cache|gate_addr|gate_port|extra_args|base_ttl_days|base_auto_update|\ - pi_config|pi_token_env|pi_token_auto|pi_agent_dir|agents|agents_dir|agents_dest) CFG["$key"]="$val" ;; + pi_config|pi_token_env|pi_token_auto|pi_agent_dir|thinking|agents|agents_dir|agents_dest) CFG["$key"]="$val" ;; *) warn "$file: unknown key: $key" ;; esac done < "$file" @@ -354,7 +366,7 @@ OPT_NO_CWD=0 OPT_SHELL=0 OPT_DRY=0 OPT_PRINT=0 OPT_NO_BUILD=0 OPT_KEEP=0 OPT_REA OPT_HOME_CACHE="" OPT_STRICT="${AISB_STRICT_ARGS:-0}" OPT_UNENFORCED=0 OPT_KEEP_ROOT=0 OPT_TTY="" OPT_INTERACTIVE="" OPT_PRINT_EP=0 OPT_PRINT_CF=0 OPT_PASSTHROUGH=0 OPT_AGENTS="" OPT_AGENTS_DIR="" -OPT_PI_CONFIG="" OPT_PI_TOKEN_AUTO="" OPT_PI_TOKEN_ENV="" +OPT_PI_CONFIG="" OPT_PI_TOKEN_AUTO="" OPT_PI_TOKEN_ENV="" OPT_THINKING="" declare -a OPT_allow=() OPT_env=() OPT_mount=() OPT_install=() OPT_label=() OPT_allow_files=() COMMAND=() @@ -365,7 +377,8 @@ KNOWN_OPTS=( workdir home-cache ephemeral-home harness install base-image cpus memory pids tmpfs-size name label keep read-only shell interactive no-tty dry-run print-policy no-build allow-unenforced strict-args insecure-keep-root - print-entrypoint print-containerfile pi-token-env no-pi-config no-pi-token-env help version + print-entrypoint print-containerfile pi-token-env no-pi-config no-pi-token-env thinking + help version ) suggest() { local given="$1" o hit=0 hits="" @@ -389,6 +402,8 @@ parse_args() { --no-pi-token-env) OPT_PI_TOKEN_AUTO=0; shift ;; --pi-token-env) need "$1" "${2:-}"; OPT_PI_TOKEN_ENV="$2"; shift 2 ;; --pi-token-env=*) OPT_PI_TOKEN_ENV="${1#*=}"; shift ;; + --thinking) need "$1" "${2:-}"; OPT_THINKING="$2"; shift 2 ;; + --thinking=*) OPT_THINKING="${1#*=}"; shift ;; --harness=*) OPT_HARNESS="${1#*=}"; shift ;; --allow) need "$1" "${2:-}"; OPT_allow+=("$2"); shift 2 ;; --allow=*) OPT_allow+=("${1#*=}"); shift ;; @@ -513,6 +528,17 @@ resolve_opts() { || die "pi token variable name is not a valid identifier: '${POL[pi_token_env]}'" POL[pi_token_auto]="${OPT_PI_TOKEN_AUTO:-${AISB_PI_TOKEN_AUTO:-$(cfgv pi_token_auto 1)}}" case "${POL[pi_token_auto]}" in 0|false|no) POL[pi_token_auto]=0 ;; *) POL[pi_token_auto]=1 ;; esac + # --- thinking level (policy, not preference: see pi_config_bridge) ---------- + # pi's own default is medium, which is not what an agentic sandbox run is for, so + # ai-sandbox states one. 'host' hands the decision back to settings.json; the entrypoint + # enforces the value on every run unless it is empty. + POL[thinking]="${OPT_THINKING:-${AISB_THINKING:-$(cfgv thinking xhigh)}}" + case "${POL[thinking]}" in + off|minimal|low|medium|high|xhigh|max) ;; + host|inherit) POL[thinking]=host ;; + *) die "--thinking must be off, minimal, low, medium, high, xhigh, max or host (got '${POL[thinking]}')" ;; + esac + POL[thinking_eff]="" # set by pi_config_bridge: the level actually written, if any POL[base_ttl]="${AISB_BASE_TTL_DAYS:-$(cfgv base_ttl_days 7)}" [[ "${POL[base_ttl]}" =~ ^-?[0-9]+$ ]] || { warn "base_ttl_days must be an integer, using 7"; POL[base_ttl]=7; } @@ -745,6 +771,7 @@ write_policy_files() { printf 'AISB_ALLOW_DNS=%s\n' "$ALLOW_DNS" printf 'AISB_PROXY_IPS=%s\n' "${PROXY_IPS[*]-}" printf 'AISB_PROXY_PORTS=%s\n' "${PROXY_PORTS[*]-}" + printf 'AISB_PI_THINKING=%s\n' "${POL[thinking_eff]:-}" } > "$SESSION_MOUNT/policy.env" chmod 644 "$SESSION_MOUNT/policy.env" printf '%s\n' ${DIRECT4[@]+"${DIRECT4[@]}"} > "$SESSION_MOUNT/direct4.list" @@ -867,20 +894,56 @@ command -v setpriv >/dev/null || fail "setpriv is missing from the image; rebuil # the entrypoint has no CAP_DAC_OVERRIDE, so root could not write there anyway. It is # also the honest identity -- these are the user's own files. # -# A file is only replaced when it is missing or still identical to what we seeded last -# time, so an edit made inside the sandbox survives; a change on the host wins next run. +# The two files are managed differently, on purpose. +# +# models.json is a catalog, so it is managed whole: replaced when it is missing or still +# identical to what we seeded, left alone once the sandbox has edited it. +# +# settings.json is a flat bag of preferences, and managing it whole went badly: pi +# rewrites it itself (a theme change, "I have seen this changelog"), and from that moment +# *every* host preference was frozen out -- provider, model, thinking budgets included, +# with only the changelog line left in the file. So it is merged per key against a copy of +# what we last wrote (.aisb-settings.json.wrote): a key the sandbox changed stays theirs, +# a key it did not touch follows the host again, a key the host dropped disappears unless +# the sandbox had changed it, and a key the sandbox invented survives. +# +# defaultThinkingLevel is the exception inside that merge: it is policy the host set for +# the sandbox, so AISB_PI_THINKING is written over the result every run however the file +# was edited. Empty (from --thinking host) makes it an ordinary preference again. + +# The merge itself, as an argument rather than a heredoc: the seeding script below is +# passed to bash -c in single quotes, so nesting a here-document inside it is unreadable. +# Nothing inside that script may contain a single quote either -- one closes the argument +# and the rest of the file silently becomes code in the *outer* shell, where $lvl is not +# set. jq programs there are written unquoted (.defaultThinkingLevel=\$lvl) for that +# reason, and tests/unit/06_entrypoint.sh checks the quoting rather than trusting it. +# T = what is in the sandbox now, B = what we last wrote, N = what the host says now. +SETTINGS_MERGE_JQ=' + ($T[0] // {}) as $t | ($B[0] // {}) as $b | ($N[0] // {}) as $n + | reduce ($t | keys[]) as $k ($t; + if ($b | has($k)) and ($n | has($k) | not) and ($t[$k] == $b[$k]) + then del(.[$k]) else . end) + | . as $kept + | reduce ($n | keys[]) as $k ($kept; + if ($b | has($k)) and ($kept | has($k)) and ($kept[$k] != $b[$k]) + then . else .[$k] = $n[$k] end) +' + if [[ -d "$POLICY_DIR/pi" ]]; then setpriv --reuid="$UID_TARGET" --regid="$GID_TARGET" --clear-groups -- /bin/bash -c ' - dst="$HOME/.pi/agent" + dst="$HOME/.pi/agent"; lvl="$2"; jq_merge="$3" mkdir -p "$dst" || { echo "[aisb] pi config: cannot create $dst" >&2; exit 0; } - for f in models.json settings.json; do - src="$1/pi/$f" - [ -f "$src" ] || continue + + # --- models.json: a catalog, managed as a whole file + f=models.json; src="$1/pi/$f" + if [ -f "$src" ]; then tgt="$dst/$f"; mark="$dst/.aisb-$f.seeded" want="$(sha256sum "$src" | cut -c1-16)" cur=""; [ -f "$tgt" ] && cur="$(sha256sum "$tgt" | cut -c1-16)" prev=""; [ -f "$mark" ] && prev="$(cat "$mark" 2>/dev/null)" - if [ -z "$cur" ]; then + if [ -e "$tgt" ] && [ ! -r "$tgt" ]; then + echo "[aisb] pi config: cannot read $f -- its mode or SELinux label blocks this container, and nothing inside can repair it (remove it from the host, or use --ephemeral-home)" >&2 + elif [ -z "$cur" ]; then install -m 600 "$src" "$tgt" && printf %s "$want" > "$mark" echo "[aisb] pi config: seeded $f" elif [ "$cur" = "$prev" ]; then @@ -889,8 +952,47 @@ if [[ -d "$POLICY_DIR/pi" ]]; then else echo "[aisb] pi config: left $f alone (changed inside the sandbox; delete it to take the host version again)" fi - done - ' aisb-seed "$POLICY_DIR" || log "pi config: seeding failed" + fi + + # --- settings.json: merged per key, then the policy level on top + f=settings.json; src="$1/pi/$f" + if [ -f "$src" ]; then + tgt="$dst/$f"; wrote="$dst/.aisb-$f.wrote"; tmp="$dst/.aisb-$f.tmp" + rm -f "$dst/.aisb-$f.seeded" # the whole-file marker is .wrote now + if [ ! -e "$tgt" ]; then + install -m 600 "$src" "$tgt" && cp "$tgt" "$wrote" + echo "[aisb] pi config: seeded $f" + elif [ ! -r "$tgt" ]; then + echo "[aisb] pi config: cannot read $f -- its mode or SELinux label blocks this container, and nothing inside can repair it (remove it from the host, or use --ephemeral-home)" >&2 + else + base=/dev/null; [ -f "$wrote" ] && base="$wrote" + had=""; [ -n "$lvl" ] && had="$(jq -r ".defaultThinkingLevel // \"\"" "$tgt" 2>/dev/null)" + if merged="$(jq -n --slurpfile T "$tgt" --slurpfile B "$base" --slurpfile N "$src" \ + -f <(printf %s "$jq_merge") 2>/dev/null)" && [ -n "$merged" ]; then + if [ -n "$lvl" ]; then + merged="$(printf %s "$merged" | jq --arg lvl "$lvl" .defaultThinkingLevel=\$lvl 2>/dev/null)" + [ -z "$merged" ] && merged="$(printf null | jq --arg lvl "$lvl" .defaultThinkingLevel=\$lvl)" + fi + else + # Unparseable, so pi could not have used it either, and this file is ours to + # manage: take the host version rather than leave a broken one in place. + if [ -n "$lvl" ]; then + merged="$(jq --arg lvl "$lvl" .defaultThinkingLevel=\$lvl "$src" 2>/dev/null)" + else + merged="$(jq . "$src" 2>/dev/null)" + fi + [ -n "$merged" ] && echo "[aisb] pi config: $f was not usable JSON -- re-seeded from the host config, your in-sandbox changes to it are gone" >&2 + fi + if [ -n "$merged" ] && printf "%s\n" "$merged" > "$tmp" \ + && install -m 600 "$tmp" "$tgt" && rm -f "$tmp" && cp "$tgt" "$wrote"; then + if [ -n "$lvl" ] && [ "$had" != "$lvl" ]; then + echo "[aisb] pi config: thinking -> $lvl (policy: --thinking, or host)" + fi + fi + fi + fi + ' aisb-seed "$POLICY_DIR" "${AISB_PI_THINKING:-}" "$SETTINGS_MERGE_JQ" \ + || log "pi config: seeding failed" fi exec setpriv --reuid="$UID_TARGET" --regid="$GID_TARGET" --clear-groups \ @@ -1328,6 +1430,8 @@ pi_config_bridge() { fi # --- settings.json: preferences travel, host paths do not + local lvl="${POL[thinking]}" + [[ "$lvl" == host ]] && lvl="" if [[ -r "$src/settings.json" ]]; then if ! jq -e . "$src/settings.json" > /dev/null 2>&1; then warn "pi config: $src/settings.json is not valid JSON -- pi starts without it" @@ -1336,12 +1440,28 @@ pi_config_bridge() { jq --arg home "$homep" ' with_entries(select(((.value | type) == "string" and (.value | contains($home))) | not)) ' "$src/settings.json" > "$dst/settings.json" || die "pi config: could not rewrite $src/settings.json" - chmod 644 "$dst/settings.json" dropped="$(jq -r --arg home "$homep" '[to_entries[] | select((.value | type) == "string" and (.value | contains($home))) | .key] | join(", ")' "$src/settings.json")" [[ -n "$dropped" ]] && warn "pi config: dropped settings $dropped -- they point inside the host home, which is not mounted" fi fi + # The thinking level is applied to whatever survived, and on its own if the host has no + # settings.json at all -- a sandbox that silently thinks at pi's default of medium is + # not what --thinking is for. It is written here *and* carried to the entrypoint, which + # re-applies it to a settings.json that already exists in the sandbox home. + if [[ -n "$lvl" ]]; then + if [[ -f "$dst/settings.json" ]]; then + jq --arg lvl "$lvl" '.defaultThinkingLevel = $lvl' "$dst/settings.json" \ + > "$dst/settings.json.new" || die "pi config: could not set the thinking level" + mv "$dst/settings.json.new" "$dst/settings.json" + else + jq -n --arg lvl "$lvl" '{defaultThinkingLevel: $lvl}' > "$dst/settings.json" \ + || die "pi config: could not write the thinking level" + fi + chmod 644 "$dst/settings.json" + POL[thinking_eff]="$lvl" + log "pi config: thinking -> $lvl (policy; --thinking host to let settings.json decide)" + fi [[ -f "$dst/models.json" || -f "$dst/settings.json" ]] || { rmdir "$dst" 2>/dev/null || true; PI_GEN_DIR=""; return 0; } @@ -1697,6 +1817,12 @@ print_policy() { printf 'mounts=%s\n' "$(join ';' ${P_mount[@]+"${P_mount[@]}"})" printf 'agents=%s\n' "${POL[agents_src]:-off}" if [[ -n "${POL[agents_src]:-}" ]]; then printf 'agents_dest=%s\n' "${POL[agents_dest]}"; fi + if [[ -n "${POL[thinking_eff]:-}" ]]; then + printf 'thinking=%s (enforced on every run)\n' "${POL[thinking]}" + else + printf 'thinking=%s (not applied: %s)\n' "${POL[thinking]}" \ + "$([[ "${POL[thinking]}" == host ]] && printf 'the config decides' || printf 'pi config bridging is off')" + fi } # =========================================================================== @@ -1857,6 +1983,14 @@ cmd_doctor() { chk WARN "pi credential: \$$ptok is unset or empty -- a bridged pi run is refused (export it, add it to $(cfg_or_home), or use --no-pi-config)" fi fi + # The thinking policy: what a pi run in this directory will reason at, and whether + # anything is actually there to say so. + if [[ -n "${POL[thinking_eff]:-}" ]]; then + chk INFO "pi thinking: ${POL[thinking]} (policy, re-applied every run; --thinking host lets settings.json decide)" + else + chk INFO "pi thinking: ${POL[thinking]} -- $([[ "${POL[thinking]}" == host ]] && printf 'pi takes it from the bridged settings.json' || printf 'not written, pi config bridging is off')" + fi + # The .agents bridge, reported read-only like the pi bridge: doctor never mounts. if [[ -z "${POL[agents_src]:-}" ]]; then chk INFO "agents: your skills are not shared (agents=0, --no-agents, or no ${POL[agents_dir]})" diff --git a/config/config.example b/config/config.example index 5812768..57b327b 100644 --- a/config/config.example +++ b/config/config.example @@ -62,6 +62,12 @@ net = allow # pi_token_env = AI_API_TOKEN # pi_token_auto = 1 # +# The thinking level a sandboxed pi reasons at: off, minimal, low, medium, high, xhigh +# or max. pi's own default is medium; ai-sandbox states xhigh and re-applies it on every +# run, so one run cannot lower the thinking of the next. 'host' stops forcing it and lets +# settings.json decide. pi only -- other harnesses read their own config. +# thinking = xhigh +# # Where to read the host config from (default ~/.pi/agent). # pi_agent_dir = ~/.pi/agent diff --git a/docs/recipes.md b/docs/recipes.md index 18ed5a3..5001e0c 100644 --- a/docs/recipes.md +++ b/docs/recipes.md @@ -50,12 +50,37 @@ pi --allow aipi.reeseapps.com --pi-token-env GW_TOKEN # pass it yourself, which is what you want in shared scripts pi --allow aipi.reeseapps.com --env AI_API_TOKEN --no-pi-token-env -# no bridge: pi starts with its own defaults inside the volume +# no bridge: pi starts with its own defaults inside the volume (and no thinking policy) pi --no-pi-config --allow aipi.reeseapps.com ``` -Edits made inside the sandbox are kept; changes to the host config are picked up on -the next run; deleting `~/.pi/agent/models.json` inside the sandbox re-seeds it. +`models.json` is kept in step with the host unless you edit it in the sandbox, where your +edit then wins; deleting it inside the sandbox re-seeds it. `settings.json` is merged per +key, so a preference you changed in the sandbox stays changed and every one you did not +touch follows the host again. + +### How hard it thinks + +`defaultThinkingLevel` is policy, not preference: `xhigh` unless you say otherwise, and +re-applied every run rather than remembered from whatever the last sandbox set. + +```bash +# the everyday case: xhigh +pi --allow aipi.reeseapps.com + +# one-off: the top of pi's range, or no reasoning at all for a mechanical job +pi --thinking max --allow aipi.reeseapps.com +pi --thinking off --net allow --allow registry.npmjs.org -- npm ci + +# let settings.json decide, the way it did before this was policy +pi --thinking host --allow aipi.reeseapps.com + +# per repo, chosen by .ai-sandbox/config +# thinking = high +``` + +Nothing is written for `codex`, `claude` or a command after `--`: those read their own +configuration and ai-sandbox does not touch it. ## Your skills and standing instructions diff --git a/docs/threat-model.md b/docs/threat-model.md index c4b5dc2..8ed184a 100644 --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -88,10 +88,14 @@ Be honest about these, and design your work accordingly. uses is your real one. So the bridged config is a convenience, not a narrower credential: prefer a per-sandbox token from your gateway if it can mint one, and note that a bridged run whose token variable is unset or empty is refused rather - than started. Sandboxed edits to `~/.pi/agent` persist in the home volume and are - kept over later host changes (see the README), which is convenience with a - consequence: a hostile run could rewrite the sandboxed pi config for later runs of - that volume. `--ephemeral-home` removes that. + than started. Sandboxed edits to `~/.pi/agent` persist in the home volume and win + over later host changes (see the README), which is convenience with a consequence: a + hostile run could rewrite the sandboxed pi config for later runs of that volume — + point `baseUrl` at a host it controls, swap a model, change the provider. Merging + `settings.json` per key limits this (a run can only pin the keys it actually writes, + and everything else keeps following your host config), and `defaultThinkingLevel` is + re-forced every run so a run cannot quietly lower the reasoning of the next. It is + not eliminated: `--ephemeral-home` removes it, and `inspect` shows what is bridged. ## Residual risk rating diff --git a/tests/e2e/12_sandbox.sh b/tests/e2e/12_sandbox.sh index 3c5994a..315a25f 100644 --- a/tests/e2e/12_sandbox.sh +++ b/tests/e2e/12_sandbox.sh @@ -245,6 +245,55 @@ sb $DARK --harness pi -- bash -c 'rm -f "$HOME/.pi/agent/models.json"; echo rese sb $DARK --harness pi -- bash -lc 'grep -q 524288 "$HOME/.pi/agent/models.json" && echo reseeded || echo not_reseeded' assert_contains "reseeded" "deleting it inside the sandbox takes the host version again" +t_name "settings.json follows the host per key, and the thinking level is policy" +# The host file says medium and no ai-sandbox policy is in it: anything that arrives as +# xhigh can only have been written by the policy. +printf '{ "defaultProvider": "gateway", "defaultModel": "turbo", "theme": "dark/dark",\n "defaultThinkingLevel": "medium", "thinkingBudgets": { "xhigh": 65536 } }\n' > "$AGENT/settings.json" +sb $DARK --harness pi -- bash -lc 'jq -c . "$HOME/.pi/agent/settings.json"' +assert_contains '"defaultThinkingLevel":"xhigh"' "xhigh is enforced over the host file's medium" +assert_contains '"thinkingBudgets":{"xhigh":65536}' "and the rest of the preferences come along" + +# What pi does to its own settings: rewrites the file, adding a key of its own. That used +# to freeze every host preference out of the sandbox for good. +sb $DARK --harness pi -- bash -c ' + f="$HOME/.pi/agent/settings.json" + jq ".theme=\"light/light\" | .defaultThinkingLevel=\"off\" | .lastChangelogVersion=\"9.9.9\"" "$f" > "$f.n" && mv "$f.n" "$f" + echo edited' >/dev/null +# the host then moves the model and the budgets, drops the provider entirely, and says +# medium itself -- so a later --thinking host run has something to hand over +printf '{ "defaultModel": "deepthink", "theme": "sepia/sepia",\n "defaultThinkingLevel": "medium", "thinkingBudgets": { "xhigh": 131072 } }\n' > "$AGENT/settings.json" +sb $DARK --harness pi -- bash -lc 'jq -c . "$HOME/.pi/agent/settings.json"' +MERGED="$OUT" +printf '%s' "$MERGED" | grep -qF '"theme":"light/light"' && _ok "the theme edited in the sandbox survives" \ + || _bad "the theme edited in the sandbox survives" "$MERGED" +printf '%s' "$MERGED" | grep -qF '"lastChangelogVersion":"9.9.9"' && _ok "a key only pi wrote survives" \ + || _bad "a key only pi wrote survives" "$MERGED" +printf '%s' "$MERGED" | grep -qF '"defaultModel":"deepthink"' && _ok "an untouched model follows the host again" \ + || _bad "an untouched model follows the host again" "$MERGED" +printf '%s' "$MERGED" | grep -qF '"xhigh":131072' && _ok "and so do the thinking budgets" \ + || _bad "and so do the thinking budgets" "$MERGED" +printf '%s' "$MERGED" | grep -qF 'defaultProvider' && _bad "an untouched key the host dropped is dropped" \ + || _ok "an untouched key the host dropped is dropped" +printf '%s' "$MERGED" | grep -qF '"defaultThinkingLevel":"xhigh"' && _ok "thinking is forced back to xhigh after the sandbox set off" \ + || _bad "thinking is forced back to xhigh after the sandbox set off" "$MERGED" + +# (the jq programs below avoid `{a, b}` object construction: bash brace-expands an +# unquoted word containing a comma before jq ever sees it) +sb $DARK --harness pi --thinking host -- bash -lc 'jq -r .defaultThinkingLevel "$HOME/.pi/agent/settings.json"' +assert_eq "medium" "$(printf '%s' "$OUT" | tail -1)" "with --thinking host the host's own medium is handed over" + +sb $DARK --harness pi -- bash -c 'printf "{oops" > "$HOME/.pi/agent/settings.json"' >/dev/null +sb $DARK --harness pi -- bash -lc 'jq -c . "$HOME/.pi/agent/settings.json"' +assert_contains "deepthink" "a settings.json that is not JSON is re-seeded rather than left broken" +assert_contains '"defaultThinkingLevel":"xhigh"' "with the policy still applied" +sb $DARK --harness pi -- bash -lc 'jq -r .defaultThinkingLevel "$HOME/.pi/agent/settings.json"' +assert_eq "xhigh" "$(printf '%s' "$OUT" | tail -1)" "and it stays xhigh, not the medium that came back" + +sb $DARK --harness pi -- bash -c 'chmod 000 "$HOME/.pi/agent/settings.json"' >/dev/null +sb $DARK --harness pi -- bash -lc 'echo ran' +assert_contains "cannot read settings.json" "a file this container cannot read is reported, not blamed on JSON" +assert_contains "--ephemeral-home" "and the report names a way out" + unset AI_API_TOKEN sb_timeout 90 $DARK --harness pi assert_rc 1 "a run without the credential is refused before anything starts" diff --git a/tests/run.sh b/tests/run.sh index 0af2555..817de4b 100755 --- a/tests/run.sh +++ b/tests/run.sh @@ -79,7 +79,7 @@ for f in "${FILES[@]}"; do # test should depend on or accidentally share. Files that exercise the .agents # bridge point AISB_AGENTS_DIR at a fixture of their own. export AISB_AGENTS_DIR="${AISB_AGENTS_DIR:-$W/absent-agents-dir}" - unset AI_API_TOKEN AISB_PI_TOKEN_ENV AISB_PI_CONFIG AISB_PI_TOKEN_AUTO + unset AI_API_TOKEN AISB_PI_TOKEN_ENV AISB_PI_CONFIG AISB_PI_TOKEN_AUTO AISB_THINKING # NOTE: HOME is deliberately not overridden -- podman storage lives there. unset AISB_ALLOW AISB_ENV AISB_MOUNT AISB_INSTALL AISB_NET AISB_EXTRA_ARGS AISB_SESSION_OVERRIDE unset AISB_AGENTS AISB_AGENTS_DEST diff --git a/tests/unit/06_entrypoint.sh b/tests/unit/06_entrypoint.sh index b532347..c05656e 100755 --- a/tests/unit/06_entrypoint.sh +++ b/tests/unit/06_entrypoint.sh @@ -61,6 +61,83 @@ grep -c 'accept comment "aisb-direct"' "$RULES" | grep -qx 3 && _ok "exactly 3 d t_name "no DNS hole when the gate is in front" grep -E 'th dport 53' "$RULES" >/dev/null && _bad "DNS is closed in gate mode" || _ok "DNS is closed in gate mode" +t_name "the seeding script survives being passed in single quotes" +# Everything between `bash -c '` and `' aisb-seed` is one single-quoted argument of the +# outer script. One quote inside it closes that argument early and the rest of the file +# silently becomes code in the *outer* shell -- valid bash, wrong program, so neither +# `bash -n` nor shellcheck notices. Hence this test. +INNER="$WORK/seeding-inner.sh" +awk "/setpriv .*bash -c '/{f=1;next} /^ *' aisb-seed/{f=0} f" "$EP" > "$INNER" +[[ -s "$INNER" ]] && _ok "the seeding script was extracted" || _bad "the seeding script was extracted" +if grep -q "'" "$INNER"; then + _bad "it contains no single quote" "found: $(grep -n "'" "$INNER" | head -2 | tr '\n' ' ')" +else + _ok "it contains no single quote" +fi +bash -n "$INNER" && _ok "and it is valid bash on its own" || _bad "and it is valid bash on its own" +grep -q '\$lvl' "$INNER" && _ok "the thinking level is enforced inside it" || _bad "thinking level enforced" + +t_name "settings.json is merged per key, not managed whole" +# Run the merge exactly as the entrypoint does, with the program extracted from the image +# script: T = what the sandbox has, B = what we last wrote, N = what the host says now. +MJQ="$WORK/merge.jq" +awk '/^SETTINGS_MERGE_JQ=/{f=1;next} f && /^.$/{exit} f' "$EP" > "$MJQ" +[[ "$(wc -l < "$MJQ")" -gt 3 ]] && _ok "the merge program was extracted" || _bad "the merge program was extracted" +M_IN="$WORK/m"; mkdir -p "$M_IN" +cat > "$M_IN/new.json" <<'EOF' +{ "defaultProvider": "gateway", "defaultModel": "turbo", "theme": "dark/dark", + "thinkingBudgets": { "xhigh": 65536 } } +EOF +cat > "$M_IN/wrote.json" <<'EOF' +{ "defaultProvider": "gateway", "defaultModel": "turbo", "theme": "dark/dark", + "thinkingBudgets": { "xhigh": 65536 } } +EOF +cat > "$M_IN/sandbox.json" <<'EOF' +{ "defaultProvider": "gateway", "defaultModel": "turbo", "theme": "light/light", + "thinkingBudgets": { "xhigh": 65536 }, "lastChangelogVersion": "1.4.0", "mine": 7 } +EOF +merge_out() { jq -c -n --slurpfile T "$1" --slurpfile B "$2" --slurpfile N "$3" -f "$MJQ"; } +M="$(merge_out "$M_IN/sandbox.json" "$M_IN/wrote.json" "$M_IN/new.json")" +[[ "$(printf '%s' "$M" | jq -r .theme)" == "light/light" ]] \ + && _ok "a key edited in the sandbox stays edited" || _bad "sandbox edit kept: $M" +[[ "$(printf '%s' "$M" | jq -r .lastChangelogVersion)" == "1.4.0" ]] \ + && _ok "a key only pi wrote survives" || _bad "sandbox-only key: $M" +[[ "$(printf '%s' "$M" | jq -r .mine)" == "7" ]] \ + && _ok "and so does one the sandbox invented" || _bad "invented key: $M" + +# host moves two keys, drops a third, keeps one the sandbox changed +cat > "$M_IN/new2.json" <<'EOF' +{ "defaultProvider": "gateway", "defaultModel": "deepthink", + "thinkingBudgets": { "xhigh": 131072 } } +EOF +M2="$(merge_out "$M_IN/sandbox.json" "$M_IN/wrote.json" "$M_IN/new2.json")" +[[ "$(printf '%s' "$M2" | jq -r .defaultModel)" == "deepthink" ]] \ + && _ok "an untouched key follows the host again" || _bad "host key arrives: $M2" +[[ "$(printf '%s' "$M2" | jq -r .thinkingBudgets.xhigh)" == "131072" ]] \ + && _ok "including a nested one (whole key, whole value)" || _bad "nested key: $M2" +[[ "$(printf '%s' "$M2" | jq -r 'has("theme")')" == "true" ]] \ + && _ok "a key the host dropped stays if the sandbox had changed it" || _bad "dropped-but-edited key: $M2" +# the same drop against a file the sandbox never touched: it goes away, as it did when +# the whole file was re-seeded -- otherwise settings only ever accumulate +cat > "$M_IN/clean.json" <<'EOF' +{ "defaultProvider": "gateway", "defaultModel": "turbo", "theme": "dark/dark", + "thinkingBudgets": { "xhigh": 65536 } } +EOF +M3="$(merge_out "$M_IN/clean.json" "$M_IN/wrote.json" "$M_IN/new2.json")" +[[ "$(printf '%s' "$M3" | jq -r 'has("theme")')" == "false" ]] \ + && _ok "an untouched key the host dropped is dropped" || _bad "untouched dropped key: $M3" +# no base copy (the first run after this shipped, or .wrote deleted): the host wins on +# every key it has, which is what repairs a settings file pi had frozen. +M4="$(merge_out "$M_IN/sandbox.json" /dev/null "$M_IN/new.json")" +[[ "$(printf '%s' "$M4" | jq -r '.thinkingBudgets.xhigh')" == "65536" ]] \ + && _ok "with no base, host preferences come back (the frozen-file repair)" || _bad "no-base merge: $M4" +[[ "$(printf '%s' "$M4" | jq -r .lastChangelogVersion)" == "1.4.0" ]] \ + && _ok "and pi's own keys are kept" || _bad "no-base, pi key: $M4" +printf '{oops' > "$M_IN/broken.json" +jq -n --slurpfile T "$M_IN/broken.json" --slurpfile B /dev/null --slurpfile N "$M_IN/new.json" \ + -f "$MJQ" > /dev/null 2>&1 \ + && _bad "unparseable input fails loudly for the caller to handle" || _ok "unparseable input fails loudly for the caller to handle" + t_name "privilege drop is the last thing that happens" tail -5 "$EP" | grep -q 'setpriv' && _ok "exec setpriv at the end" || _bad "setpriv last: $(tail -3 "$EP" | tr '\n' ' ')" grep -q -- '--bounding-set=-all' "$EP" && _ok "bounding set cleared" || _bad "bounding set cleared" diff --git a/tests/unit/08_pi_config.sh b/tests/unit/08_pi_config.sh index da077b8..3b6e338 100644 --- a/tests/unit/08_pi_config.sh +++ b/tests/unit/08_pi_config.sh @@ -37,12 +37,15 @@ cat > "$AGENT/models.json" < "$AGENT/settings.json" < + clear_sessions + sb --keep --print-policy --net none ${1:+--thinking "$1"} + local f; f="$(gen_file settings.json)" + [[ -n "$f" ]] || { printf 'none'; return; } + jq -r '.defaultThinkingLevel // "absent"' "$f" 2>/dev/null || printf 'invalid' +} + +t_name "the thinking level is policy, written over the settings that travel" +sb --keep --print-policy --net none +assert_contains "thinking=xhigh (enforced on every run)" "xhigh is the default, not pi's medium" +assert_contains "pi config: thinking -> xhigh" "and it is announced" +# the fixture says medium, so an xhigh here can only have come from the policy +assert_eq "xhigh" "$(rendered_level '')" "the policy beats the host file's medium" +# the level is a preference, so it belongs to settings.json and nowhere else +clear_sessions +sb --keep --print-policy --net none +if grep -q 'defaultThinkingLevel' "${M:-/nonexistent}" 2>/dev/null; then + _bad "models.json is untouched by it" "the key appeared in the model catalog" +else + _ok "models.json is untouched by it" +fi +S="$(gen_file settings.json)" +grep -qF '"defaultProvider": "gateway"' "${S:-/nonexistent}" \ + && _ok "the rest of the settings still travel" || _bad "the rest of the settings still travel" +grep -qF '"xhigh": 65536' "${S:-/nonexistent}" \ + && _ok "including the host's thinking budgets" || _bad "the host's thinking budgets" +grep -qF 'sessionDir' "${S:-/nonexistent}" \ + && _bad "and a host path is still dropped while the level is added" || _ok "and a host path is still dropped while the level is added" + +t_name "every level, including 'host' and 'off', and nothing in between" +assert_eq "low" "$(rendered_level low)" "a level of your choosing" +assert_eq "max" "$(rendered_level max)" "and the top of pi's range" +assert_eq "off" "$(rendered_level off)" "off is pi's own level, not 'do not manage it'" +assert_eq "medium" "$(rendered_level host)" "host leaves the key to settings.json (medium there)" +rendered_level host >/dev/null +assert_contains "thinking=host (not applied: the config decides)" "and says so" +rendered_level bogus >/dev/null +assert_rc 1 "an unknown level is refused" +assert_contains "must be off, minimal, low, medium, high, xhigh, max or host" "with the list to pick from" +sb_env AISB_THINKING=medium --print-policy --net none +assert_contains "thinking=medium" "AISB_THINKING works" +sb_env AISB_THINKING=medium --thinking max --print-policy --net none +assert_contains "thinking=max" "the flag beats the environment" + +t_name "a host with no settings.json still gets the policy" +mv "$AGENT/settings.json" "$WORK/settings-away.json" +assert_eq "xhigh" "$(rendered_level '')" "the level does not depend on the host having the file" +clear_sessions +sb --keep --print-policy --net none +assert_contains "pi_endpoints" "the model bridge still happened alongside it" +mv "$WORK/settings-away.json" "$AGENT/settings.json" + +t_name "no bridge means no policy either, and it says so" +clear_sessions +sb --keep --no-pi-config --print-policy --net none +assert_contains "thinking=xhigh (not applied: pi config bridging is off)" "--no-pi-config declines the thinking policy too" +if [[ -n "$(gen_file settings.json)" ]]; then _bad "nothing is rendered for pi at all"; else _ok "nothing is rendered for pi at all"; fi + t_name "auth.json and the credential store are never bridged" if find "$AISB_STATE_DIR/sessions" \( -name auth.json -o -name models-store.json \) | grep -q .; then _bad "only models.json and settings.json are bridged" @@ -130,6 +194,7 @@ else fi t_name "the token is passed by name, once, and announced" +sb --keep --print-policy --net none assert_contains "passing \$AI_API_TOKEN from the host" "the auto-pass is announced" assert_eq "env_names=AI_API_TOKEN" "$(printf '%s' "$OUT" | grep '^env_names=' || true)" "exactly one variable is passed" assert_absent "$AI_API_TOKEN" "and only its name is reported"