name the sandbox's working directory, and drive phases through it
Two features, one shared reason: what the sandbox calls things decides what a run tells you afterwards. The cwd is now mounted at the path it has on the host and that is the working directory, so pwd, an error the agent quotes, a commit message and the window title a harness derives from the cwd all name this project instead of reading the same as every sandbox in the session. The mount is unchanged: same directory, same rw, same :z, still the only writable one. Only the destination changed, and a destination is where a bind mount can do damage -- it hides whatever the image already had there. So mirror_block_of() refuses the paths the image or the sandbox owns (/usr, /etc, /var, /opt, /bin, /sbin, /lib*, /boot, /proc, /sys, /dev, /root, /run, /tmp, /home, /home/sandbox*) and / itself, while /home/<user>/... mirrors: the one standard hierarchy where work lives, and the one whose per-user tree never meets the container's /home/sandbox volume or the read-only .agents share inside it. /tmp is what set the shape, and it was measured rather than assumed: with --tmpfs /tmp and -v /tmp/x:/tmp/x in one run podman puts the tmpfs on top, so pwd prints the right path, writes succeed, and the project is not there. A failure that quiet earns a refusal plus a line of output instead: workdir_note in --print-policy, a comment in --dry-run, one log line on a real run. --workdir DIR still chooses the mount point; onto a refused path it is honoured (it was asked for) and warned about. /project, the one directory baked into the image, is every fallback, so a workdir always exists -- --no-cwd included. ai-sandbox-autophase runs a .agents/phases/ pipeline with the split the sandbox forces: every executor agent and the .agents/validate.sh gate inside, the commits and the pushes outside. The gate is in the writable mount, so running it on the host is running agent-editable bash on the host; the git remote is deliberately unreachable in there, because an agent that can push can ship what you are about to review. It runs your launcher, not a copy of it -- the definition is read out of your rc files and never called, which is what lets --status and --dry-run promise no token fetch, no build, no container. Host git runs with an empty hooks path and no ext:: transport, and delivery is refused if a sandboxed unit rewrote .git/config or planted a hook. Needs --no-interactive to go with --no-tty: an unattended run must not leave stdin open on your keyboard, and the two are now separate choices on both sides. Tests and tooling, most of it forced by what the two changes exposed: mirror_block_of is extracted from the script and asked about /usr and /home/sandbox, which a test cannot create (the 06_entrypoint.sh pattern), and both real branches are driven from a scratch dir under $HOME, since $TMPDIR is always the fallback -- which is why the whole suite had only ever seen /project. e2e/12 holds a real container to the same promise, including that /project, /home/sandbox and a home-volume marker survive a mirrored run. The "nothing was started" checks now compare the set of container ids before and after instead of counting every sandbox on the machine, so a session you have open is not a test failure; the pty cases fall back to python3 where util-linux's script is not installed. shellcheck covers both programs, the completion file and the fixtures, so the completion builds are word-by-word now, and a colon in the project path is refused with the reason instead of podman's "invalid option type". 558 unit assertions, 719 with e2e; lint and bash -n clean. LLM-Generated: true
This commit is contained in:
+3
-1
@@ -22,7 +22,9 @@ trap 'kill '"$SRV"' 2>/dev/null' EXIT
|
||||
sleep 1
|
||||
|
||||
t_name "allow mode: enforcement in one run"
|
||||
sb $GATE --env AISB_PROBE_HOSTSVC="$HOSTPORT" -- bash /project/egress_probe.sh
|
||||
# the probe is a file in the project, reached at whatever path the container mounts it at
|
||||
PROJ_DEST="$([[ -n "$(mirror_block_of "$PROJ")" ]] && echo /project || echo "$PROJ")"
|
||||
sb $GATE --env AISB_PROBE_HOSTSVC="$HOSTPORT" -- bash "$PROJ_DEST/egress_probe.sh"
|
||||
assert_contains "ok_allowed_http=200" "allowlisted http works (through the gate)"
|
||||
assert_contains "ok_allowed_https=200" "allowlisted https works (CONNECT)"
|
||||
assert_contains "ok_gate_denied_domain=403" "a non-allowlisted domain is refused by the gate"
|
||||
|
||||
+39
-5
@@ -22,17 +22,51 @@ sb $DARK --harness pi -- bash -lc "command -v pi"
|
||||
assert_rc 0 "the default pi binary exists in the image"
|
||||
|
||||
t_name "the project directory is mounted, writable, and owned by the host user"
|
||||
sb $DARK -- bash -c 'pwd; touch /project/from-container.txt; echo wrote'
|
||||
assert_contains "/project" "working directory is /project"
|
||||
# Where it is mounted depends on the host path; see the mirroring block below, which is
|
||||
# the other branch of the same decision.
|
||||
DEST="$([[ -n "$(mirror_block_of "$WORK_DIR")" ]] && echo /project || echo "$WORK_DIR")"
|
||||
sb $DARK -- bash -c "pwd; touch '$DEST/from-container.txt'; echo wrote"
|
||||
assert_contains "$DEST" "working directory is the project mount"
|
||||
[[ -f "$WORK_DIR/from-container.txt" ]] && _ok "writes land on the host" || _bad "writes land on the host"
|
||||
if [[ "$(stat -c %u "$WORK_DIR/from-container.txt")" == "$(id -u)" ]]; then
|
||||
_ok "the file is owned by the host uid, not by root"
|
||||
else
|
||||
_bad "the file is owned by the host uid" "owner=$(stat -c %u "$WORK_DIR/from-container.txt") expected=$(id -u)"
|
||||
fi
|
||||
sb $DARK -- bash -c 'cat /project/host-file.txt'
|
||||
sb $DARK -- bash -c "cat '$DEST/host-file.txt'"
|
||||
assert_contains "hello from the host" "existing project files are readable"
|
||||
|
||||
t_name "the container names the directory what the host calls it"
|
||||
# The reason for mirroring: pwd, an error message and a harness' window title name *this*
|
||||
# project instead of every sandbox alike. Only a real container answers the question --
|
||||
# podman decides what a bind hides, and it is the one thing about the arrangement that can
|
||||
# go quietly wrong.
|
||||
if [[ "$DEST" == "/project" ]]; then
|
||||
assert_contains "cannot mirror" "the fallback is announced rather than silent"
|
||||
fi
|
||||
mirrorable_dir || true
|
||||
if [[ -z "$USER_MIRROR_DIR" ]]; then
|
||||
_ok "no path on this host may be mirrored ($HOME is inside a sandbox?) -- skipped"
|
||||
else
|
||||
MD="$USER_MIRROR_DIR"
|
||||
echo "hello from the host" > "$MD/host-file.txt"
|
||||
cd "$MD" || exit 1
|
||||
sb $DARK -- bash -c 'pwd; cat host-file.txt; touch from-container.txt; echo wrote; ls -d /project; test -w /home/sandbox && echo home_writable && echo marked > /home/sandbox/mirror-marker'
|
||||
assert_contains "$MD" "pwd is the host path, not /project"
|
||||
assert_contains "hello from the host" "the mirrored mount is the project, not an empty directory"
|
||||
assert_contains "wrote" "and it is writable"
|
||||
[[ -f "$MD/from-container.txt" ]] \
|
||||
&& _ok "writes land on the host through the mirrored path" || _bad "writes land on the host"
|
||||
assert_absent "cannot mirror" "nothing is announced when the path mirrors"
|
||||
assert_contains "/project" "the image's own /project is still there"
|
||||
assert_contains "home_writable" "the home volume is still mounted at /home/sandbox"
|
||||
# The home volume is the state the next run expects, so check it survived the mirror.
|
||||
sb $DARK -- bash -c 'cat /home/sandbox/mirror-marker 2>&1; ls'
|
||||
assert_contains "marked" "and the marker written through a mirrored run is still there"
|
||||
assert_contains "host-file.txt" "a mirrored run sees its own project, not the previous one"
|
||||
cd "$WORK_DIR" || exit 1
|
||||
fi
|
||||
|
||||
t_name "nothing else on the host is visible"
|
||||
echo secret > "$WORK/not-shared.txt"
|
||||
sb $DARK -- bash -c "cat '$WORK/not-shared.txt' 2>&1 || echo NOT_VISIBLE"
|
||||
@@ -145,12 +179,12 @@ assert_absent "demo-skill" "not even as a directory name"
|
||||
# visible at once, which is what a harness that reads both (pi) expects.
|
||||
mkdir -p "$WORK_DIR/.agents/skills/project-skill"
|
||||
printf 'name: project-skill\ninstructions: project only\n' > "$WORK_DIR/.agents/skills/project-skill/SKILL.md"
|
||||
sb $DARK -- bash -c 'cat /project/.agents/skills/project-skill/SKILL.md; ls "$HOME/.agents/skills" | tr "\n" " "'
|
||||
sb $DARK -- bash -c "cat '$DEST/.agents/skills/project-skill/SKILL.md'; ls \"\$HOME/.agents/skills\" | tr '\n' ' '"
|
||||
assert_contains "project only" "the project's own .agents comes in with the project mount"
|
||||
assert_contains "demo-skill" "and the user-level one is still alongside it"
|
||||
|
||||
t_name "read-only root filesystem"
|
||||
sb $DARK --read-only -- bash -c 'touch /usr/bin/nope 2>/dev/null && echo "BAD: wrote to rootfs" || echo rootfs_locked=ok; touch /project/rw-still-works && echo project_still_writable'
|
||||
sb $DARK --read-only -- bash -c "touch /usr/bin/nope 2>/dev/null && echo 'BAD: wrote to rootfs' || echo rootfs_locked=ok; touch '$DEST/rw-still-works' && echo project_still_writable"
|
||||
assert_contains "rootfs_locked=ok" "--read-only blocks writes outside mounts"
|
||||
assert_contains "project_still_writable" "the project mount stays writable"
|
||||
|
||||
|
||||
Vendored
+161
@@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env bash
|
||||
# tests/fixtures/phase-lib/lib.sh -- a stand-in for the phased-execution skill's lib.sh.
|
||||
#
|
||||
# It is a CONTRACT fixture, not a reimplementation: the function names, the child call
|
||||
# shape (`pi "${PI_ARGS[@]}" --session-dir ... --mode json "$prompt"` piped to a progress
|
||||
# renderer), the gate call (`run_validation <logfile>`) and the host-side commit + push
|
||||
# at a phase's commit point are what bin/ai-sandbox-autophase is written against, and
|
||||
# what breaks if that library changes shape. Keeping it here means the unit tests run
|
||||
# with no skill installed and no containers; tests/unit/10_autophase.sh also runs the
|
||||
# real library once, when it is installed.
|
||||
#
|
||||
# Sourced, never executed.
|
||||
|
||||
PHASE_TODO=".agents/phases/todo"
|
||||
PHASE_DONE=".agents/phases/complete"
|
||||
PHASE_REPORTS=".agents/reports"
|
||||
PHASE_SESSIONS=".agents/phase-sessions"
|
||||
MAX_FIX_ATTEMPTS="${MAX_FIX_ATTEMPTS:-3}"
|
||||
|
||||
FIXTURE_LIB_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
ts() { date '+%Y-%m-%d %H:%M:%S'; }
|
||||
warn() { echo "[$(ts)] ⚠ $*" >&2; }
|
||||
die() { echo "[$(ts)] ✗ ERROR: $*" >&2; exit 1; }
|
||||
|
||||
find_root() {
|
||||
local d; d="$(pwd)"
|
||||
while :; do
|
||||
[[ -d "$d/.agents/phases/todo" ]] && { printf '%s\n' "$d"; return 0; }
|
||||
[[ "$d" == "/" ]] && return 1
|
||||
d="$(dirname "$d")"
|
||||
done
|
||||
}
|
||||
|
||||
# A phase entry is anything in todo/ whose name starts with a digit: a phase directory, or
|
||||
# a single .md file. Globs rather than ls|grep, so a name cannot be mis-split.
|
||||
phase_entries() {
|
||||
local e
|
||||
{
|
||||
( cd "$PHASE_TODO" 2>/dev/null || exit 0
|
||||
for e in [0-9]*; do [[ -e "$e" ]] && printf '%s\n' "$e"; done )
|
||||
} | sort
|
||||
}
|
||||
|
||||
phase_next_unit() {
|
||||
local p="$1" f t
|
||||
if [[ -d "$PHASE_TODO/$p" ]]; then
|
||||
for f in "$PHASE_TODO/$p"/[0-9]*.md; do
|
||||
[[ -f "$f" ]] || continue
|
||||
t="${f##*/}"; [[ "$t" == "00_phase.md" ]] && continue
|
||||
printf '%s/%s\n' "$p" "$t"; return 0
|
||||
done
|
||||
[[ -f "$PHASE_TODO/$p/00_phase.md" ]] && { printf '%s/00_phase.md\n' "$p"; return 0; }
|
||||
elif [[ -f "$PHASE_TODO/$p" ]]; then
|
||||
printf '%s\n' "$p"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
next_unit() {
|
||||
local p u
|
||||
for p in $(phase_entries); do
|
||||
u="$(phase_next_unit "$p")"
|
||||
[[ -n "$u" ]] && { printf '%s\n' "$u"; return 0; }
|
||||
done
|
||||
return 0
|
||||
}
|
||||
|
||||
count_units() {
|
||||
local p n=0 f
|
||||
for p in $(phase_entries); do
|
||||
if [[ -d "$PHASE_TODO/$p" ]]; then
|
||||
for f in "$PHASE_TODO/$p"/[0-9]*.md; do [[ -f "$f" ]] && n=$((n + 1)); done
|
||||
elif [[ -f "$PHASE_TODO/$p" ]]; then
|
||||
n=$((n + 1))
|
||||
fi
|
||||
done
|
||||
printf '%s\n' "$n"
|
||||
}
|
||||
|
||||
unit_phase() { local u="$1"; [[ "$u" == */* ]] && printf '%s\n' "${u%%/*}" || printf '%s\n' "${u%.md}"; }
|
||||
unit_base() { local u="${1%.md}"; printf '%s\n' "${u//\//__}"; } # 01_api/02_x.md -> 01_api__02_x
|
||||
is_phase_end() { local u="$1"; if [[ "$u" == */* ]]; then [[ "${u##*/}" == "00_phase.md" ]]; else return 0; fi; }
|
||||
|
||||
unit_report_dir() {
|
||||
local u="$1" p
|
||||
if [[ "$u" == */* ]]; then p="${u%%/*}"; else p="."; fi
|
||||
printf '%s/%s\n' "$PHASE_REPORTS" "$p"
|
||||
}
|
||||
unit_report() { # <unit> <attempt> <ext>
|
||||
printf '%s/%s.a%s.%s\n' "$(unit_report_dir "$1")" "$(unit_base "$1")" "$2" "$3"
|
||||
}
|
||||
|
||||
move_unit() {
|
||||
local u="$1"
|
||||
if [[ "$u" == */* ]]; then
|
||||
mkdir -p "$PHASE_DONE/${u%%/*}"
|
||||
mv -f "$PHASE_TODO/$u" "$PHASE_DONE/${u%%/*}/${u##*/}"
|
||||
rmdir "$PHASE_TODO/${u%%/*}" 2>/dev/null || true
|
||||
else
|
||||
mkdir -p "$PHASE_DONE"
|
||||
mv -f "$PHASE_TODO/$u" "$PHASE_DONE/$u"
|
||||
fi
|
||||
}
|
||||
|
||||
sweep_finished_phases() { return 0; }
|
||||
|
||||
build_pi_args() { PI_ARGS=(--no-approve); }
|
||||
|
||||
first_prompt() { printf 'Execute the unit %s. Reply with a report.\n' "$1"; }
|
||||
fix_prompt() { printf 'Fix the failures: %s\n' "${1:-}"; }
|
||||
|
||||
run_validation() { bash .agents/validate.sh >"$1" 2>&1; }
|
||||
|
||||
# The host-side commit + push, at the phase's commit point: same shape as the real
|
||||
# library's commit_phase/push_phase (commit, then push when PHASE_PUSH=1).
|
||||
commit_phase() {
|
||||
local phase="$1"
|
||||
git rev-parse --is-inside-work-tree >/dev/null 2>&1 || return 0
|
||||
git add -A >/dev/null 2>&1 || return 1
|
||||
if ! git diff --cached --quiet; then
|
||||
git commit --no-gpg-sign -m "phase: $phase" >/dev/null 2>&1 || return 1
|
||||
fi
|
||||
# the real library hands the commit to the remote through push_phase, so this fixture
|
||||
# does too -- which is also the function the driver shadows to keep every push (and
|
||||
# every refusal to push) in one place.
|
||||
push_phase "$phase" || return 1
|
||||
return 0
|
||||
}
|
||||
|
||||
# Mirrors the library's own entry point; the driver replaces it.
|
||||
push_phase() {
|
||||
git remote >/dev/null 2>&1 || return 0
|
||||
git push >/dev/null 2>&1 || return 1
|
||||
return 0
|
||||
}
|
||||
|
||||
notify_task() { return 0; }
|
||||
|
||||
execute_unit() {
|
||||
local unit="$1" errors="" rc=0
|
||||
mkdir -p "$PHASE_DONE" "$PHASE_REPORTS" "$PHASE_SESSIONS" "$(unit_report_dir "$unit")"
|
||||
echo "[$(ts)] ━━ $unit — attempt 1/$MAX_FIX_ATTEMPTS ━━" # the real library prints this; tests read it
|
||||
build_pi_args
|
||||
local prompt out errf vrc
|
||||
prompt="$(first_prompt "$unit")"
|
||||
out="$(unit_report "$unit" 1 md)"
|
||||
errf="$(unit_report "$unit" 1 err)"
|
||||
pi "${PI_ARGS[@]}" --session-dir "$PHASE_SESSIONS" --name "$(unit_base "$unit")" \
|
||||
--mode json "$prompt" 2>"$errf" | bash "$FIXTURE_LIB_DIR/progress.sh" "$out"
|
||||
rc="${PIPESTATUS[0]}"
|
||||
(( rc == 0 )) || errors+="child exited $rc"$'\n'
|
||||
run_validation "$(unit_report "$unit" 1 validate)"; vrc=$?
|
||||
(( vrc == 0 )) || errors+=".agents/validate.sh failed"$'\n'
|
||||
[[ -n "$errors" ]] && { printf '%s' "$errors" >&2; return 1; }
|
||||
move_unit "$unit"
|
||||
if is_phase_end "$unit"; then
|
||||
commit_phase "$unit" || return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
Vendored
+23
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# tests/fixtures/phase-lib/progress.sh -- the fixture's stand-in for progress.mjs.
|
||||
#
|
||||
# Same job and same contract: read the child's JSON stream on stdin, write the final
|
||||
# assistant text to $1, exit non-zero when the stream produced no report.
|
||||
set -uo pipefail
|
||||
|
||||
out="$1"; shift
|
||||
text=""
|
||||
while IFS= read -r line; do
|
||||
case "$line" in
|
||||
*'"type": "message_end"'* | *'"type":"message_end"'*)
|
||||
t="$(printf '%s' "$line" | sed -n 's/.*"text": *"\([^"]*\)".*/\1/p')"
|
||||
[[ -n "$t" ]] && text="$t"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z "$text" ]]; then
|
||||
printf '(no final assistant message)\n' > "$out"
|
||||
exit 1
|
||||
fi
|
||||
printf '%s\n' "$text" > "$out"
|
||||
@@ -31,6 +31,84 @@ sb_timeout() {
|
||||
return 0
|
||||
}
|
||||
|
||||
# Extract one function from the script itself, the way 06_entrypoint.sh extracts the
|
||||
# entrypoint: the test then asks the real code a question instead of restating its answer.
|
||||
aisb_fn() { # aisb_fn <function-name> -> that function's source, on stdout
|
||||
awk -v fn="$1" '$0 ~ "^" fn "\\(\\) \\{" {f=1} f {print} f && /^\}$/ {exit}' "$AISB"
|
||||
}
|
||||
|
||||
# Where the sandbox would mount a host path. Empty means it mirrors the path; anything
|
||||
# else names the container path the bind would bury. Extracted rather than reimplemented:
|
||||
# the answer depends on the container's own filesystem (/usr, /tmp, /home/sandbox), and
|
||||
# most of that hierarchy cannot be created by a test.
|
||||
mirror_block_of() { # mirror_block_of <absolute host path> -> blocking path or empty
|
||||
local src; src="$(aisb_fn mirror_block_of)"
|
||||
[[ -n "$src" ]] || { echo "mirror_block_of not found in $AISB" >&2; return 2; }
|
||||
( set -uo pipefail; eval "$src"; mirror_block_of "$1" )
|
||||
}
|
||||
|
||||
# A directory whose host path the sandbox is allowed to mirror, in USER_MIRROR_DIR (empty
|
||||
# when there is none). The scratch dir ($WORK) is under $TMPDIR, and /tmp is one of the
|
||||
# paths the sandbox owns, so a run from there always takes the fallback branch -- the
|
||||
# mirroring branch needs a per-user tree, and $HOME is the one place an unprivileged test
|
||||
# can create it. Nothing is written into the real ~/.config, ~/.local or ~/.cache.
|
||||
#
|
||||
# Assigns a global instead of printing: the cleanup trap below belongs to the test file,
|
||||
# so this must not be called as $(mirrorable_dir) -- a command substitution would set the
|
||||
# trap in a subshell that exits, and takes the directory with it.
|
||||
USER_MIRROR_DIR=""
|
||||
mirrorable_dir() {
|
||||
local base="${HOME%/}" qbase
|
||||
USER_MIRROR_DIR=""
|
||||
[[ -n "$base" && -d "$base" && -w "$base" ]] || return 1
|
||||
[[ -z "$(mirror_block_of "$base")" ]] || return 1
|
||||
USER_MIRROR_DIR="$(mktemp -d "$base/.ai-sandbox-test-mirror.XXXXXX" 2>/dev/null)" || return 1
|
||||
[[ -n "$USER_MIRROR_DIR" ]] || return 1
|
||||
# Removed when the test file's subshell exits, including on a failed assert. The base is
|
||||
# quoted in now rather than read from $HOME later, so the trap cannot widen.
|
||||
printf -v qbase '%q' "$base"
|
||||
# shellcheck disable=SC2064 # immediate expansion of the captured base is the point
|
||||
trap "rm -rf -- $qbase/.ai-sandbox-test-mirror.* >/dev/null 2>&1 || true" EXIT
|
||||
return 0
|
||||
}
|
||||
|
||||
# Run a command with stdin, stdout and stderr on a pty: -t and -i are decided from the
|
||||
# real terminal, so a test about them cannot use the runner's pipes. util-linux's 'script'
|
||||
# is the usual tool; python3's pty module does the same job where script is not installed
|
||||
# (measured: all three descriptors are ttys and the exit code propagates). Sets OUT and RC
|
||||
# like sb(). Args are joined for 'script', so pass simple words, not quoted sentences.
|
||||
pty_available() { command -v script > /dev/null 2>&1 || command -v python3 > /dev/null 2>&1; }
|
||||
pty_run() {
|
||||
OUT="" RC=0
|
||||
if command -v script > /dev/null 2>&1; then
|
||||
OUT="$(script -qec "$*" /dev/null 2>&1 </dev/null)" && RC=0 || RC=$?
|
||||
elif command -v python3 > /dev/null 2>&1; then
|
||||
OUT="$(python3 -c '
|
||||
import os, pty, sys
|
||||
st = pty.spawn(sys.argv[1:])
|
||||
try: sys.exit(os.waitstatus_to_exitcode(st))
|
||||
except AttributeError: sys.exit(st >> 8)' "$@" 2>&1 </dev/null)" && RC=0 || RC=$?
|
||||
else
|
||||
OUT="no pty tool available"
|
||||
RC=127
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# "nothing was started", as a hermetic question. Counting every sandbox container on the
|
||||
# machine fails when the developer has a session open, or when an interrupted run left one
|
||||
# behind -- neither of which is this test starting a container. Take the set of ids before,
|
||||
# and ask whether it changed.
|
||||
snapshot_containers() { AISB_CONTAINER_SNAPSHOT="$(podman ps -aq --filter label=ai-sandbox=1 2>/dev/null | sort)"; }
|
||||
assert_no_new_containers() { # assert_no_new_containers <label> (after snapshot_containers)
|
||||
local now extra
|
||||
now="$(podman ps -aq --filter label=ai-sandbox=1 2>/dev/null | sort)"
|
||||
if [[ "$now" == "${AISB_CONTAINER_SNAPSHOT:-}" ]]; then _ok "$1"; return 0; fi
|
||||
extra="$(comm -13 <(printf '%s\n' "${AISB_CONTAINER_SNAPSHOT:-}") \
|
||||
<(printf '%s\n' "$now") | tr '\n' ' ')"
|
||||
_bad "$1" "container ids appeared: ${extra:-none (one disappeared instead)}"
|
||||
}
|
||||
|
||||
# sb_env VAR=val ... -- args (env prefix, without leaking into the test shell)
|
||||
sb_env() {
|
||||
local -a envs=()
|
||||
|
||||
+2
-1
@@ -30,7 +30,8 @@ command -v podman >/dev/null || { echo "podman is required to run the tests" >&2
|
||||
|
||||
# Syntax-check every shell file in the project before running anything.
|
||||
SYNTAX_BAD=0
|
||||
for f in "$AISB" "$HERE"/*.sh "$HERE"/unit/*.sh "$HERE"/e2e/*.sh "$ROOT"/install.sh; do
|
||||
for f in "$AISB" "$HERE"/*.sh "$HERE"/unit/*.sh "$HERE"/e2e/*.sh "$HERE"/fixtures/*.sh \
|
||||
"$HERE"/fixtures/phase-lib/*.sh "$ROOT"/install.sh "$ROOT"/contrib/*.bash; do
|
||||
[[ -f "$f" ]] || continue
|
||||
if ! syn="$(bash -n "$f" 2>&1)"; then
|
||||
echo "SYNTAX: $f" >&2; printf '%s\n' "$syn" | sed 's/^/ /' >&2; SYNTAX_BAD=1
|
||||
|
||||
+158
-5
@@ -5,6 +5,9 @@ source "$TESTS/lib.sh"
|
||||
PROJ="$WORK/proj"; mkdir -p "$PROJ"; cd "$PROJ" || exit 1
|
||||
export AISB_CONFIG_DIR="$WORK/cfg-empty"; mkdir -p "$AISB_CONFIG_DIR"
|
||||
A="--allow example.com"
|
||||
# Every run in this file is a preview. What that is worth is asserted at the end, against
|
||||
# the set of containers that existed before it started.
|
||||
snapshot_containers
|
||||
|
||||
t_name "isolation flags are always on"
|
||||
sb --dry-run $A
|
||||
@@ -36,12 +39,15 @@ sb --dry-run $A --no-gate
|
||||
assert_absent "HTTP_PROXY=" "--no-gate sets no proxy"
|
||||
|
||||
t_name "filesystem"
|
||||
# Where the cwd lands depends on the host path, so the expected destination is asked of
|
||||
# the same function the script uses instead of being hardcoded here.
|
||||
PROJ_DEST="$([[ -n "$(mirror_block_of "$PROJ")" ]] && echo /project || echo "$PROJ")"
|
||||
sb --dry-run $A
|
||||
assert_contains "$PROJ:/project" "cwd is mounted at /project"
|
||||
assert_contains "--workdir /project" "cwd is the working directory"
|
||||
assert_contains "$PROJ:$PROJ_DEST" "cwd is mounted at its own path or the fallback"
|
||||
assert_contains "--workdir $PROJ_DEST" "cwd is the working directory"
|
||||
assert_contains "/run/aisb:ro" "policy dir is mounted read-only"
|
||||
sb --dry-run $A --no-cwd
|
||||
assert_absent "$PROJ:/project" "--no-cwd mounts nothing"
|
||||
assert_absent "$PROJ" "--no-cwd mounts nothing"
|
||||
sb --dry-run $A --workdir /src
|
||||
assert_contains "$PROJ:/src" "--workdir changes the mount point"
|
||||
mkdir -p "$WORK/extra"
|
||||
@@ -58,6 +64,119 @@ sb --dry-run $A --ephemeral-home
|
||||
assert_contains "/home/sandbox:rw,nosuid,nodev" "--ephemeral-home uses a tmpfs home"
|
||||
assert_absent "type=volume,source=aisb-home" "--ephemeral-home drops the volume"
|
||||
|
||||
t_name "the container names the directory what the host calls it"
|
||||
# Mirroring the host path is what makes pwd, an error message and a harness' window title
|
||||
# name *this* project. It is a decision about the host path, so both branches need a real
|
||||
# directory: $TMPDIR is a path the sandbox owns, a per-user tree is not.
|
||||
mirrorable_dir || true
|
||||
MIRROR_DIR="$USER_MIRROR_DIR"
|
||||
if [[ -z "$MIRROR_DIR" ]]; then
|
||||
_ok "no writable host path here may be mirrored ($HOME is itself inside a sandbox?) -- skipped"
|
||||
else
|
||||
cd "$MIRROR_DIR" || exit 1
|
||||
sb --dry-run $A
|
||||
assert_contains "$MIRROR_DIR:$MIRROR_DIR" "the cwd is mounted at the path it has on the host"
|
||||
assert_contains "--workdir $MIRROR_DIR" "and worked in at that path"
|
||||
assert_absent "# workdir:" "nothing to announce when the path mirrors"
|
||||
sb --dry-run $A --print-policy
|
||||
assert_eq "" "$(printf '%s' "$OUT" | grep '^workdir_note=' || true)" "and no note in the policy"
|
||||
sb --dry-run $A --workdir /src
|
||||
assert_contains "$MIRROR_DIR:/src" "--workdir still picks the mount point over mirroring"
|
||||
assert_contains "--workdir /src" "--workdir is what reaches podman"
|
||||
sb --dry-run $A --workdir auto
|
||||
assert_contains "--workdir $MIRROR_DIR" "'auto' is the default, spelled out"
|
||||
cd "$PROJ" || exit 1
|
||||
fi
|
||||
|
||||
t_name "a host path the sandbox owns is not mirrored, and says so"
|
||||
# /tmp is the guaranteed case: the sandbox mounts a tmpfs there, and podman puts that
|
||||
# tmpfs on top of a bind of the same path -- so a mirrored /tmp project would come out as
|
||||
# an empty tmpfs. Measured, see docs/networking.md.
|
||||
cd /tmp || exit 1
|
||||
sb --dry-run $A
|
||||
assert_contains "--workdir /project" "/tmp falls back to the image's own directory"
|
||||
assert_contains "-v /tmp:/project" "and mounts the cwd there instead"
|
||||
assert_contains "# workdir: cannot mirror /tmp" "--dry-run explains the fallback"
|
||||
assert_contains "the sandbox owns /tmp*" "naming what it would have buried"
|
||||
assert_contains "--workdir DIR to choose" "and the way to decide otherwise"
|
||||
sb_env AI_API_TOKEN=canary-not-a-real-token --dry-run $A --env AI_API_TOKEN
|
||||
assert_absent "canary-not-a-real-token" "the note carries no values, only the decision"
|
||||
sb --dry-run $A --print-policy
|
||||
assert_contains "workdir=/project" "the policy says which directory was chosen"
|
||||
assert_contains "workdir_note=" "and why"
|
||||
assert_contains "cwd=/tmp" "while still printing the host path"
|
||||
cd "$PROJ" || exit 1
|
||||
|
||||
t_name "which host paths may be mirrored is a decision about the container's filesystem"
|
||||
# The refusals are about what lives at a path *inside* the image, and a test cannot create
|
||||
# /usr or /home/sandbox. So ask the function itself, extracted from the script, about the
|
||||
# paths it was written for -- the same trick 06_entrypoint.sh uses for the entrypoint.
|
||||
[[ -n "$(aisb_fn mirror_block_of)" ]] \
|
||||
&& _ok "the mirror decision is one function in the script" \
|
||||
|| _bad "the mirror decision is one function in the script"
|
||||
# Blocked: everything the base image owns, plus this sandbox's own mounts and home.
|
||||
for p in / /usr/src/proj /var/lib/pg/data /opt/stack/app /etc/proj /tmp/build/proj \
|
||||
/run/user/1000/proj /home /home/sandbox /home/sandbox/work /bin/proj /boot/efi \
|
||||
/dev/shm/x /lib/x86_64-linux-gnu /lib32/x /lib64/ld /proc/1/cwd /root/proj \
|
||||
/sbin/init /sys/firmware; do
|
||||
b="$(mirror_block_of "$p")"
|
||||
if [[ -n "$b" ]]; then _ok "$p is not mirrored (would bury $b)"
|
||||
else _bad "$p is not mirrored" "it was mirrored"; fi
|
||||
done
|
||||
assert_eq "/tmp" "$(mirror_block_of /tmp/a/b)" "the note names the top-level path, not the project"
|
||||
assert_eq "/home/sandbox" "$(mirror_block_of /home/sandbox/work)" "the container home is named as itself"
|
||||
assert_eq "/" "$(mirror_block_of /)" "the whole root is refused"
|
||||
# Free: per-user trees under /home, and the hierarchy the image leaves empty.
|
||||
for p in /home/you/Projects/foo /home/dev /home/sandboxer/proj /mnt/data/proj /srv/www \
|
||||
/media/usb/proj /code/x /work/proj /data/proj /home/sandboxish/x; do
|
||||
b="$(mirror_block_of "$p")"
|
||||
if [[ -z "$b" ]]; then _ok "$p mirrors to itself"
|
||||
else _bad "$p mirrors to itself" "refused because of $b"; fi
|
||||
done
|
||||
|
||||
t_name "a project path podman cannot express is refused with a reason"
|
||||
# Measured: podman answers `-v /a:b:/project` with `invalid option type "/project"`, which
|
||||
# is a strange thing to read when all you typed was `pi`. Mirroring puts the same string in
|
||||
# the destination too, so it is refused before podman is involved.
|
||||
COLON="$WORK/a:b"; mkdir -p "$COLON"; cd "$COLON" || exit 1
|
||||
sb --dry-run $A
|
||||
assert_rc 1 "a ':' in the project path is refused"
|
||||
assert_contains "podman cannot parse" "and says whose parser"
|
||||
assert_contains "--no-cwd" "and names the way out"
|
||||
sb --dry-run $A --no-cwd
|
||||
assert_rc 0 "--no-cwd needs nothing mounted, so the path is fine"
|
||||
cd "$PROJ" || exit 1
|
||||
|
||||
t_name "--workdir takes a container path or 'auto', nothing else"
|
||||
sb --dry-run $A --workdir relative/dir
|
||||
assert_rc 1 "a relative --workdir is refused"
|
||||
assert_contains "must be an absolute path or 'auto'" "and says what it wanted"
|
||||
sb --dry-run $A --workdir /
|
||||
assert_rc 1 "--workdir / is refused"
|
||||
assert_contains "over the whole container" "with the reason"
|
||||
sb --dry-run $A --workdir /src/
|
||||
assert_contains "$PROJ:/src" "a trailing slash is not part of the destination"
|
||||
# An explicit path onto something the sandbox owns is honoured -- it was asked for -- but
|
||||
# the hiding rule does not stop applying because a human chose the path, so it is warned.
|
||||
sb --dry-run $A --workdir /tmp/proj
|
||||
assert_rc 0 "--workdir onto a path the sandbox owns is still honoured"
|
||||
assert_contains "$PROJ:/tmp/proj" "podman is told what was asked for"
|
||||
assert_contains "warning: --workdir /tmp/proj" "and the covering is announced"
|
||||
assert_contains "/tmp is a tmpfs" "with the reason"
|
||||
sb --dry-run $A --workdir /opt/proj
|
||||
assert_contains "the sandbox owns /opt*" "the same for a directory the image owns"
|
||||
sb --dry-run $A --workdir /srv/app
|
||||
assert_absent "warning: --workdir" "a path nothing owns is chosen in silence"
|
||||
sb_env AISB_PROJECT_DEST=/srv/app --dry-run $A
|
||||
assert_contains "$PROJ:/srv/app" "AISB_PROJECT_DEST still chooses it"
|
||||
mkdir -p "$WORK/cfg-wd"
|
||||
printf 'workdir = /srv/cfg\n' > "$WORK/cfg-wd/config"
|
||||
# No subshell here: OUT and the counters have to survive it.
|
||||
AISB_CONFIG_DIR_SAVED="$AISB_CONFIG_DIR"; export AISB_CONFIG_DIR="$WORK/cfg-wd"
|
||||
sb --dry-run $A
|
||||
assert_contains "$PROJ:/srv/cfg" "the config file chooses it too"
|
||||
export AISB_CONFIG_DIR="$AISB_CONFIG_DIR_SAVED"
|
||||
|
||||
t_name "labels, limits, extra args"
|
||||
sb --dry-run $A --label team=x
|
||||
assert_contains "--label team=x" "--label"
|
||||
@@ -80,7 +199,41 @@ if [[ -n "$pos_cmd" && -n "$pos_img" && $pos_img -lt $pos_cmd ]]; then _ok "the
|
||||
|
||||
t_name "dry run is side-effect free"
|
||||
sb --dry-run $A
|
||||
RUNNING="$(podman ps --format '{{.Names}}' | grep -c 'aisb' || true)"
|
||||
# Ask about the container *this* invocation would have created, not about every sandbox on
|
||||
# the machine: another session that happens to be running is not this test starting one.
|
||||
# The session id is unique per run, so its label is the hermetic question.
|
||||
SID="$(printf '%s' "$OUT" | tr ' ' '\n' | sed -n 's/^aisb\.session=//p' | head -1)"
|
||||
[[ -n "$SID" ]] && _ok "the dry run names its session" || _bad "the dry run names its session"
|
||||
CREATED="$(podman ps -a --filter "label=aisb.session=$SID" --format '{{.Names}}' | wc -l | tr -d ' ')"
|
||||
DIRS="$(ls -A "$AISB_STATE_DIR/sessions" 2>/dev/null | wc -l | tr -d ' ')"
|
||||
assert_eq "$RUNNING" 0 "dry run started no container"
|
||||
assert_eq "$CREATED" 0 "dry run created no container, under any name"
|
||||
assert_eq "$DIRS" 0 "dry run left no session directory behind"
|
||||
assert_no_new_containers "and no preview in this file created one either"
|
||||
|
||||
t_name "attachment: stdin is a choice, and an unattended run does not want it"
|
||||
# -t and -i are decided from the real stdin/stdout, and the test runner's are pipes. The
|
||||
# pipe case is checked either way; the attached case needs a pty, which lib.sh builds from
|
||||
# 'script' or python3 and reports as skipped when the host has neither.
|
||||
inv_line() { printf '%s' "$OUT" | tr -d '\r' | tail -1; }
|
||||
has_opt() { inv_line | grep -qE "(^| )$1( |$)"; }
|
||||
sb_pty() { pty_run "$AISB" --dry-run --net none "$@"; }
|
||||
sb --dry-run --net none
|
||||
if has_opt "-i" || has_opt "-t"; then _bad "pipes get neither" "$(inv_line)"; else _ok "pipes get neither: nothing is attached to a terminal that is not there"; fi
|
||||
sb --dry-run --net none --interactive
|
||||
if has_opt "-i"; then _ok "--interactive still forces it on with pipes"; else _bad "--interactive still forces it on with pipes" "$(inv_line)"; fi
|
||||
if ! pty_available; then
|
||||
_ok "neither 'script' nor python3 here -- the attached cases are skipped"
|
||||
else
|
||||
sb_pty
|
||||
assert_rc 0 "on a pty"
|
||||
if has_opt "-i" && has_opt "-t"; then _ok "a terminal run is attached, which is what an interactive shell expects"; else _bad "a terminal run is attached" "$(inv_line)"; fi
|
||||
sb_pty --no-interactive
|
||||
assert_rc 0 "--no-interactive"
|
||||
if has_opt "-i"; then _bad "--no-interactive drops -i" "$(inv_line)"; else _ok "--no-interactive drops -i: the command cannot read this terminal"; fi
|
||||
if has_opt "-t"; then _ok "--no-interactive leaves the TTY alone (it is a separate choice)"; else _bad "--no-interactive leaves the TTY alone" "$(inv_line)"; fi
|
||||
sb_pty --no-tty --no-interactive
|
||||
if has_opt "-t" || has_opt "-i"; then _bad "--no-tty --no-interactive detaches both" "$(inv_line)"; else _ok "--no-tty --no-interactive detaches both"; fi
|
||||
sb_pty --strict-args --no-interactive
|
||||
assert_rc 0 "--no-interactive is a real flag, not a pass-through to the harness"
|
||||
if has_opt "-i"; then _bad "--no-interactive is parsed, not forwarded" "$(inv_line)"; else _ok "--no-interactive is parsed, not forwarded"; fi
|
||||
fi
|
||||
|
||||
@@ -207,12 +207,13 @@ assert_contains "will not reach the container" "preview warns instead of launchi
|
||||
|
||||
t_name "a bridged provider with no credential in reach is refused, not attempted"
|
||||
unset AI_API_TOKEN
|
||||
snapshot_containers
|
||||
sb --net none --harness pi
|
||||
assert_rc 1 "the run refuses to start"
|
||||
assert_contains 'asks for $AI_API_TOKEN, but it is not going to reach the container' "it names the variable and why"
|
||||
assert_contains "env = AI_API_TOKEN" "it shows the config-file way"
|
||||
assert_contains "--no-pi-config" "and the way to opt out"
|
||||
assert_eq "0" "$([[ -n "$(podman ps -q --filter label=ai-sandbox=1)" ]] && echo 1 || echo 0)" "and nothing was started"
|
||||
assert_no_new_containers "and nothing was started (not even a container someone else runs)"
|
||||
|
||||
t_name "an empty token is treated as missing"
|
||||
sb_env AI_API_TOKEN= --net none --harness pi # set, but empty
|
||||
|
||||
@@ -0,0 +1,569 @@
|
||||
#!/usr/bin/env bash
|
||||
# tests/unit/10_autophase.sh -- bin/ai-sandbox-autophase: the pipeline driver.
|
||||
#
|
||||
# Nothing here starts a container. The two things the driver is built on are both
|
||||
# stood in for, faithfully:
|
||||
#
|
||||
# * the LAUNCHER -- a sandbox-pi() function written into a fixture rc file, exactly
|
||||
# like the one people keep in ~/.bashrc.d. It records the argv it was handed (that
|
||||
# is what "the launcher, then --, then the harness command" has to mean) and then
|
||||
# runs what it was told to run, so the pipeline is exercised for real. It also
|
||||
# exports a canary token the way the real one calls `pass`, which is how these
|
||||
# tests check that no printed surface repeats a value.
|
||||
# * the PIPELINE LIBRARY -- tests/fixtures/phase-lib is a contract fixture: the same
|
||||
# function names, the same `pi ... | progress` call, the same gate call, the same
|
||||
# host-side commit+push at a phase's commit point. The last test runs the REAL
|
||||
# library as well, when it is installed.
|
||||
#
|
||||
# The git remote is a local bare repository: a push really happens, and really fails on
|
||||
# demand, with no network involved.
|
||||
source "$TESTS/lib.sh"
|
||||
|
||||
APBIN="$(dirname "$AISB")/ai-sandbox-autophase"
|
||||
LIB_FIX="$TESTS/fixtures/phase-lib/lib.sh"
|
||||
CANARY="CANARY-TOKEN-9d41f6"
|
||||
REAL_GIT="$(command -v git)"
|
||||
mkdir -p "$WORK/bin"
|
||||
|
||||
t_name "the driver exists"
|
||||
[[ -x "$APBIN" ]] && _ok "bin/ai-sandbox-autophase is executable" || _bad "bin/ai-sandbox-autophase is executable"
|
||||
|
||||
# --- the launcher, as it lives in an rc file ---------------------------------
|
||||
mkdir -p "$WORK/rc"
|
||||
cat > "$WORK/rc/00-nothing.bash" <<'EOF'
|
||||
# defines nothing the driver asked for: the search must keep looking
|
||||
unrelated_setup() { PATH="$PATH:/nowhere"; }
|
||||
EOF
|
||||
cat > "$WORK/rc/sandbox.bash" <<EOF
|
||||
sandbox-pi() {
|
||||
export AI_API_TOKEN='$CANARY'
|
||||
local trace="\${AISB_AP_TRACE:-/dev/null}"
|
||||
local -a argv0=("\$@") cmd=()
|
||||
while (( \$# )); do
|
||||
case "\$1" in
|
||||
--allow|--env|--pi-token-env|--mount) shift 2 ;;
|
||||
--no-tty|--no-interactive|--dry-run) shift ;;
|
||||
--) shift; cmd=("\$@"); break ;;
|
||||
*) shift ;;
|
||||
esac
|
||||
done
|
||||
{
|
||||
printf 'LAUNCH argc=%s cmd=%s\n' "\${#argv0[@]}" "\${cmd[0]:-none}"
|
||||
local a
|
||||
for a in "\${argv0[@]}"; do printf 'A:%s\n' "\$a"; done
|
||||
} >> "\$trace"
|
||||
(( \${#cmd[@]} )) || return 0
|
||||
"\${cmd[@]}"
|
||||
}
|
||||
EOF
|
||||
cat > "$WORK/rc/aliases.bash" <<'EOF'
|
||||
alias sandbox-alias='sandbox-pi'
|
||||
EOF
|
||||
cat > "$WORK/rc/broken.bash" <<'EOF'
|
||||
broken-sandbox() {
|
||||
local -a cmd=()
|
||||
while (($#)); do case "$1" in --) shift; cmd=("$@"); break ;; *) shift ;; esac; done
|
||||
echo "Error: podman: executable not found" >&2
|
||||
return 125
|
||||
}
|
||||
EOF
|
||||
|
||||
# --- the harness and the git shim -------------------------------------------
|
||||
cat > "$WORK/bin/pi" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
# fake pi: does the work, writes a session file, emits the stream a real pi emits.
|
||||
name=""; sessdir="."; prompt=""
|
||||
while (($#)); do
|
||||
case "$1" in
|
||||
--session-dir) sessdir="$2"; shift 2 ;;
|
||||
--name) name="$2"; shift 2 ;;
|
||||
--session|--mode|--model|--thinking) shift 2 ;;
|
||||
--no-session|--no-approve|--approve) shift ;;
|
||||
--*) shift ;;
|
||||
*) prompt="$1"; shift ;;
|
||||
esac
|
||||
done
|
||||
[[ -n "$prompt" ]] || { echo "fake pi: no prompt arrived" >&2; exit 3; }
|
||||
# the token has to reach the sandboxed agent -- recorded as a length, never a value
|
||||
printf 'token_bytes=%s\n' "${#AI_API_TOKEN}" >> "${FAKE_PI_MARKER:-/dev/null}"
|
||||
printf '%s\n---\n' "$prompt" >> "${FAKE_PI_PROMPTS:-/dev/null}"
|
||||
printf 'work by %s\n' "${name:-unit}" >> work.txt
|
||||
# an agent that tries to make the *host* run something when the harness commits
|
||||
if [[ -n "${FAKE_PI_PLANT_HOOK:-}" ]]; then
|
||||
mkdir -p .git/hooks
|
||||
printf '#!/bin/sh\ntouch "%s"\n' "$FAKE_PI_PLANT_HOOK" > .git/hooks/pre-commit
|
||||
chmod +x .git/hooks/pre-commit
|
||||
fi
|
||||
mkdir -p "$sessdir"
|
||||
{
|
||||
printf '{"type": "session", "id": "fake-%s"}\n' "${name:-s}"
|
||||
printf '{"type": "message_end", "message": {"role": "assistant", "content": [{"type": "text", "text": "report for %s"}]}}\n' "${name:-s}"
|
||||
printf '{"type": "agent_end"}\n'
|
||||
} | tee "${sessdir}/${name:-session}.jsonl"
|
||||
EOF
|
||||
cat > "$WORK/bin/git" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
printf '%s\n' "$*" >> "${FAKE_GIT_TRACE:-/dev/null}"
|
||||
if [[ "$1" == push && -n "${FAKE_GIT_FAIL:-}" ]]; then
|
||||
echo "fatal: unable to access '${FAKE_GIT_FAIL}': Could not resolve host" >&2
|
||||
exit 1
|
||||
fi
|
||||
exec "$REAL_GIT" "$@"
|
||||
EOF
|
||||
chmod +x "$WORK/bin/pi" "$WORK/bin/git"
|
||||
export GIT_AUTHOR_NAME=test GIT_AUTHOR_EMAIL=t@example.invalid
|
||||
export GIT_COMMITTER_NAME=test GIT_COMMITTER_EMAIL=t@example.invalid
|
||||
|
||||
# --- a phased project, with a local bare repo as its remote ------------------
|
||||
# Every project gets its own bare remote: they all commit a branch called main, and a
|
||||
# shared remote would turn the second project's push into a non-fast-forward that has
|
||||
# nothing to do with what is being tested.
|
||||
mkproj() { # <dir> [phases] -> the remote it pushed its init commit to
|
||||
local p="$1" n="${2:-1}" REMOTE
|
||||
# Named after the project, not counted: a counter incremented in a command
|
||||
# substitution would stay in the subshell and hand every project the same remote,
|
||||
# which is a non-fast-forward waiting to happen.
|
||||
REMOTE="$WORK/remotes/$(basename "$p").git"
|
||||
mkdir -p "$WORK/remotes"
|
||||
"$REAL_GIT" init -q --bare "$REMOTE" >/dev/null 2>&1
|
||||
mkdir -p "$p/.agents/phases/todo/01_one"
|
||||
printf '# task one\n\nDo the thing.\n' > "$p/.agents/phases/todo/01_one/01_task.md"
|
||||
printf '# final pass one\n\nWrap up phase 01_one.\n' > "$p/.agents/phases/todo/01_one/00_phase.md"
|
||||
if [[ "$n" == 2 ]]; then
|
||||
mkdir -p "$p/.agents/phases/todo/02_two"
|
||||
printf '# task two\n\nDo the next thing.\n' > "$p/.agents/phases/todo/02_two/01_task.md"
|
||||
printf '# final pass two\n\nWrap up phase 02_two.\n' > "$p/.agents/phases/todo/02_two/00_phase.md"
|
||||
fi
|
||||
printf '#!/usr/bin/env bash\nexit 0\n' > "$p/.agents/validate.sh"
|
||||
printf '.agents/phase-sessions/\n*.log\n' > "$p/.gitignore"
|
||||
( cd "$p" && "$REAL_GIT" init -q -b main . && "$REAL_GIT" remote add origin "$REMOTE" \
|
||||
&& "$REAL_GIT" add -A && "$REAL_GIT" commit -q --no-gpg-sign -m 'init' \
|
||||
&& "$REAL_GIT" push -q -u origin main ) >/dev/null 2>&1
|
||||
printf '%s\n' "$REMOTE" # so a test can count what actually reached the remote
|
||||
}
|
||||
|
||||
TRACE="$WORK/launcher.trace"
|
||||
GTRACE="$WORK/git.trace"
|
||||
MARKER="$WORK/token.marker"
|
||||
PROMPTS="$WORK/prompts.log"
|
||||
: > "$TRACE"; : > "$GTRACE"; : > "$MARKER"; : > "$PROMPTS"
|
||||
DEF_RC="$WORK/rc/sandbox.bash:$WORK/rc/aliases.bash:$WORK/rc/00-nothing.bash"
|
||||
|
||||
# A prefix assignment in front of a shell function sticks around for the rest of the
|
||||
# script, so ap_run reads every T_* into a local and then unsets it: each run starts
|
||||
# from the defaults unless the test asked otherwise, and cannot inherit the last one.
|
||||
ap_run() { # [args...]
|
||||
local cmd="${T_CMD-sandbox-pi}" rc="${T_RC-$DEF_RC}" lib="${T_LIB-$LIB_FIX}"
|
||||
local validate="${T_VALIDATE-sandbox}" sargs="${T_SANDBOX_ARGS---no-tty --no-interactive}"
|
||||
local push="${T_PUSH-1}" pre="${T_PREFLIGHT-1}" notes="${T_NOTES-1}" gfail="${T_GITFAIL-}"
|
||||
local ppush="${T_PHASE_PUSH-}" dryout="${T_DRYOUT-}" plant="${T_PLANT-}"
|
||||
unset T_CMD T_RC T_LIB T_VALIDATE T_SANDBOX_ARGS T_PUSH T_PREFLIGHT T_NOTES T_GITFAIL T_PHASE_PUSH T_DRYOUT T_PLANT
|
||||
OUT="$(env "PATH=$WORK/bin:$PATH" \
|
||||
"AISB_AP_LIB=$lib" "AISB_AP_RC_FILES=$rc" "AISB_AP_PI_CMD=$cmd" \
|
||||
"AISB_AP_VALIDATE=$validate" "AISB_AP_SANDBOX_ARGS=$sargs" \
|
||||
"AISB_AP_PUSH=$push" "AISB_AP_PREFLIGHT=$pre" "AISB_AP_NOTES=$notes" \
|
||||
"AISB_AP_TRACE=$TRACE" "FAKE_GIT_TRACE=$GTRACE" "FAKE_PI_MARKER=$MARKER" \
|
||||
"FAKE_PI_PROMPTS=$PROMPTS" "REAL_GIT=$REAL_GIT" "PHASE_NOTIFY=0" \
|
||||
${gfail:+"FAKE_GIT_FAIL=$gfail"} ${ppush:+"PHASE_PUSH=$ppush"} ${dryout:+"AISB_AP_DRYOUT=$dryout"} \
|
||||
${plant:+"FAKE_PI_PLANT_HOOK=$plant"} \
|
||||
"$APBIN" "$@" 2>&1)" && RC=0 || RC=$?
|
||||
return 0
|
||||
}
|
||||
gt() { grep -c "^$1" "$GTRACE" 2>/dev/null || true; } # count of git calls by verb
|
||||
trace_has() { grep -qF -- "$1" "$TRACE"; }
|
||||
|
||||
PROJ="$WORK/proj"; mkproj "$PROJ" 1 >/dev/null
|
||||
cd "$PROJ" || exit 1
|
||||
|
||||
t_name "the launcher is resolved from the rc files, and is never run to find it"
|
||||
ap_run --status
|
||||
assert_rc 0 "--status"
|
||||
assert_contains "sandbox-pi" "names the configured command"
|
||||
assert_contains "function, from $WORK/rc/sandbox.bash" "found as a function, in the file that defines it"
|
||||
assert_absent "$CANARY" "the token never appears"
|
||||
if [[ -s "$TRACE" ]]; then _bad "resolving does not run the launcher" "the launcher was called"; else _ok "resolving does not run the launcher"; fi
|
||||
ap_run --dry-run
|
||||
assert_absent "$CANARY" "--dry-run does not either"
|
||||
|
||||
t_name "an alias is a launcher too"
|
||||
T_CMD=sandbox-alias ap_run --status
|
||||
assert_rc 0 "--status with an alias"
|
||||
assert_contains "alias, from $WORK/rc/aliases.bash" "found as an alias"
|
||||
|
||||
t_name "a program on PATH, and a whole command line"
|
||||
T_CMD=pi ap_run --status
|
||||
assert_rc 0 "a program on PATH"
|
||||
assert_contains "program, from $WORK/bin/pi" "and where it came from"
|
||||
T_CMD='ai-sandbox --allow api.example.com --env API_KEY' ap_run --status
|
||||
assert_rc 0 "a command line"
|
||||
assert_contains "arguments, from --pi-cmd" "used verbatim"
|
||||
T_CMD='ai-sandbox --env API_KEY=SECRETLITERALVALUE --allow api.example.com' ap_run --status
|
||||
assert_absent "SECRETLITERALVALUE" "a literal --env value in --pi-cmd is not echoed back"
|
||||
assert_contains "--env API_KEY" "while the name still is"
|
||||
|
||||
t_name "an unresolvable launcher is refused with what was searched"
|
||||
T_CMD=no-such-sandbox-cmd ap_run --status
|
||||
assert_rc 1 "nothing by that name anywhere"
|
||||
assert_contains "cannot resolve the sandbox launcher" "says what failed"
|
||||
assert_contains "$WORK/rc/00-nothing.bash" "lists the rc files it searched"
|
||||
T_CMD='' ap_run --status
|
||||
assert_rc 1 "an explicitly empty launcher is an error, not a silent fallback"
|
||||
assert_contains "no sandbox launcher" "and says so"
|
||||
|
||||
t_name "a child is the launcher, then --, then the harness command"
|
||||
: > "$TRACE"
|
||||
PROJ_A="$WORK/pa"; mkproj "$PROJ_A" 1 >/dev/null
|
||||
cd "$PROJ_A" || exit 1
|
||||
ap_run --dry-run
|
||||
assert_rc 0 "--dry-run"
|
||||
assert_contains "-- pi <pi args> <prompt>" "the pi command rides after --"
|
||||
assert_contains "--no-tty" "no TTY for the JSON stream to corrupt"
|
||||
assert_contains "--no-interactive" "and your keystrokes cannot reach the agent"
|
||||
: > "$TRACE"
|
||||
T_SANDBOX_ARGS='' ap_run --dry-run
|
||||
assert_absent "--no-tty" "--sandbox-args replaces the defaults rather than adding to them"
|
||||
assert_rc 0 "and an explicitly empty value means none, not the default"
|
||||
: > "$TRACE"
|
||||
ap_run --max-units 1
|
||||
assert_rc 0 "one unit"
|
||||
trace_has "A:--no-interactive" "the launcher got the run flags"
|
||||
if grep -q '^LAUNCH argc=[0-9]* cmd=pi$' "$TRACE"; then
|
||||
_ok "pi is the in-container command"
|
||||
else
|
||||
_bad "pi is the in-container command" "$(grep '^LAUNCH' "$TRACE" | head -3)"
|
||||
fi
|
||||
argc="$(grep '^LAUNCH argc=.* cmd=pi$' "$TRACE" | head -1 | sed 's/.*argc=\([0-9]*\).*/\1/')"
|
||||
[[ -n "$argc" && "$argc" -ge 8 ]] && _ok "the whole harness command line reached the launcher ($argc arguments)" \
|
||||
|| _bad "the whole harness command line reached the launcher" "argc=$argc"
|
||||
if grep -A1 '^A:--$' "$TRACE" | grep -q '^A:pi$'; then
|
||||
_ok "-- is the separator, and nothing after it is a sandbox flag"
|
||||
else
|
||||
_bad "-- is the separator, and nothing after it is a sandbox flag"
|
||||
fi
|
||||
|
||||
t_name "the prompt arrives as one argument, with the sandbox rules in it"
|
||||
if grep -qF 'Sandbox rules for this run' "$PROMPTS"; then
|
||||
_ok "the agent is told it is inside a sandbox"
|
||||
else
|
||||
_bad "the agent is told it is inside a sandbox" "$(tail -5 "$PROMPTS")"
|
||||
fi
|
||||
grep -qF 'git push' "$PROMPTS" && _ok "it is told the remote is unreachable, by name" \
|
||||
|| _bad "it is told the remote is unreachable, by name"
|
||||
grep -qF '01_one/01_task.md' "$PROMPTS" && _ok "the unit it is running is in the prompt" \
|
||||
|| _bad "the unit it is running is in the prompt"
|
||||
: > "$PROMPTS"
|
||||
T_NOTES=0 ap_run --unit 01_one/00_phase.md
|
||||
assert_rc 0 "one unit, notes off"
|
||||
if grep -qF 'Sandbox rules' "$PROMPTS"; then _bad "--no-notes drops the block"; else _ok "--no-notes drops the block"; fi
|
||||
grep -qF '01_one/00_phase.md' "$PROMPTS" && _ok "the library's own prompt still arrives" \
|
||||
|| _bad "the library's own prompt still arrives" "$(head -3 "$PROMPTS")"
|
||||
|
||||
t_name "the token reaches the agent"
|
||||
if grep -q "token_bytes=${#CANARY}" "$MARKER"; then
|
||||
_ok "exported by the launcher, present in the child"
|
||||
else
|
||||
_bad "exported by the launcher, present in the child" "$(tail -1 "$MARKER" 2>/dev/null)"
|
||||
fi
|
||||
|
||||
t_name "the gate runs where the agent runs"
|
||||
: > "$TRACE"
|
||||
PROJ_B="$WORK/pb"; mkproj "$PROJ_B" 1 >/dev/null
|
||||
cd "$PROJ_B" || exit 1
|
||||
ap_run --unit 01_one/01_task.md
|
||||
assert_rc 0 "one task"
|
||||
if grep -q '^A:.agents/validate.sh$' "$TRACE"; then
|
||||
_ok ".agents/validate.sh is handed to the launcher"
|
||||
else
|
||||
_bad ".agents/validate.sh is handed to the launcher" "$(tail -5 "$TRACE")"
|
||||
fi
|
||||
: > "$TRACE"
|
||||
T_VALIDATE=host ap_run --unit 01_one/00_phase.md
|
||||
assert_rc 0 "one unit, host gate"
|
||||
if grep -q '^A:.agents/validate.sh$' "$TRACE"; then _bad "host mode does not consult the launcher"; else _ok "host mode does not consult the launcher"; fi
|
||||
assert_contains "outside the sandbox" "and the choice is announced, because it is the risky one"
|
||||
T_VALIDATE=host ap_run --status
|
||||
assert_contains "on the host (see --validate-in)" "and --status says where the gate would run"
|
||||
assert_contains "gate will run on the HOST" "and says so loudly: the gate file is agent-writable"
|
||||
T_VALIDATE=sideways ap_run --status
|
||||
assert_rc 1 "and a gate location that is neither is refused"
|
||||
|
||||
t_name "a completed phase is committed and pushed by the host"
|
||||
PROJ1="$WORK/p1"; R1="$(mkproj "$PROJ1" 1)"
|
||||
: > "$GTRACE"; cd "$PROJ1" || exit 1
|
||||
ap_run
|
||||
assert_rc 0 "the pipeline ran to completion"
|
||||
if grep -q 'report for 01_one__00_phase' "$PROJ1/.agents/reports/01_one/01_one__00_phase.a1.md" 2>/dev/null; then
|
||||
_ok "the final pass ran last, and its report is on disk"
|
||||
else
|
||||
_bad "the final pass ran last, and its report is on disk" "$(ls "$PROJ1/.agents/reports/01_one" 2>/dev/null)"
|
||||
fi
|
||||
if grep -q '^push' "$GTRACE"; then _ok "a push went out for the phase"; else _bad "a push went out for the phase"; fi
|
||||
assert_contains "up to date with origin/main" "and the end-of-run check confirmed the remote has it"
|
||||
if [[ -d "$PROJ1/.agents/phases/complete/01_one" && ! -d "$PROJ1/.agents/phases/todo/01_one" ]]; then
|
||||
_ok "the phase moved to complete/"
|
||||
else
|
||||
_bad "the phase moved to complete/" "$(ls "$PROJ1/.agents/phases"/* 2>/dev/null)"
|
||||
fi
|
||||
if ( cd "$PROJ1" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then
|
||||
_ok "the host made the phase commit"
|
||||
else
|
||||
_bad "the host made the phase commit" "$( cd "$PROJ1" && "$REAL_GIT" log --oneline | head -3 )"
|
||||
fi
|
||||
assert_eq 2 "$( git --git-dir="$R1" rev-list --count --all )" "the remote has init + the phase"
|
||||
|
||||
t_name "the driver owns delivery: push at the phase boundary, re-check, push again at the end"
|
||||
PROJ2="$WORK/p2"; R2="$(mkproj "$PROJ2" 2)"
|
||||
: > "$GTRACE"; cd "$PROJ2" || exit 1
|
||||
ap_run
|
||||
assert_rc 0 "two phases ran"
|
||||
commits="$(gt commit)"; pushes="$(gt push)"
|
||||
[[ "$commits" == 2 ]] && _ok "both phases committed on the host" || _bad "both phases committed on the host" "commits=$commits"
|
||||
[[ "$pushes" == "$commits" ]] && _ok "every commit is delivered, not left for a later run ($pushes)" \
|
||||
|| _bad "every commit is delivered, not left for a later run" "commits=$commits pushes=$pushes"
|
||||
assert_contains "phase 01_one" "the push is attributed to the phase that made it"
|
||||
assert_contains "up to date with origin/main" "and the next phase's start re-checks before running it"
|
||||
assert_eq 3 "$( git --git-dir="$R2" rev-list --count --all )" "the remote has init + both phases"
|
||||
|
||||
# PHASE_PUSH is the library's switch; it means the same thing here, so it is honoured
|
||||
# rather than overridden -- and honoured means nothing goes out, in either direction.
|
||||
PROJ3="$WORK/p3"; R3="$(mkproj "$PROJ3" 2)"
|
||||
: > "$GTRACE"; cd "$PROJ3" || exit 1
|
||||
T_PHASE_PUSH=0 ap_run
|
||||
assert_rc 0 "the pipeline still ran"
|
||||
if grep -q '^push' "$GTRACE"; then _bad "PHASE_PUSH=0 pushes nothing"; else _ok "PHASE_PUSH=0 pushes nothing"; fi
|
||||
assert_contains "push disabled" "and says so"
|
||||
assert_eq 1 "$( git --git-dir="$R3" rev-list --count --all )" "the remote is untouched"
|
||||
|
||||
t_name "--no-push leaves the commits local, and says so"
|
||||
PROJ4="$WORK/p4"; R4="$(mkproj "$PROJ4" 1)"
|
||||
: > "$GTRACE"; cd "$PROJ4" || exit 1
|
||||
ap_run --no-push
|
||||
assert_rc 0 "the pipeline ran"
|
||||
if grep -q '^push' "$GTRACE"; then _bad "nothing is pushed" "$(grep '^push' "$GTRACE")"; else _ok "nothing is pushed"; fi
|
||||
assert_contains "push disabled" "and that is stated, not silent"
|
||||
if [[ "$(gt commit)" == 1 ]] && ( cd "$PROJ4" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then
|
||||
_ok "the work is still committed locally"
|
||||
else
|
||||
_bad "the work is still committed locally" "$( cd "$PROJ4" && "$REAL_GIT" log --oneline | head -3 )"
|
||||
fi
|
||||
assert_eq 1 "$( git --git-dir="$R4" rev-list --count --all )" "the remote is untouched"
|
||||
|
||||
t_name "a failed push stops the run, with the hand-fix command"
|
||||
PROJ5="$WORK/p5"; mkproj "$PROJ5" 1 >/dev/null
|
||||
: > "$GTRACE"; cd "$PROJ5" || exit 1
|
||||
T_GITFAIL="https://user:pushcanary@remote.invalid/repo.git" ap_run
|
||||
assert_rc 1 "the run stops on a failed push"
|
||||
assert_contains "the push FAILED" "says what failed"
|
||||
assert_contains "finish by hand" "and how to finish it"
|
||||
assert_absent "pushcanary" "a credential in the remote URL is not repeated in the output"
|
||||
: > "$GTRACE"
|
||||
ap_run # the remote answers this time
|
||||
assert_contains "unpushed commit" "the next run notices what is still local"
|
||||
assert_contains "pushed:" "and takes it out"
|
||||
# and a run that could not deliver never reports success
|
||||
PROJ5B="$WORK/p5b"; mkproj "$PROJ5B" 1 >/dev/null; cd "$PROJ5B" || exit 1
|
||||
T_GITFAIL="https://user:pushcanary@remote.invalid/repo.git" ap_run
|
||||
assert_rc 1 "a run that could not deliver does not report success"
|
||||
assert_contains "the push FAILED" "and says which part failed"
|
||||
if ( cd "$PROJ5B" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then
|
||||
_ok "its commit is still local and safe"
|
||||
else
|
||||
_bad "its commit is still local and safe"
|
||||
fi
|
||||
if ( cd "$PROJ5" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then
|
||||
_ok "the phase stayed complete through it"
|
||||
else
|
||||
_bad "the phase stayed complete through it"
|
||||
fi
|
||||
|
||||
t_name "nothing a preview prints carries a value"
|
||||
PROJ6="$WORK/p6"; mkproj "$PROJ6" 1 >/dev/null
|
||||
cd "$PROJ6" || exit 1
|
||||
: > "$TRACE"
|
||||
out_status="$(cd "$PROJ6" && env PATH="$WORK/bin:$PATH" AISB_AP_LIB="$LIB_FIX" \
|
||||
AISB_AP_RC_FILES="$WORK/rc/sandbox.bash" AISB_AP_PI_CMD=sandbox-pi \
|
||||
AISB_AP_TRACE="$TRACE" "$APBIN" --status 2>&1)"
|
||||
out_dry="$(cd "$PROJ6" && env PATH="$WORK/bin:$PATH" AISB_AP_LIB="$LIB_FIX" \
|
||||
AISB_AP_RC_FILES="$WORK/rc/sandbox.bash" AISB_AP_PI_CMD=sandbox-pi \
|
||||
AISB_AP_TRACE="$TRACE" "$APBIN" --dry-run 2>&1)"
|
||||
for v in "$out_status" "$out_dry"; do
|
||||
if printf '%s' "$v" | grep -qF -- "$CANARY"; then
|
||||
_bad "a preview does not echo the launcher's secret"
|
||||
else
|
||||
_ok "a preview does not echo the launcher's secret"
|
||||
fi
|
||||
if printf '%s' "$v" | grep -qF "export AI_API_TOKEN"; then
|
||||
_bad "a preview does not print the launcher's body"
|
||||
else
|
||||
_ok "a preview does not print the launcher's body"
|
||||
fi
|
||||
done
|
||||
if [[ -s "$TRACE" ]]; then _bad "a preview runs nothing"; else _ok "a preview runs nothing"; fi
|
||||
assert_eq "$( git -C "$PROJ6" rev-list --count HEAD )" "$( git -C "$PROJ6" rev-list --count HEAD )" "a preview changes no git state"
|
||||
if [[ -e "$PROJ6/.agents/reports" ]]; then _bad "a preview writes no reports"; else _ok "a preview writes no reports"; fi
|
||||
if [[ -e "$PROJ6/.agents/phases/complete/01_one" ]]; then _bad "a preview does not move units"; else _ok "a preview does not move units"; fi
|
||||
OUT="$out_status"
|
||||
assert_contains "01_one/01_task.md" "but it does list what is pending"
|
||||
assert_contains "01_one/00_phase.md" "task first, then the final pass"
|
||||
OUT="$out_dry"
|
||||
assert_contains "one container per unit" "and --dry-run says what it would do"
|
||||
|
||||
t_name "preflight refuses to start a pipeline the sandbox cannot run"
|
||||
cd "$PROJ6" || exit 1
|
||||
: > "$TRACE"
|
||||
T_CMD=broken-sandbox T_RC="$WORK/rc/broken.bash" ap_run --max-units 1
|
||||
assert_rc 1 "preflight stops the run"
|
||||
assert_contains "preflight failed" "and says so"
|
||||
assert_contains "executable not found" "with the launcher's own error, for diagnosis"
|
||||
if grep -q 'cmd=pi' "$TRACE"; then _bad "no unit runs"; else _ok "no unit runs"; fi
|
||||
T_PREFLIGHT=0 ap_run --max-units 1
|
||||
assert_rc 0 "--no-preflight is available for the runs that cannot afford it"
|
||||
|
||||
t_name "bad usage is refused before anything happens"
|
||||
cd "$PROJ6" || exit 1
|
||||
ap_run --not-a-flag
|
||||
assert_rc 2 "an unknown option"
|
||||
ap_run --max-units lots
|
||||
assert_rc 1 "--max-units wants a number"
|
||||
cd / || exit 1
|
||||
ap_run --status
|
||||
assert_rc 1 "outside a phased project"
|
||||
assert_contains "no .agents/phases/todo" "and it says what is missing"
|
||||
cd "$PROJ6" || exit 1
|
||||
|
||||
t_name "a library that does not match the contract is refused"
|
||||
printf 'find_root() { printf "%%s\\n" "$PWD"; }\nnext_unit() { :; }\n' > "$WORK/rc/half-lib.sh"
|
||||
T_LIB="$WORK/rc/half-lib.sh" ap_run --status
|
||||
assert_rc 1 "a library missing pieces is not run against"
|
||||
assert_contains "does not provide" "and the missing function is named"
|
||||
|
||||
t_name "a hook planted from inside the sandbox never runs on the host"
|
||||
# The commit is made out here, by us, after an agent had the tree writeable. A repository
|
||||
# does not merely record .git/hooks and .git/config -- it runs them. So the delivery
|
||||
# refuses when they changed, and even when it does not, our git never looks in .git/hooks.
|
||||
HOOKPROOF="$WORK/hook-ran"
|
||||
PROJ9="$WORK/p9"; R9="$(mkproj "$PROJ9" 1)"
|
||||
cd "$PROJ9" || exit 1
|
||||
: > "$GTRACE"
|
||||
T_PLANT="$HOOKPROOF" ap_run
|
||||
assert_rc 1 "a run that rewrote the repository's own git config does not deliver"
|
||||
assert_contains "refusing to deliver" "and says why"
|
||||
assert_contains "--trust-git-changes" "and names the way through"
|
||||
if [[ -e "$HOOKPROOF" ]]; then _bad "the planted pre-commit hook did not run"; else _ok "the planted pre-commit hook did not run"; fi
|
||||
if grep -q '^push' "$GTRACE"; then _bad "nothing was pushed"; else _ok "nothing was pushed"; fi
|
||||
assert_eq 1 "$( git --git-dir="$R9" rev-list --count --all )" "the remote still has only the init commit"
|
||||
ap_run --trust-git-changes
|
||||
assert_rc 0 "the named escape hatch does deliver"
|
||||
assert_contains "pushed:" "and says so"
|
||||
if [[ -e "$HOOKPROOF" ]]; then _bad "and still the hook did not run"; else _ok "and still the hook did not run: our git uses an empty hooks path"; fi
|
||||
assert_eq 2 "$( git --git-dir="$R9" rev-list --count --all )" "the phase commit is on the remote now"
|
||||
# and it is refused even when this script is not the one that would push: the point is
|
||||
# that the operator hears about it, not that a push is blocked
|
||||
PROJ9B="$WORK/p9b"; mkproj "$PROJ9B" 1 >/dev/null; cd "$PROJ9B" || exit 1
|
||||
T_PLANT="$HOOKPROOF" ap_run --no-push
|
||||
assert_rc 1 "--no-push does not make it safe to ignore"
|
||||
assert_contains "refusing to deliver" "and the same message explains it"
|
||||
if [[ -e "$HOOKPROOF" ]]; then _bad "the hook did not run here either"; else _ok "the hook did not run here either"; fi
|
||||
|
||||
t_name "what the driver composes is a real ai-sandbox invocation"
|
||||
# The fixture launcher stands in for ai-sandbox everywhere else, because this suite runs
|
||||
# without containers. Here the launcher IS ai-sandbox, in --dry-run: if the flags land on
|
||||
# the wrong side of --, or --no-interactive were forwarded to the harness, or the project
|
||||
# were not the mount, ai-sandbox itself says so.
|
||||
cat > "$WORK/rc/drylaunch.bash" <<EOF
|
||||
sandbox-dry() {
|
||||
$AISB --dry-run --net none --allow api.example.com "\$@" >> "\${AISB_AP_DRYOUT:-/dev/null}" 2>&1
|
||||
}
|
||||
EOF
|
||||
PROJ8="$WORK/p8"; mkproj "$PROJ8" 1 >/dev/null
|
||||
cd "$PROJ8" || exit 1
|
||||
DRYOUT="$WORK/ai-sandbox.dryrun"; : > "$DRYOUT"
|
||||
T_CMD=sandbox-dry T_RC="$WORK/rc/drylaunch.bash" T_PREFLIGHT=0 T_DRYOUT="$DRYOUT" \
|
||||
ap_run --max-units 1
|
||||
# the child produces no JSON (ai-sandbox printed an invocation instead), so the unit
|
||||
# does not complete -- what is under test is the command that produced that output
|
||||
if grep -q 'effective podman invocation' "$DRYOUT"; then
|
||||
_ok "ai-sandbox accepted the composed command"
|
||||
else
|
||||
_bad "ai-sandbox accepted the composed command" "$(head -4 "$DRYOUT")"
|
||||
fi
|
||||
if grep -q 'egress: net=none' "$DRYOUT"; then
|
||||
_ok "the launcher's own policy is still what is enforced"
|
||||
else
|
||||
_bad "the launcher's own policy is still what is enforced" "$(head -2 "$DRYOUT")"
|
||||
fi
|
||||
if grep -qF -- '--no-interactive' "$DRYOUT" || grep -qF -- '--no-tty' "$DRYOUT"; then
|
||||
_bad "the run's sandbox flags are consumed by ai-sandbox, not passed on to pi"
|
||||
else
|
||||
_ok "the run's sandbox flags are consumed by ai-sandbox, not passed on to pi"
|
||||
fi
|
||||
if grep -qE -- ' pi (--no-approve|--session-dir)' "$DRYOUT"; then
|
||||
_ok "pi is the in-container command, after --"
|
||||
else
|
||||
_bad "pi is the in-container command, after --" "$(tail -1 "$DRYOUT" | cut -c1-200)"
|
||||
fi
|
||||
if grep -qE -- '(^| )-i( |$)' "$DRYOUT" || grep -qE -- '(^| )-t( |$)' "$DRYOUT"; then
|
||||
_bad "nothing is attached to the terminal: no -t in the JSON stream, no -i to steal keystrokes"
|
||||
else
|
||||
_ok "nothing is attached to the terminal: no -t in the JSON stream, no -i to steal keystrokes"
|
||||
fi
|
||||
# The project is mounted at its own host path where the sandbox can mirror it, and at
|
||||
# /project where it cannot -- the driver composes neither, it just does not disturb it.
|
||||
MOUNT_AT="$([[ -n "$(mirror_block_of "$PROJ8")" ]] && echo /project || echo "$PROJ8")"
|
||||
if grep -qF -- "$PROJ8:$MOUNT_AT" "$DRYOUT"; then
|
||||
_ok "the project directory is the mount the agent writes into"
|
||||
else
|
||||
_bad "the project directory is the mount the agent writes into"
|
||||
fi
|
||||
if grep -qF -- 'bash .agents/validate.sh' "$DRYOUT"; then
|
||||
_ok "and so is the gate: validate.sh runs in the same sandbox, as the same user"
|
||||
else
|
||||
_bad "and so is the gate: validate.sh runs in the same sandbox, as the same user" "$(tail -2 "$DRYOUT" | cut -c1-200)"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The real library, real renderer, real commit -- with the same fake sandbox.
|
||||
t_name "the real phased-execution library, end to end"
|
||||
REAL_LIB="$HOME/.agents/skills/phased-execution/scripts/lib.sh"
|
||||
if [[ ! -f "$REAL_LIB" ]]; then
|
||||
_ok "the skill is not installed here -- the real-library run is skipped"
|
||||
else
|
||||
PROJ7="$WORK/p7"; mkproj "$PROJ7" 1 >/dev/null
|
||||
: > "$GTRACE"; cd "$PROJ7" || exit 1
|
||||
# shellcheck disable=SC2034 # assert_rc() in tests/lib.sh is what reads RC
|
||||
OUT="$(env PATH="$WORK/bin:$PATH" AISB_AP_LIB="$REAL_LIB" \
|
||||
AISB_AP_RC_FILES="$WORK/rc/sandbox.bash" AISB_AP_PI_CMD=sandbox-pi \
|
||||
AISB_AP_VALIDATE=sandbox AISB_AP_TRACE="$TRACE" FAKE_GIT_TRACE="$GTRACE" \
|
||||
FAKE_PI_MARKER="$MARKER" REAL_GIT="$REAL_GIT" PHASE_NOTIFY=0 QUIET=1 \
|
||||
"$APBIN" 2>&1)" && RC=0 || RC=$?
|
||||
assert_rc 0 "the real pipeline completed a phase inside the sandbox launcher"
|
||||
if grep -rq 'report for 01_one__00_phase' "$PROJ7/.agents/reports/" 2>/dev/null; then
|
||||
_ok "the real progress renderer read the child stream and wrote the report"
|
||||
else
|
||||
_bad "the real progress renderer read the child stream and wrote the report" "$(ls -R "$PROJ7/.agents/reports" 2>/dev/null | head)"
|
||||
fi
|
||||
if compgen -G "$PROJ7/.agents/reports/01_one/*.md" > /dev/null; then
|
||||
_ok "the executor reports were written where the skill puts them"
|
||||
else
|
||||
_bad "the executor reports were written where the skill puts them" "$(ls -R "$PROJ7/.agents" 2>/dev/null | head -20)"
|
||||
fi
|
||||
if ( cd "$PROJ7" && "$REAL_GIT" log --oneline | grep -q 'phase: 01_one' ); then
|
||||
_ok "the real commit_phase made the scoped commit"
|
||||
else
|
||||
_bad "the real commit_phase made the scoped commit" "$( cd "$PROJ7" && "$REAL_GIT" log --oneline | head -3 )"
|
||||
fi
|
||||
assert_contains "pushed:" "and the push went out"
|
||||
if printf '%s' "$OUT" | grep -qF -- "$CANARY"; then _bad "no token in the real pipeline output"; else _ok "no token in the real pipeline output"; fi
|
||||
if ( cd "$PROJ7" && "$REAL_GIT" ls-tree -r --name-only HEAD | grep -q '.agents/phases/complete/01_one/00_phase.md' ); then
|
||||
_ok "the moved phase file is in the commit"
|
||||
else
|
||||
_bad "the moved phase file is in the commit" "$( cd "$PROJ7" && "$REAL_GIT" ls-tree -r --name-only HEAD | head -10 )"
|
||||
fi
|
||||
if grep -qF 'Sandbox rules for this run' "$PROMPTS"; then
|
||||
_ok "and the real prompts carried the sandbox rules"
|
||||
else
|
||||
_bad "and the real prompts carried the sandbox rules"
|
||||
fi
|
||||
fi
|
||||
|
||||
cd "$WORK" || exit 1
|
||||
Reference in New Issue
Block a user