ai-sandbox/docs/networking.md

Networking: how --net allow actually holds

Everything in this file was measured on the reference host (Fedora 44, podman 5.8.9, rootless, pasta as the rootless network helper, SELinux enforcing) with tests/spike_network.sh. Re-run that script on a new machine before trusting the model; podman's rootless networking is the part of the stack most likely to differ.

The two mechanisms

--net allow stacks two independent controls:

1. A netfilter ruleset inside the container's own network namespace. The container starts with --cap-add=NET_ADMIN and the image's entrypoint runs as root only long enough to install an inet aisb table with policy drop on the output hook, then it execs the real command through

   setpriv --reuid=1000 --regid=1000 --clear-groups --inh-caps=-all --bounding-set=-all --no-new-privs
   

   After that exec the command has no capabilities and no root. Removing a rule needs CAP_NET_ADMIN, opening a new network namespace needs CAP_SYS_ADMIN, and a setuid helper can't hand either back because no-new-privileges is set on the container. The ruleset is therefore a one-way door.

2. An egress gate on the host loopback. With pasta, a container can be given a magic address that means the host's loopback:

   --network=pasta:--map-host-loopback,100.64.0.1
   

   Connecting to 100.64.0.1:PORT inside the container lands on the host's 127.0.0.1:PORT. The gate is a squid container started with --network=host (so it has internet and the host's resolver) bound to 127.0.0.1:<free port>, with http_access deny all after the allowlist. The sandbox gets HTTP(S)_PROXY pointing at 100.64.0.1:<that port>, and the netfilter ruleset allows nothing else out.

The gate itself runs as the image's unprivileged proxy user (--user proxy, --cap-drop=ALL, pid_filename /tmp/...): squid normally starts as root and drops privileges itself, which needs SETUID/SETGID we would rather not grant. Binding a high loopback port does not need them. squid 6 treats obsolete directives as hard errors, so the generated config avoids dns_v4_first and max_mem_obj_size and does not redeclare the built-in manager ACL.

Together: the harness can only talk to the gate, and the gate only relays to domains you allowlisted, resolved by the gate itself. CDNs, SNI and CONNECT-time filtering all work because squid terminates the CONNECT request and resolves the name itself.

Publishing a port (ingress)

--publish is the one control that runs the other way: podman/pasta puts a listener on the host and forwards what arrives into the container, so a client on the host can reach a server the sandboxed process listens on. Nothing about the egress model changes -- the same run that answers your browser still refuses every domain outside the allowlist (tests/e2e/13_publish.sh asserts exactly that, in one container).

Binding and parsing decisions, and why:

decision	reason
default bind address 127.0.0.1	measured: a bare -p 8000:8000 binds *:8000, i.e. every interface. A port in a committed config must not quietly be a port on the LAN. publish_addr or an explicit addr: changes it, and a non-loopback address warns.
one port per entry, no ranges, no /udp	this list is a policy a human reviews; a range is a line nobody reads.
* ("every interface") is refused; 0.0.0.0 is accepted	podman's shorthand hides what it opens. Making someone type the address is the whole warning.
an address requires three parts	0.0.0.0:8000 means two different things in podman's grammar. Refusing it with a message beats guessing.
--net none + --publish refuses	measured: podman prints one line about discarding the mapping and starts the container regardless. "It ran" with nothing listening is the worst outcome here.
a busy host port refuses the run	measured before the container starts, because podman's own error arrives after any build.
the ruleset accepts the published ports on the input hook	see below: it changes nothing on the reference host, and it is the audit trail on every host.

Where the packet actually arrives was the interesting measurement. The input chain is policy drop, so a port forward looks doomed from the reading of it -- and it is not:

Probe	Result	Consequence
--net allow --publish H:8000, server on 0.0.0.0, curl from the host	200, with enforce=nft in the same run	the existing ruleset does not block a published port
nft -a list chain inet aisb input while that port serves traffic	aisb-publish counter: 0 packets	pasta forwards the connection into the namespace over loopback, so iifname "lo" accept carries it
server bound to 127.0.0.1 inside, port published	host: connection refused; inside: 200	pasta delivers to the container's own address, not to its loopback → dev servers must bind 0.0.0.0
ss -ltnp on the host while published	LISTEN 127.0.0.1:H ... pasta.avx2	the listener is pasta's, on loopback, and it is gone when the container exits
container's own view of its interface	wlp4s0 with the host's LAN address	pasta mirrors the host address into the namespace, which is why a remote host cannot reach the published port: only pasta's own socket on the host accepts, and it accepts on the address we named

So publish_rules() in the entrypoint is deliberately not what makes publishing work here. It is there because (a) a different host, or a bridge network, can deliver the packet on the container's own interface, where policy drop would leave a port that looks configured and answers nothing, and (b) nft list table inet aisb should answer "what can reach this container" instead of leaving that question to a flag on the host.

Measurements

Probe	Result	Consequence
--network=<internal> container, egress	blocked (no default route)	a blackout network is available
--network=<internal> container, DNS	blocked	internal mode is a real blackout
container → container on the same user-defined network	does not work (Failed to connect ... Could not connect to server)	rules out the "sandbox + sidecar proxy on a shared internal network" design
--network=host in a rootless container	works, has internet	makes the host-loopback gate possible
--network=pasta (default), egress	works	pasta is a plain internet tap, no filtering
container → host 127.0.0.1 via --map-host-loopback,100.64.0.1	works	the gate address
host → container via -p 127.0.0.1:H:C, --net allow, ruleset installed	works; the input hook sees 0 packets for it	ingress arrives on lo — see "Publishing a port" above
-p H:C with no address	binds *:H on the host	the default bind address in ai-sandbox is 127.0.0.1, not podman's
--map-host-loopback,127.0.0.1	rejected by passt (Invalid address to remap to host)	the magic address must not itself be loopback
--network=pasta:--map-gw	works (the gateway address already maps to the host by default)	--map-host-loopback chosen for explicitness
squid as root with --cap-drop=ALL	setgid: (1) Operation not permitted, gate dies	run the gate as proxy from the start
squid 6 with dns_v4_first / max_mem_obj_size	ERROR: Directive ... is obsolete, config refused	generate only current directives
sandbox in allow mode, curl with the proxy env removed	connection times out	netfilter, not just HTTP_PROXY, is doing the gating
sandbox in allow mode, raw TCP socket to a literal IP, DNS to 8.8.8.8	both blocked	nothing bypasses the ruleset by ignoring proxy variables
root in a rootless container, nft add table with podman's default caps	fails (Operation not permitted)	NET_ADMIN is not in podman's default cap set, so it must be added deliberately
root + --cap-add=NET_ADMIN, drop tcp dport 80 inside the container	egress to port 80 times out	in-container netfilter really does gate the pasta path
--cap-drop=ALL, nft add table	fails	a non-privileged process cannot install or remove rules
--pids-limit 64 --memory 256m --cpus 0.5	pids.max=64, memory.max=268435456 visible inside	limits are real (note: /proc/cpuinfo still lists all cores; use nproc/cgroup limits, not cpuinfo)
-v $dir:/project with SELinux enforcing, --userns=keep-id:uid=1000,gid=1000	write denied	bind mounts need :z on SELinux hosts
same, with :z	write succeeds, file owned by the host uid (1000:1000)	project files keep sane ownership
--tmpfs /tmp and -v /tmp/x:/tmp/x in one run	the tmpfs wins: mount shows tmpfs on /tmp/x, pwd prints /tmp/x, writes succeed	a project mounted at its own path under /tmp would be an empty, writable, correctly-named directory that is not the project — so mirror_block_of() refuses /tmp rather than degrading quietly
-v $dir:$dir (a per-user path) beside the /tmp tmpfs, the /home/sandbox volume and the .agents share	pwd is the host path, writes land on the host owned by the host uid, /project + the home volume + the share all still there	mirroring buries nothing outside the denied set; asserted per release in tests/e2e/12_sandbox.sh

Why not the other designs

• Internal network + sidecar proxy container. The obvious design, and it is what most write-ups suggest. On this host, two containers on the same user-defined network cannot reach each other at all, so the sandbox could never reach its own proxy. Rejected on measurement, not taste.
• --network=none plus a unix-socket proxy. Structurally the strongest (no IP stack at all) and worth revisiting, but only curl and a handful of tools speak --unix-socket; everything else (git, npm, pip, node's fetch) would need a TCP-to-socket forwarder inside the container, and DNS would need its own path. The current design gets the same practical containment with the two controls above.
• DNS filtering only. A resolver allowlist is trivially bypassed by connecting to a literal IP, and it breaks the SNI-based allowlist that real agent traffic needs. Used only as the fallback in --no-gate mode, where it is paired with IP-pinned netfilter rules.
• --network=slirp4netns. Not installed here; pasta is the default helper in podman 5 and is what the code targets (it falls back to whatever helper podman picks, since the code only ever asks for the pasta driver).

Failure modes we chose

• If the ruleset cannot be installed (no NET_ADMIN, no nft, weird kernel) the container refuses to start with exit 77 rather than running unfiltered. --allow-unenforced exists for debugging and says so in the flag name.
• If the policy file is missing the entrypoint exits 78 before running anything.
• The gate container is created per session, bound to loopback only, and removed when the sandboxed command exits. A gate that outlives its session would be an open door with nobody watching it. gate up is the explicit, named exception.
• In gate mode DNS is closed inside the container. That is deliberate: it stops a sandboxed process from resolving names and then talking to them behind the proxy's back. Tools that insist on resolving names themselves need --allow <ip/cidr> entries instead.