# Networking: how `--net allow` actually holds

Everything in this file was measured on the reference host (Fedora 44, podman
5.8.9, rootless, `pasta` as the rootless network helper, SELinux enforcing) with
[`tests/spike_network.sh`](../tests/spike_network.sh). Re-run that script on a new
machine before trusting the model; podman's rootless networking is the part of the
stack most likely to differ.

## The two mechanisms

`--net allow` stacks two independent controls:

1. **A netfilter ruleset inside the container's own network namespace.**
   The container starts with `--cap-add=NET_ADMIN` and the image's entrypoint runs
   as root *only long enough* to install an `inet aisb` table with
   `policy drop` on the output hook, then it execs the real command through

   ```
   setpriv --reuid=1000 --regid=1000 --clear-groups --inh-caps=-all --bounding-set=-all --no-new-privs
   ```

   After that exec the command has no capabilities and no root. Removing a rule
   needs `CAP_NET_ADMIN`, opening a new network namespace needs `CAP_SYS_ADMIN`,
   and a setuid helper can't hand either back because `no-new-privileges` is set on
   the container. The ruleset is therefore a one-way door.

2. **An egress gate on the host loopback.** With `pasta`, a container can be given
   a magic address that *means* the host's loopback:

   ```
   --network=pasta:--map-host-loopback,100.64.0.1
   ```

   Connecting to `100.64.0.1:PORT` inside the container lands on the host's
   `127.0.0.1:PORT`. The gate is a squid container started with `--network=host`
   (so it has internet and the host's resolver) bound to `127.0.0.1:<free port>`,
   with `http_access deny all` after the allowlist. The sandbox gets
   `HTTP(S)_PROXY` pointing at `100.64.0.1:<that port>`, and the netfilter ruleset
   allows nothing else out.

The gate itself runs as the image's unprivileged `proxy` user (`--user proxy`,
`--cap-drop=ALL`, `pid_filename /tmp/...`): squid normally starts as root and drops
privileges itself, which needs SETUID/SETGID we would rather not grant. Binding a
high loopback port does not need them. squid 6 treats obsolete directives as hard
errors, so the generated config avoids `dns_v4_first` and `max_mem_obj_size` and does
not redeclare the built-in `manager` ACL.

Together: the harness can only talk to the gate, and the gate only relays to
domains you allowlisted, resolved by the gate itself. CDNs, SNI and CONNECT-time
filtering all work because squid terminates the CONNECT request and resolves the
name itself.

## Publishing a port (ingress)

`--publish` is the one control that runs the other way: podman/pasta puts a listener on the
host and forwards what arrives into the container, so a client on the host can reach a server
the sandboxed process listens on. Nothing about the egress model changes -- the same run that
answers your browser still refuses every domain outside the allowlist
(`tests/e2e/13_publish.sh` asserts exactly that, in one container).

Binding and parsing decisions, and why:

| decision | reason |
|---|---|
| default bind address `127.0.0.1` | measured: a bare `-p 8000:8000` binds `*:8000`, i.e. every interface. A port in a committed config must not quietly be a port on the LAN. `publish_addr` or an explicit `addr:` changes it, and a non-loopback address warns. |
| one port per entry, no ranges, no `/udp` | this list is a policy a human reviews; a range is a line nobody reads. |
| `*` ("every interface") is refused; `0.0.0.0` is accepted | podman's shorthand hides what it opens. Making someone type the address is the whole warning. |
| an address requires three parts | `0.0.0.0:8000` means two different things in podman's grammar. Refusing it with a message beats guessing. |
| `--net none` + `--publish` refuses | measured: podman prints one line about discarding the mapping and starts the container regardless. "It ran" with nothing listening is the worst outcome here. |
| a busy host port refuses the run | measured before the container starts, because podman's own error arrives after any build. |
| the ruleset accepts the published ports on the `input` hook | see below: it changes nothing on the reference host, and it is the audit trail on every host. |

Where the packet actually arrives was the interesting measurement. The `input` chain is
`policy drop`, so a port forward looks doomed from the reading of it -- and it is not:

| Probe | Result | Consequence |
|---|---|---|
| `--net allow --publish H:8000`, server on `0.0.0.0`, `curl` from the host | **200**, with `enforce=nft` in the same run | the existing ruleset does not block a published port |
| `nft -a list chain inet aisb input` while that port serves traffic | `aisb-publish` counter: **0 packets** | pasta forwards the connection into the namespace over **loopback**, so `iifname "lo" accept` carries it |
| server bound to `127.0.0.1` **inside**, port published | host: connection refused; inside: 200 | pasta delivers to the container's own address, not to its loopback → dev servers must bind `0.0.0.0` |
| `ss -ltnp` on the host while published | `LISTEN 127.0.0.1:H ... pasta.avx2` | the listener is pasta's, on loopback, and it is gone when the container exits |
| container's own view of its interface | `wlp4s0` with the **host's** LAN address | pasta mirrors the host address into the namespace, which is why a *remote* host cannot reach the published port: only pasta's own socket on the host accepts, and it accepts on the address we named |

So `publish_rules()` in the entrypoint is deliberately not what makes publishing work here.
It is there because (a) a different host, or a bridge network, can deliver the packet on the
container's own interface, where `policy drop` would leave a port that looks configured and
answers nothing, and (b) `nft list table inet aisb` should answer "what can reach this
container" instead of leaving that question to a flag on the host.

## Measurements

| Probe | Result | Consequence |
|---|---|---|
| `--network=<internal>` container, egress | blocked (no default route) | a blackout network is available |
| `--network=<internal>` container, DNS | blocked | internal mode is a real blackout |
| container → container on the same user-defined network | **does not work** (`Failed to connect ... Could not connect to server`) | rules out the "sandbox + sidecar proxy on a shared internal network" design |
| `--network=host` in a rootless container | works, has internet | makes the host-loopback gate possible |
| `--network=pasta` (default), egress | works | pasta is a plain internet tap, no filtering |
| container → host `127.0.0.1` via `--map-host-loopback,100.64.0.1` | **works** | the gate address |
| host → container via `-p 127.0.0.1:H:C`, `--net allow`, ruleset installed | **works**; the `input` hook sees 0 packets for it | ingress arrives on `lo` — see "Publishing a port" above |
| `-p H:C` with no address | binds `*:H` on the host | the default bind address in `ai-sandbox` is `127.0.0.1`, not podman's |
| `--map-host-loopback,127.0.0.1` | rejected by passt (`Invalid address to remap to host`) | the magic address must not itself be loopback |
| `--network=pasta:--map-gw` | works (the gateway address already maps to the host by default) | `--map-host-loopback` chosen for explicitness |
| squid as root with `--cap-drop=ALL` | `setgid: (1) Operation not permitted`, gate dies | run the gate as `proxy` from the start |
| squid 6 with `dns_v4_first` / `max_mem_obj_size` | `ERROR: Directive ... is obsolete`, config refused | generate only current directives |
| sandbox in allow mode, `curl` with the proxy env removed | connection times out | netfilter, not just `HTTP_PROXY`, is doing the gating |
| sandbox in allow mode, raw TCP socket to a literal IP, DNS to 8.8.8.8 | both blocked | nothing bypasses the ruleset by ignoring proxy variables |
| root in a rootless container, `nft add table` with podman's default caps | **fails** (`Operation not permitted`) | NET_ADMIN is not in podman's default cap set, so it must be added deliberately |
| root + `--cap-add=NET_ADMIN`, drop `tcp dport 80` inside the container | egress to port 80 **times out** | in-container netfilter really does gate the pasta path |
| `--cap-drop=ALL`, `nft add table` | fails | a non-privileged process cannot install or remove rules |
| `--pids-limit 64 --memory 256m --cpus 0.5` | `pids.max=64`, `memory.max=268435456` visible inside | limits are real (note: `/proc/cpuinfo` still lists all cores; use `nproc`/cgroup limits, not cpuinfo) |
| `-v $dir:/project` with SELinux enforcing, `--userns=keep-id:uid=1000,gid=1000` | write **denied** | bind mounts need `:z` on SELinux hosts |
| same, with `:z` | write succeeds, file owned by the host uid (1000:1000) | project files keep sane ownership |
| `--tmpfs /tmp` **and** `-v /tmp/x:/tmp/x` in one run | the tmpfs wins: `mount` shows `tmpfs on /tmp/x`, `pwd` prints `/tmp/x`, writes succeed | a project mounted at its own path under `/tmp` would be an empty, writable, correctly-named directory that is not the project — so `mirror_block_of()` refuses `/tmp` rather than degrading quietly |
| `-v $dir:$dir` (a per-user path) beside the `/tmp` tmpfs, the `/home/sandbox` volume and the `.agents` share | `pwd` is the host path, writes land on the host owned by the host uid, `/project` + the home volume + the share all still there | mirroring buries nothing outside the denied set; asserted per release in `tests/e2e/12_sandbox.sh` |

## Why not the other designs

* **Internal network + sidecar proxy container.** The obvious design, and it is
  what most write-ups suggest. On this host, two containers on the same
  user-defined network cannot reach each other at all, so the sandbox could never
  reach its own proxy. Rejected on measurement, not taste.
* **`--network=none` plus a unix-socket proxy.** Structurally the strongest (no IP
  stack at all) and worth revisiting, but only `curl` and a handful of tools speak
  `--unix-socket`; everything else (git, npm, pip, node's fetch) would need a
  TCP-to-socket forwarder inside the container, and DNS would need its own path.
  The current design gets the same practical containment with the two controls above.
* **DNS filtering only.** A resolver allowlist is trivially bypassed by connecting
  to a literal IP, and it breaks the SNI-based allowlist that real agent traffic
  needs. Used only as the fallback in `--no-gate` mode, where it is paired with
  IP-pinned netfilter rules.
* **`--network=slirp4netns`.** Not installed here; `pasta` is the default helper in
  podman 5 and is what the code targets (it falls back to whatever helper podman
  picks, since the code only ever asks for the `pasta` driver).

## Failure modes we chose

* If the ruleset cannot be installed (no NET_ADMIN, no `nft`, weird kernel) the
  container **refuses to start** with exit 77 rather than running unfiltered.
  `--allow-unenforced` exists for debugging and says so in the flag name.
* If the policy file is missing the entrypoint exits 78 before running anything.
* The gate container is created per session, bound to loopback only, and removed
  when the sandboxed command exits. A gate that outlives its session would be an
  open door with nobody watching it. `gate up` is the explicit, named exception.
* In gate mode DNS is **closed** inside the container. That is deliberate: it stops
  a sandboxed process from resolving names and then talking to them behind the
  proxy's back. Tools that insist on resolving names themselves need
  `--allow <ip/cidr>` entries instead.