ExSandbox.Egress.Acceptor (ExSandbox v1.2.0)

Copy Markdown View Source

The listener that lives inside one sandbox's network namespace (005 T060a1/T060a3, contracts/egress.md).

Why this exists, and what it replaced

Egress.Pool -- the module now called ExSandbox.Egress.Decision, back when it still held a socket -- bound 127.0.0.1 in the host namespace and was designed as one pool for every sandbox — 013-FR-014c argued for blast radius, not process count, and a process per sandbox is the heaviest way to get it.

That design cannot work, and the reason is not a bug to fix. An nft redirect is DNAT to the local machine as the namespace sees it, so it can only ever reach a socket in that namespace. Measured: with the pool listening on the host and the redirect installed in the sandbox's namespace, the tenant's connect returned OK and the pool never saw the connection.

The one alternative that preserved a single pool — dnat to the gateway — was measured and does not work: pasta is a userspace stack that terminates and re-originates connections, so the pool reads ORIGINAL_DST=127.0.0.1:<pool port>, its own address, and would judge every connection against that. See egress-path-measurements.md option (b).

So the acceptor moves to where the redirect lands. The blast-radius argument survives in substance — no acceptor holds a platform credential, and no sandbox has a route to any other — but the shape it justified does not.

Why this is no longer a separate OS process (2026-08-29)

It used to be. The reasoning was that the BEAM runs in the host namespace, a socket it opens is a host socket, and no option to :gen_tcp.listen/2 changes which namespace a socket belongs to. The first two are true. The third is true and irrelevant, which is the part that was missed for a year.

setns(2) with CLONE_NEWNET affects only the calling thread. So the socket can be created in the sandbox's namespace on a thread of its own and the descriptor handed back, and :gen_tcp.listen/2 will adopt it with {:fd, Fd}. The socket never moves; it was never here. Measured:

listener adopted from the namespace fd   {:ok, {{0, 0, 0, 0}, 9200}}
connect from the HOST namespace          {:error, :econnrefused}
connect from INSIDE the namespace        received its bytes
SO_ORIGINAL_DST on the accepted socket   readable

The econnrefused is the load-bearing half: that port does not exist here.

⚠️ The namespace is named by /proc/<holder-pid>/ns/net, and holder_pid is the namespace holder, never pasta's own pid. See ExSandbox.Egress.Pasta: the pidfile records pasta's host-side process, and entering that one puts the listener in the host namespace, where it would bind a host port and see none of the sandbox's traffic. That hazard is unchanged by dropping the helper -- only the mechanism that consumes the pid changed, from nsenter -t to a path under /proc.

What dropping the helper deleted

The helper could not be asked a question in-process, so everything it needed had to become a protocol: an AF_UNIX verdict socket with its own wire format, a second one for DNS with its own framing, the sandbox's identity passed on argv, a readiness line parsed off stdout, and the discipline that any failure to obtain a verdict is a refusal because the platform might be unreachable. None of that was incidental complexity -- all of it was the cost of the process boundary. decide/3 is now an ordinary function call, so the boundary and its protocols are gone rather than simplified.

What is enforced here, and what is not

Nothing. The decision is ExSandbox.Egress.Decision.decide/3's, unchanged and shared, so there is exactly one implementation of "may this sandbox reach this destination" and moving the listener did not fork it.

⚠️ The identity is different, though, and the difference is load-bearing. The host pool attributed a connection by peername masked to a /30, because every sandbox reached the same socket and they had to be told apart. This acceptor serves one namespace: nothing else can reach it, so the sandbox's identity is the acceptor's own existence rather than anything read off the connection. source_key is supplied at start and is not derived from the peer — a per-namespace listener that trusted peername would be reading a value the tenant partly controls in order to answer a question it has already answered by connecting at all.

Summary

Types

How to reach the namespace this acceptor serves.

Functions

Returns a specification to start this module under a supervisor.

Decides one accepted connection and either relays it or closes it.

The acceptor's own /30, expressed as an address Policy.source_key/1 masks back to that /30 — so the shared decision function is reached with the identity this acceptor was started for.

What the shared decision says about a connection from this sandbox.

Types

spec()

@type spec() :: %{
  source_key: ExSandbox.Egress.Policy.source_key(),
  holder_pid: pos_integer(),
  port: :inet.port_number()
}

How to reach the namespace this acceptor serves.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

handle_connection(socket, state)

@spec handle_connection(:gen_tcp.socket(), map()) :: :ok

Decides one accepted connection and either relays it or closes it.

⚠️ The destination is read from the kernel with SO_ORIGINAL_DST, never from the client. This is a transparent proxy: the sandbox believes it is talking to the destination directly, so there is no frame in which it could state one -- which is what keeps the claim unforgeable. Any design where the sandbox tells the acceptor where it wants to go reintroduces exactly that claim.

⚠️ Every non-permit outcome closes the socket, including the ones that are not policy decisions -- an undecodable destination, an unreadable option. Those are host or kernel faults rather than denials, and they are deliberately treated the same way at the socket. A fault that let the connection through would be an enforcement point that stops enforcing precisely when something is wrong with it. The reason is logged so an operator can tell a denial from a malfunction; that distinction is lost in the logs, never in the outcome.

sandbox_address(arg)

The acceptor's own /30, expressed as an address Policy.source_key/1 masks back to that /30 — so the shared decision function is reached with the identity this acceptor was started for.

Public because ExSandbox.Egress.Binding derives the same address, and two copies of this arithmetic would be two things that must agree forever. The symptom of them drifting is a sandbox judged against a neighbouring sandbox's allowlist — a cross-tenant policy error with no local sign of being wrong.

verdict(map, destination, registry)

What the shared decision says about a connection from this sandbox.

Delegates to ExSandbox.Egress.Decision.decide/3 with the source_key this acceptor was started for — see the moduledoc on why the key is supplied rather than read from the peer.

⚠️ Returns the tagged verdict rather than a boolean because handle_connection/2 has to log why it refused, and false cannot say whether the allowlist denied the destination or the sandbox has no registered policy at all.

⚠️ A boolean permits?/3 stood beside this until 2026-08-29, justified in its own docstring as "one implementation, so the two cannot drift". Nothing called it. A wrapper with no caller cannot drift from anything, and the justification described a risk that only existed if it were used -- the same shape as the pool listener this module replaced, kept for a reason that had stopped being true.