Agent Sandboxes: Isolated Runtimes for Testing AI Agent Behavior

You’re building an AI agent that browses the web, writes files, calls APIs, and executes shell commands. You run it locally against a test prompt. It works. You run it again with a slightly different system prompt and it starts deleting things it shouldn’t, or it hammers an external API until you hit a rate limit, or it quietly writes to a database you forgot was pointed at production. None of these failures are obvious until they happen. That’s the problem a virtual sandbox solves: it gives the agent a place to act, fail, and recover without those failures having consequences outside the sandbox boundary.

Getting this right changes the shape of your development loop. Instead of running agents carefully against live systems and hoping nothing breaks, you can run them aggressively against controlled inputs, observe exactly what they do, and iterate on prompts, tool definitions, and policies until the behavior is what you actually want. You catch the footguns early. You stop treating every test run as a potential incident.

What is an Agent Sandbox?

An agent sandbox is an isolated execution environment designed to run AI agent behavior under controlled conditions. The agent can still call tools, execute code, read and write files, and make decisions. What changes is that those actions happen inside a boundary: the sandbox intercepts, logs, and optionally blocks anything that would reach a live system.

The definition matters because “sandbox” gets used loosely. A mock test suite is not a sandbox. A staging environment is not a sandbox. A sandbox is a runtime that lets the agent behave as it would in production, including making real decisions and taking real actions, while preventing those actions from having real consequences. The distinction is important because agents that only run against mocks often behave differently when they encounter real tool responses, latency, or ambiguous outputs.

In practice, an agent sandbox sits between the agent and the outside world. It can simulate tool responses, record every action the agent takes, enforce permission boundaries (no network calls, no writes outside a specific directory, no calls to external APIs), and restore the environment to a known state after each run. That last capability, rollback, is what separates a sandbox from a simple logging wrapper.

One wrinkle worth naming early: since 2026 the phrase does double duty. “Agent Sandbox” is also the proper name of a Kubernetes project that implements this pattern as a set of custom resources. The general pattern and the named project are related, but they are not the same thing.

How Does an Agent Sandbox Work?

The Isolation Layer

The isolation model is what makes a sandbox useful rather than just decorative, and there is more than one way to build it. The trade is always boundary strength against cost, and it is the axis the whole field organizes around.

The mechanism behind each row, and where each boundary actually holds, is covered rung by rung in virtual sandboxes. What matters here is the fit. Agent-written code is untrusted by construction, not merely buggy, which is a harder threat model than most sandboxing choices are made against: the bottom two rows are reasonable for code you reviewed and thin for code a model produced while you were not watching.

Whichever layer you pick, a working sandbox enforces the boundary in several places at once: a private filesystem whose writes don’t persist unless you snapshot them, outbound calls that are blocked or allowlisted, an unprivileged process that can’t escalate, and capped CPU and memory so a runaway agent can’t consume everything. The key property across all of them is reproducibility. The same inputs produce the same starting conditions every time.

Observability Inside the Sandbox

Isolation without observability is just a black box. The value comes from seeing exactly what the agent did, in what order, and why: every tool call with its arguments and response, the planning trace rather than only the final action, state diffs across the run, and timing per step.

{
  "step": 2,
  "action": "tool_call",
  "tool": "write_file",
  "args": { "path": "/etc/passwd", "content": "..." },
  "result": "blocked",
  "reason": "path outside allowed write boundary"
}

That entry is the one you care about. The sandbox caught an attempted write to a sensitive path, blocked it, and logged it. Without the sandbox, that action either succeeds in production or fails silently in a mock. With it, you have a concrete artifact you can use to tighten the agent’s tool permissions or adjust the prompt.

Sandbox the Agent, or the Code It Writes?

There are two patterns, and picking the wrong one is the most common architectural mistake in this space. In the first, the agent runs inside the sandbox and you talk to it over the network: it has a shell, a filesystem, package managers, and a real machine to work on. In the second, the agent runs outside on your infrastructure and ships only the code it generates into a sandbox for execution, reading the result back.

The second pattern is lighter and is what most code-execution products are built for. It works when the unit of untrusted work is a self-contained snippet: run this Python, return stdout. It stops working the moment the agent needs to install a dependency, start a server, hit its own endpoint, read the stack trace, and try again, because each of those steps needs state that survives between calls.

The first pattern costs more per session and gives the agent a whole environment to be wrong in, which is exactly why the isolation boundary underneath it has to be real. The rough rule: sandbox the code when the agent is generating functions, and sandbox the agent when it is doing work that a human would need a computer for.

What Is the Agent Actually Allowed to Do?

Host isolation and permission are two different questions, and the second one is where most setups stop short. A container or a microVM answers “where does this run so it can’t hurt the host.” It says nothing about which credentials the agent holds, which hosts it can reach, or which files it can see. An agent that cannot escape its VM can still push to the wrong branch, empty a bucket, or post an API key to a domain it chose itself.

Three controls carry most of the weight:

• Egress. Allowlist rather than blocklist. The awkward part is that the domains an agent legitimately needs (package registries, git hosts, model APIs) are also the shape of a working exfiltration path, so the allowlist is a real design decision rather than a checkbox. A denied request should fail fast rather than hang, because agents read a timeout as “retry” and a refusal as “stop.”
• Credentials. Scope what is mounted to the task. A sandbox holding a long-lived token with organization-wide scope has moved the blast radius, not reduced it.
• Filesystem reach. Mount the one repository the task needs, not the parent directory it happens to sit in.

What makes all three worth configuring is where the control lives. If the policy is enforced from inside the sandbox, the code you don’t trust is also the code enforcing the rules, and a prompt injection that reaches a shell can rewrite them. The policy has to be set from outside, and readable but not writable from within. That asymmetry, more than the strength of the isolation layer, is what keeps a restricted environment restricted.

Agent Sandbox on Kubernetes

Agent Sandbox is a Kubernetes project, running under SIG Apps, that adds a Sandbox custom resource for exactly this workload: it gives you a declarative API for a single, stateful pod with a stable identity and optional persistent storage, and a controller that handles creation, scheduled deletion, pausing and resuming. It exists because the existing Kubernetes abstractions are shaped for replicated, stateless workloads, and an agent session is a singleton that has to keep its filesystem.

Four resources make up the API:

• Sandbox is the core resource. It requires a podTemplate and takes optional volumeClaimTemplates for storage, plus shutdownPolicy and shutdownTime, an RFC3339 timestamp that acts as a hard deadline on the environment’s lifetime. That deadline is the answer to runaway tasks and unbounded compute cost.
• SandboxTemplate is the reusable blueprint: a podTemplate plus an optional networkPolicy carrying ingress and egress rules with standard Kubernetes NetworkPolicy semantics.
• SandboxClaim is the transactional request. It references a template by spec.sandboxTemplateRef.name and can be satisfied immediately from a warm pool, so the caller never handles provisioning logic.
• SandboxWarmPool keeps pre-initialized sandboxes running. It takes spec.replicas and a template reference, hands a ready pod to an incoming claim, then replenishes itself. The project’s stated goal is creation in under a second, which is the number that decides whether an interactive agent feels usable.

The project does not implement isolation itself. It delegates that to a Kubernetes RuntimeClass, so the same API runs over gVisor or over Kata Containers with hardware virtualization and a dedicated kernel per sandbox. On GKE the documented path sets runtimeClassName: gvisor in the template, alongside automountServiceAccountToken: false, runAsNonRoot: true, and dropping all capabilities.

The honest caveat is that this is orchestration, not a sandbox in itself. You still bring the isolation runtime, nodes that can run it, and a cluster to operate. Right answer if Kubernetes is already your platform, considerable overhead if it isn’t.

Agent Sandbox in Practice

Repeated Simulations and Configuration Comparison

The most practical use of an agent sandbox is running the same scenario repeatedly with different configurations. You want to know whether a new system prompt, a different model, or an adjusted tool definition changes how the agent behaves on a class of inputs. Without a sandbox you’re comparing runs that happened at different times against different system states. With one, you control the starting conditions and isolate the variable you’re testing.

The workflow is short: define a scenario, snapshot the baseline, run configuration A and record the trace, restore, run configuration B, restore, compare. Did it reach the correct outcome? In fewer steps? Did it attempt anything unsafe? Because each run gets its own isolated environment, those comparisons also run in parallel without interfering with each other.

Detecting Unsafe Agent Behavior

Agents fail in ways that are hard to anticipate. They hallucinate tool arguments. They get stuck in loops. They take actions that are technically within their permissions but semantically wrong. A sandbox is one of the few places you can observe these failure modes without paying for them in production.

Detection works by defining what safe looks like and flagging the deviations:

• Permission boundaries: any action outside the defined permission set is blocked and logged. An attempt to call a tool it shouldn’t have is a signal, not noise.
• Action frequency: an agent calling the same tool 50 times in one run is probably looping. Enforce call limits and surface it in the trace.
• Baseline comparison: diff new traces against a reference run you trust and flag structural deviations.

When to Use an Agent Sandbox

Use a sandbox when any of the following apply:

• You’re iterating on prompts or tool definitions and want to know whether a change improves or degrades behavior on a class of inputs, without running against live systems.
• Your agent calls external APIs or writes to persistent storage and one bad run could cause rate limiting, data corruption, or unintended side effects.
• You’re evaluating a new model or planning strategy and need to compare traces under identical starting conditions.
• You’re testing edge cases or adversarial inputs you wouldn’t want to run against production, such as prompts designed to trigger unsafe tool use.
• You need to reproduce a failure mode seen in production without recreating the full production environment.
• You’re running untrusted or agent-generated code and need hardware-level isolation to keep it away from the host and other tenants.

The common thread is that you have something to learn from the run and something to lose if the run goes wrong. A sandbox separates those two concerns.

Development Sandbox vs. Agent Sandbox

The sandbox pattern predates agents. A development sandbox is the same idea pointed at a person: an isolated environment where one engineer installs dependencies, rewrites a config layer, and reproduces a race condition without touching production or blocking the shared staging box. Sandbox development, like a cloud development environment, means everyone gets their own resettable environment instead of queuing for a single shared one.

The lifecycle is the loop the agent workflow already uses: provision from a known baseline, configure, experiment, inspect, then reset or promote. That last step separates a real dev sandbox from a long-lived dev box, because a box someone has been mutating for six months is in an unknown state, and an unknown state is useless for debugging.

A shared test environment still has a job: integration checks against a stable representation of the full system. It is a poor fit for exploratory work or intermittent bugs, because it is never fully yours. Three costs are specific to one sandbox per engineer: drift against the production image, data (copying production data everywhere is a privacy problem, so you need synthetic generation or anonymized snapshots), and secrets, which need scoping and rotation per environment rather than a shared env file.

What changes when the operator is an agent is the strictness. A human engineer reads the error, notices the environment is wrong, and stops. An agent keeps going.

Does Sandboxing Make Your Agent Worse?

Usually not in capability, sometimes in latency, and almost always for the first week while the allowlist is wrong. This is the most common objection practitioners raise, and it deserves a straight answer rather than a dismissal.

Where the tax is real: cold start on every fresh environment, the round trip of shipping code in and results back if you chose the sandbox-the-code pattern, and the failure that actually hurts, which is a legitimate request being denied. An agent that cannot reach a package registry does not report a policy problem. It works around the problem, badly, and you read the wrong conclusion in the trace. Nearly every “sandboxing tanked my agent” story is this one.

Where the tax is imaginary: an agent with a real environment, a persistent filesystem, and the ability to install what it needs is more capable than one calling stateless functions, not less. Isolation and capability are not opposites. The confusion comes from conflating the boundary with the restrictions layered on top of it, which are separate settings.

Two things make the difference in practice. Fail fast rather than hanging, so a blocked call reads as a refusal and not as a flaky network. And keep the environment warm and persistent between runs, so the agent is not paying setup cost on every task.

Common Challenges and Trade-offs

A sandbox reduces risk. It doesn’t eliminate it. A few failure modes are handled poorly, and it’s worth being direct about them.

Distribution shift: agents tested against synthetic or historical inputs behave differently against real ones. The sandbox is only as good as the scenarios you put into it.

Emergent behavior at scale: an agent that behaves correctly on a single run may not when it is one of thousands of concurrent instances, or when it has a much larger context to work with. Sandbox runs are typically single-instance and short-horizon.

Prompt injection from external data: if the agent reads web pages, documents, or email, those sources can carry adversarial content. A sandbox can contain what the injection is able to do. It cannot generate the adversarial test inputs for you.

Model non-determinism: even with identical inputs, some models produce different outputs across runs. A controlled environment does not make the model deterministic.

These aren’t reasons to skip the sandbox. They’re reasons to treat sandbox results as necessary but not sufficient. Sandbox testing, production monitoring, and human review are complements, not substitutes.

Agent Sandboxes on Fly.io

Fly.io’s Sprites are hardware-isolated sandbox environments built for this workload. Each Sprite is a microVM with its own kernel, dedicated CPU and memory, its own network namespace, and an ext4 filesystem that behaves like real local disk. There’s no shared runtime and no noisy neighbors. The boundary is hardware, not a policy layer you have to trust.

Persistence works the other way around from most sandboxes. A Sprite’s filesystem syncs continuously to durable storage, so it survives with no snapshot step: install a model SDK and a test harness once and they’re still there on the next run. When nothing is using the Sprite it pauses and compute billing stops with it, and a warm resume brings it back with processes still in place.

Checkpoints sit on top of that as the deliberate save point, and they run copy-on-write, so taking one is fast and doesn’t interrupt work in progress. The comparison loop is three commands per configuration:

sprite checkpoint create --comment "baseline: deps installed, fixtures loaded"
sprite exec -- python agent_runner.py --config a.yaml --trace /out/trace-a.json
sprite restore v1

Checkpoints get sequential IDs, so the restore names the one you took. sprite restore replaces the writable filesystem with that checkpoint’s contents and restarts the environment. It is destructive, and the state it overwrites is not saved for you, so copy out anything worth keeping first.

The permission side is a separate, opt-in control. A Sprite’s outbound network starts unrestricted; you narrow it by applying a DNS-based network policy from outside the Sprite. Once one is in force, a domain that isn’t allowed gets a DNS refusal and fails fast instead of hanging, and the default rule set already covers the git hosts, package registries, and model APIs a working agent needs. The part that matters is the asymmetry: the policy is readable from inside the Sprite and only writable from outside it, so agent-written code can’t widen its own allowlist.

Full command reference and the SDKs live in the Fly.io docs, and Sprites has the product detail.

Frequently Asked Questions

What is an agent sandbox?

An agent sandbox is an isolated runtime environment that lets developers test AI agent behavior without affecting production systems, live data, or external services. On Fly.io, Sprites provide that boundary as hardware-isolated microVMs, so an agent can call tools and execute code without its actions reaching anything real.

Why do developers use a sandbox for AI agents?

Developers use a sandbox for AI agents to evaluate planning, tool use, memory handling, and failure modes under controlled conditions before deploying agents to real environments. Running each evaluation in its own Fly.io Sprite keeps those runs isolated from one another, so one agent’s mistakes cannot affect another run or the host.

What features does an AI agent sandbox typically include?

An AI agent sandbox typically supports repeated simulations, logging, rollback, constrained permissions, and observable execution so teams can validate changes to prompts, policies, and tools. Fly.io Sprites cover these with whole-filesystem checkpoints for rollback, a persistent ext4 disk, and an opt-in network policy that limits which external hosts the agent can reach.

How does an agent sandbox help detect unsafe agent behavior?

An agent sandbox measures reliability and flags unsafe actions by running agents against controlled inputs and recording how they respond, without risking harm to live systems. When a network policy is applied to a Fly.io Sprite, attempts to reach hosts outside the allowlist fail fast with a DNS refusal, which turns an exfiltration attempt or a misconfigured tool into a clear signal instead of a silent success.

Can an agent sandbox be used to compare different AI configurations?

Yes. An agent sandbox supports testing multiple configurations under realistic scenarios, allowing developers to directly compare how different setups affect agent performance. On Fly.io, the pattern is to checkpoint a baseline environment, run one configuration, then restore that checkpoint so the next configuration starts from identical conditions.

Does an agent sandbox keep its files between runs?

Yes, on Fly.io. A Sprite’s filesystem syncs continuously to durable storage, so installed packages, fixtures, and repositories survive between runs and across the pauses that happen while a Sprite sits idle. That is separate from checkpoints, which are deliberate save points you create when you want to be able to roll back.

Can an agent sandbox block an agent from reaching the internet?

Yes, though it is not the default. A Fly.io Sprite starts with unrestricted outbound access, and egress is narrowed by applying a network policy from outside the Sprite. Once one is in force, only allowlisted domains resolve, raw IP connections are blocked unless the address came from an allowed domain, and the agent inside cannot rewrite the rules.

How do you reset an agent sandbox to a known state?

Restoring a checkpoint returns the environment to a known state. On Fly.io, sprite restore replaces a Sprite’s writable filesystem with the contents of a saved checkpoint and restarts the environment, which is how teams reset between evaluation runs. Restoring is destructive, so anything worth keeping should be copied out or checkpointed first.

What is a development sandbox?

A development sandbox is an isolated environment where one engineer builds, debugs, and resets code changes without affecting production or shared test infrastructure. On Fly.io, a Machine gives each engineer that boundary with its own filesystem, private networking, and a known baseline to reset to, so sandbox development does not require coordinating with anyone else.

How does a development sandbox differ from a shared test environment?

A development sandbox isolates one engineer’s work, so unstable changes cannot disrupt other people or ongoing test runs, while a shared test environment accumulates drift and turns one bad deploy into a team-wide block. On Fly.io the per-engineer version stays affordable because Machines automatically stop and start based on traffic, so a dev sandbox is not billing compute overnight.