The Pod Is the Wrong Unit for an Agent

A Kubernetes scheduler knows two workload shapes. A Deployment is long-lived, replicated and interchangeable, so any replica can serve any request and killing one costs you nothing. A Job runs to completion and then stops existing. Both shapes assume you can state the lifetime up front, and the whole control plane is built on that assumption.

Now describe an agent run. It wakes when work arrives, executes for a few seconds, possibly spawns children to do subtasks in parallel, and then sits on a human approval gate for forty minutes before doing another eight seconds of work. InfoQ’s writeup of the kagent project’s argument puts the mismatch plainly: agents “do not behave like the microservices these abstractions were designed around”.

The usual conclusion from that observation is a utilization argument. Agents are bursty and mostly idle, a dedicated pod per agent is therefore wasteful, so pool them. That conclusion is correct and I think it is also shallow, because utilization is the symptom rather than the disease.

My claim is that an agent is not one workload at all. It is three things with three different lifetimes, and binding them to a pod forces all three to share the pod’s lifetime. Utilization is what you notice first because it shows up on a bill, but the reason your agent platform is awkward is that one scheduling primitive is being asked to answer three unrelated questions about survival.

The three lifetimes#

The clearest decomposition I have read is Anthropic’s, in their writeup of scaling managed agents:

We virtualized the components of an agent: a session (the append-only log of everything that happened), a harness (the loop that calls Claude and routes Claude’s tool calls to the relevant infrastructure), and a sandbox (an execution environment where Claude can run code and edit files).

Read those three as lifetimes rather than as components and the pod problem falls out immediately.

The session has to outlive everything. It is the only durable record of what happened, and it needs to survive the process that wrote it, the node that process was on, and the deploy that replaced the code. Its natural lifetime is the length of the task, which might be days.

The harness should be destroyable at any moment. It is a loop, it holds no truth that is not in the session, and the correct lifetime for it is however long the current step takes. Anything longer is you accidentally storing state in a loop.

The sandbox is the only piece that needs a real isolation boundary, and it needs it for exactly as long as untrusted code is executing. Not for the forty minutes an approval is pending, and not for the length of the conversation.

Pod-per-agent gives all three the pod’s lifetime. The session dies with the pod unless you bolt a volume onto it. The harness cannot be rescheduled without losing the run, which quietly turns a node drain into lost work. And the sandbox, the piece with the shortest honest lifetime of the three, stays allocated through every minute the agent spends waiting on a human.

What unbundling actually bought#

Anthropic moved the harness out of the container and had it call execution environments through a single interface, execute(name, input) → string. The reported result:

Using this architecture, our p50 TTFT dropped roughly 60% and p95 dropped over 90%.

Two honest caveats on that number before anyone quotes it at a planning meeting. It measures time to first token, which is latency and not utilization, so it is not evidence for the cost argument that usually gets made here. And it is a vendor’s number about their own architecture, with no independent replication.

What it does show is where the latency was. It was not model speed. It was the wait for a scheduling decision to complete before the loop could start, and decoupling the loop from that decision is what removed it.

The detail I find more persuasive than the percentage is a smaller one. With the harness outside, a container failure “was caught as a tool-call error and passed back to Claude”, and the harness itself became restartable through wake(sessionId). That is the real test of whether you have the unit right. When the sandbox is the deployment unit, its death is an incident. When it is not, its death is an error value that the agent can read and route around.

Two answers, and they disagree about the pod#

Here is where it gets interesting for anyone actually choosing a platform, because two serious projects share this diagnosis and prescribe opposite things.

The Kubernetes Agent Sandbox project keeps the pod and changes the controller above it. Kubernetes handles stateless replicas and numbered stateful sets well, and the gap it names is a workload wanting “a single, stateful pod with a stable identity and persistent storage”. On why you should not just assemble that yourself from parts you already have:

While these can be approximated by combining StatefulSets (size 1), Services, and PersistentVolumeClaims, this approach is cumbersome and lacks specialized lifecycle management like hibernation.

So the Sandbox CRD adds the lifecycle verbs that a Deployment has no vocabulary for: deep hibernation that saves state to persistent storage, resuming a sandbox when a connection arrives, and a SandboxWarmPool of pre-warmed sandboxes to allocate from. The idle problem is answered by making idle nearly free.

The kagent and agent-substrate line goes the other way. It keeps the pod as a worker and moves the agent off it entirely, running a control plane above Kubernetes with WorkerPool, Workers, ActorTemplate and Actor deliberately mirroring NodePool, nodes, pod specs and pods. The point of the indirection is to use “a fixed pool of long-lived Pods to support far more logical agents than would be practical with a dedicated, continuously running Pod for each one”. The idle problem is answered by ensuring nothing is idle, because the pods are never agent-shaped in the first place.

One project makes the pod smarter, the other makes the pod dumber and adds a scheduler on top. What they agree on is the part worth taking: the thing you schedule stopped being the thing you isolate.

Where this loses#

The pooling answer spends a boundary to buy the utilization. If your pod was doing security work, and on a lot of clusters it is the only boundary anyone can actually point at, then multiplexing logical agents onto a shared worker means one agent’s blast radius is now the worker and everything else running on it. That trade is worse for agents than for ordinary workloads, because agent instructions come from a model reading text that someone outside your company may have written. You are exchanging a boundary you can describe for a saving you cannot yet quantify, and I would want the number before signing.

Which brings me to the weakest part of the case. The InfoQ piece carries no measurement at all. No utilization figures, no idle percentages, no pod counts. The wastefulness of pod-per-agent is asserted rather than demonstrated, and it is asserted by projects that exist to sell you the alternative. I believe it because it matches what I have watched happen to any bursty workload squeezed onto a scheduler built for services, but believing something for structural reasons is a weaker position than measuring it.

And most teams should not do any of this yet. At ten concurrent agents, pod-per-agent is fine. A Deployment your team already knows how to debug at three in the morning beats a control plane you adopted for elegance, and running an extra operator has a cost that never appears in the architecture diagram. The unbundling starts earning its keep at the point where idle sandboxes show up as a line item you have to explain, or where a routine node drain has started killing runs that someone was waiting on. Below that line, a wasteful system I understand beats an efficient one I do not.

The takeaway#

Before picking a platform, do the exercise on one agent run and write down three answers. What has to survive a process restart. What has to survive a node drain. What has to be destroyed the instant the code stops running.

If all three answers come back as “the pod”, the pod is doing three jobs and it is going to do at least one of them badly. In practice most teams find the first answer is the session and it is currently not durable, the second answer is nothing at all, and the third answer is the sandbox, which today is being kept alive for the entire length of the conversation.

There is a cheap version of this test that takes an afternoon. Kill a pod in the middle of an agent run and see what you lose. If you lose the run, your session is not durable and your harness is holding state that belongs somewhere else, and no amount of scheduling cleverness above that will fix it.

Working out which piece of a workload has to survive what is most of platform engineering, and agents have made it an urgent question rather than an academic one. If that is the sort of work your team is hiring for, here is what I do.