AIdataNewsGlobal Brief: Stateful, Bursty Agents and the Google Cloud View of Data Center CPUs

For readers tracking the shift from model experiments to production agent fleets, the central infrastructure question is no longer only accelerator capacity. According to Google Cloud, the computing layer underneath long-lived, intermittent agent sessions is moving back into focus. Mark Lohmeyer, vice president and general manager for AI and computing infrastructure at Google Cloud, described this transition in an interview with EE Times.

He opened with a practical distinction. "Chat is relatively straightforward—there’s a question and an answer, and it relies heavily on inference across GPUs or TPUs," he said. "With agents, the nature of those interactions is changing quite a bit."

The implication for data centers is structural rather than cosmetic. A human can ask one question and wait; an agent may split a high-level intent into subtasks, coordinate a group of smaller workers, call other APIs, retrieve website information, and write and execute code. Much of that surrounding coordination is processor work, and the sessions can remain active for days or weeks. Because context and state must survive across pauses, the workload is not simply a stream of short inference calls. It is a long-lived task package with periods of silence, then sudden activity.

"Agents are often bursty, and often idle," Lohmeyer said. "They don’t do anything for [say] a day or two until you come back to them, or maybe until some API responds… It’s unique from older CPU workloads, in that it needs to be stateful over a long period of time, but is often idle during that period of time, and then when it’s not idle, it’s highly bursty."

That combination makes both performance and security harder to manage. "Demand is already very high and is growing rapidly," he said. "So, how do we provide the infrastructure that can support that growth and do it cost-effectively?"

Google’s Arm-based Axion CPU

Two Workload Streams Are Being Repriced

Lohmeyer divided the agentic pattern into two parts. One part is new: the compute required to run the agent itself, which needs infrastructure designed around stateful, bursty execution. The other part is not new: website access and API access already existed when humans used applications and services, but agents can perform those actions more intensively and at larger scale. "Agentic workloads will have a meaningful impact on both of those elements," he said.

The existing component grows because agents impose heavier demands than human users. A company can add many agents quickly, while human headcount and user behavior cannot scale at the same pace. That changes how much CPU work is expected around inference accelerators, not only how many model tokens are produced.

Performance considerations also push toward locality. "Ideally, you’d like to have those agents and their context, the applications and data, relatively close to each other from a latency perspective, for performance reasons," he said. Yet enterprises are unlikely to become single-zone or single-cloud operators. Lohmeyer said companies continue to operate in a distributed world, with most using multiple clouds and private clouds for now. That keeps high-performance networking important within data centers and across data centers.

"Dynamic, stateful agents will need to have a new type of computing infrastructure that supports them," he said. "They’re accessing all the existing apps and data, so the demand there is growing, and all this needs to be connected together through the network. Google is investing in all of these elements."

The relative importance of processor and accelerator capacity is still changing. Lohmeyer said the optimal CPU-to-GPU compute ratio for agentic AI is evolving with the workload, but the CPU share is definitely increasing as the industry moves from training foundation models to agentic inference. "We expect both [CPU and TPU/GPU] workloads to continue to grow rapidly, but the proportion of CPUs is going from a relatively small portion to a meaningful portion, and we think that will continue to evolve as agents expand into new use cases and new environments," he said.

Hibernation, Storage, and MicroVMs

The proposed answer is not a single chip class, but co-design across the stack. Lohmeyer linked CPUs, storage, memory, and networking to the demands of agentic inference. "Google Cloud has always had a philosophy of workload-optimized infrastructure," he said. "This is more important than ever in the agentic era."

Because agents may remain idle for long periods, one practical approach is to let them effectively hibernate. Their state can be moved into storage, freeing the CPU to serve other agents, then reloaded when a task resumes. "This requires a new type of orchestration in software, and in this environment, having really high-performance storage that’s attached to that server, so I can restart it quickly, is important," he said.

Security isolation is another reason the orchestration layer matters. Many agent sessions may be packed onto the same server. Google Cloud uses microVMs, described as a slimmed-down version of a virtual machine, to create a secure boundary between them. The scale comparison is concrete: rather than 100 traditional VMs, the same hardware might run 1,000 microVMs. "Security and governance for agents is absolutely critical, and Google is investing a lot in every layer of the stack," he said.

At the lowest layer, virtualization is key. Lohmeyer said Google plans to support multiple ways to sandbox agents, with the microVM particularly important. Because a microVM has a streamlined feature set, performance can improve, and its smaller attack surface helps security compared with traditional virtual machines. The company is also investing in agent gateways that can provide egress controls at the network level, along with other higher-level controls.

Looking ahead, the ability to run thousands of microVMs for agents could affect core density, storage, how storage is attached, and the orchestration software used to manage the environment. The expected benefit is higher hardware utilization, which should lower the cost per agent. "This will allow our customers to scale their business, but do so with a good business model, good profitability, but without having to sacrifice performance as they are scaling," he said. "These capabilities will be very important going forward."

Key Data Points

• Source framing: Mark Lohmeyer’s infrastructure view was communicated to EE Times.
• Interaction model: chat is described as a question-and-answer pattern relying heavily on inference across GPUs or TPUs, while agent interactions are changing the nature of those exchanges.
• Session pattern: agents can be bursty and idle, with stateful sessions lasting days or weeks.
• Demand direction: CPU share is moving from a relatively small portion to a meaningful portion as training shifts toward agentic inference.
• Isolation comparison: 100 traditional VMs versus 1,000 microVMs on the same hardware.
• Architecture response: workload-optimized infrastructure, high-performance attached storage, virtualization, egress controls, and agent gateways.

Table: Where Agent Sessions Differ From Chat

Dimension	Simple chat view	Agent-based view
Basic flow	Question and answer	Intent split into subtasks among agents
Heavy work	Inference on GPUs or TPUs	API calls, website access, code execution, CPU coordination
Session life	Shorter interaction	Can span days or weeks with state
Activity profile	More continuous request pattern	Often idle, then highly bursty
Infrastructure pressure	Accelerator inference capacity	CPU, storage, networking, orchestration, and isolation

FAQ

Why would CPUs matter more if agents are powered by AI models?

Because agent systems do more than generate text. The model may produce intent, but the surrounding work—calling APIs, browsing, running code, keeping state, and resuming after idle periods—places load on CPU infrastructure, storage, memory, and networking.

What does “stateful” mean in this context?

It means the agent session retains context over time. A user may return after a day or two, or an external API may respond later, and the system must resume without treating each call as a brand-new stateless request.

Why is microVM isolation relevant?

Because many agent sessions can run on the same server. A smaller, streamlined virtualization boundary aims to improve performance while reducing attack surface relative to traditional VMs.

What should AIdataNewsGlobal readers watch next?

The practical next indicators are CPU-to-accelerator ratios in agent deployments, storage attached close enough to enable fast resume, and governance tools such as sandboxing and egress controls.

The broader signal is that the agentic era does not simply add more inference calls. It changes the shape of demand, making idle periods, long-lived context, and bursty coordination central to infrastructure design. The value of the interview, for AIdataNewsGlobal readers, is that it frames the debate around workload-optimized systems rather than one single silicon answer.