This week: AWS and UC Berkeley open-sourced Infinity, an agent runtime that lets AI agents nap for free while they wait, light enough that AWS says 75,000 of them fit in a Raspberry Pi 5's memory.

Elsewhere: Where that 75,000 number actually comes from, how an agent can wait three days without costing a cent of compute, and how to try it yourself.

The New Prime Signal

Here's an unglamorous secret about AI agents: they spend a lot of their lives waiting. As the AWS announcement puts it, "A coding agent may start a build that runs for twenty minutes, a triage agent may wait hours for the next pull request on a repository, and an assistant may wait until the next morning for an approval." In most setups, that waiting agent keeps holding a process and its memory the whole time, like someone keeping a restaurant table for three hours over one cold coffee. Infinity, a new open-source project from AWS and UC Berkeley, asks a simple question: what if the agent just got up and left, and came back when the food arrived?

What happened: On October 23, AWS published "Introducing Infinity: a runtime for extreme agent concurrency," written by Shadaj Laddad, research lead for the Hydro team at AWS. The team says it is "releasing Infinity under the Apache 2.0 open source license as part of the Hydro project, a collaborative effort between AWS and UC Berkeley." The headline claim is a fun one: "we can fit 75,000 concurrent agents in the 8 GB of a Raspberry Pi 5." AWS is clear that this is not a finished product. It calls Infinity "an early research release" and says it is "sharing it to get early feedback." The code is on GitHub, written in Rust.

Why it matters: Agents are moving from "answer my question" to "keep an eye on this for a week." That kind of job is mostly waiting, and paying for idle waiting adds up fast. Infinity's pitch is that a waiting agent should cost almost nothing. When an agent calls a tool, Infinity saves the conversation and suspends the agent, and when the tool's answer shows up, it wakes the agent back up. That also means agents can live on serverless platforms, where code only runs when there's work to do. In AWS's words, "So even if a tool call takes days, nothing needs to run while the agent waits." The repo puts it more bluntly: "Infinity does for agents what async did for threads."

The fine print: Before anyone orders a pallet of Raspberry Pis, look at how that 75,000 figure was reached. AWS ran a benchmark with 80,000 agents "on a single thread of an AMD EPYC 9R14 with in-memory stores," backed by a simulated model rather than a real one. Memory grew "at about 100 KB per idle agent," and "At this rate, about 75,000 agents fit in 8 GB, which is the memory of a Raspberry Pi 5." So it is a calculation based on memory use, not 75,000 agents literally running on a Pi. The other big asterisk: the thinking still happens in a large AI model somewhere else. Infinity holds the agents' notes, not their brains, so the Pi is the waiting room, not the doctor. AWS also lists open questions it hasn't solved, including how models cope with events arriving out of order, how to schedule which agents go first, and "Event authenticity," meaning how to trust messages from the internet. All the numbers here are AWS's own, and nobody outside the team has had much time to test them.

This is a refreshingly practical kind of AI news. No new model, no benchmark war, just someone noticing that agents waste most of their time and memory standing around, and borrowing a decades-old programming trick to fix it. If it holds up outside the lab, the payoff is cheaper, longer-running agents that can babysit a slow build or a busy repository without running up a bill. And yes, "75,000 agents in a Raspberry Pi's memory" is a great line, as long as you remember the agents are mostly asleep.

The idea isn't new. It's the same trick Rust, JavaScript, and Python already use for async code, pointed at AI agents.

How it works: When ordinary async code waits on something slow, the language saves just enough state to pick up later and frees up the worker for other jobs. Infinity treats an agent the same way. AWS notes that at the moment an agent calls a tool, "its conversation history contains everything needed to continue the agent once the result is available." So Infinity runs each agent in short "slices": load the history, get one response from the model, save it, send off at most one tool call, and stop. Results, user messages, and other events land in a queue and wake the right agent up. For events the model didn't ask for, like a new pull request, Infinity slips in what it calls "synthetic tool calls," so the model sees the event in a format it already understands. Agents can also spawn helper agents that report back the same way. On AWS, the reference setup uses SQS queues and Lambda functions, so an agent "waiting on a three-day CI pipeline uses no compute until the pipeline reports its result."

Try it: The code lives in the hydro-project/infinity repository under the Apache 2.0 license, and the project's quickstart site walks you through building a first agent. AWS says "you can build a simple agent running on Infinity in just a few minutes!" You'll want to be comfortable with Rust and bring your own model provider. The repo also mentions a Reactive Agent Protocol that it says runs existing MCP tool servers unchanged, and Infinity Code, a coding agent built on the runtime. For the full design, including the open questions, read the AWS post. It's an early research release, so expect rough edges and file the issues you find.