After Addy Osmani’s introduction to loop engineering, Robert Ross, writing at The Thought Drop, opens up the machinery. What looks like one agent loop is really three nested loops:
Agent loops are often oversimplified. They’re presented as a single loop, when really it’s three loops in a trench coat that make up an “agentic” experience for a customer. I’m here to write (yes, I wrote this, insane right?) yet-another-blog about agent loops. The example code blocks are also pseudo-code and for illustrating these ideas. Also I’ve omitted streaming, which complicates the post but the shape of these stays the same.
Those are the inference loop, which manages model calls and conversation history; the tool loop, which turns model output into actions; and the human loop, which approves, rejects, or redirects consequential work.
Ross’s “brain in a jar” analogy explains why the tool loop changes a model into an agent:
LLMs are brains in a jar. They provide no functional value on their own. The tools you give an LLM are what make it an agent.
When you tell a model “here are the tools you have” in your outer inference loop, the model may try to “use” them in its inference (response). This is the same thing as a brain sending an electrical signal telling your index finger to hover over the enter key of the email you desperately want to send Laney. Tom, we need to set boundaries my man.
The separate tool definitions you include in your API request are usually serialized into the system prompt field of the token stream the model processes. And it may infer the usage of multiple tools in one turn. (Hence: Tool Loop).
The human loop is the final layer—and the hardest to build:
The Human Loop is arguably the hardest part to implement in agentic systems. You can’t have a piece of code block for hours. What if the server restarts? What if you have thousands of other requests coming in you need to respond to? The first two loops (inference and tool) are simple enough. The human loop ups the ante of difficulty. This is why durable execution frameworks exist, like Temporal.
But the human loop is necessary, because it’s the only thing stopping Tom from actually sending that message to Laney. IT WAS TWO YEARS AGO TOM, MOVE ON!
