The Living Graph¶

Agent frameworks make you choose between two things you should not have to choose between: a structure you can inspect, and a structure that can change.

This page argues the choice is false, and that it only looks forced because of an unexamined assumption — that the graph describes the future.

The dichotomy¶

Plan-and-execute designs ask a model to decompose a task into steps, then run them. The appeal is obvious: you get a structure before spending a token on execution, and you can show it to someone. The problem is equally obvious once you have watched one run. Step three returns something nobody anticipated — a column that doesn't exist, a service that returns 403, a document that says the opposite of what the plan assumed — and the plan is now fiction. The framework re-plans, discarding what it learned, or it forces the stale plan forward.

ReAct loops give up on planning. Think, act, observe, repeat. They adapt beautifully, because adaptation is all they do. What they leave behind is a transcript: a flat, append-only list of messages. You cannot diff two transcripts meaningfully. You cannot ask a transcript which step caused which other step. You cannot hand it to an auditor and point at the moment a decision was made.

One gives structure without adaptation. The other gives adaptation without structure.

The assumption underneath¶

Both designs treat a graph as a plan — a claim about what will happen.

Plans must be complete before they are useful, which is why static planners have to guess. Plans become invalid when reality disagrees, which is why they are brittle. And because a plan is a prediction, keeping one accurate during execution means continuously rewriting it — which is why frameworks that try end up maintaining two sources of truth: the plan, and what actually ran.

Drop the assumption. Let the graph describe the past instead.

A graph of what has already happened is never wrong. It cannot be invalidated by a surprise, because a surprise is just another node. It does not need to be complete to be useful — a partial history is still a history. And there is nothing to reconcile, because there is only one record.

The apparent cost is that you no longer know the shape of the work up front. That is not a cost. You never knew it. Static planners only appeared to, and the appearance is precisely what made them brittle.

Loops become appends¶

Here is the whole mechanism.

When a step fails, the intuitive repair is to go back and try again — an edge from the failure to the node that produced it. That introduces a cycle, and cycles are what make execution history hard to read: you can no longer distinguish the first attempt from the fourth, or tell which attempt produced the result you kept.

Instead, append. The retry becomes a new node whose parent is the failure.

%%{init: {"flowchart": {"rankSpacing": 30, "nodeSpacing": 26, "padding": 8}}}%%
graph LR
    subgraph before["Cycle — the retry erases the failure"]
        direction TB
        A1(Draft SQL) --> B1(Failed)
        B1 -. retry .-> A1
    end

    subgraph after["Append — the failure is kept as a cause"]
        direction TB
        A2(Draft SQL v1) --> B2("Failed: no column region")
        B2 --> C2(Discover: region lives in dim_store)
        C2 --> D2(Draft SQL v2 with join)
        D2 --> E2("Validated result")
    end

    %% Same stroke vocabulary as the homepage diagram — see docs/index.md for why
    %% color goes in the stroke and never the fill. No copper `start` class here:
    %% on the homepage copper marks the goal the user asked for, and this diagram
    %% has no originating goal in frame.
    classDef fail stroke:#c9705f,stroke-width:2.5px
    classDef ok stroke:#5e8f6c,stroke-width:2.5px
    class B1,B2 fail
    class E2 ok

Both graphs encode the same events. Only the second can answer why does the final query contain a join.

That is the difference between a retry counter and a causal chain. A counter tells you something failed three times. A chain tells you what each failure taught the run, and which lesson produced the answer you shipped.

What acyclicity buys¶

Because expansion only ever appends, the graph is acyclic by construction. There is no cycle detection, no checker, no invariant enforced at a boundary — the operation that would create a cycle does not exist in the API.

That is worth more than tidiness. It means the graph is a partial order over events, and a partial order over events is exactly what an audit log is. So:

every node maps to one OpenTelemetry span
every edge is a causal claim: this happened, and it is why that happened
the thing that ran and the record of what ran are the same object

Most systems maintain execution state and observability separately, then spend real effort keeping them consistent. Here there is nothing to keep consistent. There is one structure, and tracing is a projection of it.

For anyone working under audit — clinical, financial, regulated in any way — this is the property that matters. "Show me why the system did that" is answered by pointing at a subgraph, not by correlating a trace against a checkpoint against a log.

The frontier¶

If the graph grows from outcomes, something must decide which outcomes deserve growing. That decision is the frontier, and it is the sharpest design question in the system.

A node is eligible for expansion when all three hold:

it is terminal — it has produced an outcome, success or failure
it is unexpanded — no expansion decision has been recorded against it
an obligation it was created to satisfy is still open — something the task requires that no node in the graph yet provides

The frontier is the set of eligible nodes. Expansion takes one, hands it to a policy along with the graph so far, and receives a decision.

The second condition is worth dwelling on, because the obvious version of it is broken. Unexpanded cannot mean "has no children," because the most interesting decision a policy makes is to produce none — this branch is finished, not failing, just done. Under the obvious definition such a node still has no children, so it stays eligible forever and the branch never retires.

So the decision itself is recorded, whether or not it produced anything. A branch ends because something says it ended, and the reason survives in the graph rather than being inferable only from an absence. Execution Semantics works through the full set of decisions a policy can return.

Note that failure is not special. A failed node is eligible on exactly the same terms as a successful one, which is why "retry" needs no dedicated machinery: a retry is just what expansion happens to produce when the parent's obligation is still open.

No waiting for the wave¶

Nodes that are eligible together run together, and the scheduler acts on the first outcome to arrive rather than waiting for the rest.

This matters more than it sounds. Suppose three researchers start at once and the first to return reveals that the run needs a currency normalizer nobody anticipated. The normalizer is justified now — it can be added and launched while the other two are still running. A join over a fixed list cannot do that, because both the list and the barrier were chosen before any result existed.

A wave-wide barrier would quietly reintroduce the thing this design is trying to avoid: a commitment, made before the evidence arrived, about what the shape of the work would be.

Termination, honestly¶

This is where the design is weakest, and where the temptation to hand-wave is strongest.

The graph does have a natural termination condition: the frontier empties when every branch declines to expand. That is real, and most runs reach it.

But nothing guarantees it is reached. The expansion policy consults a language model, and a language model can always propose one more thing to try. A run that never finds its answer can keep finding plausible next steps indefinitely. The natural stopping condition exists; it is simply not enforceable.

So budget and circuit-breaking are not safety features bolted to the side. They are the only termination guarantee the system has, and they belong in the type system — an expansion path that can reach a model call without passing a budget check is a severity-one bug, not a missing config option.

That is the honest trade for everything above. A static plan terminates because it is finite. This design buys adaptability by giving that up, and must pay for termination explicitly.

There is a partial answer to the comprehensibility half of the problem, if not the termination half. The expansion policy does not have to be a single model call trusted to behave. It can be two things: a deterministic policy that owns the invariants and is always a legal fallback, and a model policy fenced inside it — able to propose only registered kinds of work and edges to nodes that already exist, with malformed proposals rejected and the rejection recorded. The model decides what is useful; code decides what is possible.

That does not make a run predictable. It does make the set of things a run can do enumerable, which is a weaker property and the one actually needed to answer "what will this do next."

Why this shape fits serverless¶

Cloud Run allows a request up to 60 minutes, and a request dispatched through Cloud Tasks is capped at 30, defaulting to 10. Frameworks that model a run as one long-lived process fight that ceiling, and lose at exactly the moment a run gets interesting.

There is no long-lived process here. One expansion is: read the frontier, execute a node, append the result. That is a short request against durable state. A run is many such requests, and a crash resumes by reading the frontier — the same operation the scheduler performs normally, so recovery is not a separate code path that runs only during incidents.

That handles work that is slow. It does not handle work that is blocked, and the distinction is the more important one. A node awaiting a human approval, a webhook, or a remote agent is not computing anything; it is simply not finished. Holding a process open for it would put a ceiling on how long an agent may wait, and the ceiling would be counted in minutes.

So a blocked node enters waiting: alive, holding no worker and no open request, carrying a correlation key, and returning to the frontier when the matching external event is appended. An agent can wait six hours without anything running for six hours. Under a process-shaped design that is not a tuning problem — it is impossible.

The constraint that breaks other designs is one this design already had to satisfy. That is not luck. It is what falls out when the graph is the only authoritative state and every node is a fact rather than a running thing.

What this costs¶

Three things, plainly:

Graphs grow monotonically within a run. Nothing is ever removed, so long runs accumulate nodes and need a retention story. Append-only is cheap to write and expensive to keep.

Termination is your problem. See above. This is the real one.

You cannot show the user a plan, because there isn't one. A progress bar over unknown total work is a genuine UX problem, and pretending otherwise would be dishonest — though it is worth saying that plan-based progress bars were lying anyway.

Next: Prior Art — where this comes from and who did it first. Very little of the execution model is new. The application of it is.