Skip to content

State and Persistence

Design, not implementation — Phase 3

src/swarmplane/persistence/ is a package skeleton. The model below is decided but unimplemented; none of it has been proven by a running system.

The log is primary; the graph is a projection

The stored truth is an immutable, ordered event log. Every run persists:

  • the original goal and ownership scope
  • every node and edge as it was created
  • every lifecycle transition
  • every effect attempt and its result
  • every expansion disposition, and the event that triggered it

The graph anyone queries — nodes, edges, current states, the frontier — is a projection of that log, not a separate table that the log describes.

This is the difference between claiming an audit record and having one. A projection can be rebuilt and re-checked against its source. A mutating node record cannot be un-mutated, and a run that ended in the wrong place leaves no evidence of the state it passed through.

It also gives "append-only" a precise meaning. Events are appended. Topology follows from them. Nothing is edited in place.

Crash recovery is the ordinary path

There is no separate recovery routine, which matters more than it sounds: a code path that only executes during incidents is a code path that is never exercised until it matters.

On restart:

  • succeeded nodes stay succeeded. Their outcomes are already in the log.
  • running nodes return to eligible, unless they left an effect unaccounted for. The process died before it could record an outcome, so as far as the log is concerned the work never produced one. Dependably true of the outcome; not always true of the effect — see below.
  • waiting nodes stay waiting. They were holding nothing to begin with.
  • terminal events that landed before their disposition are replayed before any new work launches.

That last step is safe because dispositions are idempotent. A disposition already recorded against a node is never recorded twice, so replaying the event that triggered it does not duplicate the nodes it created.

The invariant underneath all of it:

An outcome is durable before it is allowed to create more work.

An effect can outlive the process that issued it

Idempotent dispositions make expansion safe to replay. They say nothing about execution, and the two are different operations.

A node that calls an MCP server, dispatches over A2A or emits a surface has done something outside this process. If that effect commits and the process dies before the outcome is appended, a rule that returns every running node to eligible issues it a second time. The more of the design that crosses a protocol seam, the more of it is exposed here.

So effects are ordered against their own record the way outcomes are ordered against expansion:

An intent is durable before the effect is allowed to happen.

Appending the attempt first leaves restart with three distinguishable states rather than one ambiguous one. No attempt means the effect did not happen. An attempt with a result means it happened, and how it ended. An attempt with no result means nobody knows.

Only the third is a problem, and the runtime does not guess at it. An effect declares itself repeatable by carrying a read-only marker or an idempotency key its target honours, and a repeatable effect returns to eligible like anything else. An effect that declares nothing is treated as unknown rather than as safe — silence is the wrong thing to read as consent — and its node enters waiting on a correlation key naming the effect, resuming when an external event settles what actually happened.

That trades liveness for correctness: a crash mid-effect can park a run on a person, and until the runtime can ask a target whether an effect landed there is no cheaper answer. The alternative is a graph that records a write as having happened once when it happened twice, which would make the audit claim false in precisely the case where it is load-bearing.

Retention

Append-only is cheap to write and expensive to keep. A long run accumulates events monotonically, and the projection has to be rebuilt from them.

Neither compaction nor snapshotting is designed yet. The obvious approach — periodic snapshots of the projection, with the log truncated behind them — trades away the ability to replay a run from its beginning, which is part of what the audit claim rests on. That trade has not been made.

Interface

GraphStore is the seam, with an in-memory implementation for the demo and tests, and a Postgres implementation for deployment. Nothing above depends on which is in use.

The in-memory implementation is not a toy for the demo's sake — the warehouse agent must run from a clean clone with no cloud account, so the in-memory store is a first-class target that the same property tests run against.