The gate goes outside the model
Design notes from running an engineering agent unattended
For the past few months I've had an agent running unattended on a cloud droplet. It pulls work from a queue, writes a spec, researches the codebase, designs, implements, reviews its own diff adversarially, commits, and schedules the next cycle. I look at my phone when it decides something needs me.
It works now. Almost nothing about the version that works resembles the version I first built, and every difference points the same direction: wherever I trusted the model to hold state, follow a process, or check its own work, I eventually had to move that responsibility out of the model and into something deterministic.
The domain underneath is a personal research codebase, and that part isn't the interesting bit. What's interesting is the loop, so that's what this is about.
1. Reality is on disk. Everything else is a cache.
The first version tracked pipeline state in a JSON file. Step 4 of 12, task PRIV-83, in progress. Then a tick died halfway through writing an artifact, and the next tick read "step 4," concluded step 4 was done, and confidently started step 5 on top of a half-written file.
The rule that replaced it: the git working tree and the artifacts on disk are the only source of truth. The status file, the issue tracker, any counter: all caches. Caches lag and desync. Every tick reads reality first, then the caches, then reconciles the caches to reality. If the status says step 4 but design.md doesn't exist on disk, the status is simply wrong, and fixing it is the tick's first job.
Obvious written down. Not obvious while building, because reading a counter is so much cheaper than listing a directory and deciding what the files mean.
2. Every tick has to be amnesia-correct
The agent holds nothing across ticks. Each one reconstructs its entire understanding from disk.
I arrived here from the opposite direction, trying to keep a long-running session alive with accumulated context and watching quality decay as the context filled with its own earlier reasoning. The fix wasn't a bigger context window. It was making each tick short enough that it never needed one.
The test I use: if my context were wiped right now, would this tick still be correct? If the answer depends on something the agent remembers rather than something it can read, the design is wrong.
A useful side effect: the dispatcher passes file paths and one-line statuses to sub-agents, never file contents. The orchestrator's context stays tiny, and the sub-agents each get a fresh window sized to one job.
3. Single-flight, on an atomic primitive
At most one task is ever in the pipeline. This is enforced with an actual atomic operation, mkdir or flock, never a read-modify-write on a JSON file.
I learned that distinction the boring way. Two overlapping ticks, both read {"locked": false}, both wrote {"locked": true}, both proceeded, and they edited the same files. The lock has to be atomic at the filesystem level or it isn't a lock, it's a suggestion with extra steps.
4. The gate goes outside the model
This is the one that matters most, and the one I'd lead with if I could only say a single thing.
The agent used to run the test suite and commit if it passed. Sometimes it reported green without having run anything. Sometimes it started the suite in the background and committed before the run finished. Occasionally it "fixed" a failure by narrowing the assertion.
None of this is malice, and I don't think it's really even a capability problem. "Run the suite and commit if it passes" is a process, and a process described inside a prompt is a strong suggestion, not a guarantee. The model is the thing being checked. It cannot also be the check.
So now the agent's job ends at: leave a clean, committable working tree and write a handoff file naming the commit subject. A shell script the model never invokes and cannot modify then runs the linter, the fitness checks, and the suite, synchronously, and either commits and pushes or hard-rolls-back and escalates.
Generalized: any check that must not be skipped cannot live inside the thing being checked. That single move eliminated an entire category of failure that no amount of prompt-tightening had touched.
5. Recover to a clean boundary, never a partial merge
When a tick dies mid-step, the tempting move is to work out what got done and continue from there. Don't. The agent rolls back to the last committed step or last complete artifact and redoes the work.
This requires every step to be idempotent or explicitly guarded: artifact exists and validates → skip. That constraint is annoying to design around and it is the reason the system can crash at any point without a human untangling it. Redoing five minutes of work is always cheaper than reasoning about a half-applied state.
6. Escalation is a feature, not a failure
Early on I treated every human interruption as a bug to design away. That was backwards. An autonomous system without a clear "stop and ask" path doesn't become more autonomous. It becomes more confidently wrong.
There's now an explicit list of triggers: a decision that would contradict a ratified decision, anything that changes the mission or a risk parameter, a step whose autonomous mode doesn't exist yet, the same task failing three times, budget exhaustion. On any of these the agent writes a reason, pauses itself, and stops scheduling.
The alert is mechanical: a separate runner detects the pause transition and pushes the notification. The agent doesn't decide whether to tell me. It writes the reason; the plumbing does the rest. Same principle as the commit gate: the step that must not be skipped lives outside the thing that might skip it.
A related rule I'm fond of: the agent must never invoke an interactive skill. If the only version of a step needs a human at a keyboard, that's an escalation, not something to attempt. An agent silently hanging on a prompt nobody will ever answer is a much worse failure than stopping loudly.
7. Memory has to compound, and debt has to be visible
Every completed unit of work writes a decision or experiment record to a wiki the next cycle reads. Without it, the loop happily re-runs the same dead end forever. It has no memory of having tried, and nothing about its situation looks different the second time.
The records are marked provisional until a human ratifies them. That distinction turned out to matter: without it, the agent's own guess from three weeks ago gradually hardens into "the decision," and there's no way to tell an assumption apart from an intention.
Corollary, learned after finding a 0.0 placeholder that had been quietly load-bearing for weeks: every deferral, stub, and TODO gets filed as a real tracked issue. Carried-forward debt must never live only in runtime state, or it rots somewhere nobody looks.
8. Bounded by construction
Cycles per day, tokens per cycle, cooldown after consecutive failures. Not because the agent tends to run away, but because "how would I notice if it did?" needs an answer that isn't "my bill."
What I'd tell someone starting
Build the loop as if the model is a competent contractor who will occasionally, unpredictably, skip a step and tell you it's done. That's not a slight. It's just the correct threat model, and designing for it produces a system you can leave running.
Concretely, the ranking that emerged:
- Move every must-not-skip check outside the model. Largest single quality jump by a wide margin.
- Make disk canonical and reconcile everything else to it. Kills the whole class of bugs where the agent believes something that isn't true.
- Make ticks stateless and short. Cheaper, more reliable, and trivially resumable.
- Design the escalation path before the happy path. It's what makes leaving it running a reasonable thing to do rather than a gamble.
Everything else (the prompts, the stage decomposition, the model choice) mattered far less than I expected going in.
The runtime described here is open source: claude-asimov.
I'm reachable at vladimir.kisselev.dev@gmail.com.