KarstenSamaschke
Zurück zu allen Artikeln

Notizen aus der Praxis

The Image Shipped, and My Agent Never Noticed

My agent waited for a new image of my harness, the image shipped, and the update sat untouched until I asked again. The overnight fix is an alarm clock with a deadline, and the woken agent has to check before it acts.

Karsten Samaschke · · 14 Min. Lesezeit

The Image Shipped, and My Agent Never Noticed

Yesterday one of my agents had a simple job: wait until a new image of my harness was published, then update my agents to it. The image did not exist yet, so the agent reported "waiting" and ended its turn, which was the right answer. Then the image was published, and nothing happened. The update sat there until I noticed that nothing had moved and asked again.

Nothing had crashed, because ending the loop on "waiting" was exactly how my harness was built. The textbook fix is a doorbell, a webhook from the release system that rings when the image exists. What landed overnight is cruder, an alarm clock with a hard deadline, and I think that is the right call for two reasons. A doorbell needs one integration for every kind of thing an agent might wait for, while an alarm clock works for any condition the agent can describe. More importantly, a woken agent must not trust the ring anyway: before it touches a rollout, it has to check whether the image really exists and whether the rollout already ran while it sat idle, and it must not keep checking forever.

The alarm clock is core #105 in the public VanillaCore Agent Harness, and its problem statement describes exactly this failure: a self-reported wait kept no executable next step and no wakeup time, so the resolved dependency did not restart the authorized task and the operator had to ask again. A second change, #90, covers the opposite case, a "waiting" or "blocked" that was wrong in the first place, and that case needs a different answer.

An hourglass with a dark graphite frame stands centered on a navy desk, most of its pale sand still in the upper bulb, with a blank paper tag with a muted light-blue edge lying in front of it

How the work got stuck

After each working turn, a Harness agent records what should happen next through the plugin_task_continuation tool. It can plan a concrete next step, report that it is waiting, report a blocker, or mark the task as completed. Before these changes, only the first answer scheduled another turn on its own, and waiting and blocked ended the loop unless new input arrived, and the heartbeat documentation said so openly: waiting states need an external event or explicit new input before anything resumes.

That design leaves the scheduling decision to the model. Its self-report is the only signal the loop reads, and three of its four possible answers stop the loop. For "completed", the evening post covered the countermeasure: since #87, a small tool-free assessor rereads a premature "completed" against the original task. "Waiting" and "blocked" were deliberately left out of that check, which left two different ways for a task to go quiet.

One of them is a wrong claim. A model that concludes it lacks an approval, although the original task already granted it, reports "blocked", and the task ends there. The description of #90 calls this a mistaken blocker claim that became terminal, because no check ever read it again. The other one is a correct claim with nowhere to go. A model that rightly waits for a release image can write a perfectly good next step into its reply, but that sentence is prose in a transcript. Nothing stored which condition to check or when to look, so when the image was published, nothing in the system knew that it was supposed to look again.

Why the fix comes in two parts

These two cases need different answers, and I think separating them is the most useful decision of the night. A wrong claim needs a check, because the agent should not be the only judge of whether it is stuck. A correct claim needs state that the host owns, because a promise in a reply to come back later is nothing a scheduler can run.

#90, merged at 20:25, handles the first case with the assessor that already existed. The code change is small: the assessor used to skip every checkpoint other than "completed", and now it skips only an explicit continue plan. Its prompt gained one instruction, which says that the agent's self-reported state is advisory and no independent evidence of a real blocker, and that the assessor must not invent a missing approval either. Everything else works as #87 built it. The assessment needs an anchor in the original task, it still respects the original authorization, expiry, follow-up depth, cancellation, pause and queued operator input, and an unavailable or uncertain assessment does not trigger a continuation. A genuine blocker still produces no follow-up.

#105, merged at 07:28 together with the plugin side in plugins #8, handles the second case. An agent that really depends on something external can now attach a wait object to its "waiting" plan, consisting of a reason, a recheck interval and an absolute deadline. At registration, the host writes the reason, the next step, the computed due time and the deadline into the same checkpoint as the turn itself. The interval is only an input, and what the host keeps is the moment at which the next check becomes due. From that point on, the wait belongs to the Harness instead of the transcript, and a restart does not lose it.

The PR names its own tradeoff: the wait is a timer fallback and not a release webhook integration. The Harness does not subscribe to release infrastructure, installs no Forgejo webhook, and core does not poll the network. A timer works for any dependency an agent can describe, without one integration per source, and its price is that a wakeup proves nothing. The task-waits docs say exactly that: a recheck is a normal working turn with the original mandate and deadline attached, its job is to verify the condition instead of assuming it, and the wakeup grants no extra authority.

Each wait is bounded, and I checked the bounds against the code at the window-end commit. The interval runs from one minute to 24 hours. At registration, the deadline must lie more than one interval ahead and no more than seven days ahead, and it can never extend the deadline the original task already had. Rechecks count against the existing limit of twenty automatic follow-ups. A registration buys exactly one recheck, so if the condition still does not hold, the agent has to register a new wait inside the same deadline. That rules out an endless polling loop and makes the agent decide again on current evidence each time.

Delivery reuses a path the Harness already trusts. On the first idle scheduler tick after the due time, the recheck is claimed through the same durable inbox barrier that ordinary operator input passes, so overlapping ticks cannot claim it twice. If the checkpoint cannot be written to disk, the queued recheck is held and never executed early. When the deadline passes or the follow-up limit is used up, the task becomes blocked and the Harness posts a visible notice for the operator without starting another model turn, so a wait that never resolves ends in something a person sees.

Plain "waiting" without a wait object still exists for work that genuinely depends on the operator, and the tool response now tells the agent that no automatic wakeup is registered. Writing a next step next to a plain wait schedules nothing. I like this detail more than the scheduler, because it corrects the agent's picture of the system at exactly the moment the agent would otherwise rely on its own promise. Registered waits are also excluded from the assessor override of #90, which keeps the two halves from fighting: a scheduled recheck cannot be replaced by an immediate continuation.

An illustrative walkthrough

The following run replays that kind of update wait step by step to show the mechanics, so it is constructed rather than a log of yesterday's run. The reason and the next step come from the example in the task-waits documentation, and all times are invented.

An agent has a task that ends in a rollout once an image containing the approved source revision has been published. At 09:00, it looks, finds no such image, and reports "waiting". Before #105, the loop stops at this point. The image is published at 09:40, and the task stays untouched until the operator asks at noon why nothing has happened.

With #105, which first shipped in v0.1.56, the agent registers a wait at 09:00 instead. Its reason says that an image with the approved source revision has been published, and its next step says to read the release source and image digest and to reconcile a completed rollout before taking any action. Its interval is 30 minutes, and its deadline is 15:00. The host persists all of this with the checkpoint and stores 09:30 as the due time. If the Harness restarts at 09:10, the wait comes back with the rest of the checkpoint.

At 09:30, the next idle tick claims one recheck, and with the default tick of 15 seconds, the turn starts shortly after the due time if the agent is idle. The agent reads the release source, finds no matching image and registers a new 30-minute wait under the same 15:00 deadline. At 10:00, the next recheck finds the image from 09:40, reads its digest, checks whether a rollout has already completed and only then continues the original task under the original mandate. The task resumed 20 minutes after the image appeared, which is the latency a timer costs, and the operator did not have to ask. If the image never appears, the agent keeps registering waits until 15:00 or until the twenty follow-ups are used up, and then the task becomes blocked with a visible notice.

A variant of the same run shows the other half. Suppose that at 09:00 the agent reports "blocked, waiting for deployment approval", although the original task already authorized the rollout. Where an assessor is configured, it now rereads that claim against the original task and treats the agent's statement as advisory. If a step of the task is still missing, the Harness records exactly one continuation for it. Before #90, the claim would have ended the task.

Infographic: what happens after an agent reports waiting or blocked, before and after core #90 and #105. Plain waiting or blocked is rechecked against the original task; a registered wait is persisted and gets one recheck on the first eligible idle tick, must be registered again if the condition still does not hold, and turns into a visible block at the deadline

Both paths in the graphic end in a state that someone can see. A wrong claim gets a second reading, and a correct wait either resumes the task or becomes a block at its deadline.

What this does not solve

Its reason field is text for the agent to inspect, not an expression that the Harness evaluates. Whether the condition holds is still the agent's judgment, and the agent can get it wrong on a recheck just as it can on any other turn. The assessor is a model as well and can misjudge a task, which is why an uncertain assessment does not continue. The PR for #105 also says plainly that there is no formal proof of how far natural-language authority reaches.

The timer is slow by design. A recheck fires only when the agent is idle and a tick runs, so the real delay is the interval plus tick and idle time. Pause, stop, queued operator work, active turns, externally managed worker sessions and unavailable persistence all suppress wakeups, and API runs and the nightly controller cannot register waits at all. Tasks that were already waiting before the upgrade are not migrated and get no timer.

The evidence is the PRs and the code, not a live run. #90 reports 515 of 515 tests passing, including regressions in which a premature "completed", "waiting" or "blocked" each produce exactly one durably recorded continuation while terminal boundaries produce none. #105 reports 581 of 581, including restart serialization, concurrent ticks at the same due time, the held disk-failure case and deadline escalation. Both changes were checked by their author, and neither got an additional independent review round. One documentation file also lags behind the code: docs/work-heartbeat.md at the window-end commit still says that waiting and blocked checkpoints are not reassessed, while docs/completion-check.md and the code say they are, and I go with the code.

What I would take from it

If you build an agent loop that has to survive waiting, the condition, the next check and the end of the wait have to live in state that the host owns and runs, and the model's own report has to be input for a check instead of the decision itself. The Harness still lets the model judge whether a release exists, but a task can no longer disappear because the model said "waiting". A timer is cruder than a webhook, and I still prefer it here, because its deadline produces a visible block that an operator can debug, while the old behavior produced a silence that nobody noticed until someone asked.

Elsewhere in the window

The other merges of the night are only loosely connected to this story. #91, merged at 20:27, follows a related rule for dispatched executions interrupted mid-run: it reconciles them from retained tool evidence in the session archive instead of replaying an opaque turn, and an explicit recovery grant is consumed atomically, though at this revision it is an API in core and not yet a chat command. #103 renders the nightly improvement report from complete validated receipts instead of replies cut at 4,000 characters, and it labels outcomes as reported, not verified. Most remaining merges move code into harness/plugins (#98, #99, #100), add trusted local policy hooks that the PRs explicitly call no sandbox (#93, #101, #102), or change Matrix behavior (#92, #94, #95, #96, #97, #104), where the opt-in activity messages are off by default and none of the PRs claims a live Matrix acceptance test.

What merged and what shipped

Sixteen core merges, #90 to #105, landed on main, and the head at the end of the window is 25bee2d from #105. harness/plugins merged #4 to #8 without a release of its own, and core #105 pins the plugins revision that contains the wait plans. Core published fourteen releases in the window, from v0.1.43 at 20:35 to v0.1.56 at 07:33. I checked ancestry instead of timestamps: #90 and #91 first shipped in v0.1.43, #103 in v0.1.54, and #105 only in v0.1.56, not in v0.1.55.

As in the previous posts, I describe Harness source changes here. Nothing in this post claims features or compatibility in Hermes Agent, Claude Code, Codex or OpenClaw, and nothing proves that any installation already runs these versions. Each release note names its source commit, a container image and a digest, which documents an image release and says nothing about deployment. Most PRs in the window state that merging or building does not activate a deployment, and #91, #102, #104 and #105 name GitOps as the owner of rollout. I ran no benchmark and no live test.

This post covers 26 October 2026, 20:00, to 27 October 2026, 08:00, Berlin time (CET), which is 19:00 to 07:00 UTC. It continues the evening post, which ended with core #89 and v0.1.42. Core merges from #106 and releases from v0.1.57 on came after 08:00 and are not part of it. When I checked, the public harness organization listed three repositories, core, plugins and deployment-base, and deployment-base had no merge and no release in this window. That describes the public inventory at that time and says nothing about private repositories, and a query for open pull requests returned none.

Sources used