Notizen aus der Praxis

Blocked by a Limit That Wasn't There

A worker in my Hayate Agent could stop a job by claiming a limit that did not exist, and the checker could not tell. Two pull requests now give the checker the host's own tool evidence and turn a false block into a correction for the same worker.

Karsten Samaschke · Freitag, 6. November 2026 · 11 Min. Lesezeit

行く秋や
「進めぬ」という
手に証

Autumn departing.
"I cannot go on," one says,
the proof in its hand.

Some jobs in my Hayate Agent (formerly the VanillaCore Agent) stopped because a worker believed something false about its own situation. Pull request #281 names three such cases: a worker claimed the updater lacked a capability it actually had, a worker claimed the review tool would not run in its working directory, and a worker demanded more evidence than its task asked for. Each of them ended its step with "blocked". The independent checker, a second model that has to confirm every step before the next one may start, then had to decide whether that block was real. It decided from the task and the worker's written report, because it never saw which commands the worker had run or what they returned. A confident report could stop a job even when the commands from that same step showed the claim was false.

I don't want a checker that takes a confident sentence at face value, and I want one that waves things through even less. When a worker's claim is wrong, the checker should say why, cite what the host actually recorded, and send that note back to the same worker in the same session, so the step continues instead of starting over. That is harder than it sounds. The note must not turn into new authority, the correction loop has to end, a restart must not lose the correction, and a genuine block, such as a missing permission, still has to stop the job. Pull request #281, merged at 20:18 on Thursday, builds that. Pull request #283, merged at 05:59 on Friday, closes the next hole: a worker that ended its step without a proper final report never reached the checker at all, and a restart in the middle of an assessment could run the worker again.

All times are Europe/Berlin. The window is 2026-11-05 20:00 to 2026-11-06 08:00.

What the checker could not see

A delegated job in the Hayate Agent can run as a sequence of steps. A worker executes one step and ends its reply with a status line: complete, blocked, or too hard for its model. Since earlier pull requests on Thursday, a "complete" and a "blocked" claim both go to the checker. If the checker answers "continue", the worker gets up to two more rounds in the same step, with the checker's hint passed along as untrusted data. If the checker answers "blocked", the job pauses with a visible reason.

Before #281, the checker received the task, the step and the worker's reply. The host already counted the worker's tool calls, but it did not pass what those calls were or what they returned. A worker that wrote "the updater does not support this option" was judged on that sentence. The checker had nothing to compare it with.

Correct the same worker, with the host's evidence

#281 records the tool calls of each worker turn on the host side: the tool name, a shortened form of the call (up to 400 characters), a shortened result (up to 600 characters) and whether it failed, for the last twelve calls of the turn. That list goes to the checker as untrusted data. The checker's instructions now separate two kinds of "blocked". A claim that rests on a false or unverified assumption, and that the host's tool observations contradict or do not support, is correctable: the checker answers "continue" and names one concrete correction that cites the host evidence, or the exact read-only facts the worker has to report first. Only a genuine external dependency, a missing permission or a safety boundary counts as blocked. The checker gets the job's deadline with that rule, and it is told never to grant new scope.

The correction is stored with the job before the worker sees it. A restarted host resumes the exact step with the pending correction, in the same session and on the same route, without a new plan and without repeating earlier steps. The existing limit of two extra rounds still applies. If the checker sends the same correction twice, the worker evidently did not act on it, and the job stops with one short notice for the operator instead of looping. A corrupt stored correction, for example a round count outside the allowed range, blocks the job visibly. A "blocked", "uncertain", invalid or technically failed verdict never authorizes the next step, and a manual pause during the check still wins: the correction never reaches the worker.

The PR's tests replay the three cases above, which the test file calls actual operator failures. In each, a worker claims to be blocked while the host recorded a tool result that contradicts the claim, for example the updater's help text listing exactly the option the worker said was missing. The checker sees that result, sends the correction, the same worker completes the step, and step two starts only afterwards.

An empty answer is not a success

#283 starts from a different gap. The step logic only sent a reply to the checker when it ended with "complete" or "blocked". A reply without a status line, an empty reply, or a status line followed by more text ended the step without any assessment. A worker could have done all the work and simply not reported it, and the job stopped anyway.

Now every reply goes to the checker, except one that reports the step as too hard for its model, which still goes straight to escalation. The checker gets a new prompt that judges only the current step and states that a missing status line is not an external blocker. If the evidence for the step is insufficient, the checker asks the same worker for the missing read-only facts. If the host evidence already shows the step is done, the checker still does not mark it complete. It sends a report-only correction instead: "Independent evidence passed. Report the existing verified result and final STEP-STATUS only. Do not repeat tools or completed effects." The step counts only when the worker's report comes back with its status line and the checker confirms it. Tool observations from earlier rounds of the step now stay in the evidence, up to 36 entries, instead of being replaced by the latest round.

The other half of #283 is about failures of the checker itself. An assessment that comes back uncertain, invalid or technically failed gets at most two attempts per worker result, and a second attempt never runs the worker again. The settled worker result, its evidence and the number of attempts are saved before each attempt, so a restart neither resets that budget nor replays the worker's effects. Before a worker starts a step, the host checks that the stored plan leaves room for that evidence, and refuses to start if it does not. A failed delivery of the plan status to Matrix no longer stops the job.

The checker's model is now configurable on its own. The host takes a configured validator model first, then an explicit review model, then the reviewer role, then the installation's default model. A model that is explicitly configured but unavailable fails visibly instead of switching silently. The same selection applies when the host assesses whether a supervised task is complete. The PR names two limits: paused tasks and expired mandates from before the change are not reopened automatically, and the user-facing recovery notices are a separate issue that this PR does not fix.

A test that passed half the time

Seventeen minutes after #283, #284 merged a three-line test change. According to its description, a native release preparation exposed a race in one of the new #283 tests. The job's paused state is saved before the dispatcher delivers its status messages, and the test checked for the message as soon as it saw the pause. #284 waits a bounded time for that delivery, like the other restart tests do. It changes no production code.

What a scripted run shows

The AI agent that drafts these changelog posts ran the test files that each pull request added or changed against the code directly before and after its merge, with Node 22 on a local machine. These are controlled local tests with fake model runtimes and temporary state, not a live system.

Change and tests	Before the merge	After the merge
#281 step tests and paused-child supervision, 40 tests	34 pass, 6 fail (065683c)	40 pass (f985096)
#283 step tests and the new step-prompt tests, 51 tests	the step test file fails to load (f985096)	50 pass, 1 fails (9accea8)
#284 the checkpoint-budget test, 10 runs each	5 pass, 5 fail (9accea8)	10 pass (0a8a265, which adds only the README after #284)

The six failures before #281 are the new correction cases: the three false-assumption replays, the restart with a stored correction, the repeated-correction stop and the corrupt-metadata block. The two other new tests, no correction for a genuine block and the manual pause, already pass on the old code, as they should. Before #283, the new step test file cannot load because the function that selects the checker's model does not exist yet. The single failure after #283 is the test that #284 fixes, and it fails in about half of the runs. At the last merge of the window, 0a8a265, the step tests and the paused-child supervision tests pass 51 of 51. The authors report 1,529 tests with 1,526 passing and 3 skipped for #281, and 1,542 of 1,542 for #283.

What the checker still cannot know

The evidence the checker sees comes from the host, not from the worker's prose, and that is the point of #281. It is still evidence the worker produced by choosing which commands to run. The observations are shortened excerpts with a fixed limit, and the new checker prompt says itself that a missing observation does not prove that nothing happened. The checker is a model too, and it can be wrong; its correction is untrusted data that grants nothing. I accept those limits, because before #281 the checker had nothing but the worker's own sentence to check, and a confident wrong sentence was enough to stop the job.

The rest of the night

#285 in core, merged at 06:30, and #50 in plugins, merged at 06:46, replace the internal migration and publication status in the README files with a user-facing introduction. Both descriptions say they synchronize a README already reviewed on GitHub and change no runtime, deployment layout or credentials. The plugins change also adjusts its product-naming test.

What merged and what was tagged

When the window closed, core main was at 0a8a265 and plugins main at 94d6dfb. The deployment repository had no merges in the window. Each new core release tag sits on the merge commit of one of the night's pull requests: v0.1.191 on #281 at 20:25, v0.1.192 on #283 at 06:04, v0.1.193 on #284 at 06:23 and v0.1.194 on #285 at 06:35, and each later tag contains the earlier merges. A tag marks the source of a published image, and a published image is not a deployment. The description of #283 plans the normal image publication after the merge and an update of four named agents; whether any running agent was updated is not verified here.