KarstenSamaschke
Zurück zu allen Artikeln

Notizen aus der Praxis

The Task That Stopped Without Saying Anything

When my Hayate Agent's independent recovery check failed three times, the task stopped as blocked with no message, and the budget reset built for stuck tasks refused it. Since #251 that end produces one notice naming the reset, even with technical diagnostics switched off. Tasks already stuck the old way stay silent.

Karsten Samaschke · · 9 Min. Lesezeit

The Task That Stopped Without Saying Anything

三度目で
黙るタスクが
声を出す

At the third try,
the task that went silent
speaks up.

My Hayate Agent (formerly the VanillaCore Agent) keeps long tasks on disk, so a restart or an interruption does not erase them. Before it continues such a task, it asks an independent check whether the task should go on. When that check breaks for technical reasons or returns an invalid answer, it gets three attempts. Until this afternoon, a task that used up all three simply stopped. It was marked as blocked, no message went to the conversation, and the one command built for stuck tasks, the budget reset, refused it with "Task is not host budget-blocked". The task stayed there until someone went looking for it. The pull request author reports two live tasks with exactly this signature.

What makes this tricky is that the harness had been made quieter on purpose almost four hours earlier. Pull request #246 switched the host's technical recovery notices off by default, and I agree with that default: most of them are diagnostics, not decisions. A stop that only a human can lift is not a diagnostic, though, and pull request #251, merged at 15:34, draws that line: exhausted recovery now ends with one notice that names the way out, whatever the diagnostics setting says.

Hayate Agent cover: small slate 疾風 logo and "Hayate Agent" at the top, the red two-line Japanese headline 何も言わずに / 止まったタスク (the task that stopped without saying anything), the white English headline "The Task That Stopped Without Saying Anything" and github.com/hayate-agent at the bottom, on an indigo-black field with a faint wave pattern along the lower edge

Where the silence came from

The recovery check is deliberately strict. A missing assessment never lets a task fall back to just working on. An inconclusive answer blocks the task immediately, without any retry. For technical failures and invalid answers, every attempt is reserved and saved before the check runs, so a crash uses up an attempt instead of granting a free one. The next attempt becomes due one minute after the previous one was reserved, and after the third failure the harness stops trying.

Two code paths reached that point: a task that already had three failed attempts on record, and a task whose third check had just failed technically or returned an invalid answer. Before #251, both only parked the task. Its state became blocked with a placeholder next step ("Independent supervisor admission unavailable"), but the task got no termination reason, no notice ID and no message. For this kind of stop, the budget reset requires the termination reason recovery_assessment_failed and at least three used attempts, so it rejected exactly the tasks it was meant for. An older test even asserted this behavior under the title "three real recovery failures retain diagnostics but do not require operator budget reset".

What #251 changes

Both paths now call the notice function the harness already had, with the reason "exhausted", after parking the task. That records the termination reason recovery_assessment_failed, stores a deduplication ID and queues one message to the conversation, or to the configured home address if the conversation has none. If neither exists, the state is still recorded, but there is nobody to message. The message, in German, says that all three assessment attempts are used up, and it quotes the sentence an authenticated administrator writes to reset the budget and continue. It also states that the deadline and the workers stay unchanged. This one notice goes out even with recovery diagnostics switched off, the same way the existing task-budget escalation does. All other technical recovery notices stay opt-in.

The PR is explicit about what it leaves alone. It adds no new retry logic, starts no work and replays no worker, and it extends no deadline. Stopped, paused, expired and deliberately blocked tasks are untouched. Tasks that were already saved in the old silent state are not migrated either: the new test asserts that the reset still rejects such a legacy task. Anyone who has one of those on disk still has to find it by hand.

What a scripted run shows

The AI agent that drafts these changelog posts ran each merge's added and changed test files against the code directly before and after that merge, with Node 22 on a local machine. The tests drive the real harness class, mostly with fake runtimes, in-memory channels and a temporary state file; the #253 tests also run the real PI adapter locally. None of them runs a live system.

Change and tests Before the merge After the merge
#251 exhausted recovery, 4 files, 51 tests 38 pass, 13 fail 51 pass
#252 validator context, 8 tests 6 pass, 2 fail 8 pass
#253 worker contract, 5 tests file does not load 5 pass

For #251, the two main new tests each fail three recovery checks across restarts of the harness from its saved state, once with technical failures and once with invalid answers. On the old code, the saved task has no termination reason where the test expects recovery_assessment_failed. On the new code, the task ends with exactly that reason, one queued notice that is delivered once, no fourth assessment and no worker turn, and the reset then commits without moving the deadline. For #253, the before column means the new test cannot even import, because the module it tests does not exist before the merge. At the last merge of the window, 0815b55, all six files together pass 64 of 64.

The PR authors report full serial runs of 1,422 tests for #251 and 1,430 for #252, each with an independent review without findings. Those are their reports. The run described here covered only the files above.

Telling workers the truth about their tools

The two later merges are about the same family of problem: a part of the system acting on a picture that does not match reality.

#253, merged at 16:55, deals with child sessions. When the agent hands a job to a worker or a subagent, that child is not allowed to schedule task continuations, start loops or start subagents of its own, and a dispatcher worker also cannot dispatch further work. The tool guard enforced that, but the child's prompt still carried the parent's instructions about continuing tasks, and the denied tools could still be advertised to the model. A worker could plan around a tool it would never be allowed to call. Now one denied-tool list per child role feeds the guard, the tool list the PI runtime shows the model and the capability description, and children get a short contract instead: their final reply is their outcome, and the parent decides what happens next. The PR says this still needs live activation and does not claim to fix every worker stall.

#252, merged at 16:12, gives the independent completion check more to work with. It now sees the outcomes of worker runs bound to that exact task, and it keeps the task's recent conversation messages in the same task-bound section, so both survive when the input has to be shortened. Unrelated runs are only counted, worker results are marked as untrusted claims, and when the input gets too long the check keeps these sections with fixed caps and a list of what was shortened.

The rest of the day

The other merges are smaller. #250 saves a /stop to disk before the runtime is interrupted, so the stop is recorded before anything is aborted. #249 makes /new and /reset list up to twelve retained tasks, waits, dispatcher jobs, loops, heartbeats and standing goals that remain after the reset, plus a count of the rest. #247 and #248 change how technical failures during recovery are retried and reported, and plugins #47 gives the completion check the matching evidence. #245 and plugins #46 stop the activity display from claiming a model is running before the runtime has reported which model it actually selected. #243 makes the task classifier classify a task that contains instructions instead of following them, and #244 only changes a test adapter.

What merged and what was tagged

The window ran from 08:00 to 20:00 on November 4, Europe/Berlin time, and all times in this post use that zone. Core merged eleven pull requests, #243 through #253, plugins merged #46 and #47, and deployment-base merged nothing. Each core merge is the target of its own release tag, from v0.1.169 for #243 at 09:11 to v0.1.179 for #253 at 17:00. When the window closed, core main was at 0815b55, plugins main at 99b4963 and deployment-base at 9776088. Each release reports a managed image with a digest, which was not checked against a registry for this post. A tag is not evidence of a rollout either: whether any running Hayate Agent uses these changes is not verified here, and the public GitHub snapshots under hayate-agent were not checked for them.

Verdict

Quiet defaults are right for diagnostics and wrong for decisions. A task that can only move again when a human resets it has to say so, once, with the exact sentence that does it, provided there is someone to tell. That is what #251 adds, and nothing more. The tasks that already went silent before it are still silent.

Sources used