KarstenSamaschke
Zurück zu allen Artikeln

Notizen aus der Praxis

One Night, 22 Merges: The VanillaCore Agent Harness Learned When to Stop

Between Saturday 20:00 and Sunday 08:00 Berlin time, the VanillaCore Agent Harness merged 22 pull requests. The most useful change undid another: a default five-minute heartbeat, gone two hours later.

Karsten Samaschke · · 10 Min. Lesezeit

One Night, 22 Merges: The VanillaCore Agent Harness Learned When to Stop

Between Saturday 20:00 and Sunday 08:00 Berlin time, the public VanillaCore Agent Harness repository merged 22 pull requests into main and published eight release images. The most useful change of the night undid another one: at 21:01 a default five-minute work heartbeat landed, and two hours later it was gone again, replaced by something smaller and more honest (#45, #49).

If you run Hermes Agent, Claude Code, Codex or OpenClaw for anything longer than a quick question, you probably know at least one of two failure modes. The agent stops halfway and waits for you, or something keeps nudging it and it burns tokens to tell you it is still thinking. My take on this night of Harness work is simple: an agent that keeps working is only worth running if it knows when to stop, and if its documentation says where its guarantees end. Most of the 22 merges move in that direction, and the ones that do not at least say so in writing.

A stopped metronome with its pendulum resting vertical next to a closed notebook on a dark desk

The heartbeat that never shipped

#45 gave established conversations with a reply destination (API runs excluded) an automatic five-minute contextual work heartbeat and migrated existing sessions into it, with a durable opt-out. That is the classic polling answer to "the agent stopped": wake it up periodically and hope it has something to do.

#49 removed the automatic initialization again. In its place, a working turn records a structured continuation plan through plugin_task_continuation with one of four states: continue, waiting, blocked or completed. Only "continue" with a concrete next step can create a follow-up, at most one per turn, and a chain stops after 20 follow-ups. Failed and cancelled turns do not continue, API runs are excluded, and queued user input always goes first (work-heartbeat docs at the window-end commit). Explicit /heartbeat schedules still work; nothing is installed automatically anymore.

The release timeline makes this clean for users. The last release before the window, v0.1.7, predates #45, and the first release in the window, v0.1.8, already contains #49. No tagged release ever carried the default five-minute heartbeat. The docs also say plainly what the new mechanism is not: it is not a task registry, a waiting state needs an external event or new input to resume, and the tool records a plan rather than inferring completion from prose. I prefer that sentence to any amount of "autonomous" marketing.

Nightly improvement: a foundation, labeled as one

#48 adds an optional nightly improvement controller. It reads an improvement.yaml, works inside a local-hour night window, reserves work turns durably before the model runs, only starts when other conversations are idle, and sends a morning report through the Harness outbox when a destination is configured (otherwise the report stays local). It is off by default with a budget of 20 turns; the docs show 200 as an example, and the code at the window-end commit accepts up to 1,000.

The PR title says "not full evaluator loop", and the body is even more direct: objective evaluation, structured experiment records, enforced isolated worktrees and gated PR/merge were unfinished, prompt instructions are not enforcement, and nobody should activate unattended merge on this foundation. The improvement docs at the window-end commit are more precise about what the merge already contains: an optional pinned evaluator compares baseline and candidate commits in detached worktrees and writes evidence records, but its verdict is not wired into an enforced adoption or merge gate, and the docs call it neither a security boundary nor unattended release authority.

Three follow-ups make the scheduling less fragile. #49 starts nightly work via local-time Cron and persists start intent across busy sessions and restarts. #55 keeps the next night's start intent alive when the morning report is delayed, so a restart no longer silently skips a night. #56 catches up a missed start when the Harness comes up inside the night window, without resetting the night's budget, and its regression covers the DST switch that happened in this very window. #53 keeps the bundled continuation and evaluator tools reachable when an operator configures an explicit tool allowlist, while an explicit empty list still means no tools.

Context handover before the turn, not after

Long sessions fill the context window, and every agent runtime has to decide what happens then. #40 separates two things that used to blur: automatic handover now runs before queued input reaches the working model, seeds a fresh session from a summary and passes the original prompt through unchanged, while compaction stays a post-turn operation. The incoming prompt's size is estimated byte-wise; the PR explicitly does not claim semantic task classification. Automatic handover is enabled by default in the code at the window-end commit (autoHandover: true), although an older paragraph further down in the same document still calls it opt-in.

#60 fixes the interaction between the two: with autoHandover on, automatic post-turn compaction is deferred so it cannot shrink usage and burn the cooldown right before the handover would have triggered (context-maintenance docs). #54 removes a global bottleneck: slow context maintenance in one conversation no longer blocks the start of turns in unrelated conversations, while ordering inside one conversation is preserved.

Memory that is searchable without a manual reindex

Three memory fixes are small, and each one would annoy you in practice. #43 schedules embeddings right after new or revised memories are written, so they become semantically searchable without an explicit reindex; an unreachable embedding endpoint defers retries to the periodic scan instead of spinning. #50 puts a shared budget of 16 embedding requests per scan interval on that background work, counting failed requests too. #52 stops trimming trailing whitespace from stored memory payloads, so what you write is what you read back after a restart.

Pinned identity and skills: integrity, explicitly not a sandbox

#42 lets infrastructure config pin SHA-256 hashes for files like SOUL.md or skills. Pins are checked at startup, before every working turn, before persona loading and before skill reload, and a missing or changed file fails closed. Runtime-writable local settings cannot override the manifest, and an admin-only /protected-content status command shows expected and actual hashes.

Eighty minutes later, #47 fixed a confirmed P1 in exactly this feature: the default config omitted protectedContent, so manifests loaded from harness.yaml were dropped and verification was silently disabled. Fail-open in an integrity feature is the worst kind of bug, and it was caught and fixed in the same evening, with a regression test that failed before the fix.

The protected-content docs are blunt about the limits. This is integrity checking, not a sandbox or filesystem write protection. A process that can change both files and manifest can defeat the pins after a restart, trusted in-process plugins can bypass the checks, and pinning individual files does not stop someone from dropping an unpinned skill next to them. Put manifest and files on infrastructure-owned read-only mounts, or do not rely on this.

Gateway fixes for message delivery

Four merges harden the messaging gateway lifecycle. #41 refuses to replace a gateway while a reply is in flight, so durable outbox retries stay queued instead of getting lost. #46 keeps a newer activation request alive while an older replacement is still pending, and #51 discards slower, superseded preparations that finish out of order. #44 merges per-channel gateway settings field by field, so a partial overlay no longer wipes the installation's privacy defaults. That last one is the privacy-relevant fix of the batch.

Loops the agent can manage itself

#59 gives the agent a bundled plugin_loop tool with status, set, pause, resume and stop, sharing the same controller as the /loop slash command. A loop is scoped to the current conversation, needs an interval of at least one minute, and can be bounded by a finite run count. The loop docs add the details I care about: a running turn is never interrupted, no deadline is inferred from prompt text, and a mutation only reports success after persistence; if the state write fails, the tool reports failure instead of pretending the loop is durable.

The big bundle and the small stuff

#57 is the large one: 22 locally prepared commits, 50 files, merged as "reviewed nightly reliability fixes". Most of the commits are narrow: memory worker lifecycle and revalidating queued memories before embedding, a hard timeout for direct scheduler scripts, keeping the old Cron schedule when a reload fails validation, gateway pending invalidation and overlay preservation, SSE keepalive backpressure, unique staging files for durable writes, archive pagination, timer bound checks, targeted Unicode and UTF-8 guards, denying pending approvals when a turn aborts, and evaluator process cleanup. I will not pretend each of those is a headline. The PR lists its own open limits: review capture cancellation is unresolved, process-group termination is not a sandbox, and the scheduler timeout covers only the direct child. #58 then adds a commit the bundle missed, so progress heartbeats share the 50-entry progress budget.

#60 also adds a local /update status and /update apply [skills|cron|all] admin command. Read the update docs before getting excited: it verifies pinned content and reloads already provisioned local skills and Cron jobs, but it fetches nothing remote, does not pull Git and does not replace core code. Core always reports "rollout required". The same PR coalesces parallel progress lines for same-name tools and renews Matrix typing indicators. #61 replaces the play triangle in tool-start telemetry with a plain middle dot, which is exactly as dramatic as it sounds.

What merged and what shipped

All 22 PRs, #40 to #61, are merged into main within the window; the window-end head is 4111cd0 from #61. Eight release images were published in the window, and every merge is contained in one of them:

"Shipped" here means a published release image, nothing more. The PRs themselves repeatedly state that merging and building do not activate a running installation, and none of them claims a rollout. Nothing in this post claims that any installation runs these versions.

Scope of this post

This post covers 2026-10-24 20:00 CEST to 2026-10-25 08:00 CET, which is 18:00 to 07:00 UTC across the daylight-saving switch. All local times above are Berlin time. It covers harness/core, the only repository currently listed in the public harness organization that already existed during the window. When checked on 2026-10-25, the organization page listed three public repositories, but harness/plugins and harness/deployment-base were created after the cutoff on 2026-10-25, so their changes belong to a later write-up. That is a statement about the currently listed public inventory, not about private repositories or past visibility. #62 merged at 08:15 and v0.1.16 followed at 08:27, both after the cutoff, and a query for open pull requests returned none.

Verdict

Twenty-two merges in thirteen hours (the clock change added one) sounds like a firehose. Read them one by one and the pattern is restraint: polling got replaced by a bounded continuation plan, nightly self-improvement got a budget and a written warning label instead of a merge button, and integrity checks got a document that tells you exactly how to defeat them. If you are choosing an agent runtime to leave running overnight, that is the bar I would hold every one of them to, Hermes, Claude Code, Codex and OpenClaw included: show me where it stops, and show me where it says it cannot protect you.

Sources used