Field notes · 2026-08-25 · RunVouch
7 ways unattended AI agents fail silently — and one check for each
Seven ways a scheduled AI agent fails without an error — from a run that never starts to a run that does nothing — and the one detector that catches each.
I ran a crypto trading bot and a handful of scheduled AI agents unattended for about two years. Almost none of the incidents that cost me money or time produced an error. The process exited zero, the log said done, and the thing I actually wanted did not happen. That is what "silent failure" means in practice: the agent ran, and nothing was wrong except the outcome.
RunVouch is a dead man's switch, cost cap and outcome check for unattended AI agents: it expects a check-in on a cadence, watches cost and behaviour during the run, and alerts you when a run is missed, broken, empty or runaway. Below are the seven failure modes I hit most often, and the single detector I now use for each. The list is ordered roughly from "never started" to "ran perfectly and still failed".
1. The agent never ran at all
The boring one, and still the most common. The machine rebooted and crontab was not restored. The Claude Code Routine was paused during a plan change. The n8n instance was upgrading at 03:00. Nothing failed, because nothing executed, so there was nothing to log.
You cannot detect the absence of a run from inside the run. You need something outside that expects it. This is the classic dead man's switch pattern, the same one healthchecks.io documents for cron jobs: the job pings on completion, and the monitor alerts when the ping is late.
Detector: MISSED. Register the agent with a cadence and RunVouch alerts when the window passes with no check-in.
rv agent nightly-report --cadence 24h
2. The agent ran and crashed, but the wrapper swallowed it
My bot ran under a shell script that did python bot.py || true so a bad night would not stop the loop. That line hid three weeks of crashes. Headless Claude Code is better behaved — the docs state that claude -p exits with a non-zero code when the run fails — but only if something looks at the exit code.
Detector: FAILED. Wrap the command with rv run and the exit status is reported with the check-in, so || true further up the chain no longer matters.
rv run nightly-report -- claude -p "Write the daily summary" --bare
3. The agent ran, exited 0, and produced nothing
This is the one the target query is really about: the agent ran but did nothing. A model refusal, a missing API key that made every tool return an empty result, a prompt that changed meaning after a system update. The run completes, the exit code is 0, the output file is empty or is yesterday's file untouched.
Exit codes measure the process. They do not measure the work. The fix is to name the artefact the run is supposed to produce and check that it exists and is fresh.
Detector: NO_EVIDENCE. Point RunVouch at the output and it flags a run that finished without producing it.
rv run nightly-report --evidence-file out.html -- ./generate.sh
4. The agent got stuck calling the same tool with the same input
I once watched an agent call a search tool with the identical query 140 times because the tool returned a transient error and the model's reaction to "try again" was, reasonably, to try again. It never crashed. It just burned a token budget and, on a worse day, would have hammered an API that rate-limits by account.
Retry loops are visible in the transcript long before they are visible in the bill. Claude Code exposes every tool call via hooks such as PostToolUse and PostToolUseFailure, which is what the RunVouch plugin listens to.
Detector: RETRY_STORM. The same tool with identical input eight or more times in a run raises an alert, and can pause the agent via webhook. RunVouch never kills a running process; pausing means the next scheduled run does not start until you look.
5. The agent ran and cost ten times what it usually costs
Cost overruns in agents are rarely a single expensive call. They are a run that takes a longer path than usual: more files read, a subagent that spawns subagents, a context that compacts and re-reads. With --output-format json the headless CLI reports total_cost_usd per invocation, so the number exists; the question is whether anyone compares it to a limit before the second night.
Detector: BUDGET_RUN and BUDGET_DAY. Cap per run and per day. The plugin reads tokens and cost from the transcript, so there is nothing to instrument.
rv agent nightly-report --cadence 24h --cap-run-cost 2 --evidence
6. The agent ran, but the output slowly stopped being right
The slowest failure I have had was the trading bot's data feed changing its timestamp format. Every run succeeded. Every run produced output. The output was computed on stale candles and got slightly worse each week until I noticed by accident. There was no single bad run to alert on.
You catch this by comparing runs to their own history rather than to a fixed threshold. I use a seven-run median absolute deviation on duration, cost, tool count and evidence size: a run that is far outside its recent band is worth a look, even if it is "fine" in isolation.
Detector: DRIFT. Seven-run MAD over run metrics, with an alert when a run leaves the band.
7. The agent started and simply never finished
An MCP server that hangs on connect. A browser tool waiting on a page that never loads. A claude -p run holding open a background process — which, per the docs, is now capped at ten minutes by default, but only for that particular case. The run has started, so a naive "did it start" check is satisfied, and it never sends the completion ping that a cadence check would miss until the next window.
Detector: STALLED. A run that checked in as started and exceeds its expected duration without finishing gets its own alert, separate from MISSED, because the fix is different: something is hung, not absent.
Why one detector per failure mode
The temptation is to build one dashboard and eyeball it. I did that. The problem is that each of these seven failures looks normal on most dashboards, because most dashboards show process health, and every one of these is a process that was healthy. Each mode needs a question that is specific enough to be answerable by a machine: did a check-in arrive by the deadline, was the exit non-zero, does the artefact exist, was the same call repeated, is the cost over the cap, is this run outside its own band, has the run been open too long.
Workflow tools have partial answers. n8n's Error Trigger node handles mode 2 well and, by its own docs, only runs when an automatic workflow errors, so it does nothing for modes 1, 3, 5 and 6. Cron pingers handle mode 1. Cost dashboards handle mode 5 after the fact. What was missing for me was the cheap outcome checks in the middle, which is why RunVouch exists.
It works with Claude Code Routines and headless claude -p via the plugin, with OpenClaw, n8n and plain cron via the rv CLI or the Python and Node clients, and as an MCP server listed in the official registry as com.runvouch/runvouch. Alerts go to Telegram, Slack or a webhook. The CLI fails open: if RunVouch is unreachable, your agent still runs. Three agents are free, Solo is $9, Team is $29, the core is MIT and you can self-host it; hosted data stays in the EU.
FAQ
What does "AI agent silent failure" actually mean?
A scheduled agent that exits successfully, or never starts, without producing the result it was scheduled for. No exception, no non-zero exit, no alert. The seven modes above are the ones I have personally hit; each is detectable, but only from outside the run.
Why not just check the exit code and log to a file?
Exit codes catch mode 2 only. Modes 1, 3, 4, 6 and 7 all produce a zero exit or no exit at all. Logs help you diagnose after you know something is wrong; they do not tell you something is wrong. You need an expected check-in, an expected artefact, and a comparison to previous runs.
Will RunVouch stop a runaway agent?
It alerts and can pause the agent through a webhook so the next run does not start. It never kills a running process — killing mid-run is how you get half-written files, and I would rather be woken up than have a tool decide that for me.
Try it: free for 3 agents · Docs: Claude Code · cron