Docs › CrewAI scheduled crew monitoring with RunVouch
CrewAI scheduled crew monitoring with RunVouch
CrewAI scheduled crew monitoring for the partial answer: a crew has max_iter per agent and max_rpm; neither is a budget. A nightly crew that keeps re-planning burns tokens until the iteration limit, then returns whatever it has.
How it runs on CrewAI
Around crew.kickoff() with the Python client, plus run.tool() from a step callback for loop detection.
Store the key
An environment variable on the host or container that runs the crew.
Wrap the job
import runvouch
from crewai import Crew
def on_step(step):
run.tool(step.tool or "llm", {"input": step.tool_input or ""}, cost=getattr(step, "cost", 0))
crew = Crew(agents=[...], tasks=[...], step_callback=on_step)
with runvouch.vouch("weekly-competitor-brief", evidence=lambda: {"brief": len(out.raw) > 500}) as run:
out = crew.kickoff()rv fails open: if RunVouch is unreachable the job still runs and you get one warning line.
Register the cadence and caps
rv agent nightly-report --cadence 24h --grace 30m --max-runtime 1h --evidence --cap-run-cost 2
Register the agent once, from anywhere with the key. Cadence is what turns a schedule that stopped into an alert; --evidence makes a run without evidence a failure; the caps pause the agent when it overspends.
What goes silent on CrewAI
- Reaching
max_iteris not an error; the crew returns a partial answer. Evidence (length, file, URL) is what fails it. - Per-agent token usage is available in
crew.usage_metrics; pass it as cost so the daily cap works.
What CrewAI does not tell you
CrewAI reports token usage per crew run in usage_metrics and stops an agent at max_iter by returning its best answer so far, without raising. There is no built-in budget, no notion of a schedule, and no notification; the crew's verbose log is printed to the console of whatever started it.
What RunVouch detects
| Alert | What it means here |
|---|---|
| MISSED | cadence plus grace passed and no run started |
| FAILED | non-zero exit or a reported failure, with the stderr excerpt |
| NO_EVIDENCE | the run said ok but the file, URL or assertion you required is missing |
| STALLED | a run started and never ended within max runtime |
| RETRY_STORM | the same tool called with identical input many times in one run |
| BUDGET_RUN / BUDGET_DAY | cost cap crossed; the agent is paused until you resume it |
| DRIFT | duration or output size far off its 7-run baseline |
Set up in two minutes
- Get a free key (3 agents, no card) and store it where this page says.
- wrap the job: copy the snippet above into the scheduled job.
- Register the cadence once:
rv agent nightly-report --cadence 24h --grace 30m --evidence, or let the first run create the agent and set the cadence on the dashboard. - Send one test alert:
curl -X POST https://api.runvouch.com/v1/settings/test-alert -H "X-API-Key: $RUNVOUCH_KEY". The next missed, failed or empty run reaches the same channels.
Also running agent frameworks?
AutoGen. OpenAI Agents SDK. Pydantic AI and smolagents. Dify and Flowise. The full list of runtimes is on integrations, and the same agent can report from several of them: one job, one cadence, wherever it happens to run.
Need a key? Get a free key · Stuck? contact