Case studyAgentEval · LLM · FastAPI · Python · Reliability Testingtoolmonkey.vercel.app ↗8 min read
ToolMonkey
Chaos Monkey for LLM Tool-Calling Agents
Problem
Every team building an LLM agent eventually hits the same wall in production. A tool times out and the agent keeps going with no data. A tool returns a plausible but wrong answer and the agent uses it confidently. A tool returns malformed JSON and the agent either crashes or quietly hallucates a response. The agent completes the task and the user gets a confident, completely wrong answer.
There is no standard way to test agent reliability before shipping. Unit tests check that tools work. They do not check what the agent does when tools fail. Most teams discover their agent failure modes through production incidents, not through testing.
I was reading about Netflix's Chaos Monkey, their system for deliberately breaking production infrastructure to find weaknesses before they become outages. The question I could not stop thinking about was: does anyone do this for LLM agents? I built ToolMonkey to answer that question.
Decision
The core idea is to create a controlled simulation environment where an AI agent is given tasks that require tool calls, but the tools are rigged to fail in four deterministic ways. The system then measures how the agent behaves under each failure mode and produces a structured reliability report.
What I built in V1: A backend simulation engine with 6 fake deterministic tools (search, calculator, database, weather, summarizer, code execution), 4 failure injection modes, a Groq-orchestrated agent loop, a 4-metric eval scoring system, and a 3-screen Next.js frontend. 15 predefined task scenarios with known correct answers so the system can score not just behavior but correctness.
What I built in V2: A provider-agnostic adapter layer so users can plug in their own model and API key. Instead of testing ToolMonkey's internal Groq agent, users can now stress-test their own model (OpenAI, Anthropic, Gemini, DeepSeek, Groq) against the same 15 scenarios and get a reliability report on their specific configuration. A compare mode lets you run two models side by side on identical test conditions and see the reliability delta.
What I rejected:
Real tool integrations were considered and dropped. Real APIs are non-deterministic, rate-limited, and require secrets management. The entire value of ToolMonkey comes from controlled, reproducible failure injection. If tools are real, you cannot guarantee a specific failure mode fires on every run. Fake deterministic tools are a feature, not a limitation.
Per-chunk LLM tagging for failure categorization was also considered. The simpler approach of pre-constructed failure strings injected before the agent sees the tool response turned out to be more reliable and faster, with no additional API calls at ingestion time.
System Architecture
The system has four main layers.
Failure injection layer: When a tool call is made, the failure engine intercepts the response before the agent sees it. In V1 this is a middleware intercept. In V2, failure strings are pre-constructed and injected directly into the normalized response. Four modes are available: timeout (15 second delay), wrong answer (plausible but incorrect data), malformed JSON (broken or incomplete response structure), and silent failure (empty string or null response).
Orchestration layer: A Groq Llama 3.3 70B agent receives the task, selects which tool to call, processes the (potentially corrupted) response, decides whether to retry or flag uncertainty, and synthesizes a final answer. The agent loop is intentionally not hardened, which is the point. You want to observe default behavior, not behavior that has been manually patched to handle failures.
Eval layer: Four metrics were defined in writing before a single line of scorer code was written. Task Completion Rate measures whether the agent produced a usable final answer. Failure Detection Rate measures whether the agent explicitly noticed that a tool failed. Retry Efficiency measures whether retries actually led to recovery. Silent Failure Rate measures whether the agent produced a confident answer using bad or missing data without flagging anything. Silent Failure Rate has a target of below 10%. It is the most dangerous metric because it is the one the user never sees.
Adapter layer (V2): An abstract base class defines the interface all providers must implement: translate_request(), translate_response(), get_provider_name(). Five adapters are built: Groq, OpenAI, Anthropic, Gemini, DeepSeek. Each handles the provider-specific authentication format, request schema, and response parsing. A key validation endpoint fires a minimal real API call to confirm the user's key works before running a full simulation.
Key pool management: All internal API calls rotate across multiple keys from separate accounts using a KeyPoolManager singleton. Round-robin rotation with a 65 second cooldown on 429 responses. The critical rule: multiple keys from the same account share the same quota pool and provide zero benefit. Every key must come from a separate account.
Technical Choices
Groq Llama 3.3 70B as the orchestrator. The original design used Gemini. Mid-build, it became clear that Gemini's free tier returns limit: 0 on all models for India-based accounts. This is a regional restriction, not a rate limit. All four Gemini keys across four separate accounts hit the same wall simultaneously. Groq was switched in as the orchestrator with no architecture changes because the direct REST pattern was already in place. The adapter layer in V2 was partially motivated by this experience: never hardcode a single provider into the core flow.
Direct REST over SDK. Every API call in the codebase uses requests.post() directly, not any provider SDK. This was the lesson carried forward from Filtr. SDKs add version compatibility coupling and can route through gRPC in ways that make debugging harder. REST calls are explicit, debuggable, and provider-portable.
Metrics defined before scorer code. This was a deliberate process constraint. The four metrics and their formulas were written down and agreed on before eval/scorer.py was created. The reason is that post-hoc metric design is biased toward metrics that make the system look good. Pre-defined metrics force you to measure what matters, not what flatters.
Semantic evaluator with majority vote. V2 added a Groq-based semantic evaluator that uses three separate LLM calls to judge whether a failure was detected, then takes the majority vote. One call is not sufficient because LLM judgment on edge cases is noisy. Three calls with majority vote reduces variance meaningfully.
Session-only key handling. User keys in V2 are never stored. They travel with the request, are used for the duration of the simulation, and are discarded. This was both a privacy decision and a practical one: storing user API keys requires auth, encryption, and key management infrastructure that is out of scope for V1.
Metrics
V1 results on scenario C1 (What is 847 multiplied by 23?) across all four failure modes:
| Failure Mode | Completion | Detection | Silent Failure | Retries |
|---|---|---|---|---|
| none | 100% | N/A | 0% | 0 |
| wrong_answer | 100% | 0% (MISSED) | 0% | 0 |
| malformed_json | 100% | 0% (MISSED) | 0% | 0 |
| silent_failure | 100% | 100% | 25% | 2 |
Aggregate Health Score: 72.5 out of 100 (Amber zone)
The result that validated the entire project: the agent completed 100% of tasks while only detecting 33% of injected failures and silently failing 25% of the time. Task completion rate is a vanity metric for agents. An agent that finishes every task but uses garbage data on 1 in 4 runs is not reliable. It is dangerous.
What Broke
The Gemini regional quota wall. This was the single most disruptive problem in the build. The original architecture had Gemini as the orchestrator. Four keys across four separate Google accounts were set up. All four returned limit: 0 simultaneously. This is not a rate limit that resets. It is a regional restriction applied to all India-based accounts across all models. The entire orchestration layer had to be rebuilt around Groq in the middle of Day 1.
The lesson is not "use Groq instead of Gemini." The lesson is that regional API restrictions are a real infrastructure risk that does not appear in API documentation. Any system designed around a single provider's free tier for a specific region is fragile. The V2 adapter layer is the architectural response to this: no single provider is load-bearing.
The security incident. An env.txt file with real API keys was accidentally staged for commit. GitHub's push protection caught it before the push completed. All Groq and Cohere keys were rotated immediately. Gemini keys were not in the file and were not exposed.
The root cause was creating env.txt as a scratch file while setting up the environment, then accidentally staging it with git add. The guardrail added after: always run git diff --staged before every commit and scan for gsk_, AIza, or any real key string pattern.
UptimeRobot sending HEAD requests. The initial /health endpoint only accepted GET. UptimeRobot's free tier sends HEAD requests for its monitoring pings. FastAPI returned 405 Method Not Allowed, UptimeRobot marked the service as down, and the Render instance started spinning down between pings. Fixed by changing the route decorator to accept both GET and HEAD: @app.api_route('/health', methods=['GET', 'HEAD']).
Malformed JSON not being detected. The wrong_answer and malformed_json failure modes both showed 0% failure detection in testing. The agent received a response with no result field and hallucinated a correct-looking answer rather than flagging the missing data. This was not a bug in the failure injection. It was the expected behavior of an unhardened agent, which is exactly what ToolMonkey is supposed to surface. The insight was documented as a finding, not patched away.
Retrospective
The most important decision I made was defining the four eval metrics before writing the scorer. It sounds like a minor process detail but it completely changed what got measured. In every AI project I have seen, metrics get defined after the system is built, which means they get defined around what the system already does well. Defining them first means measuring what actually matters for production reliability, including metrics that make the system look bad.
The V2 adapter layer was the right architectural call but it was also the most time-consuming part of the build. Each provider has a different authentication scheme, request format, and response structure. Anthropic uses an x-api-key header. Gemini uses a query parameter. OpenAI and DeepSeek share a schema but DeepSeek has reasoning model variants that need special handling. Building five adapters that all conform to the same abstract interface took significant time. In retrospect I would have built the base adapter and one implementation first, validated the interface, and then built the remaining four.
If I were starting V3 today, the first thing I would add is score history with timestamped runs. The current architecture produces a score and discards it. The most valuable thing for a builder is not their score on a single run but how their score changes as they modify their agent. A comparison over time is what turns ToolMonkey from an interesting experiment into a tool people come back to.